Pygments; need a regex to replace \b

[btiffin]

Hello,

I'm coding up a Pygments lexer for OpenCOBOL and require a solid replacement for the word boundary \b regex operator. COBOL includes hyphen along with underscore as part of valid identifiers.

I'm feeling lazy today, and don't feel like experimenting or thinking.

code:

(?<=(\w|[-]))   # start of word? (?=(\w|[-]))   # end of word?

These aren't right. I need something that doesn't consume any characters but detects transitions ala \b.

Any gurus in the crowd?
Cheers

[OneOffDriveByPoster]

I think it may be a bad idea, but is it possible to modify the LOCALE so that it is like the C locale but with '-' in the character class you want?

[btiffin]

Update: Being lazy just didn't work out. Ignoring the problem eventually got to me.

(and thanks for the one option OneOffDriveByPoster)

Pygments lexers usually use

code:

r'\b(keyword1|keyword2|otherword)\s*\b'

to scan for special highlight words. I ended up with

code:

r'(^|(?<=[^0-9a-z_\-]))(keyword1|keyword2|otherword)\s*($|(?=[^0-9a-z_\-]))'

for a version of word boundary zero length (and trailing space chewing) match that includes hypen.
re.IGNORECASE | re.MULTILINE in effect.

I'm more than open to criticism or belittling betterments; as Python regex is not really my thing.

And to see the fruits of the labour, overtly promoting OpenCOBOL again; be warned
http://opencobol.add1tocobol.com/#what-is-ocdoc

Cheers
Edit; added the opencobol FAQ link

[btiffin]

I finally got round to making a pull request for Pygments that adds the COBOL highlighting. Georg was helpful. Turns out adding dash to a word boundary is as easy as

code:

r"\b(?!-)"

Yayy for smart people that see trees AND forests. No need for the potentially (and needlessly) CPU sucking ?<= back scanning operator.

Cheers