Add per-word highlighting to user defined words in syntax highlighting

[akalenuk]

The idea behind per-word highlighting is to show things that are different different and thing which are the same - the same. So 'edx' is visually much different from 'ebx' and 'uint_16' from 'uint_64'. I've been using per-word highlighting for reviewing code in C, C++ and Assembly and it has always been a great help.

Here's an example of what I mean: http://awesomescreenshot.com/0bd3072r87 The above is plain code and below is the same piece per-word highlighted.

On the other hand, in high level languages we woudn't mix 'if' for 'else' anyway, but we'd like this words to denote a code structure, so syntax highlighting, not per-word, is preferable.

I've made a hack, that brings complementary per-word highlighting over syntax highlighting provided by 'pygments'.

Here's the code for a colorer:
https://github.com/akalenuk/codrspace/blob/master/apps/codrspace/pygments/over_pygments.py

And that's about how I think of patching the templatetag, so one could chose per-word coloring for user defined words by adding 'more_colors' to a 'code' tag:
akalenuk@29ea2fc#diff-1

It's all ready and working in my fork (the example from above is a locally hosted Codrspace screenshot) and I'm ready to send a pull request any moment. Yet I'm not sure about if this hack is the right way to go. I mean, it isn't orthogonal to pygments and it is not a fair patch on it. It just parasites on its output. If I were to review my own code, I would mark this as a potential malleability issue.

So, what do you think?

[durden]

Thanks for working on this. It's an interesting idea. However, I'm wondering if this should be an issue integrated into pygments itself. That is a much better long-term solution.

Maybe you could propose the idea to the pygments developers and see what their thoughts are.

We could support this on the codrspace side as well similar to how you developed it. However, I don't think it's feasible to run a forked version of pygments to accomplish this. Maybe we just support a different 'language' in the [code] tag that bypasses pygments completely. This is still a bit of a hack, but it might be an easier way to accomplish this if there's demand for this mode.

The other solution is you could write a stand-alone script to syntax highlight your own posts BEFORE submitting it to codrspace. Then you might be able to post that html output directly into codrspace. The caveat here is that you'll need to make sure any html you generate is compatible with the 'safe' tags we allow in the codrspace text entry.

[akalenuk]

No-no, I'm not going to support a fork just for a sole feature :-) It's just something to show it in action.

For now I've made it orthogonal to existing pygments highlighting. Look: http://awesomescreenshot.com/05d30c4m8c You can just switch it with a special flag and it would work with any language pygments supports! You can have both syntax highlighting for language core features and per-word for user defined entities such as variables or methods. If I do it with a special language, orthogonality would not be possible.

I'd dig deeper into pigments to see if it would be easy enough to integrate. This might take some time though.