Fixes #163 - tokens with backslashes

[robstoll]

Fixes #163 - fixed token file generation for tokens with backslashes in their corresponding literal part

[sharwell]

Can you also add a regression test? I want to make sure that it not only parses, but is recognizing the correct sequence of input characters as a token.

[robstoll]

Sure. Could you please point out where I should add the test? I could not find (quickly) a test relating to .tokens file generation.

[sharwell]

Probably in TestJavaCodeGeneration.java.

[robstoll]

Thanks for the quick answer, I must have overlooked this file. I will create a test in the coming days.

[robstoll]

Found some time to write the tests. Please let me know if these regression tests should not be enough

[sharwell]

This is not a valid change. See #163 for details.

[robstoll]

You should at least merge the regression test

testTokenVocabWithUnescapedBackslashThrowsNullPointerException

And you will see that the current implementation is wrong. I am happy if you fix it otherwise but it is still a bug IMO

[sharwell]

The error is a duplicate of #157

[robstoll]

Fair enough, I figured that are two different things. Anyway, I am looking forward to the merge of your pull request #165 and will see if it works for me as well