Encoding problems when .gitattributes cannot convert UTF8 characters

[ivanbili]

I encountered a problem when in one of the files in my repo, instead of using the following default hyphen (-)
https://www.compart.com/en/unicode/U+002D
I used the following non default hyphen (‑):
https://www.compart.com/en/unicode/U+2011

In my .gitattributes file, the encoding is specified as zos-working-tree-encoding=ibm-1047.

When cloning such a repo on z/OS, the first thing I notice is that the file is treated as modified. When opening the file in vim, I see ^Z as the character instead of ‑, which means that it got converted to an illegal character. Inspecting the file using xxd, I see that there are 0x3f characters, which break tooling on z/OS.

Simply fixing this issue by doing the right conversion to a hyphen might not be good, because it will be a many-to-one conversion with loss of information. Some ideas:

• Issue a warning on the z/OS side if something like this happens when you clone the repo
• Provide a git hook or GitHub check that will prevent one from committing a character that you know will mess up conversion as per .gitattributes (this in itself is not enough because .gitattributes can change at a later point).