Windows: fix non-ASCII chat prompt corruption (ANSI codepage vs UTF-8)#366
Merged
Merged
Conversation
coli passes the chat prompt to glm.exe through the PROMPT/COLI_PROMPT environment variable. On Windows, plain getenv() is populated by the CRT from the ANSI-codepage view of the environment block, not UTF-8 — so any non-ASCII prompt text (Cyrillic, CJK, ...) is silently mangled before the byte-level tokenizer ever sees it, even though the parent process (coli's Python subprocess call) sets the value correctly via the wide env block. Add compat_getenv_utf8() in compat.h: reads the variable through GetEnvironmentVariableW and converts straight to UTF-8, bypassing the ANSI codepage entirely. No-op passthrough to getenv() on non-Windows platforms. coli_user_prompt() in glm.c now uses it for both COLI_PROMPT and PROMPT. Verified: tiny-oracle self-test still 32/32 after rebuild, and a real run against the full GLM-5.2-int4 model with a Cyrillic prompt now round-trips correctly end to end (input echoed intact, coherent Cyrillic output), where it previously produced replacement-character garbage on both input and output.
JustVugg
added a commit
that referenced
this pull request
Jul 19, 2026
fix(test): make test_stops build on Windows (mkdtemp compat shim) Conflict with #366's getenv_utf8 in compat.h resolved by keeping both blocks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
coli run/coli chatpass the prompt toglm.exethrough thePROMPT/COLI_PROMPTenvironment variable. Plaingetenv()there is populated by the CRT from the ANSI codepage view of the environment block, not UTF-8 — so any non-ASCII prompt text (Cyrillic, CJK, ...) gets silently mangled before the byte-level tokenizer ever sees it, even thoughcoli's Pythonsubprocesscall sets the value correctly through the wide (UTF-16) environment block.compat_getenv_utf8()incompat.h: reads the variable viaGetEnvironmentVariableWand converts straight to UTF-8, bypassing the ANSI codepage entirely. No-op passthrough togetenv()on non-Windows platforms — zero impact there.coli_user_prompt()inglm.cnow uses it for bothCOLI_PROMPTandPROMPT.How I found it
Ran a fresh Windows 11 native build/install per the README and hit this while testing
coli runwith a Russian prompt: both the echoed input and the generated output came back as�replacement-character garbage. Traced it to the env-var read path incoli_user_prompt()(glm.c) — narrowgetenv()round-tripping non-ASCII text through the system ANSI codepage.Test plan
glm.exe(MinGW-w64, gcc 16.1.0) with the fix."Привет! Кто ты?"): before the fix, input echo and output were both garbled; after the fix, the prompt round-trips intact and the model produces coherent Cyrillic output end to end.getenv_utf8is a plaingetenv()alias outside_WIN32).