Help wanted (native Windows): delivery sqlite JSON round-trip writes invalid/wrong output on windows-latest

[fujibee]

Looking for a contributor with a native Windows + Git Bash environment 🙏

We've made the Codex bridge and the in-memory sqlite captures work on Windows (#179, #180), and quarantined the watcher/bridge process-gap tests (#182). What's left is a cluster of delivery.sh failures that only reproduce on native Windows / windows-latest and need someone who can run and debug there — the maintainers don't have a Windows box.

These are real bugs, not test fragility: the delivery set JSON round-trip (readfile() → sqlite json_set/json_object → writefile()) produces invalid JSON or wrong values on Windows, even after the CRLF-in-captured-output fix (#180). They are currently red on the non-required windows-latest (full, experimental) leg (not skipped — we want them visible until fixed).

Symptoms (failing tests on windows-latest, all green on Linux/macOS)

A. Invalid JSON written by delivery set (the file fails json_valid / python json.load):

• delivery set monitor: round-trips multibyte (UTF-8) settings without a short-write reject (test_delivery.bats)
• delivery set monitor: existing settings with single-quoted hook commands stays valid JSON (#134)
• hook on: is idempotent (test_hook.bats — json.load throws → count empty)

B. Wrong agmsg-entry count or extracted value (idempotency check sees ≠1 entry, or json_extract returns the wrong scalar — suspected: the stored hook command path is in a different form, e.g. C:\… vs /c/…, so the instr(command, <skill-dir>) match in agmsg_entries misses):

• delivery set monitor: idempotent / delivery set both: idempotent across repeats
• delivery: monitor is idempotent for SessionEnd entry
• delivery set monitor: preserves unrelated settings
• delivery set turn (copilot): writes .github/hooks/agmsg.json with version + Stop entry
• delivery set monitor (copilot): does NOT delete an existing turn hook
• delivery set turn (copilot): idempotent across repeats

C. Large-settings (E2BIG) round-trip (extracted first/last entry or array length wrong, or set off exits non-zero):

• delivery set monitor: handles a large settings.local.json without E2BIG (#95)
• delivery set both: handles a large settings.local.json across strip+add+prune (#95)
• delivery set off: idempotent strip on a large settings.local.json (#95)

(Related but possibly separate: install: actas-claim runs in a fresh shell with no inherited env — actas-claim.sh exits non-zero under env -i on Windows. Likely a PATH/env issue.)

Suspected root causes (need on-Windows confirmation)

1. CRLF inside the file content / JSON values. #180 stripped CR from captured command output, but if a CR rides into the settings blob through readfile()/writefile() (or the input file has CRLF), the resulting JSON string can contain a raw \r, which is invalid JSON. Check the bytes of the file delivery set monitor writes on Windows (xxd/od -c).
2. Path form in stored hook commands. agmsg_entries/strip match agmsg-owned entries via instr(command, '<skill-dir>'). If the command is stored with a Windows path (C:\Users\…, via cygpath in sql_readfile_path) while the matcher uses the Git Bash form (/c/Users/…), the match misses → wrong count.

How to reproduce

1. Windows + Git for Windows (Git Bash) + sqlite3 on PATH (e.g. choco install sqlite).
2. git clone + ./install.sh --cmd agmsg into a temp HOME.
3. bats tests/test_delivery.bats — or the focused tests above. They pass on Linux/macOS, fail on Git Bash.
4. Inspect the written .claude/settings.local.json / .github/hooks/agmsg.json bytes and the commandWindows/command values.

Already fixed (context)

• fix(codex): Git Bash compatibility for the Codex bridge (portable hash + bash-exec) #179 — Git Bash compat: portable sha1 (no shasum), codex-bridge.js spawns .sh via bash (EFTYPE).
• fix: strip CR from sqlite3 output so Windows Git Bash works (#130) #180 — strip CR from sqlite3 :memory: captured output across scripts (agmsg_sqlite_mem).
• Windows: quarantined experimental-leg tests (process/path/bridge gaps after #179/#180) #182 — separate quarantined cluster: watcher process-mgmt, codex-bridge resolve, symlink, commandWindows.

Happy to pair / review PRs. Comment here if you can help! 🐙

[Tsuki8825]

Hi, I have a Windows 11 + Git Bash environment and can help with testing.

My environment:

• Windows 11
• Git Bash: C:\Program Files\Git\bin\bash.exe
• sqlite3: 3.51.1 from Miniconda
• agmsg: v1.0.6
• Using Codex and Claude Code

After updating to v1.0.6, my previous delivery.sh set turn / mode turn issue from #143 is fixed in real usage. Codex and Claude Code can send/receive messages successfully.

I am a beginner, so I may not be able to debug deeply, but I can run specific commands/tests on my Windows environment and share the output if that helps.

[fujibee]

@Tsuki8825

Thank you — a Windows 11 + Git Bash setup is exactly what we're missing, and it's great to hear v1.0.6 cleared your #143 issue in real use. 🙏

Beginner-friendly is perfect: the most useful first data point is whether your sqlite3 build (Miniconda 3.51.1) slips a carriage return (\r) into the JSON during the read/modify/write round-trip that delivery set performs. Could you paste the output of this? It's self-contained and only writes to a temp file:

f="$(mktemp)"
sqlite3 :memory: "SELECT writefile('$f', json_object('matcher','','command','/c/Users/test/check-inbox.sh'))"
echo '--- raw bytes (we are looking for \r / 0d before 0a) ---'
od -c "$f" | head
echo '--- json_valid? expect 1 ---'
sqlite3 :memory: "SELECT json_valid(readfile('$f'));"
echo '--- how sqlite prints a plain row ---'
printf 'SELECT 1;\n' | sqlite3 :memory: | od -c
sqlite3 --version

If od -c shows a \r before the \n, that pins the root cause to the sqlite CLI's text-mode output on Windows, and we can fix it centrally. If the JSON is clean, the bug is elsewhere (likely a C:\… vs /c/… path mismatch) and I'll send a follow-up probe.

No pressure on timing — this alone is a big help. Thanks again for stepping up!

[Tsuki8825]

I ran the probe on my Windows 11 + Git Bash environment.

One note: with the exact command using f="$(mktemp)", my native Windows sqlite3 did not appear to write to the POSIX /tmp/... path. The output was:

--- raw bytes (we are looking for \r / 0d before 0a) ---
0000000
--- json_valid? expect 1 ---

--- how sqlite prints a plain row ---
0000000 1 \r \n
0000003
3.51.1 2025-11-28 17:28:25 281fc0e9afc38674b9b0991943b9e9d1e64c6cbdb133d35f6f5c87ff6af38a88 (64-bit)

I also ran the same probe using cygpath -w for the temp file path, and that produced this:

posix temp: /tmp/tmp.iuweUEb2wc
windows temp: C:\Users\gzd01\AppData\Local\Temp\tmp.iuweUEb2wc
55
--- raw bytes (we are looking for \r / 0d before 0a) ---
0000000 { " m a t c h e r " : " " , " c
0000020 o m m a n d " : " / c / U s e r
0000040 s / t e s t / c h e c k - i n b
0000060 o x . s h " }
0000067
--- json_valid? expect 1 ---
1
--- how sqlite prints a plain row ---
0000000 1 \r \n
0000003
--- sqlite path/version ---
/c/Users/gzd01/miniconda3/Library/bin/sqlite3
3.51.1 2025-11-28 17:28:25 281fc0e9afc38674b9b0991943b9e9d1e64c6cbdb133d35f6f5c87ff6af38a88 (64-bit)> @Tsuki8825

Thank you — a Windows 11 + Git Bash setup is exactly what we're missing, and it's great to hear v1.0.6 cleared your #143 issue in real use. 🙏

Beginner-friendly is perfect: the most useful first data point is whether your sqlite3 build (Miniconda 3.51.1) slips a carriage return (\r) into the JSON during the read/modify/write round-trip that delivery set performs. Could you paste the output of this? It's self-contained and only writes to a temp file:

f="$(mktemp)"
sqlite3 :memory: "SELECT writefile('$f', json_object('matcher','','command','/c/Users/test/check-inbox.sh'))"
echo '--- raw bytes (we are looking for \r / 0d before 0a) ---'
od -c "$f" | head
echo '--- json_valid? expect 1 ---'
sqlite3 :memory: "SELECT json_valid(readfile('$f'));"
echo '--- how sqlite prints a plain row ---'
printf 'SELECT 1;\n' | sqlite3 :memory: | od -c
sqlite3 --version
If od -c shows a \r before the \n, that pins the root cause to the sqlite CLI's text-mode output on Windows, and we can fix it centrally. If the JSON is clean, the bug is elsewhere (likely a C:\… vs /c/… path mismatch) and I'll send a follow-up probe.

No pressure on timing — this alone is a big help. Thanks again for stepping up!

[fujibee]

@Tsuki8825

Thank you, this is exactly the data we needed — and your cygpath -w instinct nailed it. 🙏 Two things are now confirmed from your output:

1. With a Windows path (C:\…), writefile() produces clean, valid JSON (json_valid = 1, no stray \r).
2. With a POSIX path (/tmp/…), writefile() silently writes 0 bytes (your first run showed an empty file). So native Windows sqlite3 can only read/write files given a C:\… path.

That second point is probably the heart of the remaining failures, and honestly you've already given us the key insight — this is enough for us to dig in from here. 🙌

If you're up for one more optional probe, it would help us confirm the exact spot, but please don't feel you have to — it's a bit fiddly, and it's totally fine to stop here or let someone else pick it up. No pressure at all:

# from the repo root
proj="$(mktemp -d)"
bash scripts/delivery.sh set monitor claude-code "$proj"
f="$proj/.claude/settings.local.json"

echo '--- 1. real output bytes (is the JSON valid + complete?) ---'
od -c "$f" | head -40

echo '--- 2. read back via a POSIX path (a number = ok) ---'
sqlite3 :memory: "SELECT json_array_length(json_extract(readfile('$f'), '\$.hooks.SessionStart'));"

echo '--- 3. read back via a Windows path ---'
winf="$(cygpath -w "$f")"
sqlite3 :memory: "SELECT json_array_length(json_extract(readfile('$winf'), '\$.hooks.SessionStart'));"

If it's at all unclear or doesn't run cleanly, no worries — just say so and we'll take it from here. Either way, you've already moved this forward a lot, and we really appreciate it.

[Tsuki8825]

I ran the optional probe from the repo root.

Output:

Delivery mode set to 'monitor' for /tmp/tmp.O4P2V9w7TB (claude-code)
Future sessions: SessionStart hook will auto-launch the watcher.

AGMSG-DIRECTIVE: For this running session, invoke the Monitor tool now with:
command: /c/Users/gzd01/agmsg/scripts/watch.sh agmsg-1782089488-1563 /tmp/tmp.O4P2V9w7TB claude-code
description: agmsg inbox stream
persistent: true

Pass the command above verbatim — the session_id is already filled in.
--- probe paths ---
proj=/tmp/tmp.O4P2V9w7TB
f=/tmp/tmp.O4P2V9w7TB/.claude/settings.local.json
--- 1. real output bytes (is the JSON valid + complete?) ---
0000000 { " h o o k s " : { " S e s s i
0000020 o n S t a r t " : [ { " m a t c
0000040 h e r " : " " , " h o o k s " :
0000060 [ { " t y p e " : " c o m m a n
0000100 d " , " c o m m a n d " : " ' /
0000120 c / U s e r s / g z d 0 1 / a g
0000140 m s g / s c r i p t s / s e s s
0000160 i o n - s t a r t . s h ' ' c
0000200 l a u d e - c o d e ' ' / t m
0000220 p / t m p . O 4 P 2 V 9 w 7 T B
0000240 ' " } ] } ] , " S e s s i o n E
0000260 n d " : [ { " m a t c h e r " :
0000300 " " , " h o o k s " : [ { " t y
0000320 p e " : " c o m m a n d " , " c
0000340 o m m a n d " : " ' / c / U s e
0000360 r s / g z d 0 1 / a g m s g / s
0000400 c r i p t s / s e s s i o n - e
0000420 n d . s h ' ' c l a u d e
0000440 - c o d e ' ' / t m p / t m p
0000460 . O 4 P 2 V 9 w 7 T B ' " } ] }
0000500 ] } }
0000503
--- 2. read back via a POSIX path (a number = ok) ---

--- 3. read back via a Windows path ---
winf=C:\Users\gzd01\AppData\Local\Temp\tmp.O4P2V9w7TB.claude\settings.local.json
1
agmsg: instance-id falling back to bare session_id (agent pid unresolved for type=claude-code); parallel --continue/--resume isolation is degraded

Summary: it looks like the file was written correctly, but readback only worked with the Windows path. The POSIX-path readback printed a blank line, while the Windows-path readback returned 1.

@Tsuki8825

Thank you, this is exactly the data we needed — and your cygpath -w instinct nailed it. 🙏 Two things are now confirmed from your output:

1. With a Windows path (C:\…), writefile() produces clean, valid JSON (json_valid = 1, no stray \r).
2. With a POSIX path (/tmp/…), writefile() silently writes 0 bytes (your first run showed an empty file). So native Windows sqlite3 can only read/write files given a C:\… path.

That second point is probably the heart of the remaining failures, and honestly you've already given us the key insight — this is enough for us to dig in from here. 🙌

If you're up for one more optional probe, it would help us confirm the exact spot, but please don't feel you have to — it's a bit fiddly, and it's totally fine to stop here or let someone else pick it up. No pressure at all:

from the repo root

proj="$(mktemp -d)"
bash scripts/delivery.sh set monitor claude-code "$proj"
f="$proj/.claude/settings.local.json"

echo '--- 1. real output bytes (is the JSON valid + complete?) ---'
od -c "$f" | head -40

echo '--- 2. read back via a POSIX path (a number = ok) ---'
sqlite3 :memory: "SELECT json_array_length(json_extract(readfile('$f'), '$.hooks.SessionStart'));"

echo '--- 3. read back via a Windows path ---'
winf="$(cygpath -w "$f")"
sqlite3 :memory: "SELECT json_array_length(json_extract(readfile('$winf'), '$.hooks.SessionStart'));"
If it's at all unclear or doesn't run cleanly, no worries — just say so and we'll take it from here. Either way, you've already moved this forward a lot, and we really appreciate it.

[fujibee]

Resolved by #187 (landed in 1.1.0). The delivery sqlite round-trip failures on windows-latest were a test-harness path-form gap: the tests read the written settings back through readfile() using a POSIX path, which native Windows sqlite3 cannot open, so the assertions failed even though the file the script wrote was valid. The tests now route those paths through cygpath -w (a no-op off Windows). Thanks @Tsuki8825 for the on-Windows diagnostics that pinned the root cause. The separate actas-claim pid-liveness item was addressed by #196 (#134). Closing as done.