fix(sync): materialize git-backed blobs so large pull/fetch stops re-inflating (promptctl-sync-pull-l07p)

[brandon-fryslie]

Fixes the lit sync pull/fetch large-pull slowness over a git-backed remote (ticket promptctl-sync-pull-l07p) — the same git-blob ranged-read pathology PR #274 fixed for lit init adopt, now fixed at the source so every sync path benefits.

Root cause

On a git-backed remote, Dolt's NBS table files live inside git blob objects, which have no random access. dolt's GitBlobstore served a ranged read by streaming the whole blob through git cat-file and discarding the prefix (store/blobstore/git_blobstore.go: sliceInlineBlob). NBS reads a table file with many small ranged reads — footer (a negative-offset tail read), index, then chunk-by-chunk — so each read re-inflated nearly the entire ~18.5 MB archive blob from byte 0. A large DOLT_FETCH/DOLT_PULL was O(reads × blobsize): tens of GB of redundant zlib and thousands of subprocess spawns. (Incremental syncs pull a small delta and were already fine; #274 already routed the cold init-adopt through clone.)

Decision (the ticket asked for one)

Three options were on the table:

1. Fix it at the source in dolt's blobstore — materialize each git blob to a local file once, serve ranges via Seek.
2. lit-side clone-then-merge for "large" pulls.
3. lit-side blob-materialization shim.

Chosen: option 1. It's the single seam ([LAW:single-enforcer]) where "git blob, no random access" meets "NBS table file, expects random access," so one change fixes fetch and pull and receive and clone/adopt at once. Option 2 requires a "large vs small pull" branch ([LAW:no-mode-explosion]/[LAW:dataflow-not-control-flow]), duplicates transfer logic, and leaves fetch/receive slow. Option 1 is also literally what dolt's own TODO(gitblobstore) recommends.

How it ships

The fix lives entirely inside the pinned upstream dolthub/dolt. This PR adds a transient replace directive pointing github.com/dolthub/dolt/go at a one-commit fork that carries the patch:

replace github.com/dolthub/dolt/go => github.com/brandon-fryslie/dolt/go v0.40.5-0.20260630102541-e6f5f8e3a886

The upstream PR is open at dolthub/dolt#11264. Once it merges, drop the replace and bump the dolthub/dolt/go pin — the fork is temporary.

Regression guard

The authoritative regression test lives with the code it guards, in the fork / upstream PR: TestGitBlobstore_RangedReads_MaterializeBlobOnce asserts that N scattered ranged reads of one blob invoke BlobReader exactly once (pre-fix: N; post-fix: 1). On the lit side, the full ./internal/... suite — including the real-remote TestAdoptRemoteByCloneBootstrapsAndReAdopts and TestSyncReconcile* integration tests — passes green against the patched dolt, confirming the change is behavior-preserving (same bytes, served from a file instead of a re-inflated stream).

https://claude.ai/code/session_01TMiknvhzuS1tM8frgeDwmH

[github-actions]

Coding Agent Review

Reviewed 1 scope(s): go-deps.

go-deps — Reviewed go.mod replace directive and go.sum entries swapping github.com/dolthub/dolt/go for the brandon-fryslie fork. The change is clean: correct syntax, proper go.sum entries, informative WHY comment with upstream PR reference and removal plan. No LAW violations introduced. Dependency direction remains one-way.

✅ Approved

Reviewed by config auto→deepseek / claude-code / deepseek-v4-pro.

Cost: $0.0182 · 198,295 in / 5,064 out tokens · claude-code/deepseek-v4-pro · est.