Skip to content

feat(b1): ingestion service gains /ingest and KEEPS a /convert shim (PR A of 3, register #51) - #72

Open
tornidomaroc-web wants to merge 2 commits into
mainfrom
feat/b1a-service-ingest-with-convert-shim
Open

feat(b1): ingestion service gains /ingest and KEEPS a /convert shim (PR A of 3, register #51)#72
tornidomaroc-web wants to merge 2 commits into
mainfrom
feat/b1a-service-ingest-with-convert-shim

Conversation

@tornidomaroc-web

Copy link
Copy Markdown
Owner

PR A of 3. SERVICE-ONLY. Zero files under src/.

services/ingestion/ gains /ingest (convert → chunk → embed → persist chunks → write the terminal documents status, authenticating as the uploading user via a forwarded JWT layered on the public anon key, never service-role) while KEEPING /convert byte-identical to its pre-(b1) contract. src/app/api/ingest/route.ts is deliberately UNTOUCHED and production Next still calls /convert.

Why three PRs and not one

Railway auto-deploys main and Vercel deploys main, independently and in an order nobody controls. A single merge that renamed the endpoint and repointed the caller had a real skew window: if Vercel won the race, the new route's /ingest call hit an image that did not have it yet — a 404, !pyResponse.ok, and every upload failing until the Docker build landed.

PR Contents Effect on production
A0 (#71, merged) CI only: build the image, import the module inside it None
A (this PR) /ingest added, byte-identical /convert shim kept New image goes live. Production Next still calls /convert.
B route.ts repointed to /ingest Next starts using /ingest, which is already live and already verified
C /convert shim deleted Register #50's blob endpoint closed again. Not optional.

Landing the service first, with both endpoints live, makes the window zero-length.

The shim's three omissions are load-bearing

  • No chunk/embedding length guard. The old contract truncates via zip and the old caller tolerates a chunk with no embedding key. The guard is a better behaviour and a different contract.
  • No Form(...) parameters. The old caller posts file alone; an extra required field would 422 every production upload.
  • No Supabase client and no persistence of any kind. The old caller does its own inserts; a write here would double-write every chunk for the shim's whole life.

Response is {markdown, filename, chunks[]}, with chunks produced by the same _chunk_text that /ingest uses.

This PR's green checkmark is not evidence

tsc and db-types are the two required checks. This PR contains zero TypeScript and zero SQL, so both pass vacuously — neither inspects a single line of Python. Railway's "Wait for CI" is OFF, so the deploy does not wait for them either.

ingestion-image (#71, register #52) is the only check that inspects the code this PR changes. It is deliberately not a required check — it has three third-party network dependencies in its hot path, and register #43's rule applies: "a gate that reds main on a coin flip is worse than no gate." This is also the first run whose image contains supabase==2.29.0, so its import assertion is finally doing the work it was built for.

PR B is gated on V1 through V12

docs/b1-verification-protocol.md is added here as the pinned, binding definition of "verified" — written before the work so the acceptance criteria cannot be renegotiated afterwards. It contains PR #70's eight verification points verbatim, the corrections to them (§2), the exact SQL (§4), and the V1-V12 binary (§8). PR B may not merge until every one of V1-V12 holds. There is no partial credit.

Four rollback triggers

  1. GET /health does not return HTTP 200 within 10 minutes of this merge — roll back, do not debug forward.
  2. V4 fails — a production upload through the /convert shim does not reach status='ready'. Live user harm, and it means the shim is not contract-identical after all.
  3. V5/N3 fails — a production Ask returns nothing or an ungrounded answer. The Ask path is the core product path and this PR rewrites the module that also serves /embed.
  4. N9 fails — an unauthenticated POST /convert returns anything other than 401. A 200 means the ingestion endpoint is publicly open: register docs: advance the three stale header lines to reality (main @ 63d498c, PR #44) #45's exact failure, and worse live harm than a failed upload, because a broken upload is visible and bounded while an open endpoint is invisible and unbounded.

Rollback is the Railway dashboard → Deployments → last pre-PR-A deployment → Rollback. It is valid for the entire PR A window and dies the moment PR B merges, because after that the pre-PR-A image does not have /ingest.

A citation lesson worth recording

During review, all 19 code/file line references in the new protocol were verified against main and reported as matching. That was the wrong baseline for one of them. A citation that points into a file the same PR edits is not an external reference, and verifying it against main rather than against the assembled tree is a distinct failure class. This PR's runbook edit turns 3 lines into 13, shifting everything below by +10, which silently invalidated the protocol's own docs/supabase-migration-runbook.md:112-115 citation (now :122-125). It was caught only by re-running the reference check against the assembled tree before pushing. Anyone assembling a docs change should verify self-referential citations post-assembly, not pre-assembly.


Scope: services/ingestion/main.py, services/ingestion/requirements.txt (adds supabase==2.29.0), docs/supabase-migration-runbook.md, docs/b1-verification-protocol.md (new), docs/PROGRESS.md. Five files. Zero under src/. No migration, no DB change, no branch-protection change, no Railway or Vercel dashboard change. Opens register #51 as IN PROGRESS. Register #17 stays OPEN.

Do not merge until item 8 / V3 passes (no service-role key on Railway; anon key decodes "role":"anon").

🤖 Generated with Claude Code

…PR A of 3, register #51)

PR A of the three-PR (b1) sequence, SERVICE-ONLY. services/ingestion/main.py
gains /ingest (convert -> chunk -> embed -> persist chunks -> write the terminal
documents status, as the uploading user via a forwarded JWT on the public anon
key) while KEEPING /convert byte-identical to its pre-(b1) contract.

src/app/api/ingest/route.ts is deliberately UNTOUCHED. Production Next still
calls /convert; repointing it is PR B, deleting the shim is PR C.

Adds docs/b1-verification-protocol.md, the pinned definition of "verified" for
this sequence, and corrects the two runbook lines that still told the operator
to verify Railway holds no SUPABASE_* vars.

Zero files under src/. No migration, no DB change, no branch-protection change,
no Railway or Vercel dashboard change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
knowflow Ready Ready Preview, Comment Jul 28, 2026 4:13pm

The row this PR added at docs/PROGRESS.md:150 was a SINGLE-CELL row in a
four-column table: 2 pipes, where the header (| # | Item | Why deferred |
Where addressed |) and every other row from #1 to #53 have 5 or more. Its
Item, Why deferred and Where addressed cells were empty and its entire body
rendered inside the narrow # column.

Repaired here rather than in a later PR for one reason: this PR introduced the
defect and this PR is still open, so fixing it here means the malformed row
never reaches main at all.

Separate commit, no amend and no force-push, so the defect stays visible in
this PR's history, the same way the vacuous-green fix was handled on #71.

ONE FACTUAL CORRECTION rides along, disclosed rather than folded in silently.
The old status narrative opened "PR A of 3 landed 2026-07-26" and marked
"A = ... (landed)". PR A has not landed: gh pr view 72 reports state OPEN with
mergedAt null and no merge commit, and git branch --contains c27163a lists only
the two feature branches, not main. PROGRESS.md:9 already contradicted it,
correctly describing the same work as the active branch. A §4 row asserting a
merge that did not happen is the false-check class registers #39 and #23 exist
to police, so the status cell now reads "opened 2026-07-26; PR A NOT YET MERGED
(PR #72 open as of 2026-07-28)" and Where addressed records A as PR #72, OPEN,
not merged.

Content otherwise preserved and redistributed as the neighbouring rows do:
status in #, the substance in Item, the three-PR rationale in Why deferred, and
the sequence with its gates, closing rule and related registers in Where
addressed.

One line replaced by one line. CRLF preserved. §7 untouched: zero additions and
zero deletions inside it. No change to services/, none to src/, no migration,
no DB change, no branch-protection change, no Railway or Vercel dashboard
change; PR #73 and PR #70 untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant