ci: add the ingestion image build+import gate (PR A0 of 3, registers #52, #53)

[tornidomaroc-web]

PR A0 of the three-PR (b1) sequence. CI-ONLY. Zero files under src/, zero files under services/. No service code, no migration, no DB change, no dashboard change.

Adds one workflow, .github/workflows/ingestion-image.yml, plus this repo's required PROGRESS.md records.

This PR's first version shipped the exact defect it exists to end

Kept in the history (commit 64e4e80, fixed in ac14b46) rather than force-pushed away. See "The vacuous-green defect" below. If you read only one section, read that one.

What this PR proves

• The ingestion image builds, using Railway's own build context — Root Directory /services/ingestion, empty Dockerfile Path, read from the dashboard on 2026-07-26 rather than assumed.
• main.py imports inside that built image, with the image's own interpreter at its own WORKDIR, and the job asserts on the printed output that it did. services/ingestion/main.py:13 imports acreate_client/AsyncClient/AsyncClientOptions from supabase at module scope, and :50 calls tiktoken.get_encoding at module scope, so a bad resolve is a boot-time crash, not a runtime one.
• The container boots and serves /health — strictly stronger than an import, because it catches ASGI/lifespan startup failures a bare import does not.

All four endpoints (/health, /embed, /convert, /ingest) share one uvicorn process, so a boot crash takes the Ask path down with the upload path. That is the failure this closes.

What this PR does NOT prove

It does not prove the Railway environment is correct. CI has no SUPABASE_URL or SUPABASE_ANON_KEY, so the CI container's /health correctly reports "supabase_configured": false. The job asserts only "ok":true.

That separation is load-bearing and must not be collapsed later: this job is the CI twin of the live boot probe. The environment probe has no CI twin and cannot have one — only a live GET /health against the deployed service can assert supabase_configured: true.

It also proves nothing about ingestion behaviour. It never runs an upload, never touches Supabase, never embeds anything.

The vacuous-green defect

The first version of this workflow (64e4e80) reported the ingestion-image job green in 38 seconds with its import step emitting zero bytes of stdout — no ROUTES:, no IMPORT OK, no IMPORT FAILED. Run 30218194135.

Cause, proven by direct experiment rather than inferred. python - reads its program from stdin, and docker run without -i does not attach stdin to the container. Python read EOF, executed an empty program, exited 0, and the step passed having asserted nothing:

=== docker run WITHOUT -i (what the workflow did) ===
exit=0  <-- the script never ran

=== docker run WITH -i ===
STDIN REACHED THE CONTAINER
exit=3  <-- the script ran

A workflow written to end vacuous green checks shipped a vacuous green check.

Severity stated accurately, not dramatically. The job was not proving nothing about the import — the boot probe transitively proves the module imports, since uvicorn cannot serve /health without importing main:app. What the dead step actually cost was the route assertions, the ROUTES: evidence line, and the network-vs-ImportError diagnostic banner.

Two fixes (ac14b46), and the second is the one that matters:

1. Add -i, so the heredoc reaches the interpreter.
2. Pipe through tee and make a trailing grep -q "IMPORT OK" the thing that actually fails the step. GitHub runs steps with bash -e and NOT -o pipefail (shell: /usr/bin/bash -e {0}), so tee would otherwise mask a non-zero docker run.

Both paths were tested locally under bash -e before pushing — with -i the step exits 0 and prints IMPORT OK; with -i removed the step now exits 1, so the regression cannot silently return. A comment at the step records why -i is load-bearing so a future reader cannot innocently remove it.

The generalised rule: a check step must assert on its own OUTPUT, not merely on its exit code. Register #52 now carries it.

Audit of the other four steps under the same lens, so the fix is not mistaken for a sweep:

Step	Exposed?
Check out the repository	No — self-verifying downstream; a bad checkout fails the build on a missing Dockerfile.
Build the ingestion image	No — its exit code and its effect are the same fact. COPY requirements.txt / COPY main.py make a wrong context or missing file fail loudly. Nothing further to assert on.
Import the module inside the image	YES — the defect. Fixed.
Boot the container and probe /health	No — already asserts on payload (grep -q '"ok":true'), in an if body where bash -e applies.
Container logs (always)	No — || true by design, claims nothing.

The import step was the only exposed one. No other fix was invented.

Runtime: measured, and my estimate was wrong by an order of magnitude

Measured: 38 seconds (build ~24 s). Projected before the run: 5-9 minutes. The projection is recorded rather than deleted so the next estimate carries a known error bar. It was wrong because every dependency in markitdown[all] resolves to a prebuilt manylinux wheel and GitHub's path to PyPI is very fast — nothing compiles from source.

Two riders on that number:

• It was measured against main's requirements.txt, which does not yet contain supabase==2.29.0. PR A's run will be slower — though on this evidence nowhere near the original projection.
• It was measured on the defective run, whose import step was a no-op. The corrected step adds a real container start, so the true figure is slightly higher again.

timeout-minutes: 20 now carries far more headroom than intended. Left as-is: a generous ceiling costs nothing, and a hung third-party pull still fails loudly instead of burning the 6-hour default.

V11 is satisfied only by a green run on main of the CORRECTED workflow. The 38 s figure came from a run whose central assertion did not execute; it pins the runtime and nothing else.

Deliberately NOT a required check

This is not added to branch protection, and that is intentional.

Three third-party network dependencies sit in this job's hot path — Docker Hub (python:3.11-slim), PyPI (markitdown[all] + the supabase tree), and Azure blob storage (tiktoken's cl100k_base, see #53). db-types now has one, digest-pinned.

Register #43 is precisely this class. It redded main twice on docker: toomanyrequests with no commit to this repo, and its residual (1) records that Docker Hub is also anonymous-rate-limited per IP with GitHub runners sharing NAT pools. #43's governing quotes apply unchanged:

"A gate that reds main on a coin flip is worse than no gate."

"a required check that reds on a third party's traffic trains people to re-run it."

Promotion rule, binding, recorded in register #52: required-check status is considered only after 20 consecutive green runs on main with zero re-runs. Any red caused by a third party's network resets the counter to zero, and weakens rather than strengthens the case. Never being promoted is an acceptable outcome. Mechanically it also cannot be required until a green run on main exists (typecheck.yml:11-12).

Register rows: why two new ones, and why #51 is skipped

• test(DO NOT MERGE): db-types drift probe #52 is its own row rather than folded into ci: generate database.types.ts from migrations and gate on drift #51 because the vacuous-green gap predates (b1) and outlives it. ci: generate database.types.ts from migrations and gate on drift #51 closes when PR C lands; test(DO NOT MERGE): db-types drift probe #52's promotion obligation must not close with it. It now also carries the first instance of its own thesis — this PR's defect.
• feat: wire <Database> into the three Supabase client factories #53 (the tiktoken cold-start network dependency) is its own row because it is a production dependency, not a CI one, and it is pre-existing on main. Recording it under a (b1) row would misattribute it as a (b1) regression. Verified in the vendored package: tiktoken_ext/openai_public.py:77 resolves cl100k_base to an openaipublic.blob.core.windows.net URL, and tiktoken/load.py:36 read_file_cached is cache-first — a fresh container's cache is empty, so the fetch is real on every cold start. No fix attempted here; both candidates change the production image.
• ci: generate database.types.ts from migrations and gate on drift #51 is deliberately skipped and reserved for (b1)'s own row, which PR A adds. PR feat(b1): Railway ingestion service owns chunk persistence + terminal document status (forwarded user JWT, no service-role key) #70's body and the (b1) branch already name it ci: generate database.types.ts from migrations and gate on drift #51; renumbering now would create drift with text that already exists.

Other design notes

• The base image is not digest-pinned in the workflow, deliberately. services/ingestion/Dockerfile says FROM python:3.11-slim. Pinning in CI but not in the Dockerfile would make CI green on an image production never builds — a silent false-green, worse than the flakiness it avoids. If the base should be pinned, pin it in the Dockerfile so both paths move together.
• No paths: filter, deliberately. A path-filtered job never reports a status on PRs it skips; if this is ever promoted to required, a rule demanding a context nothing publishes blocks every unrelated PR indefinitely.
• Needs no secret. No registry credential, no Supabase key, not INGESTION_TOKEN. permissions: contents: read is the entire grant; the job cannot reach, read, or mutate production.
• Costs nothing. This repository is public, and GitHub bills Actions minutes only for private repositories.

Known forward reference

The workflow's comment header cites docs/b1-verification-protocol.md, which does not exist yet — PR A creates it. Accurate for the sequence, but a dangling path for the A0-to-A window. Flagged rather than silently left.

Logged for later, deliberately out of scope here

actions/checkout's post-job cleanup emits fatal: No url found for submodule path 'profile-readme' in .gitmodules and a git failed with exit code 128 warning. Non-fatal, does not affect any result, and near-certainly pre-existing. Not investigated in this PR, including whether tsc and db-types show it too.

Checks

tsc and db-types will pass — vacuously, since this PR contains zero TypeScript and zero SQL. That is the entire point of register #52. The only meaningful signal here is ingestion-image, and it is meaningful only now that its import step actually executes.

Deliberately not in this PR

docs/b1-verification-protocol.md (PR A) · any change to services/ingestion/ · any change to src/ · any branch-protection change · any dashboard change · closing PR #70.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
knowflow	Ready	Preview, Comment	Jul 26, 2026 8:29pm