Skip to content

fix: remove the vestigial vercel.json ingestion service (live duplicate)#67

Merged
tornidomaroc-web merged 1 commit into
mainfrom
fix/remove-duplicate-vercel-ingestion-service
Jul 23, 2026
Merged

fix: remove the vestigial vercel.json ingestion service (live duplicate)#67
tornidomaroc-web merged 1 commit into
mainfrom
fix/remove-duplicate-vercel-ingestion-service

Conversation

@tornidomaroc-web

Copy link
Copy Markdown
Owner

Security fix. Removes experimentalServices.ingestion from vercel.json, which was serving a live duplicate of the ingestion service. Also logs the incident as register #45 (OPEN). Two files: vercel.json (−4) + docs/PROGRESS.md (+8/−3).

The finding

GET https://tryknowflow.com/_/ingestion/health returns 200 {"ok":true,"embed_provider":"voyage",...}. The experimentalServices.ingestion block (entrypoint: services/ingestion, routePrefix: /_/ingestion) was active, serving a second copy of the FastAPI ingestion service on Vercel — alongside the real one on Railway (INGESTION_SERVICE_URL = knowflow-production.up.railway.app).

Correction to an earlier misread: this is not stale April code. Vercel rebuilds services/ingestion from the deployed commit, so the duplicate runs the same current main.py as Railway. The danger is the duplicate existing, not a code delta.

Why it matters

  • (a) Token-shared. INGESTION_TOKEN is a Vercel project env var the Next route already needs, so the same token that authorizes Next→Railway also opens this copy's /convert and /embed (_check_auth is a single-token equality). VOYAGE_API_KEY is set on Vercel (confirmed), so a token-holder can burn Voyage embedding credits and run arbitrary documents through MarkItDown here — bypassing every Next-route guard (auth, per-KB doc limit, enforceLimit('upload'), rate limits), none of which live in the Python service. /convert here is Vercel-edge-capped at 4.5 MB inbound (unlike Railway), which bounds but does not remove the abuse.
  • (b) /health is unauthenticated — discloses embedding provider/model/dimension and is a liveness oracle.
  • (c) INGESTION_TOKEN has been reachable on an unintended public endpoint and must be rotated.

The fix

Removes only the ingestion sub-key, keeping frontend exactly as-is:

{
  "experimentalServices": {
    "frontend": {
      "routePrefix": "/",
      "framework": "nextjs"
    }
  }
}

Minimal by design — the frontend declaration is untouched, so its deploy behavior does not change.

MANDATORY preview verification before merge

On the preview deployment, confirm all:

  1. Frontend still serves — the site loads and an authenticated page renders.
  2. A real upload completes end-to-end via Railway (Next → Railway /convert → document reaches ready).
  3. <preview>/_/ingestion/health → 404 (was 200).

Do not merge until all three pass. No blind edit to live deploy config.

Post-merge sequencing — ROTATE LAST (deliberate)

  1. Merge the config fix.
  2. Confirm production https://tryknowflow.com/_/ingestion/health → 404. If it does not 404, delete the service in the Vercel dashboard — config removal may be necessary-but-insufficient if Vercel persists an already-built service.
  3. THEN rotate INGESTION_TOKEN on Railway first, then Vercel, as near-simultaneously as possible (a mismatch window 401s uploads until both match — an availability blip, not a security gap).

Rotate-last is intentional: rotating before teardown just re-exposes the fresh token on the still-live duplicate (shared env), so it would not help.

Scope

Checks

  • npm run typecheckexit 0.
  • db-types: no SQL migration touched → regenerated types byte-identical.

🤖 Generated with Claude Code

vercel.json's `experimentalServices.ingestion` block was serving a LIVE
DUPLICATE of the ingestion service at /_/ingestion — confirmed by a 200 from
`GET https://tryknowflow.com/_/ingestion/health`. It was NOT vestigial-inert,
and NOT stale April code: Vercel rebuilds `services/ingestion` from the deployed
commit, so it runs the same current main.py as the real Railway service
(INGESTION_SERVICE_URL = knowflow-production.up.railway.app).

The duplicate is token-shared: INGESTION_TOKEN is a Vercel project env var the
Next route already needs, so the same token that authorizes Next->Railway opens
this copy's /convert and /embed. VOYAGE_API_KEY is set on Vercel, so a
token-holder could burn Voyage credits and run arbitrary documents through
MarkItDown here, bypassing every Next-route guard (auth, per-KB limit,
enforceLimit, rate limits) — none of which live in the Python service. /health
is unauthenticated and discloses provider/model/dimension.

This removes ONLY the `ingestion` sub-key, keeping `frontend` exactly as-is, so
the Next.js frontend keeps deploying. Logged as register #45 (OPEN). A distinct
incident from B5b — the B5b row and PIVOT_PLAN.md:270 are deliberately untouched.

MANDATORY sequencing (rotate LAST): (1) merge; (2) confirm production
/_/ingestion/health -> 404, and DELETE the Vercel service in the dashboard if it
does not (config removal may be necessary-but-insufficient); (3) THEN rotate
INGESTION_TOKEN on Railway then Vercel. Rotating before teardown just re-exposes
the fresh token on the shared-env duplicate. Preview must be verified before
merge: frontend still serves + a real upload completes via Railway, and
<preview>/_/ingestion/health -> 404.

Header fields refreshed per the §7 policy merged in PR #66 (Active branch pointed
at a deleted branch, Main tip was one merge stale).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
knowflow Ready Ready Preview, Comment Jul 23, 2026 12:36am

@tornidomaroc-web
tornidomaroc-web merged commit ecb1ed5 into main Jul 23, 2026
4 checks passed
@tornidomaroc-web
tornidomaroc-web deleted the fix/remove-duplicate-vercel-ingestion-service branch July 23, 2026 12:51
tornidomaroc-web added a commit that referenced this pull request Jul 23, 2026
…#49 (#68)

Convert #45 in place (🔴→✅) with the measured evidence chain: production
/_/ingestion/health went 200 (no redirect) → 307 → 404 after PR #67 merged;
config removal alone sufficed, no dashboard deletion. INGESTION_TOKEN rotated
Railway-first then Vercel, verified by a real production upload reaching READY.

Log four product-design items raised by Abo Jad as registers #46-#49 (all
OPEN): settings-page premium polish, per-material actions, landing-page nav
menu, and an animated terminal demo (with the #20 "Ready in 0.4s" correction).

Append-only §7: one new block (18→19), no landed entry touched. Refresh the
three header fields. Docs-only; Phase 7 stays ⬜, register #17 stays OPEN.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant