Skip to content

feat: transparently support form-encoded bodies (work around T3 host JSON-parse) #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FiscalMindset merged 1 commit into from
Jul 2, 2026
Merged

feat: transparently support form-encoded bodies (work around T3 host JSON-parse) #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 16 additions & 10 deletions integration-stack.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,16 +196,22 @@ Running Stripe live surfaced real constraints in the **current T3 host** egress
(`host:interfaces/http` `call`) — worth recording so nobody re-diagnoses them:

1. **The host parses request bodies as JSON.** A non-JSON payload fails with
`http.parse_payload: expected value at line 1 column 1`. JSON APIs (Gemini,
OpenAI, Anthropic) and read GETs work fully; form-encoded bodies do not. The
Stripe example works around this by putting params in the query string with an
empty body.
2. **On testnet, request headers aren't always forwarded.** Form-encoded Stripe
WRITES are flaky because the `content-type` header is intermittently dropped
(reads are 100% reliable). The example retries and reports honestly if the
egress is dropping headers on a given run. Neither is a Blindfold design
issue — auth and key protection are unaffected; these are host-egress
maturity gaps on testnet.
`http.parse_payload: expected value at line 1 column 1`. **✅ Solved
client-side:** the proxy now detects a `application/x-www-form-urlencoded`
body, moves its params into the URL query string, and sends an empty body —
so the host's JSON parser is never triggered and the agent's code is
unchanged (it can POST a normal form). Form-encoded providers (Stripe,
Twilio) accept params in the query string, so this is transparent. JSON APIs
are untouched.
2. **On testnet, request headers aren't always forwarded (residual host bug).**
Even with the params in the query string, Stripe still requires a
`content-type` header, and the testnet host **intermittently drops it** — so
a form write succeeds ~4/5 of the time and fails the rest with "check that
your POST content type…". Mitigations: the proxy always injects the correct
`content-type` for form providers (so it's present when the host does forward
it), and clients retry. The residual drop is genuinely host-side (reads are
100%); it's the one item worth raising with the T3 team. Auth and key
protection are unaffected either way.

### basic + sigv4 proven LIVE (contract 0.5.4)

Expand Down
3 changes: 2 additions & 1 deletion packages/blindfold/src/providers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ const PROVIDERS: ProviderDef[] = [
upstream: (p) => `https://api.stripe.com${stripPrefix(p, "/stripe/")}`,
secretKey: "stripe_secret_key",
auth: () => ({ scheme: "bearer" }),
defaultHeaders: { "stripe-version": "2024-06-20" },
defaultHeaders: { "stripe-version": "2024-06-20", "content-type": "application/x-www-form-urlencoded" },
},

// ---- Dev infra: GitHub. GitHub REJECTS requests with no User-Agent (403),
Expand Down Expand Up @@ -162,6 +162,7 @@ const PROVIDERS: ProviderDef[] = [
upstream: (p) => `https://api.twilio.com${stripPrefix(p, "/twilio/")}`,
secretKey: "twilio_auth_token",
auth: () => ({ scheme: "basic", username: process.env.TWILIO_ACCOUNT_SID || "" }),
defaultHeaders: { "content-type": "application/x-www-form-urlencoded" },
},

// ---- Cloud: AWS SES (SigV4 — secret SIGNS, never transmitted). ------------
Expand Down
21 changes: 19 additions & 2 deletions packages/blindfold/src/proxy.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,11 +132,28 @@ async function handle(
}
}

// Work around the T3 host egress parsing every request body as JSON (a raw
// form-encoded body fails with `http.parse_payload: expected value…`).
// Form-encoded APIs (Stripe, Twilio, AWS query APIs) accept the same params
// in the query string, so move a form body into the URL and send no body.
// The agent's code is unchanged — it can POST a normal form and this adapts
// it. JSON bodies are left untouched (the host parses those fine).
let outboundUrl = upstream;
let outboundBody: string | undefined = body.length ? body.toString("utf8") : undefined;
const contentType = (headers.find(([k]) => k.toLowerCase() === "content-type")?.[1] ?? "").toLowerCase();
if (outboundBody && contentType.includes("application/x-www-form-urlencoded")) {
const u = new URL(outboundUrl);
for (const [k, v] of new URLSearchParams(outboundBody)) u.searchParams.append(k, v);
Comment on lines +143 to +146

@gitar-bot gitar-bot Bot Jul 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Bug: Forced form content-type can mangle non-form bodies for Stripe/Twilio

providers.ts now injects content-type: application/x-www-form-urlencoded by default for Stripe and Twilio (providers.ts:117,165). Because the query-rewrite triggers purely on the content-type string (proxy.ts:143-144), any body an agent sends to these providers without its own content-type header will be treated as form-encoded and run through new URLSearchParams(outboundBody). If the agent sends a non-form body (e.g. JSON), it will be silently parsed as a single garbage key and appended to the URL, corrupting the request rather than failing clearly. This is low-likelihood since these providers are form-only, but the coupling between the default header and the rewrite condition is fragile — consider gating the rewrite on the provider actually being form-based and/or validating that the body parses as form data.

Was this helpful? React with 👍 / 👎

outboundUrl = u.toString();
outboundBody = undefined;
Comment on lines +145 to +148

@gitar-bot gitar-bot Bot Jul 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Edge Case: Large form bodies may exceed URL length limits after rewrite

The rewrite appends every form parameter to the URL (proxy.ts:146) with no size check. Servers, proxies, and the T3 host typically enforce URL length limits (commonly ~8 KB). A form body that is comfortably within HTTP body limits (e.g. a Stripe object with many metadata fields, or a long Twilio message Body) can produce a URL that exceeds these limits, causing the request to fail with a 414/400 rather than the intended write. Consider guarding the rewrite with a size threshold and falling back (or surfacing a clear error) when the resulting query string would be too large.

Was this helpful? React with 👍 / 👎

safeLog("info", { msg: "form_body_to_query", provider: provider.id });
Comment on lines +145 to +149

@gitar-bot gitar-bot Bot Jul 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Security: Form params relocated to URL query can leak into access logs

Moving the request body into the URL query string (proxy.ts:145-147) changes where potentially sensitive payload data lives. Request bodies are rarely logged, but query strings are routinely captured in host/upstream access logs, TLS-terminating proxies, and error-tracking systems. For these providers the params can include PII and sensitive fields (Stripe customer emails/metadata/tokens, Twilio phone numbers and message Body text). While Blindfold's own telemetry strips the query (upstream.replace(/\?.*$/, "") at proxy.ts:164,189), the T3 host and upstream provider still receive and may log the full URL. This is an inherent trade-off of the workaround, but it's worth documenting in integration-stack.md so operators understand payload data now travels in the URL, and worth confirming the T3 host does not log full request URLs.

Was this helpful? React with 👍 / 👎

}
Comment on lines +144 to +150

@gitar-bot gitar-bot Bot Jul 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Bug: Twilio POST params moved to query string may not be accepted

The form-body→query-string workaround is applied to every form-urlencoded provider (proxy.ts:144), and providers.ts now forces content-type: application/x-www-form-urlencoded for Twilio. However, the PR's proof only exercises Stripe (5× POST /stripe/v1/customers, 34/34 Stripe tests). Twilio's REST API generally reads write parameters (e.g. To, From, Body for POST /Messages) from the form-encoded request body, not the query string. Sending an empty body with params in the query may cause Twilio writes to fail with missing-parameter errors. The claim "Form providers accept params in the query string" (integration-stack.md:203) is asserted but not demonstrated for Twilio. Please verify a real Twilio write (e.g. sending an SMS) succeeds through the proxy before relying on this path; if Twilio requires body params, this provider should be excluded from the query-string rewrite.

Was this helpful? React with 👍 / 👎


const forwardReq: ForwardRequest = {
method: req.method ?? "GET",
url: upstream,
url: outboundUrl,
headers,
body: body.length ? body.toString("utf8") : undefined,
body: outboundBody,
secret_key: providerSecretKey,
auth: provider.auth,
};
Expand Down
Loading