Skip to content

feat: transparently support form-encoded bodies (work around T3 host JSON-parse)#6

Merged
FiscalMindset merged 1 commit into
mainfrom
feat/form-body-egress-workaround
Jul 2, 2026
Merged

feat: transparently support form-encoded bodies (work around T3 host JSON-parse)#6
FiscalMindset merged 1 commit into
mainfrom
feat/form-body-egress-workaround

Conversation

@FiscalMindset

@FiscalMindset FiscalMindset commented Jul 2, 2026

Copy link
Copy Markdown
Owner

What & why

Tackles improvement #5 (the T3 host-egress limits) as far as is possible client-side.

#1 — host parses request bodies as JSON → SOLVED client-side. The T3 host egress fails any non-JSON body with http.parse_payload: expected value…, which blocked form-encoded APIs (Stripe, Twilio). The proxy now detects an application/x-www-form-urlencoded body, moves its params into the URL query string, and sends an empty body — the host's JSON parser is never triggered. Form providers accept query params, so it's transparent: the agent POSTs a normal form and it just works. JSON APIs are untouched.

#2 — content-type intermittently dropped → mitigated, residual is host-side. Stripe still requires a content-type header even with query params, and the testnet host intermittently drops it. We now always inject the correct content-type for form providers, and clients retry. Genuinely host-side (reads are 100%) — this is the one item left for the T3 team.

Proof

Normal form-body POST /stripe/v1/customers through the proxy, 5×:

#0 200 id=cus_UoHcm8taIA0a3P
#1 200 id=cus_UoHcbL8n4cnmM5
#2 400 (content-type dropped by host)
#3 200 id=cus_UoHcHq9VMAIH6j
#4 200 id=cus_UoHc4701oHiOo7
RESULT 4/5 form-body writes succeeded   (was ~1/3 before)

The one failure is the residual host header-drop (#2). Provider tests: 34/34 green.

Notes

  • Gated strictly on form-urlencoded content-type; JSON APIs (OpenAI/Anthropic/Gemini) unaffected.
  • No secrets committed.

Summary by cubic

Add transparent support for form-encoded requests by moving form bodies into the URL query string, avoiding the T3 host’s JSON parser and restoring Stripe/Twilio writes. Also injects required content-type headers for these providers; JSON APIs are unaffected.

  • New Features
    • In packages/blindfold/src/proxy.ts, detect application/x-www-form-urlencoded, move params to the URL, and send an empty body.
    • In packages/blindfold/src/providers.ts, set content-type: application/x-www-form-urlencoded for Stripe and Twilio.
    • Updated integration-stack.md with the host-egress workaround and the residual header-drop note (advise retries).

Written for commit 96fdf4b. Summary will update on new commits.

Review in cubic

…JSON-parse)

The T3 host egress parses every request body as JSON, so a form-encoded body
fails with `http.parse_payload: expected value…`. Solve it client-side: the
proxy now detects an application/x-www-form-urlencoded body, moves its params
into the URL query string, and sends an empty body — the host's JSON parser is
never triggered and the agent's code is unchanged (it can POST a normal form).
Form providers (Stripe, Twilio) accept query params, so it's transparent; JSON
APIs are untouched.

Also inject content-type for Stripe/Twilio so it's always present. Stripe
form-body writes now succeed ~4/5 (was ~1/3). The residual failure is the
testnet host intermittently dropping the content-type header on POSTs — a
genuine host-side bug (reads are 100%), mitigated by client retry and
documented in integration-stack.md.
@FiscalMindset FiscalMindset merged commit bd415d8 into main Jul 2, 2026
4 checks passed
@FiscalMindset FiscalMindset deleted the feat/form-body-egress-workaround branch July 2, 2026 07:32
Comment on lines +144 to +150
if (outboundBody && contentType.includes("application/x-www-form-urlencoded")) {
const u = new URL(outboundUrl);
for (const [k, v] of new URLSearchParams(outboundBody)) u.searchParams.append(k, v);
outboundUrl = u.toString();
outboundBody = undefined;
safeLog("info", { msg: "form_body_to_query", provider: provider.id });
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Bug: Twilio POST params moved to query string may not be accepted

The form-body→query-string workaround is applied to every form-urlencoded provider (proxy.ts:144), and providers.ts now forces content-type: application/x-www-form-urlencoded for Twilio. However, the PR's proof only exercises Stripe (5× POST /stripe/v1/customers, 34/34 Stripe tests). Twilio's REST API generally reads write parameters (e.g. To, From, Body for POST /Messages) from the form-encoded request body, not the query string. Sending an empty body with params in the query may cause Twilio writes to fail with missing-parameter errors. The claim "Form providers accept params in the query string" (integration-stack.md:203) is asserted but not demonstrated for Twilio. Please verify a real Twilio write (e.g. sending an SMS) succeeds through the proxy before relying on this path; if Twilio requires body params, this provider should be excluded from the query-string rewrite.

Was this helpful? React with 👍 / 👎

Comment on lines +145 to +149
const u = new URL(outboundUrl);
for (const [k, v] of new URLSearchParams(outboundBody)) u.searchParams.append(k, v);
outboundUrl = u.toString();
outboundBody = undefined;
safeLog("info", { msg: "form_body_to_query", provider: provider.id });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Security: Form params relocated to URL query can leak into access logs

Moving the request body into the URL query string (proxy.ts:145-147) changes where potentially sensitive payload data lives. Request bodies are rarely logged, but query strings are routinely captured in host/upstream access logs, TLS-terminating proxies, and error-tracking systems. For these providers the params can include PII and sensitive fields (Stripe customer emails/metadata/tokens, Twilio phone numbers and message Body text). While Blindfold's own telemetry strips the query (upstream.replace(/\?.*$/, "") at proxy.ts:164,189), the T3 host and upstream provider still receive and may log the full URL. This is an inherent trade-off of the workaround, but it's worth documenting in integration-stack.md so operators understand payload data now travels in the URL, and worth confirming the T3 host does not log full request URLs.

Was this helpful? React with 👍 / 👎

Comment on lines +145 to +148
const u = new URL(outboundUrl);
for (const [k, v] of new URLSearchParams(outboundBody)) u.searchParams.append(k, v);
outboundUrl = u.toString();
outboundBody = undefined;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Edge Case: Large form bodies may exceed URL length limits after rewrite

The rewrite appends every form parameter to the URL (proxy.ts:146) with no size check. Servers, proxies, and the T3 host typically enforce URL length limits (commonly ~8 KB). A form body that is comfortably within HTTP body limits (e.g. a Stripe object with many metadata fields, or a long Twilio message Body) can produce a URL that exceeds these limits, causing the request to fail with a 414/400 rather than the intended write. Consider guarding the rewrite with a size threshold and falling back (or surfacing a clear error) when the resulting query string would be too large.

Was this helpful? React with 👍 / 👎

Comment on lines +143 to +146
const contentType = (headers.find(([k]) => k.toLowerCase() === "content-type")?.[1] ?? "").toLowerCase();
if (outboundBody && contentType.includes("application/x-www-form-urlencoded")) {
const u = new URL(outboundUrl);
for (const [k, v] of new URLSearchParams(outboundBody)) u.searchParams.append(k, v);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Bug: Forced form content-type can mangle non-form bodies for Stripe/Twilio

providers.ts now injects content-type: application/x-www-form-urlencoded by default for Stripe and Twilio (providers.ts:117,165). Because the query-rewrite triggers purely on the content-type string (proxy.ts:143-144), any body an agent sends to these providers without its own content-type header will be treated as form-encoded and run through new URLSearchParams(outboundBody). If the agent sends a non-form body (e.g. JSON), it will be silently parsed as a single garbage key and appended to the URL, corrupting the request rather than failing clearly. This is low-likelihood since these providers are form-only, but the coupling between the default header and the rewrite condition is fragile — consider gating the rewrite on the provider actually being form-based and/or validating that the body parses as form data.

Was this helpful? React with 👍 / 👎

@gitar-bot

gitar-bot Bot commented Jul 2, 2026

Copy link
Copy Markdown
Code Review ⚠️ Changes requested 0 resolved / 4 findings

Implements a form-encoded to query-string proxy workaround to bypass T3 host JSON parsing, but risks potential parameter leakage in logs, URL length overflow, and service-specific incompatibility with Twilio or mangled content types.

⚠️ Bug: Twilio POST params moved to query string may not be accepted

📄 packages/blindfold/src/proxy.ts:144-150 📄 packages/blindfold/src/providers.ts:162-165 📄 integration-stack.md:203-205

The form-body→query-string workaround is applied to every form-urlencoded provider (proxy.ts:144), and providers.ts now forces content-type: application/x-www-form-urlencoded for Twilio. However, the PR's proof only exercises Stripe (5× POST /stripe/v1/customers, 34/34 Stripe tests). Twilio's REST API generally reads write parameters (e.g. To, From, Body for POST /Messages) from the form-encoded request body, not the query string. Sending an empty body with params in the query may cause Twilio writes to fail with missing-parameter errors. The claim "Form providers accept params in the query string" (integration-stack.md:203) is asserted but not demonstrated for Twilio. Please verify a real Twilio write (e.g. sending an SMS) succeeds through the proxy before relying on this path; if Twilio requires body params, this provider should be excluded from the query-string rewrite.

💡 Security: Form params relocated to URL query can leak into access logs

📄 packages/blindfold/src/proxy.ts:145-149 📄 packages/blindfold/src/proxy.ts:164 📄 packages/blindfold/src/proxy.ts:189

Moving the request body into the URL query string (proxy.ts:145-147) changes where potentially sensitive payload data lives. Request bodies are rarely logged, but query strings are routinely captured in host/upstream access logs, TLS-terminating proxies, and error-tracking systems. For these providers the params can include PII and sensitive fields (Stripe customer emails/metadata/tokens, Twilio phone numbers and message Body text). While Blindfold's own telemetry strips the query (upstream.replace(/\?.*$/, "") at proxy.ts:164,189), the T3 host and upstream provider still receive and may log the full URL. This is an inherent trade-off of the workaround, but it's worth documenting in integration-stack.md so operators understand payload data now travels in the URL, and worth confirming the T3 host does not log full request URLs.

💡 Edge Case: Large form bodies may exceed URL length limits after rewrite

📄 packages/blindfold/src/proxy.ts:145-148

The rewrite appends every form parameter to the URL (proxy.ts:146) with no size check. Servers, proxies, and the T3 host typically enforce URL length limits (commonly ~8 KB). A form body that is comfortably within HTTP body limits (e.g. a Stripe object with many metadata fields, or a long Twilio message Body) can produce a URL that exceeds these limits, causing the request to fail with a 414/400 rather than the intended write. Consider guarding the rewrite with a size threshold and falling back (or surfacing a clear error) when the resulting query string would be too large.

💡 Bug: Forced form content-type can mangle non-form bodies for Stripe/Twilio

📄 packages/blindfold/src/proxy.ts:143-146 📄 packages/blindfold/src/providers.ts:117 📄 packages/blindfold/src/providers.ts:165

providers.ts now injects content-type: application/x-www-form-urlencoded by default for Stripe and Twilio (providers.ts:117,165). Because the query-rewrite triggers purely on the content-type string (proxy.ts:143-144), any body an agent sends to these providers without its own content-type header will be treated as form-encoded and run through new URLSearchParams(outboundBody). If the agent sends a non-form body (e.g. JSON), it will be silently parsed as a single garbage key and appended to the URL, corrupting the request rather than failing clearly. This is low-likelihood since these providers are form-only, but the coupling between the default header and the rewrite condition is fragile — consider gating the rewrite on the provider actually being form-based and/or validating that the body parses as form data.

🤖 Prompt for agents
Code Review: Implements a form-encoded to query-string proxy workaround to bypass T3 host JSON parsing, but risks potential parameter leakage in logs, URL length overflow, and service-specific incompatibility with Twilio or mangled content types.

1. ⚠️ Bug: Twilio POST params moved to query string may not be accepted
   Files: packages/blindfold/src/proxy.ts:144-150, packages/blindfold/src/providers.ts:162-165, integration-stack.md:203-205

   The form-body→query-string workaround is applied to every form-urlencoded provider (proxy.ts:144), and providers.ts now forces `content-type: application/x-www-form-urlencoded` for Twilio. However, the PR's proof only exercises Stripe (5× POST /stripe/v1/customers, 34/34 Stripe tests). Twilio's REST API generally reads write parameters (e.g. `To`, `From`, `Body` for POST /Messages) from the form-encoded request body, not the query string. Sending an empty body with params in the query may cause Twilio writes to fail with missing-parameter errors. The claim "Form providers accept params in the query string" (integration-stack.md:203) is asserted but not demonstrated for Twilio. Please verify a real Twilio write (e.g. sending an SMS) succeeds through the proxy before relying on this path; if Twilio requires body params, this provider should be excluded from the query-string rewrite.

2. 💡 Security: Form params relocated to URL query can leak into access logs
   Files: packages/blindfold/src/proxy.ts:145-149, packages/blindfold/src/proxy.ts:164, packages/blindfold/src/proxy.ts:189

   Moving the request body into the URL query string (proxy.ts:145-147) changes where potentially sensitive payload data lives. Request bodies are rarely logged, but query strings are routinely captured in host/upstream access logs, TLS-terminating proxies, and error-tracking systems. For these providers the params can include PII and sensitive fields (Stripe customer emails/metadata/tokens, Twilio phone numbers and message `Body` text). While Blindfold's own telemetry strips the query (`upstream.replace(/\?.*$/, "")` at proxy.ts:164,189), the T3 host and upstream provider still receive and may log the full URL. This is an inherent trade-off of the workaround, but it's worth documenting in integration-stack.md so operators understand payload data now travels in the URL, and worth confirming the T3 host does not log full request URLs.

3. 💡 Edge Case: Large form bodies may exceed URL length limits after rewrite
   Files: packages/blindfold/src/proxy.ts:145-148

   The rewrite appends every form parameter to the URL (proxy.ts:146) with no size check. Servers, proxies, and the T3 host typically enforce URL length limits (commonly ~8 KB). A form body that is comfortably within HTTP body limits (e.g. a Stripe object with many metadata fields, or a long Twilio message `Body`) can produce a URL that exceeds these limits, causing the request to fail with a 414/400 rather than the intended write. Consider guarding the rewrite with a size threshold and falling back (or surfacing a clear error) when the resulting query string would be too large.

4. 💡 Bug: Forced form content-type can mangle non-form bodies for Stripe/Twilio
   Files: packages/blindfold/src/proxy.ts:143-146, packages/blindfold/src/providers.ts:117, packages/blindfold/src/providers.ts:165

   providers.ts now injects `content-type: application/x-www-form-urlencoded` by default for Stripe and Twilio (providers.ts:117,165). Because the query-rewrite triggers purely on the content-type string (proxy.ts:143-144), any body an agent sends to these providers without its own content-type header will be treated as form-encoded and run through `new URLSearchParams(outboundBody)`. If the agent sends a non-form body (e.g. JSON), it will be silently parsed as a single garbage key and appended to the URL, corrupting the request rather than failing clearly. This is low-likelihood since these providers are form-only, but the coupling between the default header and the rewrite condition is fragile — consider gating the rewrite on the provider actually being form-based and/or validating that the body parses as form data.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants