fix(http): route the remaining bare urlopen() call sites through an http(s)-only opener (BE-3313)#546
Conversation
…y opener (BE-3313) The remaining bare ``urllib.request.urlopen()`` calls used urllib's global default opener, which installs FileHandler, FTPHandler and DataHandler. None of them attach a credential header, so unlike the openers #542 pinned there is no redirect-replay exposure — this is defense-in-depth for a URL that stops being trusted (a gallery URL that becomes configurable), not a live exploit. Adds a shared ``_PLAIN_OPENER`` / ``plain_urlopen()`` in comfy_cli.http for the uncredentialed case and routes all seven sites through it: the two templates gallery fetches, the three jobs local-server calls, and the two run/ sites (``/prompt`` submit, ``/object_info`` preflight) that the ticket's grep missed because they import ``from urllib import request``. HTTPRedirectHandler is passed explicitly, so these keep following redirects exactly as the default opener did.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Comment |
There was a problem hiding this comment.
🔍 Cursor Review — Consolidated panel
Triggered by @mattmillerai.
Found 2 finding(s).
| Severity | Count |
|---|---|
| 🟢 Low | 2 |
Panel: 6/8 reviewers contributed findings.
Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)
…or read (BE-3313) Two findings from the cursor-review panel, both on call sites this PR moved onto the shared openers. The /prompt submit can carry a Comfy Org credential inside extra_data (api_key_comfy_org, or auth_token_comfy_org passed through), but it went out via the redirect-following _PLAIN_OPENER. Nothing leaked today only because urllib drops the body when it follows a redirect -- confirmed against a real redirecting server: 301/302/303 arrive as a bodyless GET and 307/308 already raise. That also means redirect-following never actually queued a workflow here; it just turned into a confusing "HTTP 200 without a prompt_id". Routing the submit through no_redirect_urlopen() gives it the same refuse-a-30x policy as every other credentialed call and a clear error instead, rather than depending on a CPython quirk for credential safety. Reading inside a `with` also stops the connection lingering until GC while the run moves to the websocket. fetch_object_info capped its success read at 64 MiB but called e.read() with no limit, so a server only had to return an error status to bypass the bound. Both reads now share _MAX_OBJECT_INFO_BYTES. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ELI-5
When Python opens a URL the "normal" way (
urllib.request.urlopen()), it hands you an opener that knows how to speakhttp://and alsofile://,ftp://anddata:. So if a URL ever came from somewhere untrustworthy, "fetch this" could quietly turn into "read a file off my disk."#542 fixed that for every opener that carries a password. This PR does the same for the last few call sites that had no password attached — they still shouldn't be able to read local files. Nothing about how they behave changes; they just can't speak those extra languages anymore.
What changed
Adds a shared
_PLAIN_OPENER+plain_urlopen()tocomfy_cli/http.pyfor the uncredentialed case (the credentialed counterpart of_AUTHED_OPENER/authed_urlopen), and routes every remaining bare-urlopensite through it:templates._fetch_galleryGALLERY_URLtemplates._fetch_template_workflow_TEMPLATE_WORKFLOW_URLjobs._http_get_jsonhttp://{host}:{port}jobs._local_cancel→POST /queuehttp://{host}:{port}jobs._local_cancel→POST /interrupthttp://{host}:{port}run/execution.queue→POST /prompthttp://{host}:{port}run/preflight.fetch_object_info→GET /object_infohttp://{host}:{port}grep -rn "urlopen(" comfy_cli/now returns zero call sites on the global default opener.Severity: low — same framing as #530 / #542
None of these attach
Authorization/X-API-Key, so there is no credential-replay exposure. The URLs are also trusted today (a module-level constant; loopback host/port). This is defense-in-depth against a future caller passing an attacker-influenced URL, not a live exploit.Judgment calls
1. Two extra call sites, beyond the five the ticket enumerated.
run/execution.pyandrun/preflight.pyimportfrom urllib import requestand callrequest.urlopen(...), so a grep forurllib.request.urlopenmisses them. They are the same class of gap (global default opener, no credentials, loopback URL), so leaving them would have shipped a fix that only looked complete. Flagging as scope the ticket didn't ask for.2. One shared opener rather than a module-level opener per site. The ticket sketched
_TEMPLATES_OPENER,_JOBS_OPENER, etc. I went with a single_PLAIN_OPENERbecause all seven sites want identical policy (http(s)-only, follows redirects, no credentials), and because the existing tests patch oneurlopenseam that covers both the preflight and execution paths — four separate openers would have fragmented that seam and turned a mechanical test update into a rewrite. Happy to split them if you'd rather have per-site openers.3. Redirects are still followed — deliberately.
build_http_only_opener()installs no redirect handler unless you pass one, so a bare opener would have converted these into no-redirect openers and started raisingHTTPErroron a 30x that used to be followed transparently. All seven sites follow redirects today, soHTTPRedirectHandler()is passed explicitly to preserve that. Not silently changed.4. Follow-up worth a separate look (not done here).
run/execution.queue'sPOST /promptbody can carryapi_key_comfy_orginsideextra_data— a credential in the body rather than a header. Following a redirect would replay it. That is pre-existing behavior (the default opener did exactly this), so changing it is out of scope for a defense-in-depth PR that promises to preserve behavior — but aNoRedirectHandlerthere may be warranted. Say the word and I'll file it.Tests
tests/comfy_cli/test_http_only_openers.py—_PLAIN_OPENERappended to the table-drivenOPENERSlist (so it inherits the scheme-refusal + handler-set guards), plustest_uncredentialed_openers_still_follow_redirects, mirroringtest_download_opener_still_follows_redirects, to lock in that pinning the scheme did not make it a no-redirect opener.Test patch targets in
test_run.py/test_run_json.py/test_jobs.pymoved fromurllib.request.urlopento the_PLAIN_OPENER.openseam — mechanical, andtest_queue_passes_timeout_to_urlopenstill proves the seam is live rather than vacuously green.Verification
uv run python -m pytest tests/comfy_cli— 2528 passed, 13 skippeduv run ruff format --check comfy_cli tests— clean (236 files)uv run ruff checkon changed files — clean (14 pre-existing repo-wide errors are unrelated and present on the base branch)200, final URL/final), whilefile:///etc/passwd,ftp://example.com/xanddata:text/plain,hieach raiseURLError: unknown url type.Stacked on #542
Base is
matt/be-3298-http-only-openers—build_http_only_opener()isn't onmainyet. GitHub will retarget this tomainonce #542 merges. Review only the top commit.