Skip to content

fix(run): resolve the bundled default checkpoint at runtime (BE-2994)#519

Open
mattmillerai wants to merge 2 commits into
mainfrom
matt/be-2994-runtime-default-checkpoint
Open

fix(run): resolve the bundled default checkpoint at runtime (BE-2994)#519
mattmillerai wants to merge 2 commits into
mainfrom
matt/be-2994-runtime-default-checkpoint

Conversation

@mattmillerai

Copy link
Copy Markdown
Collaborator

ELI-5

comfy run --prompt "a fox" (no --workflow) runs a tiny built-in text2img
workflow. That workflow had one specific checkpoint filename baked in. If the
machine (or Comfy Cloud, or a fresh install) didn't have that exact file, the
run failed — often with a cryptic server-side error. This PR makes the CLI look
at what checkpoints the target actually has and use one, so a zero-config run
produces an image wherever any checkpoint exists — and gives a clear,
actionable error when there are truly none.

What changed

  1. Fixed the stale pin. cql/data/default_text2img.json node "4"
    ckpt_name is now v1-5-pruned-emaonly-fp16.safetensors (the file the
    gallery default.json and the text-to-image tutorial use), replacing the
    stale v1-5-pruned-emaonly.ckpt.

  2. Runtime checkpoint resolution. New pure, offline-testable helper
    resolve_default_checkpoint(workflow, object_info) in
    cql/default_workflow.py:

    • pinned checkpoint present in the target's enum → no change;
    • pinned absent but the target has ≥1 checkpoint → substitute the first
      available
      and emit a note;
    • target positively enumerates zero checkpoints → flag it (hard error, below);
    • enum absent / object_info unfetchable ({}) → fail open (unchanged
      behavior — submit and let the server answer).

    Wired into both run paths (local execute + execute_cloud) after
    object_info is fetched and before _preflight_validate. It only acts on
    the bundled default graph (workflow_name == "default_text2img") and is
    skipped entirely when the user pinned the checkpoint via --set checkpoint=…
    / --set 4.ckpt_name=… (honored verbatim; a checkpoint_user_set flag is
    threaded through the preloaded tuple).

  3. Actionable no-checkpoint error. When the target positively has zero
    checkpoints, a new registered no_checkpoint_available error (exit 1) with a
    download hint (local) / gallery-template guidance (cloud) replaces the cryptic
    server reject. Fail-open behavior is preserved when object_info can't be
    fetched at all.

Substitution is surfaced as a checkpoint_substituted NDJSON event (--json)
and a one-line pretty notice otherwise.

Acceptance criteria

  • default_text2img.json uses v1-5-pruned-emaonly-fp16.safetensors; fixtures/tests updated.
  • Missing-default-but-has-others → succeeds using the first available + emits a note; --set checkpoint=<name> always honored verbatim.
  • Default present → unchanged, no note.
  • Zero checkpoints (empty enum) → no_checkpoint_available + exit 1; object_info {} → fail-open submit (unchanged).
  • Both --where local and --where cloud.
  • New unit tests for resolve_default_checkpoint (present / substituted / empty-enum / user-pinned) + wiring tests; ruff/pytest green.

Testing

uv run --extra dev pytest tests/comfy_cli/cql tests/comfy_cli/command/test_run_prompt.py tests/comfy_cli/command/test_run.py tests/comfy_cli/command/test_run_json.py tests/comfy_cli/command/test_run_cli.py tests/comfy_cli/output488 passed. ruff format --check clean on all touched files.

Judgment calls / notes

  • Return shape. The ticket suggested resolve_default_checkpoint(...) -> (workflow, note|None), but a bare note can't distinguish "pinned present (fine)" from "enum positively empty (error)" — both would be None. I returned a small CheckpointResolution dataclass (note / substituted_to / no_checkpoint) so the caller can drive step 3 precisely and only hard-error when it positively knows the enum is empty.
  • --print-prompt is unchanged. The dry-run returns before the server round-trip, so it prints the bundled pin, not the runtime-resolved checkpoint (consistent with --print-prompt's existing "no server hit" contract). The prompt_preview event (emitted before resolution) likewise shows the pre-substitution graph; the explicit checkpoint_substituted event documents the swap in the stream.
  • Event schema. checkpoint_substituted is not added to schemas/run_event.json's type enum — that enum is already non-exhaustive (prompt_preview/converted aren't in it either; additionalProperties: true), so I matched the established pattern rather than partially expanding it.
  • Not a capability-denial regression (falsification). The new no_checkpoint_available deny path is capability-enabling on net: runtime resolution makes --prompt succeed wherever any checkpoint exists (the opposite of a dead-end). The error fires only in the provably-empty-enum case — a state where the workflow genuinely cannot run (no checkpoint to load) — and its hint redirects to the real fix (install a model / gallery template / --set). Fail-open is preserved for every ambiguous case.

Out of scope

Restructuring the bundled default to the gallery's newer Z-Image-Turbo graph (multi-file diffusion) is a larger, separate change and a product call on the blessed default model. Runtime resolution deliberately sidesteps that choice.

`comfy run --prompt` (zero-config) hard-pinned one checkpoint and failed
wherever it was absent — Comfy Cloud and fresh local installs.

- Fix the stale pin: node "4" ckpt_name is now
  v1-5-pruned-emaonly-fp16.safetensors (the gallery/tutorial SD1.5 file).
- Add a pure, offline-testable `resolve_default_checkpoint(workflow,
  object_info)` in cql/default_workflow.py: if the pinned checkpoint is in the
  target's CheckpointLoaderSimple enum → no change; else if the enum is
  non-empty → substitute the first available checkpoint + emit a note; else
  (positively-empty enum) → flag no_checkpoint; enum absent / object_info {} →
  fail open (unchanged behavior).
- Wire it into both run paths (local + cloud) after object_info is fetched and
  before preflight, guarded to the bundled default graph and skipped when the
  user pinned the checkpoint via --set (honored verbatim). Thread a
  checkpoint_user_set flag through the `preloaded` tuple.
- Emit a `checkpoint_substituted` event (--json) / pretty notice on
  substitution; raise a new `no_checkpoint_available` error with an actionable
  download/cloud hint when the target has zero checkpoints.
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 52 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3007a649-38a3-4661-8e96-51179f601ded

📥 Commits

Reviewing files that changed from the base of the PR and between a732f5c and 93cfdfb.

📒 Files selected for processing (8)
  • comfy_cli/cmdline.py
  • comfy_cli/command/run/__init__.py
  • comfy_cli/command/run/preflight.py
  • comfy_cli/cql/data/default_text2img.json
  • comfy_cli/cql/default_workflow.py
  • comfy_cli/error_codes.py
  • tests/comfy_cli/command/test_run_prompt.py
  • tests/comfy_cli/cql/test_default_workflow.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch matt/be-2994-runtime-default-checkpoint
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch matt/be-2994-runtime-default-checkpoint

Comment @coderabbitai help to get the list of available commands.

@mattmillerai mattmillerai added agent-coded PR authored by the agent-work loop cursor-review Request Cursor bot review labels Jul 14, 2026
@mattmillerai
mattmillerai marked this pull request as ready for review July 14, 2026 20:25
@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jul 14, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 7 finding(s).

Severity Count
🟡 Medium 3
🟢 Low 4

Panel: 6/8 reviewers contributed findings.

Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)

Comment thread comfy_cli/command/run/__init__.py Outdated
Comment thread comfy_cli/cql/default_workflow.py
Comment thread comfy_cli/command/run/__init__.py Outdated
Comment thread comfy_cli/command/run/preflight.py
Comment thread comfy_cli/cql/default_workflow.py
Comment thread comfy_cli/cql/default_workflow.py
Comment thread comfy_cli/command/run/preflight.py Outdated
…2994)

Address cursor-review findings on the runtime checkpoint resolver:

- Cloud fails open on an empty checkpoint enum instead of hard-erroring
  no_checkpoint_available: Comfy Cloud provisions models per-job, so a
  zero-length cached enum can't prove the run would fail. Only the local
  path (enum reflects what's installed) treats empty as a hard stop.
- Resolve the bundled default's checkpoint BEFORE emitting the
  prompt_preview event in the submit flow so the streamed audit trail
  advertises the graph we actually submit. --print-prompt stays a
  no-server-hit dry-run (documented) and prints the graph as-is.
- Guard _checkpoint_enum against non-dict object_info (a hostile server
  returning `[]`) so it fails open rather than crashing with AttributeError.
- Match a pinned bare filename against enum entries by basename so a
  checkpoint living in a subfolder (SD1.5/…safetensors) is found instead
  of triggering a needless substitution.
- Escape the target-provided checkpoint name in the pretty substitution
  notice to prevent Rich markup injection.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@mattmillerai

Copy link
Copy Markdown
Collaborator Author

Reviews resolver pass — all 7 review threads are now resolved, and no code changes were needed.

Review threads: 5 were already fixed in 93cfdfb. The 2 remaining were deferred/declined with reasoning; rather than leave them as loose unresolved threads, both are now recorded as tracked follow-ups (renderer events in single-envelope JSON; architecture-aware checkpoint substitution) and resolved. Neither is tagged for autonomous pickup — each needs a design decision first.

Merge state: MERGEABLE, no conflict with main.

The failing test check is NOT from this PR — it is a repo-wide CI failure.

test-windows.yml ("Windows Specific Commands") has failed 20 of 20 recent runs across every branch, including many PRs sharing no code with this one (be-3271, be-3272, be-3265, be-3268, be-3266, be-3270, be-2999, be-3000, …). Last green run on main was 2026-03-11 (1405d30). A gh run rerun --failed reproduced it identically (job 87814223657), so it is deterministic, not a flake.

Root cause: the workflow pip install -e . into venv, then runs comfy install --fast-deps from that same venv. uv tries to replace pydantic-core, but the live comfy process has already imported pydantic (via mixpanel), so Windows holds the loaded DLL open:

error: failed to remove file `...\venv\Lib\site-packages\pydantic_core/_pydantic_core.cp312-win_amd64.pyd`: Access is denied. (os error 5)
→ ImportError: cannot import name '__version__' from 'pydantic_core' (unknown location)

This PR touches only cmdline.py, command/run/{__init__,preflight}.py, cql/default_workflow.py, cql/data/default_text2img.json, error_codes.py + tests — nothing in the install/dependency path. Filed as a separate high-severity follow-up; fixing CI infra inside this behavioural PR would mask a repo-wide problem.

Verified locally: 63/63 PR tests pass (test_run_prompt.py, test_default_workflow.py), ruff check clean. Full suite: 2588 passed / 9 failed — the 9 (test_config_parser.py, test_node_init.py) are pre-existing, fail identically with main's copies of those files, are untouched by this PR, and pass in CI on all platforms (local git-env artifact).

All other checks green: ruff, build, CodeQL, GPU runners, and Tests on macos/ubuntu/windows 3.10.

@mattmillerai

Copy link
Copy Markdown
Collaborator Author

🤖 The reviews loop filed Linear follow-up ticket(s) for review thread(s) deferred as out of scope for this PR:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-coded PR authored by the agent-work loop cursor-review Request Cursor bot review size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant