[queue-split 1/3, merge FIRST — backward compatible] Explicit isGPUWorker label on every prod worker Dockerfile#153
Conversation
…file Adds a literal isGPUWorker="true"/"false" LABEL to the final build stage of every one of the 38 production worker Dockerfiles (per AWSDeploy's scripts/workers.tsv manifest), their Dockerfile_M1 variants where present, and the sample_interface copy-paste template (labeled "false"). This lets NimbusImage's dispatcher route each worker to the gpu or cpu Celery queue by image label instead of guessing; an unlabeled image falls back to gpu with a warning. 25 CPU workers -> isGPUWorker="false", 13 GPU/ML workers -> isGPUWorker="true", per the classification table in AWSDeploy's task-6 constraints. Also documents the requirement in CLAUDE.md's Docker Structure section for future workers.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 68036a4c58
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| WORKDIR / | ||
|
|
||
| LABEL isUPennContrastWorker="" \ | ||
| isGPUWorker="false" \ |
There was a problem hiding this comment.
With the new dispatcher consuming isGPUWorker, this line sends the production Deconwolf image to the CPU queue even though this Dockerfile is CUDA/OpenCL-enabled and the worker interface defaults Use GPU to true. In that context, Deconwolf jobs that should run on the GPU fleet will land on CPU workers and either fall back to CPU or run much slower, so the production Dockerfile should be labeled true (the M1 variant can remain CPU-only).
Useful? React with 👍 / 👎.
1 participant
Part 1 of 3 of the cpu/gpu Celery queue split (AWSDeploy cost review Rec A): CPU-only jobs move to a cheap always-on CPU box so they stop contending for the GPU fleet.
What this PR does
Adds an explicit
isGPUWorker="true"(13 GPU workers) orisGPUWorker="false"(25 CPU workers) Docker label to every production worker Dockerfile — in the final build stage for multi-stage files — plus the 9Dockerfile_M1variants and thesample_interfacetemplate. The NimbusImage dispatcher (PR 3/3) will read this label at dispatch time to route each job to thegpuorcpuqueue; both classes are labeled explicitly so a missing label is a logged fail-safe, never the path for a known worker. Classification verified 38/38 against AWSDeploy'sscripts/workers.tsvqueue column (25 cpu / 13 gpu). Also documents the new label requirement in CLAUDE.md.Coupling / merge order
isUPennContrastWorkerpresence); a label is inert image metadata — no runtime, dependency, or entrypoint changes.doc/Build_and_Push_Worker_Images_to_ECR.md) is what actually lands the labels on:latest— that is Step 1 of the cutover runbook.How to test
Any worker runs identically to before — the label changes no behavior.
🤖 Generated with Claude Code
https://claude.ai/code/session_01VjwUGizxeWLUxYs4yGa3me