rhizome: cosmic web volume — spec + SDSS calibration plan#114
Draft
rulkens wants to merge 5 commits into
Draft
Conversation
Design for replacing the SDSS-wedge MCPM overlay with a full-sky cosmic-web density field built by running PolyPhy ourselves on 2MRS+GLADE. Three nested cubes (±150 / ±425 / ±900 Mpc) at 256³, composited with AABB fade bands. Producer pipeline lives in a new skymap-rhizome repo; skymap consumes .npy from R2 like CF-4 and MCPM today. Includes a reproduction-against-SDSS calibration milestone that gates parameter selection before any rhizome cube ships. Supersedes the runtime portion of the 2026-05-11 MCPM cosmic-web spec (the MCPM cube becomes the calibration target rather than the shipped overlay). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
First plan in the rhizome series. Scopes the standalone `skymap-rhizome` Python repository (PolyPhy submodule pinned at 5f9cef8, Taichi 1.6.0 + Python 3.10.20 stack), a literature parameter-research pass for the Wilde et al. 2023 MCPM recipe, the producer scripts (skymap .bin decoder, calibration-shell CSV extractor, PolyPhy orchestrator with named parameter constants), and a Python-side numerical+visual comparison harness. Iteration loop is 3-5 named tuning rounds with explicit STOP-and-check-with-user gates between rounds. Out of scope for this plan: 2MRS+GLADE shells, .npy -> .scfd builder, skymap runtime wiring, R2 uploads, MCPM -> rhizome migration. Those are follow-up plans. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The .npy reference cube lives under the EXTRA_FILES path (data/raw/mcpm/) not the ALLOW-filtered top-level data/ path. Patched all four references in the calibration plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Architecture simplification — drop the 2-repo split for calibration. Everything during the SDSS reproduction phase lives in the rulkens/PolyPhy fork's rhizome/ directory: bin decoder (50-line Python port), input prep (curls sdss-large.bin from R2), PolyPhy orchestrator, comparison harness, parameter log. Skymap is not modified during calibration. Skymap only grows tools when the later shells-production plan needs buildRhizomeVolume.ts and renderer wiring. v4 PointCloud format is stable enough that the tiny Python decoder duplication carries no real drift risk. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The plan previously assumed a new `skymap-rhizome` repository with PolyPhy vendored as a submodule. The architecture has been simplified twice — the final shape (mirroring the updated spec) is that the entire calibration phase lives inside the existing `rulkens/PolyPhy` fork on a new `rhizome-sdss-calibration` branch, in a `rhizome/` directory sibling to `verification/`. Skymap itself is not touched by any task. The pinned `verification/.venv/` is reused as-is; there is no new repository, no submodule, no skymap-side tools. Task structure: branch + scaffold; parameter literature research with STOP gate; TDD-shaped scripts for the four entry points (decoder, input prep, PolyPhy orchestrator, comparison harness); baseline iter 0 with STOP gate; up to 5 explicitly non-TDD tuning iterations with per-iter STOP gates and a hard cap that surfaces upstream issues rather than thrashing; clean-checkout acceptance + commit-hygiene verification. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
skymap | 6e581d7 | Commit Preview URL Branch Preview URL |
May 12 2026, 10:49 AM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/superpowers/specs/2026-05-12-rhizome-cosmic-web-volume-design.md): replace the SDSS-wedge MCPM overlay with three full-sky nested cubes (±150 / ±425 / ±900 Mpc, all 256³) built by running PolyPhy ourselves on 2MRS+GLADE. Three field handles composited with AABB fade bands. Producer pipeline lives in the existing rulkens/PolyPhy fork; skymap only grows tools when the later shells plan needs.npy → .scfdand renderer wiring.docs/superpowers/plans/2026-05-12-rhizome-sdss-calibration.md): 9-task TDD-shaped (where applicable) implementation that reproduces the published Wilde et al. 2023 MCPM SDSS cube using our own PolyPhy invocation, locks in the parameter set for later use on 2MRS+GLADE. Calibration is 100% in the fork; skymap is not modified.Test plan
~/Development/vendor/python/PolyPhy/verification/README.md) consistent with plan's assumptions about CLI flags, output format, gotchasCalibration work happens in
rhizome-sdss-calibrationbranch of the rulkens/PolyPhy fork — seeArchitecturesection of spec for the full data flow.🤖 Generated with Claude Code