U/alxmrs/experiments/kr1 v2 - #732

Draft

alxmrs wants to merge 74 commits into

mainfrom

u/alxmrs/experiments/kr1-v2

Commits on May 12, 2026

Commits on May 16, 2026

Compact multiscale zarr data pipeline
alxmrs
committed
Updating training to be preemptable on torch.
alxmrs
committed
Inter epoch checkpointing flag + updated script to better reflect sbatch comments to turn on preemption.
alxmrs
committed
v48 hparams: hist=0, pred_residuals=false, decoder.context_patches=3

authored and
alxmrs
committed
sbatch: bind host src/ + configs/ into container

authored and
alxmrs
committed
sbatch: mkdir RUN_DIR/wandb before binding into container

authored and
alxmrs
committed
launch: set --comment at submit time (scontrol update fails post-submit)

authored and
alxmrs
committed
launch: switch to rtx6000 partition (PreemptMode=REQUEUE)

authored and
alxmrs
committed
sbatch: propagate NCCL/UCX env vars into apptainer container

authored and
alxmrs
committed
v49: stage config with extended rollout curriculum

alxmrs
and
claude
committed
sbatch: hardcode NCCL/UCX env vars (slurm requeue drops shell exports)

authored and
alxmrs
committed
launch: lock NAME at submit time (date drifts on requeue/chain otherwise)

authored and
alxmrs
committed
launch_kr1_eval_halfdeg: switch to rtx6000_lzanna per docs/torch.md

authored and
alxmrs
committed
launch_kr1_eval_halfdeg: add --data.hist=0 to match v48 training

authored and
alxmrs
committed

Commits on May 18, 2026

Commits on May 20, 2026

Commits on May 21, 2026

v51: disable decoder.context_patches; log THP setting in sbatch banner

alxmrs
and
claude
committed

Commits on May 23, 2026

Commits on May 26, 2026

launch_kr1_eval_halfdeg: bump NAME_SUFFIX to mark post-resume epoch 48

alxmrs
and
claude
committed