Skip loading chunks below a dataset's scheduled from block

[elina-chertova]

Problem

The tethys (testnet) cron-scheduler went stale on 2026-06-30 (testnet_scheduler_stale). Trigger: the dataset ethereum-mainnet-3 (from: 25000000, weight: 3) was added to the tethys scheduler config. Its first run had to enumerate the dataset from scratch.

From the scheduler logs (testnet-control-plane, pod cron-scheduler-29713660):

WARN  Dataset s3://ethereum-mainnet-3 summary population timed out after 600s. 18999 chunks processed.
DEBUG load_newer_chunks finished in 614.245s
DEBUG process finished in 619.217s          # normal runs take ~5s
INFO  Assignment uploaded ... 11:51:55       # previous assignment was 11:21:32

The 614s load pushed the 20-min cron cadence to ~30 min, so the last-assignment age crossed the staleness threshold (alert value 1821s).

Crucially, all 18999 chunks it spent the 600s budget on were blocks 0 → ~2.36M — entirely below from: 25000000, so weight::prepare_chunks discards every one of them. The run never even reached the scheduled range. The scheduler was paying for an S3 GET + parquet parse on millions of chunks it then throws away.

Fix

Thread each dataset's minimum scheduled block (segments[0].from, when it's an absolute non-negative offset) down to the chunk loader, and skip the summary download for chunks that start below it — mirroring the filter weight::prepare_chunks already applies. Datasets with from: 0 (the common case) and head-relative negative offsets are unaffected (from_block = None, previous behavior preserved).

This removes the wasted work, so a newly added high-from dataset loads only its schedulable chunks and the assignment is published on time.

Test

skips_chunks_below_dataset_from in src/storage.rs asserts the skip predicate matches weight's keep rule (kept iff start >= from), including a straddling chunk and the None (no-bound) case. It fails on the pre-fix behavior (all chunks kept) and passes after — verified locally red→green.

Verified locally: cargo test --all-features (24 pass, 5 ignored S3 integration tests), cargo fmt --all -- --check, cargo clippy --all-targets --all-features -- -D clippy::correctness -D clippy::suspicious -D perf — all green.

Falsification

If after this change the scheduler still stalls loading a freshly added high-from dataset, the bottleneck is elsewhere (e.g. listing the schedulable range itself, or an S3 hang — see #44).

Related to #35 (the dataset load timeout) and #44 (bounded S3 op timeouts) — different cause: those bound how long loading can take; this removes work that should never have been done.