feat(prices): propadj (ratio) adjustment — salvage low-priced contracts (DC)#26
Merged
Conversation
Norgate publishes continuous futures only unadjusted and ADDITIVELY back-adjusted (_CCB) — no native ratio-adjusted series. For a low-priced, ~29-year contract like DC (Class III Milk, ~$15-20/cwt), additive accumulation of roll gaps drives 46.7% of DC_backadj closes <= 0 (range -9.83..23.09), which makes price-based stops, R-multiples, and percentage returns meaningless — so CMR cannot use DC's backadj at all. Add a `propadj` view derived on read from the already-stored unadj + backadj series (no producer re-run, no schema bump, cross-platform): the offset O=B-U steps only at rolls, so each roll's calendar spread is recoverable (s = O[r-1]-O[r]) and convertible to a multiplicative ratio k = (U[r-1]+s)/U[r-1]. Scaling each segment by the cumulative product of k, anchored to the most-recent segment, yields a strictly-positive series (DC 4.68..25.01 over full 1997-2026 history) that preserves within-segment percentage returns exactly and is sign-identical to backadj on every day including rolls. Recommendation: CMR reads DC (and any similarly low-priced, long-history contract) with adjustment="propadj". Restricting DC to its positive-price era (~2011+) or dropping it were the fallbacks; neither is needed. - prices.get_prices: accept adjustment="propadj"; _ratio_adjust() derivation - tests/test_propadj.py: positivity, anchor, %-return + sign fidelity, hand-computed factors, no-Delivery-Month fallback, missing-series safety, reconstructed-volume composition - README: document propadj + the DC rationale Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mspinola
added a commit
that referenced
this pull request
Jul 20, 2026
#28) * docs: fix dangling doc refs and backfill CHANGELOG The docs/plan_promote_reconstructed_volume.md spec was removed in #15 but was still referenced from config.py, store.py (a user-facing RuntimeError), and README.md — each a dead pointer. Drop the references; the surrounding text and the README reconstructed-volume section already explain the schema-v2 behavior. Backfill the Unreleased CHANGELOG, which had only #23: add the Yahoo provider + MSCI held-out markets (#24), fold #26 into the propadj entry, and record the metadata-upsert (#25) and Yahoo-only-skip (#27) fixes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: commit uv.lock for reproducible installs uv.lock was neither tracked nor ignored, so it perpetually showed as untracked. The README promotes the uv workflow (`uv pip install -e .`), so track the lock for reproducible resolution. Verified current with `uv lock --check`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test: cover dormant databento provider and COT read API The largest provider (databento, 280 LOC) had no tests and cot.py's consumer read path was untested. Add: - test_databento_provider.py — smoke tests mocking the databento SDK (an optional extra) that lock the hard-won statistics parsing: Open Interest from stat_type 9, and settlement (stat_type 3) overriding Close dated by ts_ref. - test_cot.py — get_cot report dispatch, symbol/CFTC-code lookup, and the predecessor-code stitching (scale numeric cols, re-stamp code, keep primary on overlapping dates, sort). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(prices): fail fast on Norgate outage; drop null metadata rows Two producer reliability fixes for the Norgate path. 1. Preflight NDU reachability. norgatedata retries each data call 10x then calls bare sys.exit() — which exits 0 (a scheduled --prices run looks "successful" while writing nothing and never triggers scheduler retry) and raises SystemExit past the per-symbol `except Exception`, killing the run on the first symbol. update()/update_metadata() now probe norgatedata.status() (a safe, non-exiting check) first and raise a clear RuntimeError → non-zero exit if NDU is down. 2. Guard all-null metadata rows. get_symbol_metadata catches each field error to None, so a transient Norgate failure on a covered symbol wrote an all-null spec row (and on a scoped upsert would overwrite good specs with nulls). Skip such rows with a warning. Tests: preflight aborts update()/update_metadata() when status() is False; the all-null row is skipped on a full run. Full suite 81 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
propadj(proportional / ratio) adjustment toget_prices, derived on read from the storedunadj+ additivebackadjseries — no producer re-run, no schema bump.Norgate only publishes additive back-adjustment; on a low-priced, long-history contract (DC / Class III Milk) the accumulated roll gaps drive ~47% of
backadjcloses ≤ 0 (min −9.83), which breaks close-based stops and R-multiples.propadjrecovers each roll's spread from the additive offset, compounds it into a multiplicative factor, and yields a strictly positive series with identical percent returns, anchored to actual current price.Verified
On the real store: DC
backadjmin −9.83 (47% ≤ 0) →propadjmin 4.68, 0% ≤ 0, last 15.76 (real ~$/cwt milk price). 9 tests passed.Note
Reusable for any low-priced long-history contract. Its motivating market (DC) is illiquid, so the immediate need is low — merged as general infrastructure.
🤖 Generated with Claude Code