Phase 9 Session 2 — HGT retraining on 140-ticker graph + GNN drift factor null result#8
Merged
Merged
Conversation
…_version hgt_link_pred_v2 Retrained the HGT link-prediction model on the Phase 8 Session 2 graph (140 equities, 21–22 funds, 14 directors, 4 extended; 738 supply, 2,141 ownership, 14 board edges at the 2023-Q4 train target). Training run: - Entry: python -m nexus.graph.gnn.trainer (no code changes needed — HeteroData metadata is extracted dynamically; HGTConv adapts to the larger graph) - Hyperparameters unchanged from Phase 2: hidden=64, heads=4, layers=2, dropout=0.2, lr=5e-3, weight_decay=1e-4, 300 epochs, val_every=20 - Params: 155,184 - Train BCE 0.348 → 0.197 (monotonic descent over 300 epochs) - Val AUC trajectory: 0.9658 (e=20) → 0.9807 (e=280) - Gate PASS (>= 0.60); checkpoint saved to models/hgt_link_pred.pt - Wall-clock: 7m 03s The Phase 2 30-ticker checkpoint reached 0.9803 at e=240. The 140-ticker re-run reaches comparable AUC, confirming the link-prediction objective remains informative at the expanded universe size. MODEL_VERSION in nexus/graph/backfill.py bumped v1 -> v2 to distinguish the retrained embeddings from the historical 30-ticker rows that previously occupied node_embeddings.
…er hgt_link_pred_v2 Ran python -m nexus.graph.backfill (default window 2021-06-30 → 2026-03-31). Snapshot dates match centrality_history exactly (point-in-time integrity satisfied by construction). DB-side result: - node_embeddings: 8,120 rows = 140 companies × 58 month-ends - embedding_dim = 64 (constant) - model_version = 'hgt_link_pred_v2' on every row (UPSERT against the composite PK (company_id, snapshot_date) overwrote prior v1 rows) - Wall-clock: 86s Validation spot-checks at the latest snapshot (2026-03-31): - per-component std median 0.0515 (non-degenerate; range [0.018, 0.231]) - per-row L2 norm median 1.92 - cos(NVDA, AMD) = 0.9812 > cos(NVDA, ARW) = 0.6315 (peer vs distributor) - cos(AMD, INTC) = 0.9606 > cos(AMD, AVT) = 0.9099 (x86 peers vs distributor) - cos(LRCX, AMAT) = 0.9902 > cos(LRCX, ARW) = 0.7988 (equipment peers vs distributor) Structural ordering is decisive; embeddings encode the supply-chain / ownership topology as intended. This commit is empty by design (no source change). It records the operational milestone in the same history as the code commits that flank it.
… horizons (t<1), not registered Re-tested graph_gnn_embedding_drift on the 140-ticker universe using the retrained v2 embeddings (the factor had been excluded from every backtest since Phase 8 Session 2 via .pop() in research/phase8_backtest.py because the prior 30-ticker embeddings were OOD on the expanded universe). Multi-horizon IC results (58 monthly rebalances, avg cross-section ~138.6): horizon N mean_IC t_stat HLZ (M=400 Bonferroni) ------- -- -------- ------ ---------------------- 21d 57 +0.0075 +0.382 fail (|t| < 4.123) 63d 55 +0.0101 +0.524 fail (|t| < 4.134) 126d 52 +0.0077 +0.368 fail (|t| < 4.153) |t| < 1 at every horizon. Regime/sub-window splits suppressed because the |t| > 1.5 trigger never fires. Verdict under the Phase 9 Session 1 decision matrix: NULL (|t| < 2 at all horizons) → do NOT register. The retrained embeddings are structurally meaningful (the validation spot-checks above showed peer/distributor cosine ordering passes decisively). What this run rules out is that month-over-month cosine drift in those embeddings predicts forward returns on the semiconductor universe at full N=140 power. This matches the Phase 8 N=30 finding (mean_IC -0.006, classified DECLINING) and is now confirmed at full power. Adds two research artifacts: - research/phase9_gnn_drift_backtest.py — multi-horizon IC + HLZ report - research/_validate_embeddings.py — embedding validation spot-checks, underscore-prefixed to mark as a research/operator tool rather than a permanent code path Neither script writes to the database.
…l_registry
Idempotent INSERT ... ON CONFLICT (name) DO UPDATE writes:
name = graph_gnn_embedding_drift
type = graph
tier = A
status = rejected
t_stat = +0.524 (highest |t| across tested horizons, at 63d)
hlz_passes = false
regime_profile = JSONB with per-horizon n/mean_ic/t_stat, evidence note,
and reference to the Phase 8 N=30 DECLINING finding
Recording the verdict in signal_registry so future sessions do not
re-test the factor without first reading docs/progress/phase_9.md.
The factor remains computable on demand via the existing composer in
nexus/signals/factors/graph_based.py; this commit only adds the
registry row.
…initively null Session 2 retrained HGT on the Phase 8 Session 2 140-ticker graph and re-tested the previously-excluded graph_gnn_embedding_drift factor at full power. Outcome: - HGT val AUC 0.9807 at epoch 280 (Phase 2 30-ticker: 0.9803 at e=240) - node_embeddings re-backfilled: 8,120 rows under model_version 'hgt_link_pred_v2' (140 companies x 58 monthly snapshots) - Embedding structural validation passed (peer/distributor cosine ordering decisive at three test pairs) - graph_gnn_embedding_drift IC: t=+0.382/+0.524/+0.368 at 21/63/126d; |t| < 1 at every horizon; HLZ fail by 10x - Decision matrix verdict: NULL at all horizons -> registered as status='rejected' in signal_registry - Paper trader Step 4 skipped per the hard constraint (was conditional on Step 3 approving registration); Phase 9 Session 1 result remains: CAGR +8.72%, Sharpe 0.488, Max DD -32.68% The full Session 2 evidence record (training curve, validation spot-checks, per-horizon IC + HLZ, registry JSONB profile) lives in docs/progress/phase_9.md (gitignored, local-only). This commit appends the PROGRESS.md row that points to it. Phase 9 status: 1 fundamentals factor registered as alpha (fundamental_margin_compression), 1 graph-GNN factor registered as rejected (graph_gnn_embedding_drift). Next lane open.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Retrains the HGT link-prediction model on the Phase 8 Session 2 140-ticker graph, regenerates
node_embeddingsundermodel_version='hgt_link_pred_v2', and re-tests thepreviously-excluded
graph_gnn_embedding_driftfactor at full power. The factor is definitively null and is registered asstatus='rejected'.Motivation
Since Phase 8 Session 2,
graph_gnn_embedding_drifthas been excluded from every backtest becausenode_embeddingswas populated by a model trained on the 30-tickergraph. Running that model against the 140-ticker universe produces out-of-distribution embeddings. Phase 8 Session 2 deferred the retrain; this session closes it out.
Results
Step 1 — HGT retraining
python -m nexus.graph.gnn.trainerHeteroData.metadata()extracted dynamicallyStep 2 — Embedding backfill + validation
python -m nexus.graph.backfillwrote 8,120 rows (140 companies × 58 month-ends, 2021-06-30 → 2026-03-31) undermodel_version='hgt_link_pred_v2'(UPSERTed overprior v1 rows). Wall-clock 86s.
centrality_historyexactly — point-in-time integrity satisfied by construction.Step 3 — IC backtest for
graph_gnn_embedding_drift|t| < 1at every horizon. Decision matrix verdict: NULL at all horizons → do NOT register as alpha. Recorded insignal_registryasstatus='rejected'with fullper-horizon evidence in
regime_profileJSONB so future sessions do not silently re-test the factor.The retrained embeddings are structurally meaningful (Step 2 gates passed decisively). What this rules out is that month-over-month cosine drift in those embeddings
predicts forward returns on the semiconductor universe at full N=140 power. The Phase 8 N=30 finding (mean_IC −0.006, DECLINING) is now confirmed at full power, not as a
small-sample artifact.