Skip to content

Harden loss functions and raise evaluation rigor#77

Open
v-atharva wants to merge 3 commits into
loss-functionsfrom
feat/loss-hardening-eval-rigor
Open

Harden loss functions and raise evaluation rigor#77
v-atharva wants to merge 3 commits into
loss-functionsfrom
feat/loss-hardening-eval-rigor

Conversation

@v-atharva

Copy link
Copy Markdown
Collaborator

Summary

  • Correctness (loss_functions.py): fix + vs * typo in custom_loss_13/14/15/16 (entropy and HHI were added to their lambdas instead of scaled by them); clamp 1 + r to >= eps in log_return / log_sharpe / log_sortino so returns near -1 don't produce non-positive log args; validate temp/beta > 0 at entry of every softplus-based loss and eps-floor the beta = 1/temp in Rockafellar CVaR.
  • Tests: new tests/unit/training/test_loss_functions.py (99 cases) — registry-driven shape/finiteness/gradient coverage for every registered loss, plus targeted regressions for each fix and semantic checks (HHI uniform=0 / concentrated=1, CVaR top-k matches empirical tail mean, log-return monotonicity).
  • Evaluation: add MinVarianceCalculator (long-only QP via cvxopt, optional linear shrinkage) and EqualRiskContribCalculator (Maillard-Roncalli-Teiletche fixed point); add stationary-block bootstrap helpers bootstrap_metric_ci and bootstrap_paired_diff_ci, exposed via Evaluator.calc_metric_performance_ci and calc_paired_diff_ci; register turnover in MetricLibrary and add Evaluator.calc_turnover_for_all and calc_cost_adjusted_daily_rets.
  • Training robustness: Trainer now supports periodic checkpointing (controlled by train_hparams.checkpoint_every and checkpoint_keep_last) persisting model, optimizer, scheduler, RNG, and early-stopping state atomically; new Trainer.resume_from() and find_latest_checkpoint(); scripts/run_multi_train.py accepts --resume and writes per-seed checkpoints under artifacts/results/checkpoints/<combo-seed>/.

Test plan

  • pytest tests/unit/training/test_loss_functions.py — 99 / 99 pass
  • Full pytest tests/unit — 306 pass; 5 pre-existing failures on main (unrelated: test_loading.py, test_io.py)
  • Smoke-tested MinVarianceCalculator + EqualRiskContribCalculator on synthetic returns — weights sum to 1, long-only, ERC risk contributions equal to ~1e-6
  • Smoke-tested bootstrap CI + paired diff + calc_metric_performance_ci on synthetic data
  • Smoke-tested Evaluator.calc_turnover_for_all and calc_cost_adjusted_daily_rets — turnover in [0, 1], cost-adjusted mean < gross mean
  • Smoke-tested Trainer checkpoint/resume: 4-epoch run → kill → resume from epoch 4 → 2 more epochs; loss continues decreasing across the boundary, oldest checkpoint pruned per keep_last
  • Re-run a real scripts.run_multi_train grid once dependencies (statsmodels etc.) are installed on the target env

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant