Harden loss functions and raise evaluation rigor#77
Open
v-atharva wants to merge 3 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
+vs*typo incustom_loss_13/14/15/16(entropy and HHI were added to their lambdas instead of scaled by them); clamp1 + rto>= epsinlog_return / log_sharpe / log_sortinoso returns near-1don't produce non-positive log args; validatetemp/beta > 0at entry of every softplus-based loss and eps-floor thebeta = 1/tempin Rockafellar CVaR.tests/unit/training/test_loss_functions.py(99 cases) — registry-driven shape/finiteness/gradient coverage for every registered loss, plus targeted regressions for each fix and semantic checks (HHI uniform=0 / concentrated=1, CVaR top-k matches empirical tail mean, log-return monotonicity).MinVarianceCalculator(long-only QP via cvxopt, optional linear shrinkage) andEqualRiskContribCalculator(Maillard-Roncalli-Teiletche fixed point); add stationary-block bootstrap helpersbootstrap_metric_ciandbootstrap_paired_diff_ci, exposed viaEvaluator.calc_metric_performance_ciandcalc_paired_diff_ci; registerturnoverinMetricLibraryand addEvaluator.calc_turnover_for_allandcalc_cost_adjusted_daily_rets.Trainernow supports periodic checkpointing (controlled bytrain_hparams.checkpoint_everyandcheckpoint_keep_last) persisting model, optimizer, scheduler, RNG, and early-stopping state atomically; newTrainer.resume_from()andfind_latest_checkpoint();scripts/run_multi_train.pyaccepts--resumeand writes per-seed checkpoints underartifacts/results/checkpoints/<combo-seed>/.Test plan
pytest tests/unit/training/test_loss_functions.py— 99 / 99 passpytest tests/unit— 306 pass; 5 pre-existing failures onmain(unrelated:test_loading.py,test_io.py)MinVarianceCalculator+EqualRiskContribCalculatoron synthetic returns — weights sum to 1, long-only, ERC risk contributions equal to ~1e-6calc_metric_performance_cion synthetic dataEvaluator.calc_turnover_for_allandcalc_cost_adjusted_daily_rets— turnover in[0, 1], cost-adjusted mean < gross meanTrainercheckpoint/resume: 4-epoch run → kill → resume from epoch 4 → 2 more epochs; loss continues decreasing across the boundary, oldest checkpoint pruned perkeep_lastscripts.run_multi_traingrid once dependencies (statsmodelsetc.) are installed on the target env