Skip to content

fix(ops): Corr crashes in expanding mode (N=0)#2291

Open
UditDewan wants to merge 1 commit into
microsoft:mainfrom
UditDewan:fix/corr-expanding-window-crash
Open

fix(ops): Corr crashes in expanding mode (N=0)#2291
UditDewan wants to merge 1 commit into
microsoft:mainfrom
UditDewan:fix/corr-expanding-window-crash

Conversation

@UditDewan

Copy link
Copy Markdown

Description

Corr(x, y, 0) (expanding mode) crashes with ValueError: min_periods 1 must be <= window 0.

PairRolling documents N=0 as expanding mode (same convention as Rolling), and Cov(x, y, 0) works. But Corr._load_internal applies its zero-std NaN mask with an unconditional series.rolling(self.N, min_periods=1).std(), and pandas rejects rolling(0):

File "qlib/data/ops.py", line 1495, in _load_internal
    np.isclose(series_left.rolling(self.N, min_periods=1).std(), 0, atol=2e-05)
...
ValueError: min_periods 1 must be <= window 0

This PR computes the mask with expanding(min_periods=1).std() when N == 0 and keeps the rolling path byte-for-byte identical for N > 0.

Motivation and Context

Any expression using expanding correlation, e.g. Corr($close, ChangeInstrument('SH000300', $close), 0), currently fails, while the equivalent Cov(..., 0) succeeds. The masking line was added to suppress spurious correlations of near-constant series and simply never handled the expanding branch.

How Has This Been Tested?

  • Pass the test by running: pytest qlib/tests/test_all_pipeline.py under upper directory of qlib.
  • If you are adding a new feature, test on your own test scripts.

Added tests/ops/test_pair_rolling.py (no data download needed — uses a stub Expression):

  • expanding Corr matches pandas.Series.expanding().corr() (previously: crash)
  • constant series are still masked to NaN in expanding mode
  • rolling (N > 0) results are unchanged

Screenshots of Test Results (if appropriate):

  1. Your own tests:
test_corr_expanding_masks_constant_series ... ok
test_corr_expanding_no_crash ... ok
test_corr_rolling_unchanged ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.007s

OK

Types of changes

  • Fix bugs
  • Add new feature
  • Update documentation

PairRolling documents N=0 as expanding mode and Cov(x, y, 0) works,
but Corr(x, y, 0) raises ValueError because the zero-std NaN mask
unconditionally calls rolling(0, min_periods=1), which pandas rejects
(min_periods 1 must be <= window 0).

Use expanding std for the mask when N == 0; the rolling path is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant