diff --git a/docs/testing/TEST_COVERAGE.md b/docs/testing/TEST_COVERAGE.md index 50e253cb0..7fd5a0120 100644 --- a/docs/testing/TEST_COVERAGE.md +++ b/docs/testing/TEST_COVERAGE.md @@ -336,6 +336,7 @@ For each invalid_type in [string, object, array, ...]: - Don't need to add for every operator in `find()` computed projection: (should have same behavior with aggregation) - Don't need to add for every operator in `core/operator/aggregation/stages/set`: (alias for `$addFields`, separate code path) - Don't need to add for every operator in `$lookup` and `$facet` pipeline: (too deep nesting) +- Don't need to add for window-only operators in `$setWindowFields` **Example**: generating `$add` tests adds test cases in these files: - `stages/project/test_operators_in_project.py` @@ -567,6 +568,71 @@ Same shape as §11, applied to a different axis: §11 covers expression operator --- +### 22. Window Operator Coverage + +**Rule**: Each window operator is tested for its computation correctness across different frame shapes, while shared `$setWindowFields` stage mechanics (frame parsing, partitionBy, sortBy validation, error codes) are tested once at the stage level. + +**Per-operator tests** (under `tests/core/operator/window/$operator/`): +- **Documents-mode frame computation**: operator's computed result under whole-partition ["unbounded", "unbounded"], cumulative ["unbounded", "current"], reverse-cumulative ["current","unbounded"], and sliding frame shapes [-1,1] — verifies the operator produces correct values *given* a set of documents in the frame. This is distinct from stage-level frame boundary tests which verify that the correct documents are *selected into* the frame. +- **Field paths**: operator handles nested fields (`"a.b.c"`), missing fields, field exists with `null` value (distinct from missing), array fields, arrays at intermediate path level (`"arr.nested.value"` where `arr` is an array of objects), and numeric path components (`"arr.0.field"`) +- **Numeric precision**: type mixing (Int32/Int64/Double/Decimal128), overflow, catastrophic cancellation +- **Special floats**: NaN/Infinity propagation in removable vs non-removable windows +- **Non-numeric handling**: null, missing, string, boolean, date, object, array values in the expression field +- **Argument validation**: operator rejects invalid input shapes — unknown keys in the operator spec, wrong argument type (e.g. string where object expected), missing required parameters, and extra parameters. Each operator defines its accepted shape (empty object for rank operators, expression for accumulators, named params for $shift/$expMovingAvg/etc.); anything outside that shape must error with the correct code. + +**Tested at the stage level** (under `tests/core/operator/stages/setWindowFields/`): +- Frame validation errors (documents, range, time-range): invalid bound types, malformed arrays, conflicting modes — these are operator-agnostic +- Stage-level parameter validation: verify the `$setWindowFields` stage document structure per the spec. For each parameter below, test both valid inputs (succeeds) and invalid inputs (errors with correct code): + - `partitionBy`: optional. Valid: expression (`"$field"`), literal value, omitted (entire collection = one partition). Invalid: test is not needed — any expression is accepted. + - `sortBy`: optional in some cases, required in others. Valid: object with field-direction pairs (`{"field": 1}`, `{"a": 1, "b": -1}`), omitted when using unbounded `["unbounded", "unbounded"]` documents-mode window with a non-rank operator. Invalid/errors: omitted when using (1) rank/order operators (`$rank`, `$denseRank`, `$documentNumber`), (2) bounded documents-mode windows (e.g., `[-1, 1]`, `["unbounded", "current"]`, `["current", "unbounded"]`), (3) `$linearFill` operator, (4) range-mode or time-range-mode windows. Also invalid: non-object type, empty object, direction values other than 1/-1. + - `output`: required. Valid: document with one or more output fields, each containing a window operator and optional `window` key; dotted field names create embedded documents (same semantics as `$addFields`/`$set`). Invalid: omitted, non-document type, empty document, output field with no recognized window operator. + - `window`: optional per output field (omitting defaults to unbounded whole-partition). Valid: document with exactly one of `documents` or `range`. Invalid: empty document, both `documents` and `range` specified, unknown keys inside `window`, non-document type. + - `window.documents`: valid: two-element array where each element is `"current"`, `"unbounded"`, or an integer; lower bound ≤ upper bound. Invalid: non-array, array with fewer or more than 2 elements, non-integer numeric bounds (e.g., 1.5), string values other than `"current"`/`"unbounded"`, lower bound > upper bound. + - `window.range`: valid: two-element array where each element is `"current"`, `"unbounded"`, or a number; lower bound ≤ upper bound; requires `sortBy` on a single field. Invalid: non-array, array with fewer or more than 2 elements, string values other than `"current"`/`"unbounded"`, lower bound > upper bound, `sortBy` with multiple fields. + - `window.unit`: optional; when specified, converts `range` window to time-based. Valid: `"year"`, `"quarter"`, `"month"`, `"week"`, `"day"`, `"hour"`, `"minute"`, `"second"`, `"millisecond"`. Invalid: unrecognized string, non-string type, used with `documents` mode (should be rejected or ignored — verify behavior). + - Unknown top-level keys in the `$setWindowFields` stage document are rejected with an error. +- Documents-mode frame boundaries: using `$sum` as a sample operator, verify that document-based frame boundary specifications (centered, trailing, leading, non-overlapping) correctly control which documents are selected into the window. Edge cases: empty frame, single-element frame, and frame wider than the partition. +- Range-mode frame boundaries: using `$sum` as a sample operator, verify that numeric range-based frame bounds correctly define the window of documents selected for computation. +- Time-range-mode frame boundaries: using `$sum` as a sample operator, verify that time-based range frame bounds correctly define the window of documents selected for computation. +- Partitioning: using `$sum` as a sample operator, verify that the stage correctly isolates computation within each partition when data spans multiple partitions. +- Sort interaction: using `$sum` as a sample operator, verify that the stage respects the specified sort order and that changing sort order produces correspondingly different results. +- Multiple window fields with different operators and different window specifications: verify that a single `$setWindowFields` stage correctly computes results when the `output` document contains multiple fields, each using a different window operator (e.g., `$sum`, `$avg`, `$max`) with a different `window` specification (e.g., one with documents-mode `[-1, 1]`, another with `["unbounded", "current"]`, another with range-mode bounds). Each operator must independently respect its own frame without cross-contamination. +- Default window when `window` key is omitted: using `$sum` as a sample operator, verify that omitting the `window` key from an output field defaults to unbounded-to-unbounded (whole partition) regardless of whether `sortBy` is present. Frameless operators have implicit behavior that is tested in their own folders. + +**Why per-operator frame tests are NOT duplicated stage-level tests**: Stage-level frame tests (with a representative operator like `$sum`) verify that the correct documents are selected into the frame. Per-operator frame tests for whole-partition ["unbounded", "unbounded"], cumulative ["unbounded", "current"], reverse-cumulative ["current","unbounded"], and sliding frame shapes [-1,1] verify that the operator computes the correct result *given* those documents — different operators produce different values for the same frame. This is especially important for operators where the formula interacts with frame boundaries (e.g. `$stdDevSamp` returns null for single-element frames due to N-1 divisor). + +**Shared infrastructure**: Helpers and common test data live in `tests/core/operator/window/utils/`. The `run_window_operator()` helper and `BASIC_DOCS` dataset are shared across all window operators. + +**Category applicability** — not all dimensions apply to all operators: + +- **Frameless operators** ($rank, $denseRank, $documentNumber, $shift, $expMovingAvg, $locf, $linearFill): skip documents-mode, range-mode, time-range-mode frame tests. Verify that specifying a `window` key is rejected with correct error. Test operator-specific mechanics instead. + +- **Rank operators** ($rank, $denseRank, $documentNumber): no expression input, no numeric precision or non-numeric handling tests. Takes empty object only (`$rank: {}`) — verify non-empty object, field path string, and other non-empty values are rejected with correct error. Core dimension is **tie handling** — no ties, all ties, partial ties at beginning/middle/end, multi-field sortBy tie-breaking. $rank skips positions after ties (1,1,3); $denseRank does not (1,1,2); $documentNumber assigns unique positions regardless (1,2,3). + +- **N-parameter operators** ($firstN, $lastN, $topN, $bottomN, $minN, $maxN): test `n`=1, `n`=partition-size, `n`>partition-size (returns all available), `n`=0 (error or empty per spec), invalid `n` (negative, fractional, string, null → error codes). For $top/$topN/$bottom/$bottomN, also test the operator's own `sortBy` and `output` parameters. + +- **Multi-expression operators** ($covariancePop, $covarianceSamp): take two expressions `["$x", "$y"]`. Test null/missing/type-mixing per expression position independently. $covarianceSamp with single element → null (N-1 divisor). + +- **Gap-filler operators** ($locf, $linearFill): core dimension is **gap definition and boundary behavior** — null/missing as gaps, first value in partition is gap (no prior anchor → null), last value is gap, consecutive gaps, partition boundary isolation. $linearFill requires at least two non-null numeric anchors to interpolate. + +- **Calculus operators** ($derivative, $integral): require `unit` parameter (time unit string) when sortBy is date, omit `unit` when sortBy is numeric. Test zero delta in sortBy (division by zero for $derivative), single document in frame → null. + +- **$expMovingAvg**: `N` mode vs `alpha` mode (mutually exclusive). Test N=1 (no smoothing), alpha=1 (no memory), invalid values (N=0, alpha>1). First document result equals first value. + +- **$shift**: uses `by` (int offset), `output` (expression), `default` (value when offset exceeds partition). Test positive/negative/zero `by`, large offset beyond partition → default returned. + +- **Array-output operators** ($push, $addToSet, $firstN, $lastN, $topN, $bottomN, $minN, $maxN, $concatArrays, $setUnion): empty-frame result is `[]` (not null). Test output ordering (deterministic for ordered operators, unordered for $addToSet/$setUnion), deduplication semantics ($addToSet, $setUnion), mixed BSON types in output. + +- **Order-dependent vs order-independent**: Order-independent operators ($sum, $avg, $min, $max, $count, $stdDevPop, $stdDevSamp, $addToSet, $setUnion, $median, $percentile, $minN, $maxN) — sort test verifies **same result** with different sort orders. Order-dependent operators ($first, $last, $push, $rank, $denseRank, $documentNumber, $shift, $locf, $linearFill, $derivative, $integral, $expMovingAvg, $concatArrays) — sort test verifies **different results** with different sort orders. + +- **Non-numeric handling varies by category**: Accumulators **ignore** non-numeric values (compute on remaining numerics; all-non-numeric → null). N-selectors and $shift **return** non-numeric values as-is. $push/$concatArrays **collect** values of any type. $addToSet/$setUnion **collect unique** values of any type. Gap-fillers treat non-numeric as gaps to fill. + +- **Default window when `window` key is omitted**: Tested at the stage level (see below). One representative test using `$sum` verifies that omitting the `window` key defaults to unbounded-to-unbounded (whole partition). Per-operator tests do not need to repeat this. + +- **$median and $percentile**: `method` parameter ("approximate" is the only valid value). $percentile additionally takes `p` (array of doubles in [0,1], non-empty). Test boundary percentiles (p=0, p=1, p=0.5), multiple percentiles in single call, invalid `p` values. + +--- + ## Test Category Checklist For any DocumentDB feature, ensure coverage of: @@ -592,7 +658,6 @@ For any DocumentDB feature, ensure coverage of: - [ ] **Undocumented type coercion**: per command — when the engine accepts more types than the spec documents (numeric→bool, whole-double→int, null-as-omitted, etc.), mirror the canonical matrix from the most thoroughly-tested consumer (§20) - [ ] **Numeric equivalence**: equivalent values across numeric types grouped/matched correctly (if applicable) - [ ] **BSON type distinction**: different BSON types treated as distinct (if applicable) -- [ ] **Pipeline stage interaction**: interaction with preceding/following stages (if pipeline stage) - [ ] **Pipeline stage core semantics**: primary operation, empty input, non-existent collection, sole stage (if pipeline stage) - [ ] **Pipeline stage parameter validation**: accepted types, rejected values, stage shape, parse-time validation (if pipeline stage) - [ ] **Pipeline stage document handling**: pass-through preservation or output shape verification (if pipeline stage) diff --git a/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_documents_frame_boundaries.py b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_documents_frame_boundaries.py new file mode 100644 index 000000000..09d0d7f71 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_documents_frame_boundaries.py @@ -0,0 +1,303 @@ +""" +Tests for $setWindowFields documents-mode frame boundary selection. + +Using $sum as a sample operator, verifies that document-based frame boundary +specifications correctly control which documents are selected into the window. + +Covers: centered, trailing, leading, non-overlapping frames, and edge cases +(empty frame, single-element frame, frame wider than partition). + +Note: These stage-level tests verify correct document selection into the frame. +Per-operator tests (under window/$operator/) verify the operator computes +correct results given those documents. +""" + +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + +BASIC_DOCS = [ + {"_id": 1, "partition": "A", "value": 1}, + {"_id": 2, "partition": "A", "value": 2}, + {"_id": 3, "partition": "A", "value": 4}, + {"_id": 4, "partition": "A", "value": 8}, + {"_id": 5, "partition": "A", "value": 16}, +] + + +def _run_sum_window(collection, docs, window, sort_by=None): + """Helper to run $sum with a given window spec.""" + if sort_by is None: + sort_by = {"_id": 1} + collection.insert_many(docs) + stage = { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": sort_by, + "output": { + "result": { + "$sum": "$value", + "window": window, + } + }, + } + } + return execute_command( + collection, + {"aggregate": collection.name, "pipeline": [stage], "cursor": {}}, + ) + + +# Property [Centered Frame]: symmetric window around current row + + +def test_centered_frame(collection): + """Centered window [-1, 1] includes one row before and one after current.""" + result = _run_sum_window(collection, BASIC_DOCS, {"documents": [-1, 1]}) + # Row 1: [1, 2] = 3 (no row before) + # Row 2: [1, 2, 4] = 7 + # Row 3: [2, 4, 8] = 14 + # Row 4: [4, 8, 16] = 28 + # Row 5: [8, 16] = 24 (no row after) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 3}, + {"_id": 2, "partition": "A", "value": 2, "result": 7}, + {"_id": 3, "partition": "A", "value": 4, "result": 14}, + {"_id": 4, "partition": "A", "value": 8, "result": 28}, + {"_id": 5, "partition": "A", "value": 16, "result": 24}, + ] + assertSuccess(result, expected, msg="centered window [-1, 1] selects correct documents") + + +# Property [Trailing Frame]: window includes only current and preceding rows + + +def test_trailing_frame(collection): + """Trailing window [-2, 0] includes up to 2 rows before and current.""" + result = _run_sum_window(collection, BASIC_DOCS, {"documents": [-2, 0]}) + # Row 1: [1] = 1 + # Row 2: [1, 2] = 3 + # Row 3: [1, 2, 4] = 7 + # Row 4: [2, 4, 8] = 14 + # Row 5: [4, 8, 16] = 28 + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 1}, + {"_id": 2, "partition": "A", "value": 2, "result": 3}, + {"_id": 3, "partition": "A", "value": 4, "result": 7}, + {"_id": 4, "partition": "A", "value": 8, "result": 14}, + {"_id": 5, "partition": "A", "value": 16, "result": 28}, + ] + assertSuccess(result, expected, msg="trailing window [-2, 0] selects correct documents") + + +# Property [Leading Frame]: window includes only current and following rows + + +def test_leading_frame(collection): + """Leading window [0, 2] includes current and up to 2 rows after.""" + result = _run_sum_window(collection, BASIC_DOCS, {"documents": [0, 2]}) + # Row 1: [1, 2, 4] = 7 + # Row 2: [2, 4, 8] = 14 + # Row 3: [4, 8, 16] = 28 + # Row 4: [8, 16] = 24 + # Row 5: [16] = 16 + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 7}, + {"_id": 2, "partition": "A", "value": 2, "result": 14}, + {"_id": 3, "partition": "A", "value": 4, "result": 28}, + {"_id": 4, "partition": "A", "value": 8, "result": 24}, + {"_id": 5, "partition": "A", "value": 16, "result": 16}, + ] + assertSuccess(result, expected, msg="leading window [0, 2] selects correct documents") + + +# Property [Non-Overlapping Frame]: window excludes current row + + +def test_non_overlapping_both_before(collection): + """Non-overlapping window [-3, -1] excludes current row, looks back only.""" + result = _run_sum_window(collection, BASIC_DOCS, {"documents": [-3, -1]}) + # Row 1: empty = 0 + # Row 2: [1] = 1 + # Row 3: [1, 2] = 3 + # Row 4: [1, 2, 4] = 7 + # Row 5: [2, 4, 8] = 14 + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 0}, + {"_id": 2, "partition": "A", "value": 2, "result": 1}, + {"_id": 3, "partition": "A", "value": 4, "result": 3}, + {"_id": 4, "partition": "A", "value": 8, "result": 7}, + {"_id": 5, "partition": "A", "value": 16, "result": 14}, + ] + assertSuccess(result, expected, msg="non-overlapping [-3, -1] excludes current row") + + +def test_non_overlapping_both_after(collection): + """Non-overlapping window [1, 3] excludes current row, looks forward only.""" + result = _run_sum_window(collection, BASIC_DOCS, {"documents": [1, 3]}) + # Row 1: [2, 4, 8] = 14 + # Row 2: [4, 8, 16] = 28 + # Row 3: [8, 16] = 24 + # Row 4: [16] = 16 + # Row 5: empty = 0 + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 14}, + {"_id": 2, "partition": "A", "value": 2, "result": 28}, + {"_id": 3, "partition": "A", "value": 4, "result": 24}, + {"_id": 4, "partition": "A", "value": 8, "result": 16}, + {"_id": 5, "partition": "A", "value": 16, "result": 0}, + ] + assertSuccess(result, expected, msg="non-overlapping [1, 3] excludes current row") + + +# Property [Empty Frame]: window selects no documents + + +def test_empty_frame_returns_zero(collection): + """Window far beyond partition edges selects no documents — $sum returns 0.""" + result = _run_sum_window(collection, BASIC_DOCS, {"documents": [5, 10]}) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 0}, + {"_id": 2, "partition": "A", "value": 2, "result": 0}, + {"_id": 3, "partition": "A", "value": 4, "result": 0}, + {"_id": 4, "partition": "A", "value": 8, "result": 0}, + {"_id": 5, "partition": "A", "value": 16, "result": 0}, + ] + assertSuccess(result, expected, msg="empty frame (beyond partition) returns 0 for $sum") + + +# Property [Single-Element Frame]: window [0, 0] selects only current row + + +def test_single_element_frame(collection): + """Window [0, 0] selects only the current document.""" + result = _run_sum_window(collection, BASIC_DOCS, {"documents": [0, 0]}) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 1}, + {"_id": 2, "partition": "A", "value": 2, "result": 2}, + {"_id": 3, "partition": "A", "value": 4, "result": 4}, + {"_id": 4, "partition": "A", "value": 8, "result": 8}, + {"_id": 5, "partition": "A", "value": 16, "result": 16}, + ] + assertSuccess(result, expected, msg="window [0, 0] selects only current document") + + +# Property [Wider Than Partition]: window extends beyond partition edges, clamped + + +def test_wider_than_partition(collection): + """Window [-100, 100] extends far beyond partition — equivalent to unbounded.""" + result = _run_sum_window(collection, BASIC_DOCS, {"documents": [-100, 100]}) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 31}, + {"_id": 2, "partition": "A", "value": 2, "result": 31}, + {"_id": 3, "partition": "A", "value": 4, "result": 31}, + {"_id": 4, "partition": "A", "value": 8, "result": 31}, + {"_id": 5, "partition": "A", "value": 16, "result": 31}, + ] + assertSuccess(result, expected, msg="wider-than-partition clamped to edges") + + +# Property [Default Window]: no window key defaults to whole partition + + +def test_default_window_no_window_key(collection): + """No explicit window key defaults to unbounded-unbounded (whole partition).""" + collection.insert_many(BASIC_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": {"$sum": "$value"}, + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 31}, + {"_id": 2, "partition": "A", "value": 2, "result": 31}, + {"_id": 3, "partition": "A", "value": 4, "result": 31}, + {"_id": 4, "partition": "A", "value": 8, "result": 31}, + {"_id": 5, "partition": "A", "value": 16, "result": 31}, + ] + assertSuccess(result, expected, msg="no window key defaults to whole partition") + + +def test_empty_window_object(collection): + """Empty window object {} is equivalent to no window (whole partition).""" + collection.insert_many(BASIC_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 31}, + {"_id": 2, "partition": "A", "value": 2, "result": 31}, + {"_id": 3, "partition": "A", "value": 4, "result": 31}, + {"_id": 4, "partition": "A", "value": 8, "result": 31}, + {"_id": 5, "partition": "A", "value": 16, "result": 31}, + ] + assertSuccess(result, expected, msg="empty window {} = whole partition") + + +# Property [No SortBy]: documents window works without sortBy for unbounded frames + + +def test_no_sortby_with_unbounded_documents_window(collection): + """Documents window [unbounded, unbounded] works without sortBy.""" + collection.insert_many(BASIC_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 31}, + {"_id": 2, "partition": "A", "value": 2, "result": 31}, + {"_id": 3, "partition": "A", "value": 4, "result": 31}, + {"_id": 4, "partition": "A", "value": 8, "result": 31}, + {"_id": 5, "partition": "A", "value": 16, "result": 31}, + ] + assertSuccess( + result, expected, ignore_doc_order=True, msg="no sortBy with unbounded documents window" + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_documents_frame_errors.py b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_documents_frame_errors.py new file mode 100644 index 000000000..949511ee8 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_documents_frame_errors.py @@ -0,0 +1,575 @@ +""" +Tests for $setWindowFields documents-mode window frame validation errors. + +These are operator-agnostic — frame spec validation happens before the +accumulator runs. Uses $sum as the simplest accumulator. + +Covers: null window value, window as non-object types, unknown keys in window +object, document-bound validation errors (typo strings, wrong-length arrays, +null bounds, boolean bounds, non-object window, lower > upper, fractional +bounds), unknown accumulator keys, and specifying both documents and range. +""" + +from documentdb_tests.framework.assertions import assertFailureCode +from documentdb_tests.framework.error_codes import ( + FAILED_TO_PARSE_ERROR, + MISSING_FIELD_ERROR, + TYPE_MISMATCH_ERROR, + UNRECOGNIZED_COMMAND_FIELD_ERROR, +) +from documentdb_tests.framework.executor import execute_command + +SINGLE_DOC = [{"_id": 1, "partition": "A", "value": 10}] + +# Property [Window Type Validation]: window value must be an object + + +def test_window_null_value(collection): + """window: null produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": None, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="window: null rejected") + + +def test_window_array_value(collection): + """window as array produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": [-1, 0], + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="window as array rejected") + + +def test_window_not_object(collection): + """Window value that is a string produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": "invalid", + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="window not an object rejected") + + +def test_window_unknown_key(collection): + """Unknown key in window object produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [0, 1], "bogus": 123}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="unknown key in window object rejected") + + +# Property [Document Bounds Validation]: document bounds array must be well-formed + + +def test_documents_bounds_null_lower(collection): + """Null as lower document bound produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [None, 0]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="null lower bound rejected") + + +def test_documents_bounds_null_upper(collection): + """Null as upper document bound produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [0, None]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="null upper bound rejected") + + +def test_documents_bounds_empty_array(collection): + """Empty bounds array produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": []}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="empty bounds array rejected") + + +def test_documents_bounds_three_elements(collection): + """Three-element bounds array produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [-1, 0, 1]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="three-element bounds array rejected") + + +def test_documents_bounds_boolean(collection): + """Boolean as document bound produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [True, 0]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="boolean document bound rejected") + + +def test_bound_typo_string(collection): + """Invalid bound string (not 'current' or 'unbounded') produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["invalid", 0]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="invalid bound string rejected") + + +def test_bounds_wrong_length(collection): + """Bounds array with wrong length (1 element) produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="bounds array wrong length rejected") + + +def test_fractional_document_bound(collection): + """Fractional document bound produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [-1.5, 0]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="fractional document bound rejected") + + +# Property [Bound Ordering]: lower bound must not exceed upper bound + + +def test_lower_exceeds_upper(collection): + """Document bounds with lower > upper produces error 5339900.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [1, -1]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5339900, msg="documents lower > upper rejected") + + +# Property [Conflicting Modes]: cannot specify both documents and range + + +def test_both_documents_and_range(collection): + """Specifying both documents and range produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [-1, 0], "range": [-1, 0]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="both documents and range rejected") + + +# Property [Output Field Validation]: output field spec must be well-formed + + +def test_output_field_unknown_accumulator_key(collection): + """Unknown key alongside accumulator in output field produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [-1, 0]}, + "bogus": 1, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, FAILED_TO_PARSE_ERROR, msg="unknown key in output field spec rejected" + ) + + +# Property [Stage Document Validation]: $setWindowFields stage document must be well-formed + + +def test_stage_unknown_top_level_key(collection): + """Unknown top-level key in $setWindowFields stage document produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + "unknownKey": 1, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, UNRECOGNIZED_COMMAND_FIELD_ERROR, msg="unknown top-level key in stage rejected" + ) + + +def test_stage_output_omitted(collection): + """$setWindowFields without output field produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, MISSING_FIELD_ERROR, msg="output omitted rejected") + + +def test_stage_output_non_document(collection): + """$setWindowFields with non-document output produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": "invalid", + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, TYPE_MISMATCH_ERROR, msg="non-document output rejected") + + +# Property [SortBy Required For Bounded Windows]: bounded documents-mode windows require sortBy + + +def test_bounded_documents_window_without_sortby(collection): + """Bounded documents window [-1, 1] without sortBy produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [-1, 1]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5339901, msg="bounded documents window without sortBy rejected") + + +def test_cumulative_documents_window_without_sortby(collection): + """Cumulative documents window [unbounded, current] without sortBy produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "current"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5339901, msg="cumulative documents window without sortBy rejected") diff --git a/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_multi_output.py b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_multi_output.py new file mode 100644 index 000000000..a555ebae9 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_multi_output.py @@ -0,0 +1,228 @@ +""" +Tests for $setWindowFields multi-output behavior. + +These are operator-agnostic — they test the stage's ability to handle +multiple output fields with different accumulators and window specs. + +Covers: multiple different operators in one output clause, same operator +with different window specs, multiple operators with different window specs, +and output field that overwrites _id. +""" + +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + + +def test_mixed_operators_multi_output(collection): + """Multiple different operators ($sum + $avg + $count) in one output clause.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": 30}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "total": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + }, + "avg": { + "$avg": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + }, + "cnt": { + "$count": {}, + "window": {"documents": ["unbounded", "unbounded"]}, + }, + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "total": 60, "avg": 20.0, "cnt": 3}, + {"_id": 2, "partition": "A", "value": 20, "total": 60, "avg": 20.0, "cnt": 3}, + {"_id": 3, "partition": "A", "value": 30, "total": 60, "avg": 20.0, "cnt": 3}, + ] + assertSuccess(result, expected, msg="mixed operators in one output clause") + + +def test_same_operator_different_windows(collection): + """Same operator ($sum) with different window specs in one output clause.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": 30}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "cumulative": { + "$sum": "$value", + "window": {"documents": ["unbounded", "current"]}, + }, + "whole": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + }, + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "cumulative": 10, "whole": 60}, + {"_id": 2, "partition": "A", "value": 20, "cumulative": 30, "whole": 60}, + {"_id": 3, "partition": "A", "value": 30, "cumulative": 60, "whole": 60}, + ] + assertSuccess(result, expected, msg="same operator with different windows in one output") + + +def test_output_field_name_id(collection): + """Output field named '_id' overwrites original document _id.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": 30}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "_id": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 60, "partition": "A", "value": 10}, + {"_id": 60, "partition": "A", "value": 20}, + {"_id": 60, "partition": "A", "value": 30}, + ] + assertSuccess(result, expected, msg="output field _id overwrites document _id") + + +def test_mixed_operators_different_window_specs(collection): + """Multiple different operators each with a different window specification.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": 30}, + {"_id": 4, "partition": "A", "value": 40}, + {"_id": 5, "partition": "A", "value": 50}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "sum_sliding": { + "$sum": "$value", + "window": {"documents": [-1, 1]}, + }, + "avg_cumulative": { + "$avg": "$value", + "window": {"documents": ["unbounded", "current"]}, + }, + "max_whole": { + "$max": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + }, + }, + } + } + ], + "cursor": {}, + }, + ) + # $sum sliding [-1, 1]: row1=[10,20]=30, row2=[10,20,30]=60, row3=[20,30,40]=90, + # row4=[30,40,50]=120, row5=[40,50]=90 + # $avg cumulative [unbounded, current]: row1=10, row2=15, row3=20, row4=25, row5=30 + # $max whole [unbounded, unbounded]: always 50 + expected = [ + { + "_id": 1, + "partition": "A", + "value": 10, + "sum_sliding": 30, + "avg_cumulative": 10.0, + "max_whole": 50, + }, + { + "_id": 2, + "partition": "A", + "value": 20, + "sum_sliding": 60, + "avg_cumulative": 15.0, + "max_whole": 50, + }, + { + "_id": 3, + "partition": "A", + "value": 30, + "sum_sliding": 90, + "avg_cumulative": 20.0, + "max_whole": 50, + }, + { + "_id": 4, + "partition": "A", + "value": 40, + "sum_sliding": 120, + "avg_cumulative": 25.0, + "max_whole": 50, + }, + { + "_id": 5, + "partition": "A", + "value": 50, + "sum_sliding": 90, + "avg_cumulative": 30.0, + "max_whole": 50, + }, + ] + assertSuccess( + result, + expected, + msg="each operator independently respects its own window spec without cross-contamination", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_partitioning.py b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_partitioning.py new file mode 100644 index 000000000..12afb56a8 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_partitioning.py @@ -0,0 +1,259 @@ +""" +Tests for $setWindowFields partitioning behavior. + +Using $sum as a sample operator, verifies that the stage correctly isolates +computation within each partition when data spans multiple partitions. + +Covers: partition isolation, no partitionBy (whole collection), expression-based +partitioning, null/missing partition keys. + +Note: These stage-level tests verify the stage's partitioning mechanics. +Per-operator tests verify operator computation given a partition's documents. +""" + +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + +# Property [Partition Isolation]: frames never cross partition boundaries + + +def test_partition_isolation(collection): + """$sum frame never crosses partition boundary.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1}, + {"_id": 2, "partition": "A", "value": 2}, + {"_id": 3, "partition": "A", "value": 4}, + {"_id": 4, "partition": "B", "value": 10}, + {"_id": 5, "partition": "B", "value": 20}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 7}, + {"_id": 2, "partition": "A", "value": 2, "result": 7}, + {"_id": 3, "partition": "A", "value": 4, "result": 7}, + {"_id": 4, "partition": "B", "value": 10, "result": 30}, + {"_id": 5, "partition": "B", "value": 20, "result": 30}, + ] + assertSuccess(result, expected, msg="partitions A and B compute independently") + + +def test_sliding_window_respects_partition_boundary(collection): + """Sliding window at partition edge does not include docs from other partition.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1}, + {"_id": 2, "partition": "A", "value": 2}, + {"_id": 3, "partition": "B", "value": 100}, + {"_id": 4, "partition": "B", "value": 200}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [-1, 1]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # A: id=1 [1,2]=3; id=2 [1,2]=3 + # B: id=3 [100,200]=300; id=4 [100,200]=300 + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 3}, + {"_id": 2, "partition": "A", "value": 2, "result": 3}, + {"_id": 3, "partition": "B", "value": 100, "result": 300}, + {"_id": 4, "partition": "B", "value": 200, "result": 300}, + ] + assertSuccess(result, expected, msg="sliding window does not cross partition boundary") + + +# Property [No PartitionBy]: entire collection treated as one partition + + +def test_no_partition_by(collection): + """Omitting partitionBy treats entire collection as one partition.""" + docs = [ + {"_id": 1, "group": "A", "value": 1}, + {"_id": 2, "group": "B", "value": 2}, + {"_id": 3, "group": "A", "value": 4}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "group": "A", "value": 1, "result": 7}, + {"_id": 2, "group": "B", "value": 2, "result": 7}, + {"_id": 3, "group": "A", "value": 4, "result": 7}, + ] + assertSuccess(result, expected, msg="no partitionBy — whole collection is one partition") + + +# Property [Null Partition Key]: null and missing partition keys group together + + +def test_null_partition_key(collection): + """Documents with null partition key group into one partition.""" + docs = [ + {"_id": 1, "partition": None, "value": 1}, + {"_id": 2, "partition": None, "value": 2}, + {"_id": 3, "partition": "A", "value": 10}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": None, "value": 1, "result": 3}, + {"_id": 2, "partition": None, "value": 2, "result": 3}, + {"_id": 3, "partition": "A", "value": 10, "result": 10}, + ] + assertSuccess(result, expected, msg="null partition keys group together") + + +def test_missing_partition_key(collection): + """Documents with missing partition key group into one partition (same as null).""" + docs = [ + {"_id": 1, "value": 1}, + {"_id": 2, "value": 2}, + {"_id": 3, "partition": "A", "value": 10}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "value": 1, "result": 3}, + {"_id": 2, "value": 2, "result": 3}, + {"_id": 3, "partition": "A", "value": 10, "result": 10}, + ] + assertSuccess(result, expected, msg="missing partition key groups into null partition") + + +# Property [Expression PartitionBy]: expression-based partition key + + +def test_expression_partition_by(collection): + """Expression-based partitionBy groups documents correctly.""" + docs = [ + {"_id": 1, "category": "food", "sub": "fruit", "value": 1}, + {"_id": 2, "category": "food", "sub": "meat", "value": 2}, + {"_id": 3, "category": "drink", "sub": "water", "value": 4}, + {"_id": 4, "category": "drink", "sub": "juice", "value": 8}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$category", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 3, "category": "drink", "sub": "water", "value": 4, "result": 12}, + {"_id": 4, "category": "drink", "sub": "juice", "value": 8, "result": 12}, + {"_id": 1, "category": "food", "sub": "fruit", "value": 1, "result": 3}, + {"_id": 2, "category": "food", "sub": "meat", "value": 2, "result": 3}, + ] + assertSuccess( + result, expected, ignore_doc_order=True, msg="expression partitionBy groups correctly" + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_range_frame_boundaries.py b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_range_frame_boundaries.py new file mode 100644 index 000000000..3520b339a --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_range_frame_boundaries.py @@ -0,0 +1,198 @@ +""" +Tests for $setWindowFields range-mode frame boundary selection. + +Using $sum as a sample operator, verifies that numeric range-based frame bounds +correctly define the window of documents selected for computation. + +Covers: symmetric range, asymmetric range, unbounded/current combinations, +value-ties inclusion, gaps in sortBy values, and empty range frames. + +Note: These stage-level tests verify correct document selection by range value. +Per-operator tests (under window/$operator/) verify the operator computes +correct results given those documents. +""" + +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + + +def _run_sum_range_window(collection, docs, window, sort_by=None): + """Helper to run $sum with a range window spec.""" + if sort_by is None: + sort_by = {"score": 1} + collection.insert_many(docs) + stage = { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": sort_by, + "output": { + "result": { + "$sum": "$value", + "window": window, + } + }, + } + } + return execute_command( + collection, + {"aggregate": collection.name, "pipeline": [stage], "cursor": {}}, + ) + + +# Property [Symmetric Range]: range [-N, N] includes documents within N of current sortBy value + + +def test_range_symmetric_sliding(collection): + """Range [-5, 5] includes documents whose sortBy value is within 5 of current.""" + docs = [ + {"_id": 1, "partition": "A", "score": 0, "value": 1}, + {"_id": 2, "partition": "A", "score": 3, "value": 2}, + {"_id": 3, "partition": "A", "score": 10, "value": 4}, + {"_id": 4, "partition": "A", "score": 12, "value": 8}, + {"_id": 5, "partition": "A", "score": 20, "value": 16}, + ] + result = _run_sum_range_window(collection, docs, {"range": [-5, 5]}) + # score=0: range [-5, 5] -> scores in [-5, 5] -> includes 0, 3 -> 1+2 = 3 + # score=3: range [-2, 8] -> scores in [-2, 8] -> includes 0, 3 -> 1+2 = 3 + # score=10: range [5, 15] -> scores in [5, 15] -> includes 10, 12 -> 4+8 = 12 + # score=12: range [7, 17] -> scores in [7, 17] -> includes 10, 12 -> 4+8 = 12 + # score=20: range [15, 25] -> scores in [15, 25] -> includes 20 -> 16 + expected = [ + {"_id": 1, "partition": "A", "score": 0, "value": 1, "result": 3}, + {"_id": 2, "partition": "A", "score": 3, "value": 2, "result": 3}, + {"_id": 3, "partition": "A", "score": 10, "value": 4, "result": 12}, + {"_id": 4, "partition": "A", "score": 12, "value": 8, "result": 12}, + {"_id": 5, "partition": "A", "score": 20, "value": 16, "result": 16}, + ] + assertSuccess(result, expected, msg="symmetric range [-5, 5] selects correct documents") + + +# Property [Unbounded Range]: unbounded combined with numeric bound + + +def test_range_unbounded_to_current(collection): + """Range [unbounded, current] includes all documents up to current sortBy value.""" + docs = [ + {"_id": 1, "partition": "A", "score": 1, "value": 1}, + {"_id": 2, "partition": "A", "score": 2, "value": 2}, + {"_id": 3, "partition": "A", "score": 3, "value": 4}, + {"_id": 4, "partition": "A", "score": 4, "value": 8}, + ] + result = _run_sum_range_window(collection, docs, {"range": ["unbounded", "current"]}) + expected = [ + {"_id": 1, "partition": "A", "score": 1, "value": 1, "result": 1}, + {"_id": 2, "partition": "A", "score": 2, "value": 2, "result": 3}, + {"_id": 3, "partition": "A", "score": 3, "value": 4, "result": 7}, + {"_id": 4, "partition": "A", "score": 4, "value": 8, "result": 15}, + ] + assertSuccess(result, expected, msg="range [unbounded, current] cumulative selection") + + +def test_range_current_to_unbounded(collection): + """Range [current, unbounded] includes current and all following documents.""" + docs = [ + {"_id": 1, "partition": "A", "score": 1, "value": 1}, + {"_id": 2, "partition": "A", "score": 2, "value": 2}, + {"_id": 3, "partition": "A", "score": 3, "value": 4}, + {"_id": 4, "partition": "A", "score": 4, "value": 8}, + ] + result = _run_sum_range_window(collection, docs, {"range": ["current", "unbounded"]}) + expected = [ + {"_id": 1, "partition": "A", "score": 1, "value": 1, "result": 15}, + {"_id": 2, "partition": "A", "score": 2, "value": 2, "result": 14}, + {"_id": 3, "partition": "A", "score": 3, "value": 4, "result": 12}, + {"_id": 4, "partition": "A", "score": 4, "value": 8, "result": 8}, + ] + assertSuccess(result, expected, msg="range [current, unbounded] reverse-cumulative selection") + + +# Property [Value-Ties]: range [0, 0] includes all docs with identical sortBy value + + +def test_range_zero_zero_includes_ties(collection): + """Range [0, 0] includes all documents sharing the same sortBy value.""" + docs = [ + {"_id": 1, "partition": "A", "score": 5, "value": 1}, + {"_id": 2, "partition": "A", "score": 5, "value": 2}, + {"_id": 3, "partition": "A", "score": 5, "value": 4}, + {"_id": 4, "partition": "A", "score": 10, "value": 8}, + ] + result = _run_sum_range_window(collection, docs, {"range": [0, 0]}) + # score=5 ties: all three included -> 1+2+4 = 7 + # score=10: only itself -> 8 + expected = [ + {"_id": 1, "partition": "A", "score": 5, "value": 1, "result": 7}, + {"_id": 2, "partition": "A", "score": 5, "value": 2, "result": 7}, + {"_id": 3, "partition": "A", "score": 5, "value": 4, "result": 7}, + {"_id": 4, "partition": "A", "score": 10, "value": 8, "result": 8}, + ] + assertSuccess(result, expected, msg="range [0, 0] includes all value-ties") + + +# Property [Gaps in SortBy]: documents with large gaps in sortBy values are excluded + + +def test_range_with_gaps(collection): + """Range window correctly excludes documents with large sortBy gaps.""" + docs = [ + {"_id": 1, "partition": "A", "score": 1, "value": 1}, + {"_id": 2, "partition": "A", "score": 2, "value": 2}, + {"_id": 3, "partition": "A", "score": 100, "value": 4}, + {"_id": 4, "partition": "A", "score": 101, "value": 8}, + ] + result = _run_sum_range_window(collection, docs, {"range": [-5, 5]}) + # score=1: range [-4, 6] -> includes 1, 2 -> 1+2 = 3 + # score=2: range [-3, 7] -> includes 1, 2 -> 1+2 = 3 + # score=100: range [95, 105] -> includes 100, 101 -> 4+8 = 12 + # score=101: range [96, 106] -> includes 100, 101 -> 4+8 = 12 + expected = [ + {"_id": 1, "partition": "A", "score": 1, "value": 1, "result": 3}, + {"_id": 2, "partition": "A", "score": 2, "value": 2, "result": 3}, + {"_id": 3, "partition": "A", "score": 100, "value": 4, "result": 12}, + {"_id": 4, "partition": "A", "score": 101, "value": 8, "result": 12}, + ] + assertSuccess(result, expected, msg="range excludes documents with large sortBy gaps") + + +# Property [Empty Range]: range that includes no documents + + +def test_range_empty_returns_zero(collection): + """Range window selecting no documents returns 0 for $sum.""" + docs = [ + {"_id": 1, "partition": "A", "score": 1, "value": 10}, + {"_id": 2, "partition": "A", "score": 100, "value": 20}, + ] + result = _run_sum_range_window(collection, docs, {"range": [-2, -1]}) + # score=1: range [-1, 0] -> no doc with score in [-1, 0] -> 0 + # score=100: range [98, 99] -> no doc with score in [98, 99] -> 0 + expected = [ + {"_id": 1, "partition": "A", "score": 1, "value": 10, "result": 0}, + {"_id": 2, "partition": "A", "score": 100, "value": 20, "result": 0}, + ] + assertSuccess(result, expected, msg="empty range frame returns 0 for $sum") + + +# Property [Fractional Range]: non-integer range bounds + + +def test_range_fractional_bounds(collection): + """Range with fractional bounds correctly includes documents.""" + docs = [ + {"_id": 1, "partition": "A", "score": 1.0, "value": 1}, + {"_id": 2, "partition": "A", "score": 1.5, "value": 2}, + {"_id": 3, "partition": "A", "score": 2.0, "value": 4}, + {"_id": 4, "partition": "A", "score": 3.0, "value": 8}, + ] + result = _run_sum_range_window(collection, docs, {"range": [-0.5, 0.5]}) + # score=1.0: range [0.5, 1.5] -> includes 1.0, 1.5 -> 1+2 = 3 + # score=1.5: range [1.0, 2.0] -> includes 1.0, 1.5, 2.0 -> 1+2+4 = 7 + # score=2.0: range [1.5, 2.5] -> includes 1.5, 2.0 -> 2+4 = 6 + # score=3.0: range [2.5, 3.5] -> includes 3.0 -> 8 + expected = [ + {"_id": 1, "partition": "A", "score": 1.0, "value": 1, "result": 3}, + {"_id": 2, "partition": "A", "score": 1.5, "value": 2, "result": 7}, + {"_id": 3, "partition": "A", "score": 2.0, "value": 4, "result": 6}, + {"_id": 4, "partition": "A", "score": 3.0, "value": 8, "result": 8}, + ] + assertSuccess(result, expected, msg="fractional range bounds select correct documents") diff --git a/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_range_frame_errors.py b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_range_frame_errors.py new file mode 100644 index 000000000..f79536ef1 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_range_frame_errors.py @@ -0,0 +1,365 @@ +""" +Tests for $setWindowFields range-mode window frame validation errors. + +These are operator-agnostic — frame spec validation happens before the +accumulator runs. Uses $sum as the simplest accumulator. + +Covers: invalid bound types (non-numeric string, boolean, null), wrong-length +bounds arrays, null/missing sortBy values, no sortBy, descending sort, +multiple sortBy fields, lower > upper, and non-numeric sortBy values. +""" + +from documentdb_tests.framework.assertions import assertFailureCode +from documentdb_tests.framework.error_codes import FAILED_TO_PARSE_ERROR +from documentdb_tests.framework.executor import execute_command + +SINGLE_DOC = [{"_id": 1, "partition": "A", "score": 1, "value": 10}] + +# Property [Range Bound Validation]: range bounds must be numeric or keyword + + +def test_range_non_numeric_bound(collection): + """Range bound that is non-numeric (string, not 'unbounded'/'current') produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": ["invalid", 0]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, FAILED_TO_PARSE_ERROR, msg="non-numeric, non-keyword range bound rejected" + ) + + +def test_range_boolean_bound(collection): + """Range bound that is a boolean produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [True, 10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="boolean range bound rejected") + + +def test_range_null_bound(collection): + """Range bound that is null produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [None, 10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="null range bound rejected") + + +def test_range_empty_bounds_array(collection): + """Range with empty bounds array produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": []}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="empty range bounds array rejected") + + +def test_range_three_element_bounds(collection): + """Range with three-element bounds array produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-10, 0, 10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="three-element range bounds rejected") + + +# Property [SortBy Value Validation]: sortBy field values must be numeric + + +def test_range_null_sortby_value(collection): + """Range mode with null sortBy value produces error 5429414.""" + docs = [ + {"_id": 1, "partition": "A", "score": None, "value": 10}, + {"_id": 2, "partition": "A", "score": 5, "value": 20}, + {"_id": 3, "partition": "A", "score": 10, "value": 30}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-10, 10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5429414, msg="null sortBy value in range mode rejected") + + +def test_range_missing_sortby_field(collection): + """Range mode with missing sortBy field produces error 5429414.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "score": 5, "value": 20}, + {"_id": 3, "partition": "A", "score": 10, "value": 30}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-10, 10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5429414, msg="missing sortBy field in range mode rejected") + + +def test_range_non_numeric_sortby_value(collection): + """Range mode with non-numeric sortBy value produces error 5429414.""" + docs = [ + {"_id": 1, "partition": "A", "score": "abc", "value": 10}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-10, 10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5429414, msg="non-numeric sortBy value rejected in range mode") + + +# Property [SortBy Requirement]: range mode requires single ascending sortBy + + +def test_range_no_sortby(collection): + """Range window without sortBy produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-10, 10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5339902, msg="range window without sortBy rejected") + + +def test_range_descending_sort(collection): + """Range mode with descending sort produces error 8947401.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": -1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-10, 10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 8947401, msg="descending sort rejected in range mode") + + +def test_range_multiple_sortby(collection): + """Range mode with multiple sortBy fields produces error 5339902.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1, "_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-10, 10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5339902, msg="multiple sortBy rejected in range mode") + + +# Property [Bound Ordering]: lower bound must not exceed upper bound + + +def test_range_lower_exceeds_upper(collection): + """Range bounds with lower > upper produces error 5339900.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [10, -10]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5339900, msg="lower > upper rejected in range mode") diff --git a/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_sort_interaction.py b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_sort_interaction.py new file mode 100644 index 000000000..4f4c10825 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_sort_interaction.py @@ -0,0 +1,231 @@ +""" +Tests for $setWindowFields sort interaction behavior. + +Using $sum as a sample operator, verifies that the stage respects the specified +sort order and that changing sort order produces correspondingly different +results for order-dependent window computations. + +Covers: ascending vs descending sort with cumulative frames, compound sort, +sort with ties and tiebreaker, and sort on non-_id field. + +Note: These stage-level tests verify the stage's sort mechanics. +Per-operator tests verify operator-specific sort interactions. +""" + +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + +# Property [Ascending vs Descending]: sort direction changes cumulative frame results + + +def test_ascending_sort_cumulative(collection): + """Ascending sort with cumulative window accumulates in _id order.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1}, + {"_id": 2, "partition": "A", "value": 2}, + {"_id": 3, "partition": "A", "value": 4}, + {"_id": 4, "partition": "A", "value": 8}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "current"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 1}, + {"_id": 2, "partition": "A", "value": 2, "result": 3}, + {"_id": 3, "partition": "A", "value": 4, "result": 7}, + {"_id": 4, "partition": "A", "value": 8, "result": 15}, + ] + assertSuccess(result, expected, msg="ascending sort cumulative [unbounded, current]") + + +def test_descending_sort_cumulative(collection): + """Descending sort with cumulative window accumulates in reverse _id order.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1}, + {"_id": 2, "partition": "A", "value": 2}, + {"_id": 3, "partition": "A", "value": 4}, + {"_id": 4, "partition": "A", "value": 8}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": -1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "current"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Descending: id=4, 3, 2, 1 -> cumulative: 8, 12, 14, 15 + expected = [ + {"_id": 4, "partition": "A", "value": 8, "result": 8}, + {"_id": 3, "partition": "A", "value": 4, "result": 12}, + {"_id": 2, "partition": "A", "value": 2, "result": 14}, + {"_id": 1, "partition": "A", "value": 1, "result": 15}, + ] + assertSuccess(result, expected, msg="descending sort cumulative [unbounded, current]") + + +# Property [Compound Sort]: multi-key sort determines row ordering + + +def test_compound_sort(collection): + """Compound sort {a: 1, b: -1} correctly determines row ordering.""" + docs = [ + {"_id": 1, "partition": "A", "a": 1, "b": 2, "value": 1}, + {"_id": 2, "partition": "A", "a": 1, "b": 1, "value": 2}, + {"_id": 3, "partition": "A", "a": 2, "b": 2, "value": 4}, + {"_id": 4, "partition": "A", "a": 2, "b": 1, "value": 8}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"a": 1, "b": -1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "current"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Sort: a=1,b=2(id=1) -> a=1,b=1(id=2) -> a=2,b=2(id=3) -> a=2,b=1(id=4) + # Cumulative: 1, 3, 7, 15 + expected = [ + {"_id": 1, "partition": "A", "a": 1, "b": 2, "value": 1, "result": 1}, + {"_id": 2, "partition": "A", "a": 1, "b": 1, "value": 2, "result": 3}, + {"_id": 3, "partition": "A", "a": 2, "b": 2, "value": 4, "result": 7}, + {"_id": 4, "partition": "A", "a": 2, "b": 1, "value": 8, "result": 15}, + ] + assertSuccess(result, expected, msg="compound sort {a:1, b:-1} determines ordering") + + +# Property [Sort on Non-_id Field]: sortBy on arbitrary field + + +def test_sort_on_non_id_field(collection): + """SortBy on a non-_id field correctly determines row ordering.""" + docs = [ + {"_id": 3, "partition": "A", "ts": 100, "value": 4}, + {"_id": 1, "partition": "A", "ts": 200, "value": 1}, + {"_id": 2, "partition": "A", "ts": 300, "value": 2}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"ts": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": ["unbounded", "current"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Sort by ts: id=3(ts=100), id=1(ts=200), id=2(ts=300) + # Cumulative: 4, 5, 7 + expected = [ + {"_id": 3, "partition": "A", "ts": 100, "value": 4, "result": 4}, + {"_id": 1, "partition": "A", "ts": 200, "value": 1, "result": 5}, + {"_id": 2, "partition": "A", "ts": 300, "value": 2, "result": 7}, + ] + assertSuccess(result, expected, msg="sortBy on non-_id field (ts)") + + +# Property [Sort With Ties]: tied sortBy values with compound tiebreaker + + +def test_sort_with_ties_and_tiebreaker(collection): + """Ties in sortBy resolved by compound sort tiebreaker.""" + docs = [ + {"_id": 1, "partition": "A", "score": 10, "value": 1}, + {"_id": 2, "partition": "A", "score": 10, "value": 2}, + {"_id": 3, "partition": "A", "score": 10, "value": 4}, + {"_id": 4, "partition": "A", "score": 20, "value": 8}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1, "_id": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [-1, 0]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Sort: score=10,id=1 -> score=10,id=2 -> score=10,id=3 -> score=20,id=4 + # Trailing [-1, 0]: + # Row 1: [1] = 1 + # Row 2: [1, 2] = 3 + # Row 3: [2, 4] = 6 + # Row 4: [4, 8] = 12 + expected = [ + {"_id": 1, "partition": "A", "score": 10, "value": 1, "result": 1}, + {"_id": 2, "partition": "A", "score": 10, "value": 2, "result": 3}, + {"_id": 3, "partition": "A", "score": 10, "value": 4, "result": 6}, + {"_id": 4, "partition": "A", "score": 20, "value": 8, "result": 12}, + ] + assertSuccess(result, expected, msg="ties resolved by compound sort tiebreaker") diff --git a/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_time_range_frame_boundaries.py b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_time_range_frame_boundaries.py new file mode 100644 index 000000000..40445f9ae --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_time_range_frame_boundaries.py @@ -0,0 +1,660 @@ +""" +Tests for $setWindowFields time-range-mode frame boundary selection. + +Using $sum as a sample operator, verifies that time-based range frame bounds +correctly define the window of documents selected for computation. + +Covers: day unit sliding, hour unit sliding, unbounded/current with time unit, +gaps in dates excluding documents, and multiple time units. + +Note: These stage-level tests verify correct document selection by time range. +Per-operator tests (under window/$operator/) verify the operator computes +correct results given those documents. +""" + +from datetime import datetime, timezone + +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + + +def _run_sum_time_range_window(collection, docs, window, sort_by=None): + """Helper to run $sum with a time-range window spec.""" + if sort_by is None: + sort_by = {"date": 1} + collection.insert_many(docs) + stage = { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": sort_by, + "output": { + "result": { + "$sum": "$value", + "window": window, + } + }, + } + } + return execute_command( + collection, + {"aggregate": collection.name, "pipeline": [stage], "cursor": {}}, + ) + + +# Property [Day Unit]: time-range with unit=day selects documents within N days + + +def test_time_range_day_sliding(collection): + """Time-range [-1, 1] unit=day includes documents within 1 day of current.""" + docs = [ + {"_id": 1, "partition": "A", "date": datetime(2023, 1, 1, tzinfo=timezone.utc), "value": 1}, + {"_id": 2, "partition": "A", "date": datetime(2023, 1, 2, tzinfo=timezone.utc), "value": 2}, + {"_id": 3, "partition": "A", "date": datetime(2023, 1, 3, tzinfo=timezone.utc), "value": 4}, + {"_id": 4, "partition": "A", "date": datetime(2023, 1, 5, tzinfo=timezone.utc), "value": 8}, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-1, 1], "unit": "day"}) + # Jan 1: range [Dec 31, Jan 2] -> includes Jan 1, Jan 2 -> 1+2 = 3 + # Jan 2: range [Jan 1, Jan 3] -> includes Jan 1, Jan 2, Jan 3 -> 1+2+4 = 7 + # Jan 3: range [Jan 2, Jan 4] -> includes Jan 2, Jan 3 -> 2+4 = 6 + # Jan 5: range [Jan 4, Jan 6] -> includes Jan 5 only -> 8 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 2, tzinfo=timezone.utc), + "value": 2, + "result": 7, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 3, tzinfo=timezone.utc), + "value": 4, + "result": 6, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2023, 1, 5, tzinfo=timezone.utc), + "value": 8, + "result": 8, + }, + ] + assertSuccess(result, expected, msg="time-range day unit [-1, 1] selects correct documents") + + +# Property [Hour Unit]: time-range with unit=hour selects documents within N hours + + +def test_time_range_hour_sliding(collection): + """Time-range [-2, 2] unit=hour includes documents within 2 hours of current.""" + docs = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, tzinfo=timezone.utc), + "value": 1, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 1, 1, 0, tzinfo=timezone.utc), + "value": 2, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 1, 3, 0, tzinfo=timezone.utc), + "value": 4, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2023, 1, 1, 6, 0, tzinfo=timezone.utc), + "value": 8, + }, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-2, 2], "unit": "hour"}) + # 00:00: range [22:00 prev, 02:00] -> includes 00:00, 01:00 -> 1+2 = 3 + # 01:00: range [23:00 prev, 03:00] -> includes 00:00, 01:00, 03:00 -> 1+2+4 = 7 + # 03:00: range [01:00, 05:00] -> includes 01:00, 03:00 -> 2+4 = 6 + # 06:00: range [04:00, 08:00] -> includes 06:00 only -> 8 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 1, 1, 0, tzinfo=timezone.utc), + "value": 2, + "result": 7, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 1, 3, 0, tzinfo=timezone.utc), + "value": 4, + "result": 6, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2023, 1, 1, 6, 0, tzinfo=timezone.utc), + "value": 8, + "result": 8, + }, + ] + assertSuccess(result, expected, msg="time-range hour unit [-2, 2] selects correct documents") + + +# Property [Unbounded Time Range]: unbounded combined with time unit + + +def test_time_range_unbounded_to_current(collection): + """Time-range [unbounded, current] with unit includes all preceding documents.""" + docs = [ + {"_id": 1, "partition": "A", "date": datetime(2023, 1, 1, tzinfo=timezone.utc), "value": 1}, + {"_id": 2, "partition": "A", "date": datetime(2023, 1, 2, tzinfo=timezone.utc), "value": 2}, + {"_id": 3, "partition": "A", "date": datetime(2023, 1, 3, tzinfo=timezone.utc), "value": 4}, + ] + result = _run_sum_time_range_window( + collection, docs, {"range": ["unbounded", "current"], "unit": "day"} + ) + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, tzinfo=timezone.utc), + "value": 1, + "result": 1, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 2, tzinfo=timezone.utc), + "value": 2, + "result": 3, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 3, tzinfo=timezone.utc), + "value": 4, + "result": 7, + }, + ] + assertSuccess(result, expected, msg="time-range [unbounded, current] cumulative selection") + + +# Property [Gap Exclusion]: documents far apart in time are excluded from frame + + +def test_time_range_gap_exclusion(collection): + """Time-range window excludes documents separated by large time gaps.""" + docs = [ + {"_id": 1, "partition": "A", "date": datetime(2023, 1, 1, tzinfo=timezone.utc), "value": 1}, + {"_id": 2, "partition": "A", "date": datetime(2023, 1, 2, tzinfo=timezone.utc), "value": 2}, + {"_id": 3, "partition": "A", "date": datetime(2023, 6, 1, tzinfo=timezone.utc), "value": 4}, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-7, 7], "unit": "day"}) + # Jan 1: range [Dec 25, Jan 8] -> includes Jan 1, Jan 2 -> 1+2 = 3 + # Jan 2: range [Dec 26, Jan 9] -> includes Jan 1, Jan 2 -> 1+2 = 3 + # Jun 1: range [May 25, Jun 8] -> includes Jun 1 only -> 4 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 2, tzinfo=timezone.utc), + "value": 2, + "result": 3, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 6, 1, tzinfo=timezone.utc), + "value": 4, + "result": 4, + }, + ] + assertSuccess(result, expected, msg="time-range excludes documents with large time gaps") + + +# Property [Month Unit - Variable Length]: month unit handles variable-length months correctly + + +def test_time_range_month_sliding(collection): + """Time-range [-1, 1] unit=month includes documents within 1 month of current.""" + docs = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 15, tzinfo=timezone.utc), + "value": 1, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 2, 15, tzinfo=timezone.utc), + "value": 2, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 3, 15, tzinfo=timezone.utc), + "value": 4, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2023, 5, 15, tzinfo=timezone.utc), + "value": 8, + }, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-1, 1], "unit": "month"}) + # Jan 15: range [Dec 15, Feb 15] -> includes Jan 15, Feb 15 -> 1+2 = 3 + # Feb 15: range [Jan 15, Mar 15] -> includes Jan 15, Feb 15, Mar 15 -> 1+2+4 = 7 + # Mar 15: range [Feb 15, Apr 15] -> includes Feb 15, Mar 15 -> 2+4 = 6 + # May 15: range [Apr 15, Jun 15] -> includes May 15 only -> 8 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 15, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 2, 15, tzinfo=timezone.utc), + "value": 2, + "result": 7, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 3, 15, tzinfo=timezone.utc), + "value": 4, + "result": 6, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2023, 5, 15, tzinfo=timezone.utc), + "value": 8, + "result": 8, + }, + ] + assertSuccess(result, expected, msg="time-range month unit [-1, 1] selects correct documents") + + +# Property [Minute Unit]: time-range with unit=minute selects documents within N minutes + + +def test_time_range_minute_sliding(collection): + """Time-range [-1, 1] unit=minute includes documents within 1 minute of current.""" + docs = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, tzinfo=timezone.utc), + "value": 1, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 1, 0, tzinfo=timezone.utc), + "value": 2, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 3, 0, tzinfo=timezone.utc), + "value": 4, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 5, 0, tzinfo=timezone.utc), + "value": 8, + }, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-1, 1], "unit": "minute"}) + # 00:00: range [23:59 prev, 00:01] -> includes 00:00, 00:01 -> 1+2 = 3 + # 00:01: range [00:00, 00:02] -> includes 00:00, 00:01 -> 1+2 = 3 + # 00:03: range [00:02, 00:04] -> includes 00:03 only -> 4 + # 00:05: range [00:04, 00:06] -> includes 00:05 only -> 8 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 1, 0, tzinfo=timezone.utc), + "value": 2, + "result": 3, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 3, 0, tzinfo=timezone.utc), + "value": 4, + "result": 4, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 5, 0, tzinfo=timezone.utc), + "value": 8, + "result": 8, + }, + ] + assertSuccess(result, expected, msg="time-range minute unit [-1, 1] selects correct documents") + + +# Property [Second Unit]: time-range with unit=second selects documents within N seconds + + +def test_time_range_second_sliding(collection): + """Time-range [-1, 1] unit=second includes documents within 1 second of current.""" + docs = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, tzinfo=timezone.utc), + "value": 1, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 1, tzinfo=timezone.utc), + "value": 2, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 3, tzinfo=timezone.utc), + "value": 4, + }, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-1, 1], "unit": "second"}) + # 00:00:00: range [-1s, +1s] -> includes 00:00:00, 00:00:01 -> 1+2 = 3 + # 00:00:01: range [0s, +2s] -> includes 00:00:00, 00:00:01 -> 1+2 = 3 + # 00:00:03: range [+2s, +4s] -> includes 00:00:03 only -> 4 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 1, tzinfo=timezone.utc), + "value": 2, + "result": 3, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 3, tzinfo=timezone.utc), + "value": 4, + "result": 4, + }, + ] + assertSuccess(result, expected, msg="time-range second unit [-1, 1] selects correct documents") + + +# Property [Millisecond Unit]: time-range with unit=millisecond selects documents within N ms + + +def test_time_range_millisecond_sliding(collection): + """Time-range [-1, 1] unit=millisecond includes documents within 1ms of current.""" + docs = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, 0, tzinfo=timezone.utc), + "value": 1, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, 1000, tzinfo=timezone.utc), + "value": 2, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, 3000, tzinfo=timezone.utc), + "value": 4, + }, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-1, 1], "unit": "millisecond"}) + # +0ms: range [-1ms, +1ms] -> includes +0ms, +1ms -> 1+2 = 3 + # +1ms: range [+0ms, +2ms] -> includes +0ms, +1ms -> 1+2 = 3 + # +3ms: range [+2ms, +4ms] -> includes +3ms only -> 4 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, 0, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, 1000, tzinfo=timezone.utc), + "value": 2, + "result": 3, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 1, 0, 0, 0, 3000, tzinfo=timezone.utc), + "value": 4, + "result": 4, + }, + ] + assertSuccess( + result, expected, msg="time-range millisecond unit [-1, 1] selects correct documents" + ) + + +# Property [Week Unit]: time-range with unit=week selects documents within N weeks + + +def test_time_range_week_sliding(collection): + """Time-range [-1, 1] unit=week includes documents within 1 week of current.""" + docs = [ + {"_id": 1, "partition": "A", "date": datetime(2023, 1, 1, tzinfo=timezone.utc), "value": 1}, + {"_id": 2, "partition": "A", "date": datetime(2023, 1, 8, tzinfo=timezone.utc), "value": 2}, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 15, tzinfo=timezone.utc), + "value": 4, + }, + {"_id": 4, "partition": "A", "date": datetime(2023, 2, 1, tzinfo=timezone.utc), "value": 8}, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-1, 1], "unit": "week"}) + # Jan 1: range [Dec 25, Jan 8] -> includes Jan 1, Jan 8 -> 1+2 = 3 + # Jan 8: range [Jan 1, Jan 15] -> includes Jan 1, Jan 8, Jan 15 -> 1+2+4 = 7 + # Jan 15: range [Jan 8, Jan 22] -> includes Jan 8, Jan 15 -> 2+4 = 6 + # Feb 1: range [Jan 25, Feb 8] -> includes Feb 1 only -> 8 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 1, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 1, 8, tzinfo=timezone.utc), + "value": 2, + "result": 7, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 1, 15, tzinfo=timezone.utc), + "value": 4, + "result": 6, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2023, 2, 1, tzinfo=timezone.utc), + "value": 8, + "result": 8, + }, + ] + assertSuccess(result, expected, msg="time-range week unit [-1, 1] selects correct documents") + + +# Property [Quarter Unit - Variable Length]: quarter unit handles variable-length quarters + + +def test_time_range_quarter_sliding(collection): + """Time-range [-1, 1] unit=quarter includes documents within 1 quarter of current.""" + docs = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 15, tzinfo=timezone.utc), + "value": 1, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 4, 15, tzinfo=timezone.utc), + "value": 2, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 7, 15, tzinfo=timezone.utc), + "value": 4, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2024, 1, 15, tzinfo=timezone.utc), + "value": 8, + }, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-1, 1], "unit": "quarter"}) + # Jan 15 2023: range [Oct 15 2022, Apr 15 2023] -> includes Jan 15, Apr 15 -> 1+2 = 3 + # Apr 15 2023: range [Jan 15, Jul 15] -> includes Jan 15, Apr 15, Jul 15 -> 1+2+4 = 7 + # Jul 15 2023: range [Apr 15, Oct 15] -> includes Apr 15, Jul 15 -> 2+4 = 6 + # Jan 15 2024: range [Oct 15 2023, Apr 15 2024] -> includes Jan 15 2024 only -> 8 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2023, 1, 15, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 4, 15, tzinfo=timezone.utc), + "value": 2, + "result": 7, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2023, 7, 15, tzinfo=timezone.utc), + "value": 4, + "result": 6, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2024, 1, 15, tzinfo=timezone.utc), + "value": 8, + "result": 8, + }, + ] + assertSuccess(result, expected, msg="time-range quarter unit [-1, 1] selects correct documents") + + +# Property [Year Unit - Variable Length]: year unit handles leap years correctly + + +def test_time_range_year_sliding(collection): + """Time-range [-1, 1] unit=year includes documents within 1 year of current.""" + docs = [ + {"_id": 1, "partition": "A", "date": datetime(2022, 6, 1, tzinfo=timezone.utc), "value": 1}, + {"_id": 2, "partition": "A", "date": datetime(2023, 6, 1, tzinfo=timezone.utc), "value": 2}, + {"_id": 3, "partition": "A", "date": datetime(2024, 6, 1, tzinfo=timezone.utc), "value": 4}, + {"_id": 4, "partition": "A", "date": datetime(2026, 6, 1, tzinfo=timezone.utc), "value": 8}, + ] + result = _run_sum_time_range_window(collection, docs, {"range": [-1, 1], "unit": "year"}) + # Jun 2022: range [Jun 2021, Jun 2023] -> includes 2022, 2023 -> 1+2 = 3 + # Jun 2023: range [Jun 2022, Jun 2024] -> includes 2022, 2023, 2024 -> 1+2+4 = 7 + # Jun 2024: range [Jun 2023, Jun 2025] -> includes 2023, 2024 -> 2+4 = 6 + # Jun 2026: range [Jun 2025, Jun 2027] -> includes 2026 only -> 8 + expected = [ + { + "_id": 1, + "partition": "A", + "date": datetime(2022, 6, 1, tzinfo=timezone.utc), + "value": 1, + "result": 3, + }, + { + "_id": 2, + "partition": "A", + "date": datetime(2023, 6, 1, tzinfo=timezone.utc), + "value": 2, + "result": 7, + }, + { + "_id": 3, + "partition": "A", + "date": datetime(2024, 6, 1, tzinfo=timezone.utc), + "value": 4, + "result": 6, + }, + { + "_id": 4, + "partition": "A", + "date": datetime(2026, 6, 1, tzinfo=timezone.utc), + "value": 8, + "result": 8, + }, + ] + assertSuccess(result, expected, msg="time-range year unit [-1, 1] selects correct documents") diff --git a/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_time_range_frame_errors.py b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_time_range_frame_errors.py new file mode 100644 index 000000000..5703e0832 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/stages/setWindowFields/test_setWindowFields_time_range_frame_errors.py @@ -0,0 +1,278 @@ +""" +Tests for $setWindowFields time-range mode frame validation errors. + +These are operator-agnostic — frame spec validation happens before the +accumulator runs. Uses $sum as the simplest accumulator. + +Covers: invalid time unit string, non-string unit types (integer, boolean, null), +unit with documents mode, time unit with non-date sortBy value, boolean bounds +with unit, null bounds with unit, lower > upper with unit. +""" + +from datetime import datetime, timezone + +from documentdb_tests.framework.assertions import assertFailureCode +from documentdb_tests.framework.error_codes import FAILED_TO_PARSE_ERROR +from documentdb_tests.framework.executor import execute_command + +SINGLE_DATE_DOC = [ + {"_id": 1, "partition": "A", "date": datetime(2023, 1, 1, tzinfo=timezone.utc), "value": 10} +] + +# Property [Time Unit Validation]: time unit must be a valid string + + +def test_time_range_invalid_unit(collection): + """Range with invalid time unit produces error.""" + collection.insert_many(SINGLE_DATE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"date": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-1, 0], "unit": "invalid_unit"}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="invalid time unit rejected") + + +def test_time_range_unit_integer_type(collection): + """Range with integer unit type produces error — unit must be a string.""" + collection.insert_many(SINGLE_DATE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"date": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-1, 0], "unit": 123}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="integer unit type rejected") + + +def test_time_range_unit_boolean_type(collection): + """Range with boolean unit type produces error — unit must be a string.""" + collection.insert_many(SINGLE_DATE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"date": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-1, 0], "unit": True}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="boolean unit type rejected") + + +def test_time_range_unit_null_type(collection): + """Range with null unit type produces error — unit must be a string.""" + collection.insert_many(SINGLE_DATE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"date": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-1, 0], "unit": None}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="null unit type rejected") + + +# Property [Unit with Documents Mode]: unit is not allowed with documents-mode window + + +def test_time_range_unit_with_documents_mode(collection): + """Specifying unit with documents mode produces error.""" + collection.insert_many(SINGLE_DATE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"date": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"documents": [-1, 0], "unit": "day"}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="unit with documents mode rejected") + + +# Property [Date SortBy Requirement]: time unit requires date-type sortBy value + + +def test_time_range_unit_without_date_sortby(collection): + """Range with time unit but non-date sortBy value produces error.""" + docs = [{"_id": 1, "partition": "A", "score": 10, "value": 10}] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"score": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [-1, 0], "unit": "day"}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5429513, msg="time unit with non-date sortBy rejected") + + +# Property [Bound Type Validation with Unit]: bounds must be numeric when unit is specified + + +def test_time_range_boolean_bound_with_unit(collection): + """Boolean bound with time unit produces error.""" + collection.insert_many(SINGLE_DATE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"date": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [True, 1], "unit": "day"}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="boolean bound with time unit rejected") + + +def test_time_range_null_bound_with_unit(collection): + """Null bound with time unit produces error.""" + collection.insert_many(SINGLE_DATE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"date": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [None, 1], "unit": "day"}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="null bound with time unit rejected") + + +# Property [Bound Ordering with Unit]: lower bound must not exceed upper bound + + +def test_time_range_lower_exceeds_upper_with_unit(collection): + """Time-range bounds with lower > upper produces error.""" + collection.insert_many(SINGLE_DATE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"date": 1}, + "output": { + "result": { + "$sum": "$value", + "window": {"range": [5, -5], "unit": "day"}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, 5339900, msg="lower > upper with time unit rejected") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_argument_validation.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_argument_validation.py new file mode 100644 index 000000000..a6815fa7f --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_argument_validation.py @@ -0,0 +1,508 @@ +""" +Tests for $stdDevPop argument validation in window context. + +Covers: valid expression forms (field path, operator expression, literal), +non-numeric literal inputs that return null, invalid argument shapes that +produce parse errors (unknown keys in output field spec, multiple accumulators, +empty output field, multi-key expression objects, unknown operators in +expression, and field path with empty component). +""" + +from documentdb_tests.framework.assertions import assertFailureCode, assertSuccess +from documentdb_tests.framework.error_codes import ( + EXPRESSION_OBJECT_MULTIPLE_FIELDS_ERROR, + FAILED_TO_PARSE_ERROR, + FIELD_PATH_EMPTY_COMPONENT_ERROR, + UNRECOGNIZED_EXPRESSION_ERROR, +) +from documentdb_tests.framework.executor import execute_command + +TWO_DOCS = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, +] + +SINGLE_DOC = [{"_id": 1, "partition": "A", "value": 10}] + +# Property [Valid Expression Forms]: accepted expression inputs + + +def test_stdDevPop_field_path_expression(collection): + """$stdDevPop accepts a field path expression.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 5.0}, + {"_id": 2, "partition": "A", "value": 20, "result": 5.0}, + ] + assertSuccess(result, expected, msg="field path expression accepted") + + +def test_stdDevPop_operator_expression(collection): + """$stdDevPop accepts an operator expression.""" + docs = [ + {"_id": 1, "partition": "A", "value": 5}, + {"_id": 2, "partition": "A", "value": 10}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": {"$multiply": ["$value", 2]}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Values: [10, 20] -> stdDevPop = 5.0 + expected = [ + {"_id": 1, "partition": "A", "value": 5, "result": 5.0}, + {"_id": 2, "partition": "A", "value": 10, "result": 5.0}, + ] + assertSuccess(result, expected, msg="operator expression accepted") + + +def test_stdDevPop_literal_numeric_expression(collection): + """$stdDevPop with a literal numeric value — all rows get same value, stddev is 0.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": 30}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": {"$literal": 42}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # All rows evaluate to 42 -> stdDevPop = 0 + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 0.0}, + {"_id": 2, "partition": "A", "value": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 0.0}, + ] + assertSuccess(result, expected, msg="literal numeric expression produces 0 stddev") + + +# Property [Non-Numeric Literal Inputs]: non-numeric constants return null (not errors) + + +def test_stdDevPop_empty_string_expression(collection): + """$stdDevPop with empty string (not a valid field path) — treated as non-numeric.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Empty string is a literal string constant, non-numeric -> all null + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": None}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + ] + assertSuccess(result, expected, msg="empty string treated as non-numeric constant") + + +def test_stdDevPop_array_multiple_expressions_returns_null(collection): + """$stdDevPop with array of multiple field paths in window context returns null.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": ["$x", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20, "result": None}, + ] + assertSuccess(result, expected, msg="array of multiple expressions returns null in window form") + + +def test_stdDevPop_boolean_literal_returns_null(collection): + """$stdDevPop with boolean literal — non-numeric constant, returns null.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": True, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": None}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + ] + assertSuccess(result, expected, msg="boolean literal treated as non-numeric, returns null") + + +def test_stdDevPop_null_literal_returns_null(collection): + """$stdDevPop with null literal — returns null.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": None, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": None}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + ] + assertSuccess(result, expected, msg="null literal returns null") + + +def test_stdDevPop_empty_object_returns_null(collection): + """$stdDevPop with empty object {} — treated as non-numeric, returns null.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": {}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": None}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + ] + assertSuccess(result, expected, msg="empty object treated as non-numeric, returns null") + + +# Property [Invalid Argument Shapes - Parse Errors]: inputs that produce errors at parse time + + +def test_stdDevPop_unknown_key_in_output_field_errors(collection): + """Unknown key alongside $stdDevPop in output field spec produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + "unknownKey": 1, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, FAILED_TO_PARSE_ERROR, msg="unknown key alongside $stdDevPop rejected" + ) + + +def test_stdDevPop_unknown_key_errors_on_empty_collection(collection): + """Parse-time error fires on empty collection — no documents needed.""" + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + "unknownKey": 1, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FAILED_TO_PARSE_ERROR, + msg="parse-time error fires on empty collection", + ) + + +def test_stdDevPop_multiple_accumulators_in_output_field_errors(collection): + """Multiple accumulators in same output field spec produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FAILED_TO_PARSE_ERROR, + msg="multiple accumulators in output field rejected", + ) + + +def test_stdDevPop_no_accumulator_in_output_field_errors(collection): + """Output field with no accumulator (only window key) produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="no accumulator in output field rejected") + + +def test_stdDevPop_multi_key_expression_object_errors(collection): + """$stdDevPop with multi-key expression object (ambiguous) produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": {"$add": [1, 2], "$subtract": [3, 1]}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + EXPRESSION_OBJECT_MULTIPLE_FIELDS_ERROR, + msg="multi-key expression object rejected", + ) + + +def test_stdDevPop_unrecognized_expression_operator_errors(collection): + """$stdDevPop with unrecognized expression operator produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": {"$unknownOp": "$value"}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + UNRECOGNIZED_EXPRESSION_ERROR, + msg="unrecognized expression operator rejected", + ) + + +def test_stdDevPop_field_path_empty_component_errors(collection): + """$stdDevPop with field path containing empty component produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$a..b", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FIELD_PATH_EMPTY_COMPONENT_ERROR, + msg="field path with empty component rejected", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_field_paths.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_field_paths.py new file mode 100644 index 000000000..39d143694 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_field_paths.py @@ -0,0 +1,388 @@ +""" +Tests for $stdDevPop with nested field paths, array field traversal, +expressions that return different types per document, and $project +removing fields before $setWindowFields. + +Covers: dotted field paths, missing intermediate paths, array index access, +array-of-objects traversal, top-level array fields, expressions returning +mixed types per row, and pipeline stages removing expression fields. +""" + +from datetime import datetime, timezone + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + +# Property [Dotted Field Path]: +# Tests that $stdDevPop correctly accesses nested document values via dotted paths. + + +def test_stdDevPop_dotted_field_path(collection): + """$stdDevPop with dotted field path accesses nested document value.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"metrics": {"value": 10}}}, + {"_id": 2, "partition": "A", "data": {"metrics": {"value": 20}}}, + {"_id": 3, "partition": "A", "data": {"metrics": {"value": 30}}}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$data.metrics.value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + { + "_id": 1, + "partition": "A", + "data": {"metrics": {"value": 10}}, + "result": 8.16496580927726, + }, + { + "_id": 2, + "partition": "A", + "data": {"metrics": {"value": 20}}, + "result": 8.16496580927726, + }, + { + "_id": 3, + "partition": "A", + "data": {"metrics": {"value": 30}}, + "result": 8.16496580927726, + }, + ] + assertSuccess(result, expected, msg="dotted field path accesses nested value") + + +# Property [Missing Intermediate Path]: +# Tests that missing intermediate paths are treated as missing (ignored). + + +def test_stdDevPop_missing_intermediate_path(collection): + """$stdDevPop with missing intermediate path treated as missing (ignored).""" + docs = [ + {"_id": 1, "partition": "A", "data": {"metrics": {"value": 10}}}, + {"_id": 2, "partition": "A", "data": {"other": 99}}, + {"_id": 3, "partition": "A", "data": {"metrics": {"value": 30}}}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$data.metrics.value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Doc 2 missing data.metrics.value -> ignored. stdDevPop of [10, 30] = 10.0 + expected = [ + {"_id": 1, "partition": "A", "data": {"metrics": {"value": 10}}, "result": 10.0}, + {"_id": 2, "partition": "A", "data": {"other": 99}, "result": 10.0}, + {"_id": 3, "partition": "A", "data": {"metrics": {"value": 30}}, "result": 10.0}, + ] + assertSuccess(result, expected, msg="missing intermediate path treated as missing") + + +def test_stdDevPop_top_level_missing_object(collection): + """$stdDevPop where the top-level field of a dotted path is missing.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"value": 10}}, + {"_id": 2, "partition": "A"}, + {"_id": 3, "partition": "A", "data": {"value": 30}}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$data.value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "data": {"value": 10}, "result": 10.0}, + {"_id": 2, "partition": "A", "result": 10.0}, + {"_id": 3, "partition": "A", "data": {"value": 30}, "result": 10.0}, + ] + assertSuccess(result, expected, msg="top-level field missing in dotted path = ignored") + + +# Property [Array Field Non-Numeric]: +# Tests that top-level array values are treated as non-numeric and ignored. + + +def test_stdDevPop_array_field_is_non_numeric(collection): + """$stdDevPop on a top-level array field — arrays are non-numeric, should be ignored.""" + docs = [ + {"_id": 1, "partition": "A", "value": [10, 20, 30]}, + {"_id": 2, "partition": "A", "value": 50}, + {"_id": 3, "partition": "A", "value": [40, 50]}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Only doc 2 has a numeric value. Single numeric value in whole partition -> stdDevPop = 0.0 + expected = [ + {"_id": 1, "partition": "A", "value": [10, 20, 30], "result": 0.0}, + {"_id": 2, "partition": "A", "value": 50, "result": 0.0}, + {"_id": 3, "partition": "A", "value": [40, 50], "result": 0.0}, + ] + assertSuccess(result, expected, msg="array field values are non-numeric — ignored") + + +# Property [Expression Returns Mixed Types]: +# Tests that non-numeric expression results are ignored in the computation. + + +def test_stdDevPop_expression_returns_different_types(collection): + """$stdDevPop expression returning different types per row — non-numeric ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10}, + {"_id": 2, "partition": "A", "x": -5}, + {"_id": 3, "partition": "A", "x": 30}, + {"_id": 4, "partition": "A", "x": -1}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": { + "$cond": [ + {"$gt": ["$x", 0]}, + "$x", + "not_a_number", + ] + }, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # $cond returns: 10, "not_a_number", 30, "not_a_number" + # Only numeric: [10, 30] -> stdDevPop = 10.0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "x": -5, "result": 10.0}, + {"_id": 3, "partition": "A", "x": 30, "result": 10.0}, + {"_id": 4, "partition": "A", "x": -1, "result": 10.0}, + ] + assertSuccess(result, expected, msg="expression returning mixed types — non-numeric ignored") + + +# Property [Expression Returns Null]: +# Tests that null expression results are ignored in the computation. + + +def test_stdDevPop_expression_returns_null_for_some(collection): + """$stdDevPop expression returning null for some docs, number for others.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 2}, + {"_id": 2, "partition": "A", "x": 20, "y": None}, + {"_id": 3, "partition": "A", "x": 30, "y": 2}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": {"$multiply": ["$x", "$y"]}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # $multiply: [10*2=20, 20*null=null, 30*2=60] + # Numeric values: [20, 60] -> stdDevPop = 20.0 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 2, "result": 20.0}, + {"_id": 2, "partition": "A", "x": 20, "y": None, "result": 20.0}, + {"_id": 3, "partition": "A", "x": 30, "y": 2, "result": 20.0}, + ] + assertSuccess(result, expected, msg="expression returning null for some — null results ignored") + + +# Property [Pipeline Removes Expression Field]: +# Tests behavior when $project removes the expression field before $setWindowFields. + + +def test_stdDevPop_project_removes_expression_field(collection): + """$project removing the expression field before $setWindowFields — treated as missing.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10, "extra": 99}, + {"_id": 2, "partition": "A", "value": 20, "extra": 99}, + {"_id": 3, "partition": "A", "value": 30, "extra": 99}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + {"$project": {"partition": 1, "extra": 1}}, + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + ], + "cursor": {}, + }, + ) + # All docs missing "value" -> all non-numeric -> result is null + expected = [ + {"_id": 1, "partition": "A", "extra": 99, "result": None}, + {"_id": 2, "partition": "A", "extra": 99, "result": None}, + {"_id": 3, "partition": "A", "extra": 99, "result": None}, + ] + assertSuccess(result, expected, msg="$project removing expression field -> all null results") + + +# Property [Date Value as Expression]: +# Tests that Date values in expression field are non-numeric and ignored. + + +def test_stdDevPop_date_value_as_expression_ignored(collection): + """$stdDevPop with Date value in expression field — non-numeric, ignored.""" + docs = [ + {"_id": 1, "partition": "A", "value": datetime(2023, 1, 1, tzinfo=timezone.utc)}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": datetime(2023, 6, 1, tzinfo=timezone.utc)}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Only doc 2 is numeric -> single value in whole partition -> stdDevPop = 0.0 + expected = [ + { + "_id": 1, + "partition": "A", + "value": datetime(2023, 1, 1, tzinfo=timezone.utc), + "result": 0.0, + }, + {"_id": 2, "partition": "A", "value": 20, "result": 0.0}, + { + "_id": 3, + "partition": "A", + "value": datetime(2023, 6, 1, tzinfo=timezone.utc), + "result": 0.0, + }, + ] + assertSuccess(result, expected, msg="Date values in expression field are non-numeric — ignored") + + +# Property [Numeric Path Component]: +# Tests that numeric path components access array elements or object keys. + + +def test_stdDevPop_numeric_path_component(collection): + """$stdDevPop with numeric path component accesses array element or object key.""" + docs = [ + {"_id": 1, "partition": "A", "arr": [{"field": 10}, {"field": 20}]}, + {"_id": 2, "partition": "A", "arr": [{"field": 30}, {"field": 40}]}, + {"_id": 3, "partition": "A", "arr": [{"field": 50}, {"field": 60}]}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$arr.0.field", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + {"$project": {"_id": 1, "result": 1}}, + ], + "cursor": {}, + }, + ) + # In $setWindowFields context, $arr.0.field does not resolve to array element — + # the path returns non-numeric (array) values which are ignored, resulting in null. + expected = [ + {"_id": 1, "result": None}, + {"_id": 2, "result": None}, + {"_id": 3, "result": None}, + ] + assertSuccess(result, expected, msg="numeric path component in window context returns null") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_frame_computation.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_frame_computation.py new file mode 100644 index 000000000..f36adea78 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_frame_computation.py @@ -0,0 +1,89 @@ +""" +Tests for $stdDevPop computation under documents-mode window frame shapes. + +Verifies the operator computes correct results given the 4 defined frame shapes: +whole-partition, cumulative, reverse-cumulative, and sliding. + +Note: Stage-level frame boundary tests (under stages/setWindowFields/) verify +that the correct documents are selected into the frame (centered, trailing, +leading, non-overlapping, edge cases). These per-operator tests verify the +operator produces correct values given those documents. +""" + +import pytest + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + BASIC_DOCS, + WindowTestCase, + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.parametrize import pytest_params + +STDDEVPOP_DOCUMENTS_FRAME_TESTS: list[WindowTestCase] = [ + # Property [Whole Partition]: unbounded-unbounded frame covers entire partition + WindowTestCase( + "whole_partition", + docs=BASIC_DOCS, + window={"documents": ["unbounded", "unbounded"]}, + expected=[ + {"_id": 1, "partition": "A", "value": 10, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "value": 20, "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "value": 30, "result": 14.142135623730951}, + {"_id": 4, "partition": "A", "value": 40, "result": 14.142135623730951}, + {"_id": 5, "partition": "A", "value": 50, "result": 14.142135623730951}, + ], + msg="whole partition stdDevPop should be sqrt(200)", + ), + # Property [Cumulative Frame]: expanding frame from start to current + WindowTestCase( + "cumulative", + docs=BASIC_DOCS, + window={"documents": ["unbounded", "current"]}, + expected=[ + {"_id": 1, "partition": "A", "value": 10, "result": 0.0}, + {"_id": 2, "partition": "A", "value": 20, "result": 5.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 8.16496580927726}, + {"_id": 4, "partition": "A", "value": 40, "result": 11.180339887498949}, + {"_id": 5, "partition": "A", "value": 50, "result": 14.142135623730951}, + ], + msg="cumulative stdDevPop should grow", + ), + # Property [Reverse Cumulative Frame]: shrinking frame from current to end + WindowTestCase( + "reverse_cumulative", + docs=BASIC_DOCS, + window={"documents": ["current", "unbounded"]}, + expected=[ + {"_id": 1, "partition": "A", "value": 10, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "value": 20, "result": 11.180339887498949}, + {"_id": 3, "partition": "A", "value": 30, "result": 8.16496580927726}, + {"_id": 4, "partition": "A", "value": 40, "result": 5.0}, + {"_id": 5, "partition": "A", "value": 50, "result": 0}, + ], + msg="reverse-cumulative stdDevPop should shrink", + ), + # Property [Sliding Frame]: fixed-size window that moves with current row + WindowTestCase( + "sliding_centered", + docs=BASIC_DOCS, + window={"documents": [-1, 1]}, + expected=[ + {"_id": 1, "partition": "A", "value": 10, "result": 5.0}, + {"_id": 2, "partition": "A", "value": 20, "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": 30, "result": 8.16496580927726}, + {"_id": 4, "partition": "A", "value": 40, "result": 8.16496580927726}, + {"_id": 5, "partition": "A", "value": 50, "result": 5.0}, + ], + msg="centered sliding window [-1,1]", + ), +] + + +@pytest.mark.parametrize("test", pytest_params(STDDEVPOP_DOCUMENTS_FRAME_TESTS)) +def test_stdDevPop_documents_frames(collection, test): + """$stdDevPop with various documents-mode window frames.""" + result = run_window_operator( + collection, "$stdDevPop", test.docs, test.window, sort_by=test.sort_by + ) + assertSuccess(result, test.expected, msg=test.msg) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_non_numeric_handling.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_non_numeric_handling.py new file mode 100644 index 000000000..254bbd1f9 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_non_numeric_handling.py @@ -0,0 +1,449 @@ +""" +Tests for $stdDevPop null, missing, and non-numeric value handling. + +Covers: null values, missing fields, strings, booleans, arrays, objects, +ObjectId, Regex, Binary, Timestamp, MinKey, MaxKey, mixed numeric and +non-numeric in same frame, and all non-numeric returns null. +""" + +from datetime import datetime, timezone + +from bson import Binary, MaxKey, MinKey, ObjectId, Regex, Timestamp + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + +# Property [Null and Missing]: null and missing field values are ignored in stdDev computation + + +def test_stdDevPop_null_values_ignored(collection): + """$stdDevPop ignores null values in the expression field.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": None}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "value": None, "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + ] + assertSuccess(result, expected, msg="null values ignored, stdDevPop of 10,30 = 10") + + +def test_stdDevPop_missing_field_ignored(collection): + """$stdDevPop ignores documents where the expression field is missing.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A"}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + ] + assertSuccess(result, expected, msg="missing field ignored, stdDevPop of 10,30 = 10") + + +def test_stdDevPop_nulls_among_numerics_in_frame(collection): + """$stdDevPop ignores nulls in frame and computes on remaining numerics.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10.0}, + {"_id": 2, "partition": "A", "value": None}, + {"_id": 3, "partition": "A", "value": 30.0}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Whole partition: numerics [10, 30] -> stdDevPop = 10.0 (null ignored) + expected = [ + {"_id": 1, "partition": "A", "value": 10.0, "result": 10.0}, + {"_id": 2, "partition": "A", "value": None, "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30.0, "result": 10.0}, + ] + assertSuccess( + result, expected, msg="null values ignored, stdDevPop computed on remaining numerics" + ) + + +# Property [Non-Numeric Types Ignored]: string, boolean, array, object, date, +# ObjectId, Regex, Binary values are ignored + + +def test_stdDevPop_string_values_ignored(collection): + """$stdDevPop ignores string values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": "hello"}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "value": "hello", "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + ] + assertSuccess(result, expected, msg="string values ignored") + + +def test_stdDevPop_boolean_values_ignored(collection): + """$stdDevPop ignores boolean values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": True}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "value": True, "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + ] + assertSuccess(result, expected, msg="boolean values ignored") + + +def test_stdDevPop_array_values_ignored(collection): + """$stdDevPop ignores array values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": [1, 2, 3]}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "value": [1, 2, 3], "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + ] + assertSuccess(result, expected, msg="array values ignored") + + +def test_stdDevPop_object_values_ignored(collection): + """$stdDevPop ignores object/document values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": {"nested": 99}}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "value": {"nested": 99}, "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + ] + assertSuccess(result, expected, msg="object values ignored") + + +def test_stdDevPop_objectid_and_regex_and_binary_ignored(collection): + """$stdDevPop ignores ObjectId, Regex, and Binary values.""" + oid = ObjectId("507f1f77bcf86cd799439011") + docs = [ + {"_id": 1, "partition": "A", "value": oid}, + {"_id": 2, "partition": "A", "value": Regex("^test", "i")}, + {"_id": 3, "partition": "A", "value": Binary(b"\x01\x02\x03")}, + {"_id": 4, "partition": "A", "value": 10}, + {"_id": 5, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, + "$stdDevPop", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # Only numeric: [10, 30] -> stdDevPop = 10.0 + expected = [ + {"_id": 1, "result": 10.0}, + {"_id": 2, "result": 10.0}, + {"_id": 3, "result": 10.0}, + {"_id": 4, "result": 10.0}, + {"_id": 5, "result": 10.0}, + ] + assertSuccess(result, expected, msg="ObjectId/Regex/Binary values ignored") + + +def test_stdDevPop_timestamp_minkey_maxkey_ignored(collection): + """$stdDevPop ignores Timestamp, MinKey, and MaxKey values.""" + docs = [ + {"_id": 1, "partition": "A", "value": Timestamp(1234567890, 1)}, + {"_id": 2, "partition": "A", "value": MinKey()}, + {"_id": 3, "partition": "A", "value": MaxKey()}, + {"_id": 4, "partition": "A", "value": 10}, + {"_id": 5, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, + "$stdDevPop", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # Only numeric: [10, 30] -> stdDevPop = 10.0 + expected = [ + {"_id": 1, "result": 10.0}, + {"_id": 2, "result": 10.0}, + {"_id": 3, "result": 10.0}, + {"_id": 4, "result": 10.0}, + {"_id": 5, "result": 10.0}, + ] + assertSuccess(result, expected, msg="Timestamp/MinKey/MaxKey values ignored") + + +# Property [All Non-Numeric Returns Null]: when all values are non-numeric, result is null + + +def test_stdDevPop_all_non_numeric_returns_null(collection): + """$stdDevPop returns null when all values in frame are non-numeric.""" + docs = [ + {"_id": 1, "partition": "A", "value": "a"}, + {"_id": 2, "partition": "A", "value": None}, + {"_id": 3, "partition": "A"}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": "a", "result": None}, + {"_id": 2, "partition": "A", "value": None, "result": None}, + {"_id": 3, "partition": "A", "result": None}, + ] + assertSuccess(result, expected, msg="all non-numeric in frame returns null") + + +def test_stdDevPop_all_non_numeric_diverse_types(collection): + """$stdDevPop returns null when all values are diverse non-numeric types.""" + docs = [ + {"_id": 1, "partition": "A", "value": "text"}, + {"_id": 2, "partition": "A", "value": True}, + {"_id": 3, "partition": "A", "value": datetime(2023, 1, 1, tzinfo=timezone.utc)}, + {"_id": 4, "partition": "A", "value": [1, 2]}, + {"_id": 5, "partition": "A", "value": {"a": 1}}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # No numeric values -> null + expected = [ + {"_id": 1, "partition": "A", "value": "text", "result": None}, + {"_id": 2, "partition": "A", "value": True, "result": None}, + { + "_id": 3, + "partition": "A", + "value": datetime(2023, 1, 1, tzinfo=timezone.utc), + "result": None, + }, + {"_id": 4, "partition": "A", "value": [1, 2], "result": None}, + {"_id": 5, "partition": "A", "value": {"a": 1}, "result": None}, + ] + assertSuccess(result, expected, msg="all diverse non-numeric types return null") + + +# Property [Mixed Types in Frame]: non-numeric values filtered per-frame, numerics participate + + +def test_stdDevPop_mixed_numeric_non_numeric_sliding(collection): + """$stdDevPop in sliding window with mix of numeric and non-numeric values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10.0}, + {"_id": 2, "partition": "A", "value": "skip"}, + {"_id": 3, "partition": "A", "value": 20.0}, + {"_id": 4, "partition": "A", "value": None}, + {"_id": 5, "partition": "A", "value": 30.0}, + ] + result = run_window_operator(collection, "$stdDevPop", docs, {"documents": [-2, 2]}) + # Window [-2, 2] (5-doc centered): + # Row 1: frame [10.0, "skip", 20.0] -> numerics [10, 20] -> stdDevPop = 5.0 + # Row 2: frame [10.0, "skip", 20.0, None] -> numerics [10, 20] -> 5.0 + # Row 3: frame [10.0, "skip", 20.0, None, 30.0] -> numerics [10, 20, 30] -> 8.165 + # Row 4: frame ["skip", 20.0, None, 30.0] -> numerics [20, 30] -> 5.0 + # Row 5: frame [20.0, None, 30.0] -> numerics [20, 30] -> 5.0 + expected = [ + {"_id": 1, "partition": "A", "value": 10.0, "result": 5.0}, + {"_id": 2, "partition": "A", "value": "skip", "result": 5.0}, + {"_id": 3, "partition": "A", "value": 20.0, "result": 8.16496580927726}, + {"_id": 4, "partition": "A", "value": None, "result": 5.0}, + {"_id": 5, "partition": "A", "value": 30.0, "result": 5.0}, + ] + assertSuccess(result, expected, msg="mixed types in sliding window — non-numeric ignored") + + +def test_stdDevPop_mixed_types_in_documents(collection): + """$stdDevPop over documents with diverse value types — only numerics participate.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": "hello"}, + {"_id": 3, "partition": "A", "value": True}, + {"_id": 4, "partition": "A", "value": datetime(2023, 6, 1, tzinfo=timezone.utc)}, + {"_id": 5, "partition": "A", "value": 30}, + {"_id": 6, "partition": "A", "value": None}, + {"_id": 7, "partition": "A", "value": [1, 2, 3]}, + {"_id": 8, "partition": "A", "value": {"nested": 99}}, + {"_id": 9, "partition": "A", "value": 50}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + {"$project": {"_id": 1, "result": 1}}, + ], + "cursor": {}, + }, + ) + # Only numeric: [10, 30, 50] -> mean=30, variance=(400+0+400)/3=266.67, stddev=16.33 + expected = [ + {"_id": 1, "result": 16.32993161855452}, + {"_id": 2, "result": 16.32993161855452}, + {"_id": 3, "result": 16.32993161855452}, + {"_id": 4, "result": 16.32993161855452}, + {"_id": 5, "result": 16.32993161855452}, + {"_id": 6, "result": 16.32993161855452}, + {"_id": 7, "result": 16.32993161855452}, + {"_id": 8, "result": 16.32993161855452}, + {"_id": 9, "result": 16.32993161855452}, + ] + assertSuccess(result, expected, msg="diverse non-numeric types ignored in whole partition") + + +def test_stdDevPop_mixed_types_sliding_window(collection): + """$stdDevPop sliding window over documents with diverse types — per-frame filtering.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10.0}, + {"_id": 2, "partition": "A", "value": 20.0}, + {"_id": 3, "partition": "A", "value": "skip"}, + {"_id": 4, "partition": "A", "value": True}, + {"_id": 5, "partition": "A", "value": 30.0}, + {"_id": 6, "partition": "A", "value": datetime(2023, 1, 1, tzinfo=timezone.utc)}, + {"_id": 7, "partition": "A", "value": 40.0}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Whole partition: numerics [10, 20, 30, 40] -> stdDevPop = 11.180339887498949 + expected = [ + {"_id": 1, "partition": "A", "value": 10.0, "result": 11.180339887498949}, + {"_id": 2, "partition": "A", "value": 20.0, "result": 11.180339887498949}, + {"_id": 3, "partition": "A", "value": "skip", "result": 11.180339887498949}, + {"_id": 4, "partition": "A", "value": True, "result": 11.180339887498949}, + {"_id": 5, "partition": "A", "value": 30.0, "result": 11.180339887498949}, + { + "_id": 6, + "partition": "A", + "value": datetime(2023, 1, 1, tzinfo=timezone.utc), + "result": 11.180339887498949, + }, + {"_id": 7, "partition": "A", "value": 40.0, "result": 11.180339887498949}, + ] + assertSuccess(result, expected, msg="sliding window filters non-numeric types per frame") + + +def test_stdDevPop_numeric_among_diverse_types_cumulative(collection): + """$stdDevPop cumulative window with numerics scattered among diverse types.""" + docs = [ + {"_id": 1, "partition": "A", "value": "text"}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": datetime(2023, 6, 1, tzinfo=timezone.utc)}, + {"_id": 4, "partition": "A", "value": 20}, + {"_id": 5, "partition": "A", "value": True}, + {"_id": 6, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "current"]} + ) + # Cumulative, only numerics count: + # Row 1: no numeric -> null + # Row 2: [10] -> 0 + # Row 3: [10] -> 0 (datetime ignored) + # Row 4: [10, 20] -> 5.0 + # Row 5: [10, 20] -> 5.0 (True ignored) + # Row 6: [10, 20, 30] -> 8.165 + expected = [ + {"_id": 1, "partition": "A", "value": "text", "result": None}, + {"_id": 2, "partition": "A", "value": 10, "result": 0.0}, + { + "_id": 3, + "partition": "A", + "value": datetime(2023, 6, 1, tzinfo=timezone.utc), + "result": 0.0, + }, + {"_id": 4, "partition": "A", "value": 20, "result": 5.0}, + {"_id": 5, "partition": "A", "value": True, "result": 5.0}, + {"_id": 6, "partition": "A", "value": 30, "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="cumulative window with numerics among diverse types") + + +def test_stdDevPop_numerics_among_non_numeric_sliding(collection): + """$stdDevPop sliding frame filters non-numerics and computes on remaining values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10.0}, + {"_id": 2, "partition": "A", "value": "a"}, + {"_id": 3, "partition": "A", "value": 30.0}, + {"_id": 4, "partition": "A", "value": "b"}, + {"_id": 5, "partition": "A", "value": 50.0}, + ] + result = run_window_operator(collection, "$stdDevPop", docs, {"documents": [-2, 2]}) + # Window [-2, 2] (5-doc centered): + # Row 1: [10.0, "a", 30.0] -> numerics [10, 30] -> stdDevPop = 10.0 + # Row 2: [10.0, "a", 30.0, "b"] -> numerics [10, 30] -> 10.0 + # Row 3: [10.0, "a", 30.0, "b", 50.0] -> numerics [10, 30, 50] -> 16.3299... + # Row 4: ["a", 30.0, "b", 50.0] -> numerics [30, 50] -> 10.0 + # Row 5: [30.0, "b", 50.0] -> numerics [30, 50] -> 10.0 + expected = [ + {"_id": 1, "partition": "A", "value": 10.0, "result": 10.0}, + {"_id": 2, "partition": "A", "value": "a", "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30.0, "result": 16.32993161855452}, + {"_id": 4, "partition": "A", "value": "b", "result": 10.0}, + {"_id": 5, "partition": "A", "value": 50.0, "result": 10.0}, + ] + assertSuccess(result, expected, msg="non-numeric values filtered in sliding frame") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_numeric_precision.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_numeric_precision.py new file mode 100644 index 000000000..7a02ba5da --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_numeric_precision.py @@ -0,0 +1,724 @@ +""" +Tests for $stdDevPop numeric type mixing, overflow edge cases, +algorithmic precision validation, and Decimal128 type handling. + +Covers: Int32/Int64/Double mixing, Int64 near MAX_LONG (overflow risk when +squaring), catastrophic cancellation in variance calculation, known exact +results, very small differences, consistency between window modes, +Decimal128 (NumberDecimal) values, high-precision Decimal128, mixed Decimal128 +with other numeric types, and Decimal128 special values (NaN, Infinity). +""" + +from bson import Decimal128, Int64 + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertResult, assertSuccess, assertSuccessNaN +from documentdb_tests.framework.executor import execute_command +from documentdb_tests.framework.property_checks import Gt, Lte, PerDoc +from documentdb_tests.framework.test_constants import DOUBLE_NEGATIVE_ZERO, FLOAT_NAN + +# Property [Numeric Type Mixing]: Int32, Int64, Double coexist correctly + + +def test_stdDevPop_all_int32_values(collection): + """$stdDevPop with all Int32 values produces Double result.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 8.16496580927726}, + {"_id": 2, "partition": "A", "value": 20, "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": 30, "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="all Int32 values produce correct Double result") + + +def test_stdDevPop_all_int64_values(collection): + """$stdDevPop with all Int64 values produces correct result.""" + docs = [ + {"_id": 1, "partition": "A", "value": Int64(10)}, + {"_id": 2, "partition": "A", "value": Int64(20)}, + {"_id": 3, "partition": "A", "value": Int64(30)}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Int64(10), "result": 8.16496580927726}, + {"_id": 2, "partition": "A", "value": Int64(20), "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": Int64(30), "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="all Int64 values compute correctly") + + +def test_stdDevPop_mixed_int32_int64_double(collection): + """$stdDevPop with mixed Int32 + Int64 + Double in same frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": Int64(20)}, + {"_id": 3, "partition": "A", "value": 30.0}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 8.16496580927726}, + {"_id": 2, "partition": "A", "value": Int64(20), "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": 30.0, "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="mixed Int32 + Int64 + Double type promotion works") + + +# Property [Large Value Handling]: near-overflow and large-spread values compute without overflow + + +def test_stdDevPop_large_int64_near_max(collection): + """$stdDevPop with Int64 values near MAX_LONG — squaring would overflow 64-bit.""" + docs = [ + {"_id": 1, "partition": "A", "value": Int64(9223372036854775806)}, + {"_id": 2, "partition": "A", "value": Int64(9223372036854775807)}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Both values round to the same float64 (2^63-1 and 2^63-2 differ by 1 ULP at this scale), + # so the server sees them as identical -> stdDevPop = 0.0 + expected = [ + {"_id": 1, "partition": "A", "value": Int64(9223372036854775806), "result": 0.0}, + {"_id": 2, "partition": "A", "value": Int64(9223372036854775807), "result": 0.0}, + ] + assertSuccess(result, expected, msg="Int64 near MAX_LONG does not overflow") + + +def test_stdDevPop_large_int64_spread(collection): + """$stdDevPop with widely spread Int64 values — tests numeric stability.""" + docs = [ + {"_id": 1, "partition": "A", "value": Int64(0)}, + {"_id": 2, "partition": "A", "value": Int64(4611686018427387903)}, + {"_id": 3, "partition": "A", "value": Int64(9223372036854775807)}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Verify 3 docs returned, each with a positive result (exact value depends on precision) + checks = PerDoc( + {"result": Gt(0)}, + {"result": Gt(0)}, + {"result": Gt(0)}, + ) + assertResult(result, expected=checks, msg="Large Int64 spread produces positive result") + + +def test_stdDevPop_int32_sum_of_squares_overflow(collection): + """$stdDevPop with Int32 values where sum-of-squares would overflow Int32.""" + # Max Int32 = 2147483647. Values around 50000: 50000^2 = 2.5e9 > 2^31 + docs = [ + {"_id": 1, "partition": "A", "value": 50000}, + {"_id": 2, "partition": "A", "value": 50001}, + {"_id": 3, "partition": "A", "value": 50002}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 50000, "result": 0.816496580927726}, + {"_id": 2, "partition": "A", "value": 50001, "result": 0.816496580927726}, + {"_id": 3, "partition": "A", "value": 50002, "result": 0.816496580927726}, + ] + assertSuccess(result, expected, msg="Int32 values that would overflow Int32 sum-of-squares") + + +def test_stdDevPop_very_large_value(collection): + """$stdDevPop with very large numeric value (1e308) — all identical returns 0.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1e308}, + {"_id": 2, "partition": "A", "value": 1e308}, + {"_id": 3, "partition": "A", "value": 1e308}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 1e308, "result": 0.0}, + {"_id": 2, "partition": "A", "value": 1e308, "result": 0.0}, + {"_id": 3, "partition": "A", "value": 1e308, "result": 0.0}, + ] + assertSuccess(result, expected, msg="very large identical values return 0") + + +def test_stdDevPop_alternating_large_values(collection): + """$stdDevPop with alternating sign large values — stress variance accumulator.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1e15}, + {"_id": 2, "partition": "A", "value": -1e15}, + {"_id": 3, "partition": "A", "value": 1e15}, + {"_id": 4, "partition": "A", "value": -1e15}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Mean = 0, variance = (4 * 1e30) / 4 = 1e30, stddev = 1e15 + expected = [ + {"_id": 1, "partition": "A", "value": 1e15, "result": 1e15}, + {"_id": 2, "partition": "A", "value": -1e15, "result": 1e15}, + {"_id": 3, "partition": "A", "value": 1e15, "result": 1e15}, + {"_id": 4, "partition": "A", "value": -1e15, "result": 1e15}, + ] + assertSuccess(result, expected, msg="alternating large values produce correct stddev") + + +# Property [Algorithmic Precision]: known exact results and catastrophic cancellation handling + + +def test_stdDevPop_known_exact_result(collection): + """$stdDevPop of [2,4,4,4,5,5,7,9] = exactly 2.0.""" + docs = [ + {"_id": i, "partition": "A", "value": v} for i, v in enumerate([2, 4, 4, 4, 5, 5, 7, 9], 1) + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": i, "partition": "A", "value": v, "result": 2.0} + for i, v in enumerate([2, 4, 4, 4, 5, 5, 7, 9], 1) + ] + assertSuccess(result, expected, msg="stdDevPop of [2,4,4,4,5,5,7,9] must be exactly 2.0") + + +def test_stdDevPop_identical_floats_exactly_zero(collection): + """$stdDevPop of identical float values must be exactly 0.0, not epsilon.""" + docs = [ + {"_id": 1, "partition": "A", "value": 3.0}, + {"_id": 2, "partition": "A", "value": 3.0}, + {"_id": 3, "partition": "A", "value": 3.0}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 3.0, "result": 0.0}, + {"_id": 2, "partition": "A", "value": 3.0, "result": 0.0}, + {"_id": 3, "partition": "A", "value": 3.0, "result": 0.0}, + ] + assertSuccess(result, expected, msg="identical floats produce exactly 0.0") + + +def test_stdDevPop_catastrophic_cancellation(collection): + """$stdDevPop of [1000000001, 1000000002, 1000000003] — naive sum-of-squares fails.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1000000001}, + {"_id": 2, "partition": "A", "value": 1000000002}, + {"_id": 3, "partition": "A", "value": 1000000003}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # stdDevPop of [N, N+1, N+2] = sqrt(2/3) ≈ 0.8165 regardless of N + expected = [ + {"_id": 1, "partition": "A", "value": 1000000001, "result": 0.816496580927726}, + {"_id": 2, "partition": "A", "value": 1000000002, "result": 0.816496580927726}, + {"_id": 3, "partition": "A", "value": 1000000003, "result": 0.816496580927726}, + ] + assertSuccess( + result, + expected, + msg="catastrophic cancellation handled — correct stddev for large offset values", + ) + + +def test_stdDevPop_very_small_differences(collection): + """$stdDevPop with values that differ by very small amounts.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1.0000001}, + {"_id": 2, "partition": "A", "value": 1.0000002}, + {"_id": 3, "partition": "A", "value": 1.0000003}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # The stdDevPop should be approximately 8.165e-8 — verify positive and tiny + checks = PerDoc( + {"result": [Gt(0), Lte(0.001)]}, + {"result": [Gt(0), Lte(0.001)]}, + {"result": [Gt(0), Lte(0.001)]}, + ) + assertResult( + result, expected=checks, msg="Small differences produce very small positive stddev" + ) + + +def test_stdDevPop_sequential_values_1_to_10(collection): + """$stdDevPop over sequential values 1..10 with known exact result.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1}, + {"_id": 2, "partition": "A", "value": 2}, + {"_id": 3, "partition": "A", "value": 3}, + {"_id": 4, "partition": "A", "value": 4}, + {"_id": 5, "partition": "A", "value": 5}, + {"_id": 6, "partition": "A", "value": 6}, + {"_id": 7, "partition": "A", "value": 7}, + {"_id": 8, "partition": "A", "value": 8}, + {"_id": 9, "partition": "A", "value": 9}, + {"_id": 10, "partition": "A", "value": 10}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + {"$match": {"_id": 1}}, + ], + "cursor": {}, + }, + ) + # stdDevPop of 1..10: sqrt(82.5/10) = sqrt(8.25) ≈ 2.8723 + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 2.8722813232690143}, + ] + assertSuccess(result, expected, msg="sequential 1..10 known stdDevPop result") + + +# Property [Single Element Frame]: single value produces 0 for population stddev + + +def test_stdDevPop_single_element_sliding_window(collection): + """$stdDevPop returns 0 when sliding window frame has exactly one value.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10.0}, + {"_id": 2, "partition": "A", "value": 20.0}, + {"_id": 3, "partition": "A", "value": 30.0}, + {"_id": 4, "partition": "A", "value": 40.0}, + {"_id": 5, "partition": "A", "value": 50.0}, + ] + result = run_window_operator(collection, "$stdDevPop", docs, {"documents": [0, 0]}) + # Window [0, 0] — each frame has exactly one value -> stdDevPop = 0 + expected = [ + {"_id": 1, "partition": "A", "value": 10.0, "result": 0}, + {"_id": 2, "partition": "A", "value": 20.0, "result": 0}, + {"_id": 3, "partition": "A", "value": 30.0, "result": 0}, + {"_id": 4, "partition": "A", "value": 40.0, "result": 0}, + {"_id": 5, "partition": "A", "value": 50.0, "result": 0}, + ] + assertSuccess(result, expected, msg="single element in sliding frame returns 0") + + +# Property [Decimal128 Support]: Decimal128 values and type mixing with other numerics + + +def test_stdDevPop_pure_decimal128_values(collection): + """$stdDevPop with pure Decimal128 values computes correctly.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("20")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 8.16496580927726}, + {"_id": 2, "partition": "A", "value": Decimal128("20"), "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="pure Decimal128 values compute stdDevPop correctly") + + +def test_stdDevPop_decimal128_with_double(collection): + """$stdDevPop with mixed Decimal128 and Double values.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": 20.0}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 8.16496580927726}, + {"_id": 2, "partition": "A", "value": 20.0, "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Double compute correctly") + + +def test_stdDevPop_decimal128_with_int32(collection): + """$stdDevPop with mixed Decimal128 and Int32 values.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 8.16496580927726}, + {"_id": 2, "partition": "A", "value": 20, "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Int32 compute correctly") + + +def test_stdDevPop_decimal128_with_int64(collection): + """$stdDevPop with mixed Decimal128 and Int64 values.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Int64(20)}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 8.16496580927726}, + {"_id": 2, "partition": "A", "value": Int64(20), "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Int64 compute correctly") + + +def test_stdDevPop_decimal128_all_types_mixed(collection): + """$stdDevPop with Decimal128 + Double + Int32 + Int64 all in same frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": 20.0}, + {"_id": 3, "partition": "A", "value": 30}, + {"_id": 4, "partition": "A", "value": Int64(40)}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 11.180339887498949}, + {"_id": 2, "partition": "A", "value": 20.0, "result": 11.180339887498949}, + {"_id": 3, "partition": "A", "value": 30, "result": 11.180339887498949}, + {"_id": 4, "partition": "A", "value": Int64(40), "result": 11.180339887498949}, + ] + assertSuccess(result, expected, msg="all four numeric types mixed in same frame") + + +def test_stdDevPop_decimal128_high_precision(collection): + """$stdDevPop with high-precision Decimal128 values preserves precision.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("1.000000000000000000000000000000001")}, + {"_id": 2, "partition": "A", "value": Decimal128("2.000000000000000000000000000000001")}, + {"_id": 3, "partition": "A", "value": Decimal128("3.000000000000000000000000000000001")}, + ] + collection.insert_many(docs) + extra_stages = [ + {"$addFields": {"result": {"$round": ["$result", 3]}}}, + {"$project": {"_id": 1, "result": 1}}, + ] + pipeline = [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ] + extra_stages + result = execute_command( + collection, + {"aggregate": collection.name, "pipeline": pipeline, "cursor": {}}, + ) + expected = [ + {"_id": 1, "result": 0.816}, + {"_id": 2, "result": 0.816}, + {"_id": 3, "result": 0.816}, + ] + assertSuccess(result, expected, msg="high-precision Decimal128 values produce ~0.816") + + +def test_stdDevPop_decimal128_near_zero(collection): + """$stdDevPop with very small Decimal128 values near zero.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("0.0000000000000000000000000000000001")}, + {"_id": 2, "partition": "A", "value": Decimal128("0.0000000000000000000000000000000002")}, + {"_id": 3, "partition": "A", "value": Decimal128("0.0000000000000000000000000000000003")}, + ] + collection.insert_many(docs) + extra_stages = [ + { + "$addFields": { + "valid": { + "$and": [ + {"$ne": ["$result", None]}, + {"$gte": ["$result", 0]}, + ] + } + } + }, + {"$project": {"_id": 1, "valid": 1}}, + ] + pipeline = [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ] + extra_stages + result = execute_command( + collection, + {"aggregate": collection.name, "pipeline": pipeline, "cursor": {}}, + ) + expected = [ + {"_id": 1, "valid": True}, + {"_id": 2, "valid": True}, + {"_id": 3, "valid": True}, + ] + assertSuccess(result, expected, msg="near-zero Decimal128 produces non-negative result") + + +def test_stdDevPop_decimal128_sliding_window(collection): + """$stdDevPop with Decimal128 values in a sliding window.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("20")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + {"_id": 4, "partition": "A", "value": Decimal128("40")}, + {"_id": 5, "partition": "A", "value": Decimal128("50")}, + ] + result = run_window_operator(collection, "$stdDevPop", docs, {"documents": [-1, 0]}) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 0}, + {"_id": 2, "partition": "A", "value": Decimal128("20"), "result": 5.0}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 5.0}, + {"_id": 4, "partition": "A", "value": Decimal128("40"), "result": 5.0}, + {"_id": 5, "partition": "A", "value": Decimal128("50"), "result": 5.0}, + ] + assertSuccess(result, expected, msg="Decimal128 values in sliding window") + + +def test_stdDevPop_decimal128_identical_values(collection): + """$stdDevPop with identical Decimal128 values returns 0.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("42.5")}, + {"_id": 2, "partition": "A", "value": Decimal128("42.5")}, + {"_id": 3, "partition": "A", "value": Decimal128("42.5")}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("42.5"), "result": 0.0}, + {"_id": 2, "partition": "A", "value": Decimal128("42.5"), "result": 0.0}, + {"_id": 3, "partition": "A", "value": Decimal128("42.5"), "result": 0.0}, + ] + assertSuccess(result, expected, msg="identical Decimal128 values return 0") + + +# Property [Decimal128 Special Values]: Decimal128 NaN and Infinity handling + + +def test_stdDevPop_decimal128_nan_special(collection): + """$stdDevPop with Decimal128 NaN — NaN is numeric and poisons the calculation.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("NaN")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": Decimal128("NaN"), "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="Decimal128 NaN is numeric and poisons stdDevPop to NaN") + + +def test_stdDevPop_decimal128_infinity_special(collection): + """$stdDevPop with Decimal128 Infinity special value.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("Infinity")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + collection.insert_many(docs) + extra_stages = [ + { + "$addFields": { + "has_result": {"$ne": ["$result", None]}, + } + }, + {"$project": {"_id": 1, "has_result": 1}}, + ] + pipeline = [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ] + extra_stages + result = execute_command( + collection, + {"aggregate": collection.name, "pipeline": pipeline, "cursor": {}}, + ) + expected = [ + {"_id": 1, "has_result": True}, + {"_id": 2, "has_result": True}, + {"_id": 3, "has_result": True}, + ] + assertSuccess(result, expected, msg="Decimal128 Infinity produces a non-null result") + + +def test_stdDevPop_decimal128_neg_infinity_special(collection): + """$stdDevPop with Decimal128 -Infinity special value.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("-Infinity")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + collection.insert_many(docs) + extra_stages = [ + { + "$addFields": { + "has_result": {"$ne": ["$result", None]}, + } + }, + {"$project": {"_id": 1, "has_result": 1}}, + ] + pipeline = [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ] + extra_stages + result = execute_command( + collection, + {"aggregate": collection.name, "pipeline": pipeline, "cursor": {}}, + ) + expected = [ + {"_id": 1, "has_result": True}, + {"_id": 2, "has_result": True}, + {"_id": 3, "has_result": True}, + ] + assertSuccess(result, expected, msg="Decimal128 -Infinity produces a non-null result") + + +# Property [Basic Numeric]: standard numeric inputs handled correctly + + +def test_stdDevPop_all_identical_values_returns_zero(collection): + """$stdDevPop returns 0 when all values in the frame are identical.""" + docs = [ + {"_id": 1, "partition": "A", "value": 7.0}, + {"_id": 2, "partition": "A", "value": 7.0}, + {"_id": 3, "partition": "A", "value": 7.0}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 7.0, "result": 0.0}, + {"_id": 2, "partition": "A", "value": 7.0, "result": 0.0}, + {"_id": 3, "partition": "A", "value": 7.0, "result": 0.0}, + ] + assertSuccess(result, expected, msg="identical values produce stdDevPop = 0") + + +def test_stdDevPop_negative_numbers(collection): + """$stdDevPop handles negative numbers correctly.""" + docs = [ + {"_id": 1, "partition": "A", "value": -10}, + {"_id": 2, "partition": "A", "value": 0}, + {"_id": 3, "partition": "A", "value": 10}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": -10, "result": 8.16496580927726}, + {"_id": 2, "partition": "A", "value": 0, "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": 10, "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="negative numbers handled correctly") + + +def test_stdDevPop_decimals(collection): + """$stdDevPop handles floating-point (double) values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1.5}, + {"_id": 2, "partition": "A", "value": 2.5}, + {"_id": 3, "partition": "A", "value": 3.5}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 1.5, "result": 0.816496580927726}, + {"_id": 2, "partition": "A", "value": 2.5, "result": 0.816496580927726}, + {"_id": 3, "partition": "A", "value": 3.5, "result": 0.816496580927726}, + ] + assertSuccess(result, expected, msg="floating-point values handled correctly") + + +# Property [Negative Zero]: -0.0 treated as numeric zero + + +def test_stdDevPop_negative_zero(collection): + """$stdDevPop treats -0.0 as numeric zero — participates in computation.""" + docs = [ + {"_id": 1, "partition": "A", "value": DOUBLE_NEGATIVE_ZERO}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 20}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Values: [-0.0, 10, 20] -> mean=10, variance=(100+0+100)/3=66.67, stdDevPop=8.165 + expected = [ + {"_id": 1, "partition": "A", "value": DOUBLE_NEGATIVE_ZERO, "result": 8.16496580927726}, + {"_id": 2, "partition": "A", "value": 10, "result": 8.16496580927726}, + {"_id": 3, "partition": "A", "value": 20, "result": 8.16496580927726}, + ] + assertSuccess(result, expected, msg="-0.0 treated as numeric zero in stdDevPop") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_order_independence.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_order_independence.py new file mode 100644 index 000000000..5a28eec44 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_order_independence.py @@ -0,0 +1,89 @@ +""" +Tests for $stdDevPop order independence in window context. + +Verifies that $stdDevPop produces the same result regardless of sortBy direction, +confirming it is an order-independent operator per TEST_COVERAGE.md §22. +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + BASIC_DOCS, +) +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + +# Property [Order Independence]: $stdDevPop produces same result regardless of sort direction + + +def test_stdDevPop_whole_partition_ascending_sort(collection): + """$stdDevPop whole partition with ascending sort.""" + collection.insert_many(BASIC_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + {"$sort": {"_id": 1}}, + {"$project": {"_id": 1, "result": 1}}, + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "result": 14.142135623730951}, + {"_id": 2, "result": 14.142135623730951}, + {"_id": 3, "result": 14.142135623730951}, + {"_id": 4, "result": 14.142135623730951}, + {"_id": 5, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="ascending sort produces correct stdDevPop") + + +def test_stdDevPop_whole_partition_descending_sort(collection): + """$stdDevPop whole partition with descending sort produces same result as ascending.""" + collection.insert_many(BASIC_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": -1}, + "output": { + "result": { + "$stdDevPop": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + {"$sort": {"_id": 1}}, + {"$project": {"_id": 1, "result": 1}}, + ], + "cursor": {}, + }, + ) + # Same result regardless of sort direction — order-independent operator + expected = [ + {"_id": 1, "result": 14.142135623730951}, + {"_id": 2, "result": 14.142135623730951}, + {"_id": 3, "result": 14.142135623730951}, + {"_id": 4, "result": 14.142135623730951}, + {"_id": 5, "result": 14.142135623730951}, + ] + assertSuccess( + result, expected, msg="descending sort produces same stdDevPop — order independent" + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_special_floats.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_special_floats.py new file mode 100644 index 000000000..31fef9569 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevPop/test_window_stdDevPop_special_floats.py @@ -0,0 +1,260 @@ +""" +Tests for $stdDevPop with special float values (NaN, Infinity, -Infinity). + +Covers: Infinity as numeric participant, -Infinity, NaN values, +sliding window behavior with special floats, and non-removable window +behavior (NaN propagation). +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess, assertSuccessNaN +from documentdb_tests.framework.test_constants import ( + FLOAT_INFINITY, + FLOAT_NAN, + FLOAT_NEGATIVE_INFINITY, +) + +# Property [Infinity Non-Removable Window]: Infinity in non-removable windows +# (whole partition, cumulative) produces NaN + + +def test_stdDevPop_positive_infinity_whole_partition(collection): + """$stdDevPop with Infinity in non-removable whole partition window produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="Infinity is numeric; non-removable window produces NaN") + + +def test_stdDevPop_negative_infinity_whole_partition(collection): + """$stdDevPop with -Infinity in non-removable whole partition window produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="-Infinity is numeric; non-removable window produces NaN" + ) + + +def test_stdDevPop_inf_and_neg_inf_in_same_frame(collection): + """$stdDevPop with both Infinity and -Infinity produces NaN (Inf - Inf = NaN in variance).""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY}, + {"_id": 3, "partition": "A", "value": 10}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 10, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="Inf + -Inf in frame → NaN in variance calculation") + + +def test_stdDevPop_all_infinity_values(collection): + """$stdDevPop where all values are Infinity — identical Inf: mean=Inf, Inf-Inf=NaN.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "value": FLOAT_INFINITY}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="All identical Inf: mean=Inf, variance involves Inf-Inf=NaN" + ) + + +def test_stdDevPop_infinity_cumulative_window(collection): + """$stdDevPop with Infinity in cumulative [unbounded, current] produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "current"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": 10, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="Cumulative window is non-removable; Inf produces NaN throughout" + ) + + +def test_stdDevPop_single_infinity_value(collection): + """$stdDevPop with single Infinity value in whole partition produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="Single Inf value: stdDevPop involves Inf arithmetic → NaN" + ) + + +# Property [NaN Non-Removable Window]: NaN in non-removable windows produces NaN + + +def test_stdDevPop_nan_value_whole_partition(collection): + """$stdDevPop with NaN in non-removable window produces NaN (NaN is numeric, poisons).""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevPop", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_NAN, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="NaN is numeric; non-removable window produces NaN") + + +# Property [Special Floats Sliding Window]: special floats in removable/sliding windows return null + + +def test_stdDevPop_infinity_sliding_returns_null(collection): + """$stdDevPop sliding window returns null when Infinity is in the current frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 20}, + {"_id": 4, "partition": "A", "value": 30}, + {"_id": 5, "partition": "A", "value": 40}, + ] + result = run_window_operator(collection, "$stdDevPop", docs, {"documents": [-1, 0]}) + # Row 1: frame=[Inf] -> null (Inf in removable window) + # Row 2: frame=[Inf, 10] -> null (Inf in removable window) + # Row 3: frame=[10, 20] -> clean -> 5.0 + # Row 4: frame=[20, 30] -> clean -> 5.0 + # Row 5: frame=[30, 40] -> clean -> 5.0 + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": None}, + {"_id": 2, "partition": "A", "value": 10, "result": None}, + {"_id": 3, "partition": "A", "value": 20, "result": 5.0}, + {"_id": 4, "partition": "A", "value": 30, "result": 5.0}, + {"_id": 5, "partition": "A", "value": 40, "result": 5.0}, + ] + assertSuccess( + result, expected, msg="Sliding window: null when Inf in frame, recovers when Inf leaves" + ) + + +def test_stdDevPop_neg_infinity_sliding_returns_null(collection): + """$stdDevPop sliding window returns null when -Infinity is in the current frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 20}, + {"_id": 4, "partition": "A", "value": 30}, + ] + result = run_window_operator(collection, "$stdDevPop", docs, {"documents": [-1, 0]}) + # Row 1: frame=[-Inf] -> null (Inf in removable window) + # Row 2: frame=[-Inf, 10] -> null (Inf in removable window) + # Row 3: frame=[10, 20] -> 5.0 + # Row 4: frame=[20, 30] -> 5.0 + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY, "result": None}, + {"_id": 2, "partition": "A", "value": 10, "result": None}, + {"_id": 3, "partition": "A", "value": 20, "result": 5.0}, + {"_id": 4, "partition": "A", "value": 30, "result": 5.0}, + ] + assertSuccess( + result, expected, msg="Sliding window: null when -Inf in frame, recovers when -Inf leaves" + ) + + +def test_stdDevPop_nan_sliding_returns_null(collection): + """$stdDevPop sliding window returns null when NaN is in the current frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 20}, + {"_id": 4, "partition": "A", "value": 30}, + ] + result = run_window_operator(collection, "$stdDevPop", docs, {"documents": [-1, 0]}) + # Row 1: frame=[NaN] -> null (NaN in removable window) + # Row 2: frame=[NaN, 10] -> null (NaN in removable window) + # Row 3: frame=[10, 20] -> 5.0 + # Row 4: frame=[20, 30] -> 5.0 + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_NAN, "result": None}, + {"_id": 2, "partition": "A", "value": 10, "result": None}, + {"_id": 3, "partition": "A", "value": 20, "result": 5.0}, + {"_id": 4, "partition": "A", "value": 30, "result": 5.0}, + ] + assertSuccessNaN( + result, expected, msg="Sliding window: null when NaN in frame, recovers when NaN leaves" + ) + + +def test_stdDevPop_infinity_centered_sliding(collection): + """$stdDevPop centered sliding window [-1, 1] with Infinity in middle returns null.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 4, "partition": "A", "value": 30}, + {"_id": 5, "partition": "A", "value": 40}, + ] + result = run_window_operator(collection, "$stdDevPop", docs, {"documents": [-1, 1]}) + # Row 1: frame=[10, 20] -> 5.0 (clean, edge clamp) + # Row 2: frame=[10, 20, Inf] -> null (Inf in removable window) + # Row 3: frame=[20, Inf, 30] -> null (Inf in removable window) + # Row 4: frame=[Inf, 30, 40] -> null (Inf in removable window) + # Row 5: frame=[30, 40] -> 5.0 (clean, edge clamp) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 5.0}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + {"_id": 3, "partition": "A", "value": FLOAT_INFINITY, "result": None}, + {"_id": 4, "partition": "A", "value": 30, "result": None}, + {"_id": 5, "partition": "A", "value": 40, "result": 5.0}, + ] + assertSuccess( + result, + expected, + msg="Centered sliding: null when Inf in frame, clean rows compute correctly", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_argument_validation.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_argument_validation.py new file mode 100644 index 000000000..659dfad4b --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_argument_validation.py @@ -0,0 +1,508 @@ +""" +Tests for $stdDevSamp argument validation in window context. + +Covers: valid expression forms (field path, operator expression, literal), +non-numeric literal inputs that return null, invalid argument shapes that +produce parse errors (unknown keys in output field spec, multiple accumulators, +empty output field, multi-key expression objects, unknown operators in +expression, and field path with empty component). +""" + +from documentdb_tests.framework.assertions import assertFailureCode, assertSuccess +from documentdb_tests.framework.error_codes import ( + EXPRESSION_OBJECT_MULTIPLE_FIELDS_ERROR, + FAILED_TO_PARSE_ERROR, + FIELD_PATH_EMPTY_COMPONENT_ERROR, + UNRECOGNIZED_EXPRESSION_ERROR, +) +from documentdb_tests.framework.executor import execute_command + +TWO_DOCS = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, +] + +SINGLE_DOC = [{"_id": 1, "partition": "A", "value": 10}] + +# Property [Valid Expression Forms]: accepted expression inputs + + +def test_stdDevSamp_field_path_expression(collection): + """$stdDevSamp accepts a field path expression.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 7.0710678118654755}, + {"_id": 2, "partition": "A", "value": 20, "result": 7.0710678118654755}, + ] + assertSuccess(result, expected, msg="field path expression accepted") + + +def test_stdDevSamp_operator_expression(collection): + """$stdDevSamp accepts an operator expression.""" + docs = [ + {"_id": 1, "partition": "A", "value": 5}, + {"_id": 2, "partition": "A", "value": 10}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": {"$multiply": ["$value", 2]}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Values: [10, 20] -> stdDevSamp = sqrt(50) = 7.071 + expected = [ + {"_id": 1, "partition": "A", "value": 5, "result": 7.0710678118654755}, + {"_id": 2, "partition": "A", "value": 10, "result": 7.0710678118654755}, + ] + assertSuccess(result, expected, msg="operator expression accepted") + + +def test_stdDevSamp_literal_numeric_expression(collection): + """$stdDevSamp with a literal numeric value — all rows get same value, stddev is 0.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": 30}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": {"$literal": 42}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # All rows evaluate to 42 -> stdDevSamp = 0 + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 0.0}, + {"_id": 2, "partition": "A", "value": 20, "result": 0.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 0.0}, + ] + assertSuccess(result, expected, msg="literal numeric expression produces 0 stddev") + + +# Property [Non-Numeric Literal Inputs]: non-numeric constants return null (not errors) + + +def test_stdDevSamp_empty_string_expression(collection): + """$stdDevSamp with empty string (not a valid field path) — treated as non-numeric.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Empty string is a literal string constant, non-numeric -> all null + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": None}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + ] + assertSuccess(result, expected, msg="empty string treated as non-numeric constant") + + +def test_stdDevSamp_array_multiple_expressions_returns_null(collection): + """$stdDevSamp with array of multiple field paths in window context returns null.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": ["$x", "$y"], + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 20, "result": None}, + ] + assertSuccess(result, expected, msg="array of multiple expressions returns null in window form") + + +def test_stdDevSamp_boolean_literal_returns_null(collection): + """$stdDevSamp with boolean literal — non-numeric constant, returns null.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": True, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": None}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + ] + assertSuccess(result, expected, msg="boolean literal treated as non-numeric, returns null") + + +def test_stdDevSamp_null_literal_returns_null(collection): + """$stdDevSamp with null literal — returns null.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": None, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": None}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + ] + assertSuccess(result, expected, msg="null literal returns null") + + +def test_stdDevSamp_empty_object_returns_null(collection): + """$stdDevSamp with empty object {} — treated as non-numeric, returns null.""" + collection.insert_many(TWO_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": {}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": None}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + ] + assertSuccess(result, expected, msg="empty object treated as non-numeric, returns null") + + +# Property [Invalid Argument Shapes - Parse Errors]: inputs that produce errors at parse time + + +def test_stdDevSamp_unknown_key_in_output_field_errors(collection): + """Unknown key alongside $stdDevSamp in output field spec produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + "unknownKey": 1, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, FAILED_TO_PARSE_ERROR, msg="unknown key alongside $stdDevSamp rejected" + ) + + +def test_stdDevSamp_unknown_key_errors_on_empty_collection(collection): + """Parse-time error fires on empty collection — no documents needed.""" + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + "unknownKey": 1, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FAILED_TO_PARSE_ERROR, + msg="parse-time error fires on empty collection", + ) + + +def test_stdDevSamp_multiple_accumulators_in_output_field_errors(collection): + """Multiple accumulators in same output field spec produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$value", + "$sum": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FAILED_TO_PARSE_ERROR, + msg="multiple accumulators in output field rejected", + ) + + +def test_stdDevSamp_no_accumulator_in_output_field_errors(collection): + """Output field with no accumulator (only window key) produces parse error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode(result, FAILED_TO_PARSE_ERROR, msg="no accumulator in output field rejected") + + +def test_stdDevSamp_multi_key_expression_object_errors(collection): + """$stdDevSamp with multi-key expression object (ambiguous) produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": {"$add": [1, 2], "$subtract": [3, 1]}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + EXPRESSION_OBJECT_MULTIPLE_FIELDS_ERROR, + msg="multi-key expression object rejected", + ) + + +def test_stdDevSamp_unrecognized_expression_operator_errors(collection): + """$stdDevSamp with unrecognized expression operator produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": {"$unknownOp": "$value"}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + UNRECOGNIZED_EXPRESSION_ERROR, + msg="unrecognized expression operator rejected", + ) + + +def test_stdDevSamp_field_path_empty_component_errors(collection): + """$stdDevSamp with field path containing empty component produces error.""" + collection.insert_many(SINGLE_DOC) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$a..b", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + assertFailureCode( + result, + FIELD_PATH_EMPTY_COMPONENT_ERROR, + msg="field path with empty component rejected", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_field_paths.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_field_paths.py new file mode 100644 index 000000000..70c82458a --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_field_paths.py @@ -0,0 +1,409 @@ +""" +Tests for $stdDevSamp with nested field paths, array field traversal, +expressions that return different types per document, and $project +removing fields before $setWindowFields. +""" + +from datetime import datetime, timezone + +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + +# Property [Nested Field Paths]: dotted path accesses nested document values + + +def test_stdDevSamp_dotted_field_path(collection): + """$stdDevSamp with dotted field path accesses nested document value.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"metrics": {"value": 10}}}, + {"_id": 2, "partition": "A", "data": {"metrics": {"value": 20}}}, + {"_id": 3, "partition": "A", "data": {"metrics": {"value": 30}}}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$data.metrics.value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # stdDevSamp([10, 20, 30]) = 10.0 + expected = [ + {"_id": 1, "partition": "A", "data": {"metrics": {"value": 10}}, "result": 10.0}, + {"_id": 2, "partition": "A", "data": {"metrics": {"value": 20}}, "result": 10.0}, + {"_id": 3, "partition": "A", "data": {"metrics": {"value": 30}}, "result": 10.0}, + ] + assertSuccess(result, expected, msg="dotted field path accesses nested value") + + +# Property [Missing Path Handling]: missing intermediate or top-level fields are ignored + + +def test_stdDevSamp_missing_intermediate_path(collection): + """$stdDevSamp with missing intermediate path treated as missing (ignored).""" + docs = [ + {"_id": 1, "partition": "A", "data": {"metrics": {"value": 10}}}, + {"_id": 2, "partition": "A", "data": {"other": 99}}, + {"_id": 3, "partition": "A", "data": {"metrics": {"value": 30}}}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$data.metrics.value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Doc 2 missing data.metrics.value -> ignored. stdDevSamp([10, 30]) = 14.142135623730951 + expected = [ + { + "_id": 1, + "partition": "A", + "data": {"metrics": {"value": 10}}, + "result": 14.142135623730951, + }, + {"_id": 2, "partition": "A", "data": {"other": 99}, "result": 14.142135623730951}, + { + "_id": 3, + "partition": "A", + "data": {"metrics": {"value": 30}}, + "result": 14.142135623730951, + }, + ] + assertSuccess(result, expected, msg="missing intermediate path treated as missing") + + +def test_stdDevSamp_top_level_missing_object(collection): + """$stdDevSamp where the top-level field of a dotted path is missing.""" + docs = [ + {"_id": 1, "partition": "A", "data": {"value": 10}}, + {"_id": 2, "partition": "A"}, + {"_id": 3, "partition": "A", "data": {"value": 30}}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$data.value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Doc 2 missing "data" entirely -> ignored. stdDevSamp([10, 30]) = 14.142135623730951 + expected = [ + {"_id": 1, "partition": "A", "data": {"value": 10}, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "data": {"value": 30}, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="top-level field missing in dotted path = ignored") + + +# Property [Non-Numeric Field Types]: arrays, dates, strings are ignored + + +def test_stdDevSamp_array_field_is_non_numeric(collection): + """$stdDevSamp on a top-level array field — arrays are non-numeric, should be ignored.""" + docs = [ + {"_id": 1, "partition": "A", "value": [10, 20, 30]}, + {"_id": 2, "partition": "A", "value": 50}, + {"_id": 3, "partition": "A", "value": [40, 50]}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Only doc 2 has a numeric value. Single numeric value -> stdDevSamp = null (N=1, N-1=0) + expected = [ + {"_id": 1, "partition": "A", "value": [10, 20, 30], "result": None}, + {"_id": 2, "partition": "A", "value": 50, "result": None}, + {"_id": 3, "partition": "A", "value": [40, 50], "result": None}, + ] + assertSuccess( + result, expected, msg="array field values are non-numeric — only 1 numeric gives null" + ) + + +def test_stdDevSamp_date_value_as_expression_ignored(collection): + """$stdDevSamp with Date value in expression field — non-numeric, ignored.""" + docs = [ + {"_id": 1, "partition": "A", "value": datetime(2023, 1, 1, tzinfo=timezone.utc)}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": datetime(2023, 6, 1, tzinfo=timezone.utc)}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # Only doc 2 is numeric -> single value -> stdDevSamp = null (N=1, N-1=0) + expected = [ + { + "_id": 1, + "partition": "A", + "value": datetime(2023, 1, 1, tzinfo=timezone.utc), + "result": None, + }, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + { + "_id": 3, + "partition": "A", + "value": datetime(2023, 6, 1, tzinfo=timezone.utc), + "result": None, + }, + ] + assertSuccess( + result, + expected, + msg="Date values in expression field are non-numeric — only 1 numeric gives null", + ) + + +# Property [Expression-Based Input]: computed expressions as stdDevSamp input + + +def test_stdDevSamp_expression_returns_different_types(collection): + """$stdDevSamp expression returning different types per row — non-numeric ignored.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10}, + {"_id": 2, "partition": "A", "x": -5}, + {"_id": 3, "partition": "A", "x": 30}, + {"_id": 4, "partition": "A", "x": -1}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": { + "$cond": [ + {"$gt": ["$x", 0]}, + "$x", + "not_a_number", + ] + }, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # $cond returns: 10, "not_a_number", 30, "not_a_number" + # Only numeric: [10, 30] -> stdDevSamp = 14.142135623730951 + expected = [ + {"_id": 1, "partition": "A", "x": 10, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "x": -5, "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "x": 30, "result": 14.142135623730951}, + {"_id": 4, "partition": "A", "x": -1, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="expression returning mixed types — non-numeric ignored") + + +def test_stdDevSamp_expression_returns_null_for_some(collection): + """$stdDevSamp expression returning null for some docs, number for others.""" + docs = [ + {"_id": 1, "partition": "A", "x": 10, "y": 2}, + {"_id": 2, "partition": "A", "x": 20, "y": None}, + {"_id": 3, "partition": "A", "x": 30, "y": 2}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": {"$multiply": ["$x", "$y"]}, + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + } + ], + "cursor": {}, + }, + ) + # $multiply: [10*2=20, 20*null=null, 30*2=60] + # Numeric values: [20, 60] -> stdDevSamp = sqrt(((20-40)^2+(60-40)^2)/1) = sqrt(800) = + expected = [ + {"_id": 1, "partition": "A", "x": 10, "y": 2, "result": 28.284271247461902}, + {"_id": 2, "partition": "A", "x": 20, "y": None, "result": 28.284271247461902}, + {"_id": 3, "partition": "A", "x": 30, "y": 2, "result": 28.284271247461902}, + ] + assertSuccess(result, expected, msg="expression returning null for some — null results ignored") + + +# Property [Pipeline Stage Interaction]: $project/$unset/$match before $setWindowFields + + +def test_stdDevSamp_project_removes_expression_field(collection): + """$project removing the expression field before $setWindowFields — treated as missing.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10, "extra": 99}, + {"_id": 2, "partition": "A", "value": 20, "extra": 99}, + {"_id": 3, "partition": "A", "value": 30, "extra": 99}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + {"$project": {"partition": 1, "extra": 1}}, + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + ], + "cursor": {}, + }, + ) + # All docs missing "value" -> all non-numeric -> result is null + expected = [ + {"_id": 1, "partition": "A", "extra": 99, "result": None}, + {"_id": 2, "partition": "A", "extra": 99, "result": None}, + {"_id": 3, "partition": "A", "extra": 99, "result": None}, + ] + assertSuccess(result, expected, msg="$project removing expression field -> all null results") + + +# Property [Numeric Path Component]: +# Tests that numeric path components access array elements or object keys. + + +def test_stdDevSamp_numeric_path_component(collection): + """$stdDevSamp with numeric path component accesses array element or object key.""" + docs = [ + {"_id": 1, "partition": "A", "arr": [{"field": 10}, {"field": 20}]}, + {"_id": 2, "partition": "A", "arr": [{"field": 30}, {"field": 40}]}, + {"_id": 3, "partition": "A", "arr": [{"field": 50}, {"field": 60}]}, + ] + collection.insert_many(docs) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$arr.0.field", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + {"$project": {"_id": 1, "result": 1}}, + ], + "cursor": {}, + }, + ) + # In $setWindowFields context, $arr.0.field does not resolve to array element — + # the path returns non-numeric (array) values which are ignored, resulting in null. + expected = [ + {"_id": 1, "result": None}, + {"_id": 2, "result": None}, + {"_id": 3, "result": None}, + ] + assertSuccess(result, expected, msg="numeric path component in window context returns null") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_frame_computation.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_frame_computation.py new file mode 100644 index 000000000..b390b601c --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_frame_computation.py @@ -0,0 +1,92 @@ +""" +Tests for $stdDevSamp computation under documents-mode window frame shapes. + +Verifies the operator computes correct results given the 4 defined frame shapes: +whole-partition, cumulative, reverse-cumulative, and sliding. + +Key difference from $stdDevPop: single-element frames return null (N-1 denominator +is undefined for N=1). + +Note: Stage-level frame boundary tests (under stages/setWindowFields/) verify +that the correct documents are selected into the frame (centered, trailing, +leading, non-overlapping, edge cases). These per-operator tests verify the +operator produces correct values given those documents. +""" + +import pytest + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + BASIC_DOCS, + WindowTestCase, + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.parametrize import pytest_params + +STDDEVSAMP_DOCUMENTS_FRAME_TESTS: list[WindowTestCase] = [ + # Property [Whole Partition]: unbounded-unbounded frame covers entire partition + WindowTestCase( + "whole_partition", + docs=BASIC_DOCS, + window={"documents": ["unbounded", "unbounded"]}, + expected=[ + {"_id": 1, "partition": "A", "value": 10, "result": 15.811388300841896}, + {"_id": 2, "partition": "A", "value": 20, "result": 15.811388300841896}, + {"_id": 3, "partition": "A", "value": 30, "result": 15.811388300841896}, + {"_id": 4, "partition": "A", "value": 40, "result": 15.811388300841896}, + {"_id": 5, "partition": "A", "value": 50, "result": 15.811388300841896}, + ], + msg="whole partition stdDevSamp = sqrt(250)", + ), + # Property [Cumulative Frame]: expanding frame from start to current + WindowTestCase( + "cumulative", + docs=BASIC_DOCS, + window={"documents": ["unbounded", "current"]}, + expected=[ + {"_id": 1, "partition": "A", "value": 10, "result": None}, + {"_id": 2, "partition": "A", "value": 20, "result": 7.0710678118654755}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + {"_id": 4, "partition": "A", "value": 40, "result": 12.909944487358056}, + {"_id": 5, "partition": "A", "value": 50, "result": 15.811388300841896}, + ], + msg="cumulative stdDevSamp grows; first row is null (N=1 undefined)", + ), + # Property [Reverse Cumulative Frame]: shrinking frame from current to end + WindowTestCase( + "reverse_cumulative", + docs=BASIC_DOCS, + window={"documents": ["current", "unbounded"]}, + expected=[ + {"_id": 1, "partition": "A", "value": 10, "result": 15.811388300841896}, + {"_id": 2, "partition": "A", "value": 20, "result": 12.909944487358056}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + {"_id": 4, "partition": "A", "value": 40, "result": 7.0710678118654755}, + {"_id": 5, "partition": "A", "value": 50, "result": None}, + ], + msg="reverse-cumulative stdDevSamp shrinks; last row is null (N=1 undefined)", + ), + # Property [Sliding Frame]: fixed-size window that moves with current row + WindowTestCase( + "sliding_centered", + docs=BASIC_DOCS, + window={"documents": [-1, 1]}, + expected=[ + {"_id": 1, "partition": "A", "value": 10, "result": 7.0710678118654755}, + {"_id": 2, "partition": "A", "value": 20, "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + {"_id": 4, "partition": "A", "value": 40, "result": 10.0}, + {"_id": 5, "partition": "A", "value": 50, "result": 7.0710678118654755}, + ], + msg="centered sliding window [-1,1]; edge rows have 2 elements", + ), +] + + +@pytest.mark.parametrize("test", pytest_params(STDDEVSAMP_DOCUMENTS_FRAME_TESTS)) +def test_stdDevSamp_documents_frames(collection, test): + """$stdDevSamp with various documents-mode window frames.""" + result = run_window_operator( + collection, "$stdDevSamp", test.docs, test.window, sort_by=test.sort_by + ) + assertSuccess(result, test.expected, msg=test.msg) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_non_numeric_handling.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_non_numeric_handling.py new file mode 100644 index 000000000..8dc179c75 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_non_numeric_handling.py @@ -0,0 +1,412 @@ +""" +Tests for $stdDevSamp null, missing, and non-numeric value handling. + +Covers: null values, missing fields, strings, booleans, arrays, objects, +ObjectId, Regex, Binary, Timestamp, MinKey, MaxKey, mixed numeric and +non-numeric in same frame, and all non-numeric returns null. +""" + +from datetime import datetime, timezone + +from bson import Binary, MaxKey, MinKey, ObjectId, Regex, Timestamp + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess + +# Property [Null and Missing]: null and missing field values are ignored + + +def test_stdDevSamp_null_values_ignored(collection): + """$stdDevSamp ignores null values in the expression field.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": None}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # numerics [10, 30] → stdDevSamp = sqrt(((10-20)^2+(30-20)^2)/1) = sqrt(200) = 14.142 + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "value": None, "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "value": 30, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="null values ignored, stdDevSamp of 10,30 = 14.142") + + +def test_stdDevSamp_missing_field_ignored(collection): + """$stdDevSamp ignores documents where the expression field is missing.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A"}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "value": 30, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="missing field ignored, stdDevSamp of 10,30 = 14.142") + + +# Property [Non-Numeric Types Ignored]: string, boolean, array, object, date, +# ObjectId, Regex, Binary values are ignored + + +def test_stdDevSamp_string_values_ignored(collection): + """$stdDevSamp ignores string values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": "hello"}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "value": "hello", "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "value": 30, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="string values ignored") + + +def test_stdDevSamp_boolean_values_ignored(collection): + """$stdDevSamp ignores boolean values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": True}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "value": True, "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "value": 30, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="boolean values ignored") + + +def test_stdDevSamp_array_values_ignored(collection): + """$stdDevSamp ignores array values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": [1, 2, 3]}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "value": [1, 2, 3], "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "value": 30, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="array values ignored") + + +def test_stdDevSamp_object_values_ignored(collection): + """$stdDevSamp ignores object/document values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": {"nested": 99}}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "value": {"nested": 99}, "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "value": 30, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="object values ignored") + + +def test_stdDevSamp_objectid_and_regex_and_binary_ignored(collection): + """$stdDevSamp ignores ObjectId, Regex, and Binary values.""" + oid = ObjectId("507f1f77bcf86cd799439011") + docs = [ + {"_id": 1, "partition": "A", "value": oid}, + {"_id": 2, "partition": "A", "value": Regex("^test", "i")}, + {"_id": 3, "partition": "A", "value": Binary(b"\x01\x02\x03")}, + {"_id": 4, "partition": "A", "value": 10}, + {"_id": 5, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, + "$stdDevSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # Only numeric: [10, 30] → stdDevSamp = sqrt(200/1) = 14.142 + expected = [ + {"_id": 1, "result": 14.142135623730951}, + {"_id": 2, "result": 14.142135623730951}, + {"_id": 3, "result": 14.142135623730951}, + {"_id": 4, "result": 14.142135623730951}, + {"_id": 5, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="ObjectId/Regex/Binary values ignored") + + +def test_stdDevSamp_timestamp_minkey_maxkey_ignored(collection): + """$stdDevSamp ignores Timestamp, MinKey, and MaxKey values.""" + docs = [ + {"_id": 1, "partition": "A", "value": Timestamp(1234567890, 1)}, + {"_id": 2, "partition": "A", "value": MinKey()}, + {"_id": 3, "partition": "A", "value": MaxKey()}, + {"_id": 4, "partition": "A", "value": 10}, + {"_id": 5, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, + "$stdDevSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # Only numeric: [10, 30] -> stdDevSamp = sqrt(200/1) = 14.142 + expected = [ + {"_id": 1, "result": 14.142135623730951}, + {"_id": 2, "result": 14.142135623730951}, + {"_id": 3, "result": 14.142135623730951}, + {"_id": 4, "result": 14.142135623730951}, + {"_id": 5, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="Timestamp/MinKey/MaxKey values ignored") + + +# Property [All Non-Numeric Returns Null]: when all values are non-numeric, result is null + + +def test_stdDevSamp_all_non_numeric_returns_null(collection): + """$stdDevSamp returns null when all values in frame are non-numeric.""" + docs = [ + {"_id": 1, "partition": "A", "value": "a"}, + {"_id": 2, "partition": "A", "value": None}, + {"_id": 3, "partition": "A"}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": "a", "result": None}, + {"_id": 2, "partition": "A", "value": None, "result": None}, + {"_id": 3, "partition": "A", "result": None}, + ] + assertSuccess(result, expected, msg="all non-numeric in frame returns null") + + +def test_stdDevSamp_all_non_numeric_diverse_types(collection): + """$stdDevSamp returns null when all values are diverse non-numeric types.""" + docs = [ + {"_id": 1, "partition": "A", "value": "text"}, + {"_id": 2, "partition": "A", "value": True}, + {"_id": 3, "partition": "A", "value": datetime(2023, 1, 1, tzinfo=timezone.utc)}, + {"_id": 4, "partition": "A", "value": [1, 2]}, + {"_id": 5, "partition": "A", "value": {"a": 1}}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": "text", "result": None}, + {"_id": 2, "partition": "A", "value": True, "result": None}, + { + "_id": 3, + "partition": "A", + "value": datetime(2023, 1, 1, tzinfo=timezone.utc), + "result": None, + }, + {"_id": 4, "partition": "A", "value": [1, 2], "result": None}, + {"_id": 5, "partition": "A", "value": {"a": 1}, "result": None}, + ] + assertSuccess(result, expected, msg="all diverse non-numeric types return null") + + +def test_stdDevSamp_single_numeric_in_frame_returns_null(collection): + """$stdDevSamp returns null when only one numeric value exists in frame (N-1=0).""" + docs = [ + {"_id": 1, "partition": "A", "value": "text"}, + {"_id": 2, "partition": "A", "value": None}, + {"_id": 3, "partition": "A", "value": 42}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Only one numeric (42) in the whole partition → stdDevSamp = null (N-1=0) + expected = [ + {"_id": 1, "partition": "A", "value": "text", "result": None}, + {"_id": 2, "partition": "A", "value": None, "result": None}, + {"_id": 3, "partition": "A", "value": 42, "result": None}, + ] + assertSuccess(result, expected, msg="single numeric value → stdDevSamp = null (N-1=0)") + + +# Property [Mixed Types in Frame]: non-numeric values filtered per-frame, numerics participate + + +def test_stdDevSamp_mixed_numeric_non_numeric_sliding(collection): + """$stdDevSamp in sliding window with mix of numeric and non-numeric values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10.0}, + {"_id": 2, "partition": "A", "value": "skip"}, + {"_id": 3, "partition": "A", "value": 20.0}, + {"_id": 4, "partition": "A", "value": None}, + {"_id": 5, "partition": "A", "value": 30.0}, + ] + result = run_window_operator(collection, "$stdDevSamp", docs, {"documents": [-2, 2]}) + # Window [-2, 2] (5-doc centered): + # Row 1: frame [10.0, "skip", 20.0] → numerics [10, 20] → stdDevSamp = 7.071 + # Row 2: frame [10.0, "skip", 20.0, None] → numerics [10, 20] → 7.071 + # Row 3: frame [10.0, "skip", 20.0, None, 30.0] → numerics [10, 20, 30] → 10.0 + # Row 4: frame ["skip", 20.0, None, 30.0] → numerics [20, 30] → 7.071 + # Row 5: frame [20.0, None, 30.0] → numerics [20, 30] → 7.071 + expected = [ + {"_id": 1, "partition": "A", "value": 10.0, "result": 7.0710678118654755}, + {"_id": 2, "partition": "A", "value": "skip", "result": 7.0710678118654755}, + {"_id": 3, "partition": "A", "value": 20.0, "result": 10.0}, + {"_id": 4, "partition": "A", "value": None, "result": 7.0710678118654755}, + {"_id": 5, "partition": "A", "value": 30.0, "result": 7.0710678118654755}, + ] + assertSuccess(result, expected, msg="mixed types in sliding window — non-numeric ignored") + + +def test_stdDevSamp_mixed_types_in_documents(collection): + """$stdDevSamp over documents with diverse value types — only numerics participate.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": "hello"}, + {"_id": 3, "partition": "A", "value": True}, + {"_id": 4, "partition": "A", "value": datetime(2023, 6, 1, tzinfo=timezone.utc)}, + {"_id": 5, "partition": "A", "value": 30}, + {"_id": 6, "partition": "A", "value": None}, + {"_id": 7, "partition": "A", "value": [1, 2, 3]}, + {"_id": 8, "partition": "A", "value": {"nested": 99}}, + {"_id": 9, "partition": "A", "value": 50}, + ] + result = run_window_operator( + collection, + "$stdDevSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[{"$project": {"_id": 1, "result": 1}}], + ) + # Only numeric: [10, 30, 50] → mean=30, var_samp=(400+0+400)/2=400, stdDevSamp=20.0 + expected = [ + {"_id": 1, "result": 20.0}, + {"_id": 2, "result": 20.0}, + {"_id": 3, "result": 20.0}, + {"_id": 4, "result": 20.0}, + {"_id": 5, "result": 20.0}, + {"_id": 6, "result": 20.0}, + {"_id": 7, "result": 20.0}, + {"_id": 8, "result": 20.0}, + {"_id": 9, "result": 20.0}, + ] + assertSuccess(result, expected, msg="diverse non-numeric types ignored in whole partition") + + +def test_stdDevSamp_mixed_types_sliding_window(collection): + """$stdDevSamp sliding window over documents with diverse types — per-frame filtering.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10.0}, + {"_id": 2, "partition": "A", "value": 20.0}, + {"_id": 3, "partition": "A", "value": "skip"}, + {"_id": 4, "partition": "A", "value": True}, + {"_id": 5, "partition": "A", "value": 30.0}, + {"_id": 6, "partition": "A", "value": datetime(2023, 1, 1, tzinfo=timezone.utc)}, + {"_id": 7, "partition": "A", "value": 40.0}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Whole partition: numerics [10, 20, 30, 40] → mean=25 + # var_samp = (225+25+25+225)/3 = 500/3 = 166.67, stdDevSamp = 12.9099... + expected = [ + {"_id": 1, "partition": "A", "value": 10.0, "result": 12.909944487358056}, + {"_id": 2, "partition": "A", "value": 20.0, "result": 12.909944487358056}, + {"_id": 3, "partition": "A", "value": "skip", "result": 12.909944487358056}, + {"_id": 4, "partition": "A", "value": True, "result": 12.909944487358056}, + {"_id": 5, "partition": "A", "value": 30.0, "result": 12.909944487358056}, + { + "_id": 6, + "partition": "A", + "value": datetime(2023, 1, 1, tzinfo=timezone.utc), + "result": 12.909944487358056, + }, + {"_id": 7, "partition": "A", "value": 40.0, "result": 12.909944487358056}, + ] + assertSuccess(result, expected, msg="whole partition filters non-numeric types") + + +def test_stdDevSamp_numeric_among_diverse_types_cumulative(collection): + """$stdDevSamp cumulative window with numerics scattered among diverse types.""" + docs = [ + {"_id": 1, "partition": "A", "value": "text"}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": datetime(2023, 6, 1, tzinfo=timezone.utc)}, + {"_id": 4, "partition": "A", "value": 20}, + {"_id": 5, "partition": "A", "value": True}, + {"_id": 6, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "current"]} + ) + # Cumulative, only numerics count: + # Row 1: no numeric → null + # Row 2: [10] → null (N=1, N-1=0) + # Row 3: [10] → null (datetime ignored, still one numeric) + # Row 4: [10, 20] → stdDevSamp = 7.071 + # Row 5: [10, 20] → 7.071 (True ignored) + # Row 6: [10, 20, 30] → 10.0 + expected = [ + {"_id": 1, "partition": "A", "value": "text", "result": None}, + {"_id": 2, "partition": "A", "value": 10, "result": None}, + { + "_id": 3, + "partition": "A", + "value": datetime(2023, 6, 1, tzinfo=timezone.utc), + "result": None, + }, + {"_id": 4, "partition": "A", "value": 20, "result": 7.0710678118654755}, + {"_id": 5, "partition": "A", "value": True, "result": 7.0710678118654755}, + {"_id": 6, "partition": "A", "value": 30, "result": 10.0}, + ] + assertSuccess(result, expected, msg="cumulative window with numerics among diverse types") + + +def test_stdDevSamp_numerics_among_non_numeric_sliding(collection): + """$stdDevSamp sliding frame filters non-numerics and computes on remaining values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10.0}, + {"_id": 2, "partition": "A", "value": "a"}, + {"_id": 3, "partition": "A", "value": 30.0}, + {"_id": 4, "partition": "A", "value": "b"}, + {"_id": 5, "partition": "A", "value": 50.0}, + ] + result = run_window_operator(collection, "$stdDevSamp", docs, {"documents": [-2, 2]}) + # Window [-2, 2] (5-doc centered): + # Row 1: numerics [10, 30] → stdDevSamp = sqrt(200/1) = 14.142 + # Row 2: numerics [10, 30] → 14.142 + # Row 3: numerics [10, 30, 50] → sqrt(800/2) = 20.0 + # Row 4: numerics [30, 50] → sqrt(200/1) = 14.142 + # Row 5: numerics [30, 50] → 14.142 + expected = [ + {"_id": 1, "partition": "A", "value": 10.0, "result": 14.142135623730951}, + {"_id": 2, "partition": "A", "value": "a", "result": 14.142135623730951}, + {"_id": 3, "partition": "A", "value": 30.0, "result": 20.0}, + {"_id": 4, "partition": "A", "value": "b", "result": 14.142135623730951}, + {"_id": 5, "partition": "A", "value": 50.0, "result": 14.142135623730951}, + ] + assertSuccess(result, expected, msg="non-numeric values filtered in sliding frame") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_numeric_precision.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_numeric_precision.py new file mode 100644 index 000000000..b1c09db1d --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_numeric_precision.py @@ -0,0 +1,656 @@ +""" +Tests for $stdDevSamp numeric type mixing, overflow edge cases, +algorithmic precision validation, and Decimal128 type handling. +""" + +from bson import Decimal128, Int64 + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertResult, assertSuccess, assertSuccessNaN +from documentdb_tests.framework.property_checks import Gt, Lte, PerDoc +from documentdb_tests.framework.test_constants import DOUBLE_NEGATIVE_ZERO, FLOAT_NAN + +# Property [Numeric Type Mixing]: Int32, Int64, Double coexist correctly + + +def test_stdDevSamp_all_int32_values(collection): + """$stdDevSamp with all Int32 values produces correct Double result.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "value": 20, "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30, "result": 10.0}, + ] + assertSuccess(result, expected, msg="all Int32 values produce correct Double result") + + +def test_stdDevSamp_all_int64_values(collection): + """$stdDevSamp with all Int64 values produces correct result.""" + docs = [ + {"_id": 1, "partition": "A", "value": Int64(10)}, + {"_id": 2, "partition": "A", "value": Int64(20)}, + {"_id": 3, "partition": "A", "value": Int64(30)}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Int64(10), "result": 10.0}, + {"_id": 2, "partition": "A", "value": Int64(20), "result": 10.0}, + {"_id": 3, "partition": "A", "value": Int64(30), "result": 10.0}, + ] + assertSuccess(result, expected, msg="all Int64 values compute correctly") + + +def test_stdDevSamp_mixed_int32_int64_double(collection): + """$stdDevSamp with mixed Int32 + Int64 + Double in same frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": Int64(20)}, + {"_id": 3, "partition": "A", "value": 30.0}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 10.0}, + {"_id": 2, "partition": "A", "value": Int64(20), "result": 10.0}, + {"_id": 3, "partition": "A", "value": 30.0, "result": 10.0}, + ] + assertSuccess(result, expected, msg="mixed Int32 + Int64 + Double type promotion works") + + +# Property [Large Value Handling]: near-overflow and large-spread values compute without overflow + + +def test_stdDevSamp_large_int64_near_max(collection): + """$stdDevSamp with Int64 values near MAX_LONG — squaring would overflow 64-bit.""" + docs = [ + {"_id": 1, "partition": "A", "value": Int64(9223372036854775806)}, + {"_id": 2, "partition": "A", "value": Int64(9223372036854775807)}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Both values round to the same float64 (2^63-1 and 2^63-2 differ by 1 ULP at this scale), + # so the server sees them as identical -> stdDevSamp = 0.0 + expected = [ + {"_id": 1, "partition": "A", "value": Int64(9223372036854775806), "result": 0.0}, + {"_id": 2, "partition": "A", "value": Int64(9223372036854775807), "result": 0.0}, + ] + assertSuccess(result, expected, msg="Int64 near MAX_LONG does not overflow") + + +def test_stdDevSamp_large_int64_spread(collection): + """$stdDevSamp with widely spread Int64 values — tests numeric stability.""" + docs = [ + {"_id": 1, "partition": "A", "value": Int64(0)}, + {"_id": 2, "partition": "A", "value": Int64(4611686018427387903)}, + {"_id": 3, "partition": "A", "value": Int64(9223372036854775807)}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Verify 3 docs returned, each with a positive result (exact value depends on precision) + checks = PerDoc( + {"result": Gt(0)}, + {"result": Gt(0)}, + {"result": Gt(0)}, + ) + assertResult(result, expected=checks, msg="Large Int64 spread produces positive result") + + +def test_stdDevSamp_int32_sum_of_squares_overflow(collection): + """$stdDevSamp with Int32 values where sum-of-squares would overflow Int32.""" + # Max Int32 = 2147483647. Values around 50000: 50000^2 = 2.5e9 > 2^31 + docs = [ + {"_id": 1, "partition": "A", "value": 50000}, + {"_id": 2, "partition": "A", "value": 50001}, + {"_id": 3, "partition": "A", "value": 50002}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # stdDevSamp of [N, N+1, N+2]: mean=N+1, sum_sq=2, var_samp=2/2=1, stdDevSamp=1.0 + expected = [ + {"_id": 1, "partition": "A", "value": 50000, "result": 1.0}, + {"_id": 2, "partition": "A", "value": 50001, "result": 1.0}, + {"_id": 3, "partition": "A", "value": 50002, "result": 1.0}, + ] + assertSuccess(result, expected, msg="Int32 values that would overflow Int32 sum-of-squares") + + +def test_stdDevSamp_very_large_value(collection): + """$stdDevSamp with very large numeric value (1e308) — all identical returns 0.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1e308}, + {"_id": 2, "partition": "A", "value": 1e308}, + {"_id": 3, "partition": "A", "value": 1e308}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 1e308, "result": 0.0}, + {"_id": 2, "partition": "A", "value": 1e308, "result": 0.0}, + {"_id": 3, "partition": "A", "value": 1e308, "result": 0.0}, + ] + assertSuccess(result, expected, msg="very large identical values return 0") + + +def test_stdDevSamp_alternating_large_values(collection): + """$stdDevSamp with alternating sign large values — stress variance accumulator.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1e15}, + {"_id": 2, "partition": "A", "value": -1e15}, + {"_id": 3, "partition": "A", "value": 1e15}, + {"_id": 4, "partition": "A", "value": -1e15}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # mean=0, sum_sq=4e30, var_samp=4e30/3, stdDevSamp=sqrt(4e30/3)=2e15/sqrt(3) + expected = [ + {"_id": 1, "partition": "A", "value": 1e15, "result": 1.1547005383792515e15}, + {"_id": 2, "partition": "A", "value": -1e15, "result": 1.1547005383792515e15}, + {"_id": 3, "partition": "A", "value": 1e15, "result": 1.1547005383792515e15}, + {"_id": 4, "partition": "A", "value": -1e15, "result": 1.1547005383792515e15}, + ] + assertSuccess(result, expected, msg="alternating large values produce correct stddev") + + +# Property [Algorithmic Precision]: known exact results and catastrophic cancellation handling + + +def test_stdDevSamp_known_exact_result(collection): + """$stdDevSamp of [2,4,4,4,5,5,7,9] = sqrt(32/7) = 2.138089935299395.""" + docs = [ + {"_id": i, "partition": "A", "value": v} for i, v in enumerate([2, 4, 4, 4, 5, 5, 7, 9], 1) + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # mean=5, sum_sq=32, var_samp=32/7, stdDevSamp=sqrt(32/7)=2.138089935299395 + expected = [ + {"_id": i, "partition": "A", "value": v, "result": 2.138089935299395} + for i, v in enumerate([2, 4, 4, 4, 5, 5, 7, 9], 1) + ] + assertSuccess(result, expected, msg="stdDevSamp of [2,4,4,4,5,5,7,9] must be sqrt(32/7)") + + +def test_stdDevSamp_identical_floats_exactly_zero(collection): + """$stdDevSamp of identical float values must be exactly 0.0, not epsilon.""" + docs = [ + {"_id": 1, "partition": "A", "value": 3.0}, + {"_id": 2, "partition": "A", "value": 3.0}, + {"_id": 3, "partition": "A", "value": 3.0}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 3.0, "result": 0.0}, + {"_id": 2, "partition": "A", "value": 3.0, "result": 0.0}, + {"_id": 3, "partition": "A", "value": 3.0, "result": 0.0}, + ] + assertSuccess(result, expected, msg="identical floats produce exactly 0.0") + + +def test_stdDevSamp_catastrophic_cancellation(collection): + """$stdDevSamp of [1000000001, 1000000002, 1000000003] — naive sum-of-squares fails.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1000000001}, + {"_id": 2, "partition": "A", "value": 1000000002}, + {"_id": 3, "partition": "A", "value": 1000000003}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # stdDevSamp of [N, N+1, N+2]: mean=N+1, sum_sq=2, var_samp=2/2=1, stdDevSamp=1.0 + expected = [ + {"_id": 1, "partition": "A", "value": 1000000001, "result": 1.0}, + {"_id": 2, "partition": "A", "value": 1000000002, "result": 1.0}, + {"_id": 3, "partition": "A", "value": 1000000003, "result": 1.0}, + ] + assertSuccess( + result, + expected, + msg="catastrophic cancellation handled — correct stddev for large offset values", + ) + + +def test_stdDevSamp_very_small_differences(collection): + """$stdDevSamp with values that differ by very small amounts.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1.0000001}, + {"_id": 2, "partition": "A", "value": 1.0000002}, + {"_id": 3, "partition": "A", "value": 1.0000003}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # The stdDevSamp should be approximately 1e-7 — verify positive and tiny + checks = PerDoc( + {"result": [Gt(0), Lte(0.001)]}, + {"result": [Gt(0), Lte(0.001)]}, + {"result": [Gt(0), Lte(0.001)]}, + ) + assertResult( + result, expected=checks, msg="Small differences produce very small positive stddev" + ) + + +def test_stdDevSamp_sequential_values_1_to_10(collection): + """$stdDevSamp over sequential values 1..10 with known exact result.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1}, + {"_id": 2, "partition": "A", "value": 2}, + {"_id": 3, "partition": "A", "value": 3}, + {"_id": 4, "partition": "A", "value": 4}, + {"_id": 5, "partition": "A", "value": 5}, + {"_id": 6, "partition": "A", "value": 6}, + {"_id": 7, "partition": "A", "value": 7}, + {"_id": 8, "partition": "A", "value": 8}, + {"_id": 9, "partition": "A", "value": 9}, + {"_id": 10, "partition": "A", "value": 10}, + ] + result = run_window_operator( + collection, + "$stdDevSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[{"$match": {"_id": 1}}], + ) + # stdDevSamp of 1..10: mean=5.5, sum_sq=82.5, var_samp=82.5/9, stdDevSamp=sqrt(82.5/9) + expected = [ + {"_id": 1, "partition": "A", "value": 1, "result": 3.0276503540974917}, + ] + assertSuccess(result, expected, msg="sequential 1..10 known stdDevSamp result") + + +# Property [Single Element Frame]: single value produces null for sample stddev (N-1=0) + + +def test_stdDevSamp_single_element_sliding_window(collection): + """$stdDevSamp returns null when sliding window frame has exactly one value.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10.0}, + {"_id": 2, "partition": "A", "value": 20.0}, + {"_id": 3, "partition": "A", "value": 30.0}, + {"_id": 4, "partition": "A", "value": 40.0}, + {"_id": 5, "partition": "A", "value": 50.0}, + ] + result = run_window_operator(collection, "$stdDevSamp", docs, {"documents": [0, 0]}) + # Window [0, 0] — each frame has exactly one value -> stdDevSamp = null (N-1 = 0) + expected = [ + {"_id": 1, "partition": "A", "value": 10.0, "result": None}, + {"_id": 2, "partition": "A", "value": 20.0, "result": None}, + {"_id": 3, "partition": "A", "value": 30.0, "result": None}, + {"_id": 4, "partition": "A", "value": 40.0, "result": None}, + {"_id": 5, "partition": "A", "value": 50.0, "result": None}, + ] + assertSuccess(result, expected, msg="single element in sliding frame returns null") + + +# Property [Decimal128 Support]: Decimal128 values and type mixing with other numerics + + +def test_stdDevSamp_pure_decimal128_values(collection): + """$stdDevSamp with pure Decimal128 values computes correctly.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("20")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 10.0}, + {"_id": 2, "partition": "A", "value": Decimal128("20"), "result": 10.0}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 10.0}, + ] + assertSuccess(result, expected, msg="pure Decimal128 values compute stdDevSamp correctly") + + +def test_stdDevSamp_decimal128_with_double(collection): + """$stdDevSamp with mixed Decimal128 and Double values.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": 20.0}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 10.0}, + {"_id": 2, "partition": "A", "value": 20.0, "result": 10.0}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 10.0}, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Double compute correctly") + + +def test_stdDevSamp_decimal128_with_int32(collection): + """$stdDevSamp with mixed Decimal128 and Int32 values.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 10.0}, + {"_id": 2, "partition": "A", "value": 20, "result": 10.0}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 10.0}, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Int32 compute correctly") + + +def test_stdDevSamp_decimal128_with_int64(collection): + """$stdDevSamp with mixed Decimal128 and Int64 values.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Int64(20)}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 10.0}, + {"_id": 2, "partition": "A", "value": Int64(20), "result": 10.0}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 10.0}, + ] + assertSuccess(result, expected, msg="mixed Decimal128 and Int64 compute correctly") + + +def test_stdDevSamp_decimal128_all_types_mixed(collection): + """$stdDevSamp with Decimal128 + Double + Int32 + Int64 all in same frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": 20.0}, + {"_id": 3, "partition": "A", "value": 30}, + {"_id": 4, "partition": "A", "value": Int64(40)}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # stdDevSamp([10,20,30,40]): mean=25, sum_sq=500, var_samp=500/3, stdDevSamp=sqrt(500/3) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": 12.909944487358056}, + {"_id": 2, "partition": "A", "value": 20.0, "result": 12.909944487358056}, + {"_id": 3, "partition": "A", "value": 30, "result": 12.909944487358056}, + {"_id": 4, "partition": "A", "value": Int64(40), "result": 12.909944487358056}, + ] + assertSuccess(result, expected, msg="all four numeric types mixed in same frame") + + +def test_stdDevSamp_decimal128_high_precision(collection): + """$stdDevSamp with high-precision Decimal128 values preserves precision.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("1.000000000000000000000000000000001")}, + {"_id": 2, "partition": "A", "value": Decimal128("2.000000000000000000000000000000001")}, + {"_id": 3, "partition": "A", "value": Decimal128("3.000000000000000000000000000000001")}, + ] + result = run_window_operator( + collection, + "$stdDevSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[ + {"$addFields": {"result": {"$round": ["$result", 3]}}}, + {"$project": {"_id": 1, "result": 1}}, + ], + ) + expected = [ + {"_id": 1, "result": 1.0}, + {"_id": 2, "result": 1.0}, + {"_id": 3, "result": 1.0}, + ] + assertSuccess(result, expected, msg="high-precision Decimal128 values produce ~1.0") + + +def test_stdDevSamp_decimal128_near_zero(collection): + """$stdDevSamp with very small Decimal128 values near zero.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("0.0000000000000000000000000000000001")}, + {"_id": 2, "partition": "A", "value": Decimal128("0.0000000000000000000000000000000002")}, + {"_id": 3, "partition": "A", "value": Decimal128("0.0000000000000000000000000000000003")}, + ] + result = run_window_operator( + collection, + "$stdDevSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[ + { + "$addFields": { + "valid": { + "$and": [ + {"$ne": ["$result", None]}, + {"$gte": ["$result", 0]}, + ] + } + } + }, + {"$project": {"_id": 1, "valid": 1}}, + ], + ) + expected = [ + {"_id": 1, "valid": True}, + {"_id": 2, "valid": True}, + {"_id": 3, "valid": True}, + ] + assertSuccess(result, expected, msg="near-zero Decimal128 produces non-negative result") + + +def test_stdDevSamp_decimal128_identical_values(collection): + """$stdDevSamp with identical Decimal128 values returns 0.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("42.5")}, + {"_id": 2, "partition": "A", "value": Decimal128("42.5")}, + {"_id": 3, "partition": "A", "value": Decimal128("42.5")}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("42.5"), "result": 0.0}, + {"_id": 2, "partition": "A", "value": Decimal128("42.5"), "result": 0.0}, + {"_id": 3, "partition": "A", "value": Decimal128("42.5"), "result": 0.0}, + ] + assertSuccess(result, expected, msg="identical Decimal128 values return 0") + + +def test_stdDevSamp_decimal128_sliding_window(collection): + """$stdDevSamp with Decimal128 values in a sliding window.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("20")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + {"_id": 4, "partition": "A", "value": Decimal128("40")}, + {"_id": 5, "partition": "A", "value": Decimal128("50")}, + ] + # Window [-1, 0]: row 1 has single element -> null, rows 2-5 have pairs -> stdDevSamp + # stdDevSamp of adjacent pairs with diff=10: sqrt((25+25)/1) = sqrt(50) = 7.0710678118654755 + result = run_window_operator(collection, "$stdDevSamp", docs, {"documents": [-1, 0]}) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": None}, + {"_id": 2, "partition": "A", "value": Decimal128("20"), "result": 7.0710678118654755}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": 7.0710678118654755}, + {"_id": 4, "partition": "A", "value": Decimal128("40"), "result": 7.0710678118654755}, + {"_id": 5, "partition": "A", "value": Decimal128("50"), "result": 7.0710678118654755}, + ] + assertSuccess(result, expected, msg="Decimal128 values in sliding window with stdDevSamp") + + +# Property [Decimal128 Special Values]: Decimal128 NaN and Infinity handling + + +def test_stdDevSamp_decimal128_nan_special(collection): + """$stdDevSamp with Decimal128 NaN — NaN is numeric and poisons the calculation.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("NaN")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": Decimal128("10"), "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": Decimal128("NaN"), "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": Decimal128("30"), "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="Decimal128 NaN is numeric and poisons stdDevSamp to NaN" + ) + + +def test_stdDevSamp_decimal128_infinity_special(collection): + """$stdDevSamp with Decimal128 Infinity special value.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("Infinity")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, + "$stdDevSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[ + { + "$addFields": { + "has_result": {"$ne": ["$result", None]}, + } + }, + {"$project": {"_id": 1, "has_result": 1}}, + ], + ) + expected = [ + {"_id": 1, "has_result": True}, + {"_id": 2, "has_result": True}, + {"_id": 3, "has_result": True}, + ] + assertSuccess(result, expected, msg="Decimal128 Infinity produces a non-null result") + + +def test_stdDevSamp_decimal128_neg_infinity_special(collection): + """$stdDevSamp with Decimal128 -Infinity special value.""" + docs = [ + {"_id": 1, "partition": "A", "value": Decimal128("10")}, + {"_id": 2, "partition": "A", "value": Decimal128("-Infinity")}, + {"_id": 3, "partition": "A", "value": Decimal128("30")}, + ] + result = run_window_operator( + collection, + "$stdDevSamp", + docs, + {"documents": ["unbounded", "unbounded"]}, + extra_stages=[ + { + "$addFields": { + "has_result": {"$ne": ["$result", None]}, + } + }, + {"$project": {"_id": 1, "has_result": 1}}, + ], + ) + expected = [ + {"_id": 1, "has_result": True}, + {"_id": 2, "has_result": True}, + {"_id": 3, "has_result": True}, + ] + assertSuccess(result, expected, msg="Decimal128 -Infinity produces a non-null result") + + +# Property [Basic Numeric]: standard numeric inputs handled correctly + + +def test_stdDevSamp_all_identical_values_returns_zero(collection): + """$stdDevSamp returns 0 when all values in the frame are identical.""" + docs = [ + {"_id": 1, "partition": "A", "value": 7.0}, + {"_id": 2, "partition": "A", "value": 7.0}, + {"_id": 3, "partition": "A", "value": 7.0}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 7.0, "result": 0.0}, + {"_id": 2, "partition": "A", "value": 7.0, "result": 0.0}, + {"_id": 3, "partition": "A", "value": 7.0, "result": 0.0}, + ] + assertSuccess(result, expected, msg="identical values produce stdDevSamp = 0") + + +def test_stdDevSamp_negative_numbers(collection): + """$stdDevSamp handles negative numbers correctly.""" + docs = [ + {"_id": 1, "partition": "A", "value": -10}, + {"_id": 2, "partition": "A", "value": 0}, + {"_id": 3, "partition": "A", "value": 10}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": -10, "result": 10.0}, + {"_id": 2, "partition": "A", "value": 0, "result": 10.0}, + {"_id": 3, "partition": "A", "value": 10, "result": 10.0}, + ] + assertSuccess(result, expected, msg="negative numbers handled correctly") + + +def test_stdDevSamp_decimals(collection): + """$stdDevSamp handles floating-point (double) values.""" + docs = [ + {"_id": 1, "partition": "A", "value": 1.5}, + {"_id": 2, "partition": "A", "value": 2.5}, + {"_id": 3, "partition": "A", "value": 3.5}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 1.5, "result": 1.0}, + {"_id": 2, "partition": "A", "value": 2.5, "result": 1.0}, + {"_id": 3, "partition": "A", "value": 3.5, "result": 1.0}, + ] + assertSuccess(result, expected, msg="floating-point values handled correctly") + + +# Property [Negative Zero]: -0.0 treated as numeric zero + + +def test_stdDevSamp_negative_zero(collection): + """$stdDevSamp treats -0.0 as numeric zero — participates in computation.""" + docs = [ + {"_id": 1, "partition": "A", "value": DOUBLE_NEGATIVE_ZERO}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 20}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + # Values: [-0.0, 10, 20] -> mean=10, sum_sq=200, var_samp=200/2=100, stdDevSamp=10.0 + expected = [ + {"_id": 1, "partition": "A", "value": DOUBLE_NEGATIVE_ZERO, "result": 10.0}, + {"_id": 2, "partition": "A", "value": 10, "result": 10.0}, + {"_id": 3, "partition": "A", "value": 20, "result": 10.0}, + ] + assertSuccess(result, expected, msg="-0.0 treated as numeric zero in stdDevSamp") diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_order_independence.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_order_independence.py new file mode 100644 index 000000000..e20af8e5c --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_order_independence.py @@ -0,0 +1,89 @@ +""" +Tests for $stdDevSamp order independence in window context. + +Verifies that $stdDevSamp produces the same result regardless of sortBy direction, +confirming it is an order-independent operator per TEST_COVERAGE.md §22. +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + BASIC_DOCS, +) +from documentdb_tests.framework.assertions import assertSuccess +from documentdb_tests.framework.executor import execute_command + +# Property [Order Independence]: $stdDevSamp produces same result regardless of sort direction + + +def test_stdDevSamp_whole_partition_ascending_sort(collection): + """$stdDevSamp whole partition with ascending sort.""" + collection.insert_many(BASIC_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": 1}, + "output": { + "result": { + "$stdDevSamp": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + {"$sort": {"_id": 1}}, + {"$project": {"_id": 1, "result": 1}}, + ], + "cursor": {}, + }, + ) + expected = [ + {"_id": 1, "result": 15.811388300841896}, + {"_id": 2, "result": 15.811388300841896}, + {"_id": 3, "result": 15.811388300841896}, + {"_id": 4, "result": 15.811388300841896}, + {"_id": 5, "result": 15.811388300841896}, + ] + assertSuccess(result, expected, msg="ascending sort produces correct stdDevSamp") + + +def test_stdDevSamp_whole_partition_descending_sort(collection): + """$stdDevSamp whole partition with descending sort produces same result as ascending.""" + collection.insert_many(BASIC_DOCS) + result = execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": [ + { + "$setWindowFields": { + "partitionBy": "$partition", + "sortBy": {"_id": -1}, + "output": { + "result": { + "$stdDevSamp": "$value", + "window": {"documents": ["unbounded", "unbounded"]}, + } + }, + } + }, + {"$sort": {"_id": 1}}, + {"$project": {"_id": 1, "result": 1}}, + ], + "cursor": {}, + }, + ) + # Same result regardless of sort direction — order-independent operator + expected = [ + {"_id": 1, "result": 15.811388300841896}, + {"_id": 2, "result": 15.811388300841896}, + {"_id": 3, "result": 15.811388300841896}, + {"_id": 4, "result": 15.811388300841896}, + {"_id": 5, "result": 15.811388300841896}, + ] + assertSuccess( + result, expected, msg="descending sort produces same stdDevSamp — order independent" + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_special_floats.py b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_special_floats.py new file mode 100644 index 000000000..6c36ac920 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/stdDevSamp/test_window_stdDevSamp_special_floats.py @@ -0,0 +1,284 @@ +""" +Tests for $stdDevSamp with special float values (NaN, Infinity, -Infinity). + +Covers: Infinity as numeric participant, -Infinity, NaN values, +sliding window behavior with special floats, and non-removable window +behavior (NaN propagation). + +Key difference from $stdDevPop: single element in frame → null (N-1=0, undefined), +whereas $stdDevPop single element → NaN when value is Inf (due to Inf arithmetic) +or 0 for finite values. For frames with 2+ elements, NaN/Inf poisoning behavior +is identical to $stdDevPop. +""" + +from documentdb_tests.compatibility.tests.core.operator.window.utils.window_test_case import ( + run_window_operator, +) +from documentdb_tests.framework.assertions import assertSuccess, assertSuccessNaN +from documentdb_tests.framework.test_constants import ( + FLOAT_INFINITY, + FLOAT_NAN, + FLOAT_NEGATIVE_INFINITY, +) + +# Property [Infinity Non-Removable Window]: Infinity in non-removable windows produces NaN + + +def test_stdDevSamp_positive_infinity_whole_partition(collection): + """$stdDevSamp with Infinity in non-removable whole partition window produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="Infinity is numeric; non-removable window produces NaN") + + +def test_stdDevSamp_negative_infinity_whole_partition(collection): + """$stdDevSamp with -Infinity in non-removable whole partition window produces NaN.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="-Infinity is numeric; non-removable window produces NaN" + ) + + +def test_stdDevSamp_inf_and_neg_inf_in_same_frame(collection): + """$stdDevSamp with both Infinity and -Infinity produces NaN (Inf - Inf = NaN in variance).""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY}, + {"_id": 3, "partition": "A", "value": 10}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 10, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="Inf + -Inf in frame → NaN in variance calculation") + + +def test_stdDevSamp_all_infinity_values(collection): + """$stdDevSamp where all values are Infinity — identical Inf: mean=Inf, Inf-Inf=NaN.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 3, "partition": "A", "value": FLOAT_INFINITY}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": FLOAT_INFINITY, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, expected, msg="All identical Inf: mean=Inf, variance involves Inf-Inf=NaN" + ) + + +def test_stdDevSamp_infinity_cumulative_window(collection): + """$stdDevSamp with Infinity in cumulative [unbounded, current] — row 1 is null (single + element).""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "current"]} + ) + # Row 1: frame=[Inf] -> null (single element, N-1=0, undefined) + # Row 2: frame=[Inf, 10] -> NaN (Inf poisons) + # Row 3: frame=[Inf, 10, 30] -> NaN (Inf poisons) + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": None}, + {"_id": 2, "partition": "A", "value": 10, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN( + result, + expected, + msg="Cumulative window: row 1 null (single element), rows 2-3 NaN (Inf poisons)", + ) + + +def test_stdDevSamp_single_infinity_value(collection): + """$stdDevSamp with single Infinity value in whole partition. + + Returns null (single element, N-1=0). + """ + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": None}, + ] + assertSuccess( + result, + expected, + msg="Single Inf value: stdDevSamp with N=1 returns null (N-1=0, undefined)", + ) + + +# Property [NaN Non-Removable Window]: NaN in non-removable windows produces NaN + + +def test_stdDevSamp_nan_value_whole_partition(collection): + """$stdDevSamp with NaN in non-removable window produces NaN (NaN is numeric, poisons).""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30}, + ] + result = run_window_operator( + collection, "$stdDevSamp", docs, {"documents": ["unbounded", "unbounded"]} + ) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": FLOAT_NAN, "result": FLOAT_NAN}, + {"_id": 3, "partition": "A", "value": 30, "result": FLOAT_NAN}, + ] + assertSuccessNaN(result, expected, msg="NaN is numeric; non-removable window produces NaN") + + +# Property [Special Floats Sliding Window]: special floats in removable/sliding windows return null + + +def test_stdDevSamp_infinity_sliding_returns_null(collection): + """$stdDevSamp sliding window returns null when Infinity is in the current frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 20}, + {"_id": 4, "partition": "A", "value": 30}, + {"_id": 5, "partition": "A", "value": 40}, + ] + result = run_window_operator(collection, "$stdDevSamp", docs, {"documents": [-1, 0]}) + # Row 1: frame=[Inf] -> null (single element, N-1=0) + # Row 2: frame=[Inf, 10] -> null (Inf in removable window) + # Row 3: frame=[10, 20] -> sqrt(50) = 7.0710678118654755 + # Row 4: frame=[20, 30] -> sqrt(50) = 7.0710678118654755 + # Row 5: frame=[30, 40] -> sqrt(50) = 7.0710678118654755 + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_INFINITY, "result": None}, + {"_id": 2, "partition": "A", "value": 10, "result": None}, + {"_id": 3, "partition": "A", "value": 20, "result": 7.0710678118654755}, + {"_id": 4, "partition": "A", "value": 30, "result": 7.0710678118654755}, + {"_id": 5, "partition": "A", "value": 40, "result": 7.0710678118654755}, + ] + assertSuccess( + result, + expected, + msg="Sliding window: null for single element and when Inf in frame," + " recovers when Inf leaves", + ) + + +def test_stdDevSamp_neg_infinity_sliding_returns_null(collection): + """$stdDevSamp sliding window returns null when -Infinity is in the current frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 20}, + {"_id": 4, "partition": "A", "value": 30}, + ] + result = run_window_operator(collection, "$stdDevSamp", docs, {"documents": [-1, 0]}) + # Row 1: frame=[-Inf] -> null (single element, N-1=0) + # Row 2: frame=[-Inf, 10] -> null (-Inf in removable window) + # Row 3: frame=[10, 20] -> sqrt(50) = 7.0710678118654755 + # Row 4: frame=[20, 30] -> sqrt(50) = 7.0710678118654755 + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_NEGATIVE_INFINITY, "result": None}, + {"_id": 2, "partition": "A", "value": 10, "result": None}, + {"_id": 3, "partition": "A", "value": 20, "result": 7.0710678118654755}, + {"_id": 4, "partition": "A", "value": 30, "result": 7.0710678118654755}, + ] + assertSuccess( + result, + expected, + msg="Sliding window: null for single element and when -Inf in frame," + " recovers when -Inf leaves", + ) + + +def test_stdDevSamp_nan_sliding_returns_null(collection): + """$stdDevSamp sliding window returns null when NaN is in the current frame.""" + docs = [ + {"_id": 1, "partition": "A", "value": FLOAT_NAN}, + {"_id": 2, "partition": "A", "value": 10}, + {"_id": 3, "partition": "A", "value": 20}, + {"_id": 4, "partition": "A", "value": 30}, + ] + result = run_window_operator(collection, "$stdDevSamp", docs, {"documents": [-1, 0]}) + # Row 1: frame=[NaN] -> null (single element, N-1=0) + # Row 2: frame=[NaN, 10] -> null (NaN in removable window) + # Row 3: frame=[10, 20] -> sqrt(50) = 7.0710678118654755 + # Row 4: frame=[20, 30] -> sqrt(50) = 7.0710678118654755 + expected = [ + {"_id": 1, "partition": "A", "value": FLOAT_NAN, "result": None}, + {"_id": 2, "partition": "A", "value": 10, "result": None}, + {"_id": 3, "partition": "A", "value": 20, "result": 7.0710678118654755}, + {"_id": 4, "partition": "A", "value": 30, "result": 7.0710678118654755}, + ] + assertSuccessNaN( + result, + expected, + msg="Sliding window: null for single element and when NaN in frame," + " recovers when NaN leaves", + ) + + +def test_stdDevSamp_infinity_centered_sliding(collection): + """$stdDevSamp centered sliding window [-1, 1] with Infinity in middle returns null.""" + docs = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": FLOAT_INFINITY}, + {"_id": 4, "partition": "A", "value": 30}, + {"_id": 5, "partition": "A", "value": 40}, + ] + result = run_window_operator(collection, "$stdDevSamp", docs, {"documents": [-1, 1]}) + # Row 1: frame=[10, 20] -> sqrt(50) = 7.0710678118654755 (clean, edge clamp) + # Row 2: frame=[10, 20, Inf] -> null (Inf in removable window) + # Row 3: frame=[20, Inf, 30] -> null (Inf in removable window) + # Row 4: frame=[Inf, 30, 40] -> null (Inf in removable window) + # Row 5: frame=[30, 40] -> sqrt(50) = 7.0710678118654755 (clean, edge clamp) + expected = [ + {"_id": 1, "partition": "A", "value": 10, "result": 7.0710678118654755}, + {"_id": 2, "partition": "A", "value": 20, "result": None}, + {"_id": 3, "partition": "A", "value": FLOAT_INFINITY, "result": None}, + {"_id": 4, "partition": "A", "value": 30, "result": None}, + {"_id": 5, "partition": "A", "value": 40, "result": 7.0710678118654755}, + ] + assertSuccess( + result, + expected, + msg="Centered sliding: null when Inf in frame, clean rows compute correctly", + ) diff --git a/documentdb_tests/compatibility/tests/core/operator/window/utils/__init__.py b/documentdb_tests/compatibility/tests/core/operator/window/utils/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/documentdb_tests/compatibility/tests/core/operator/window/utils/window_test_case.py b/documentdb_tests/compatibility/tests/core/operator/window/utils/window_test_case.py new file mode 100644 index 000000000..f49e45df9 --- /dev/null +++ b/documentdb_tests/compatibility/tests/core/operator/window/utils/window_test_case.py @@ -0,0 +1,85 @@ +"""Shared test case and helpers for window operator tests.""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Any + +from documentdb_tests.framework.executor import execute_command +from documentdb_tests.framework.test_case import BaseTestCase + + +@dataclass(frozen=True) +class WindowTestCase(BaseTestCase): + """Test case for window operator tests.""" + + docs: list[dict] | None = None + window: dict[str, Any] | None = None + sort_by: dict[str, Any] = field(default_factory=lambda: {"_id": 1}) + partition_by: str = "$partition" + extra_stages: list[dict[str, Any]] | None = None + + +BASIC_DOCS: list[dict[str, Any]] = [ + {"_id": 1, "partition": "A", "value": 10}, + {"_id": 2, "partition": "A", "value": 20}, + {"_id": 3, "partition": "A", "value": 30}, + {"_id": 4, "partition": "A", "value": 40}, + {"_id": 5, "partition": "A", "value": 50}, +] + + +def run_window_operator( + collection, + operator: str, + docs: list[dict], + window: dict[str, Any], + sort_by: dict[str, Any] | None = None, + partition_by: str = "$partition", + extra_stages: list[dict[str, Any]] | None = None, +) -> Any: + """Build and execute a $setWindowFields pipeline. + + Args: + collection: The collection to operate on. + operator: The window operator string (e.g. "$stdDevPop"). + docs: Documents to insert into the collection. + window: The window specification (e.g. {"documents": ["unbounded", "current"]}). + sort_by: The sortBy specification. Defaults to {"_id": 1}. + partition_by: The partitionBy expression. Defaults to "$partition". + extra_stages: Additional pipeline stages to append after $setWindowFields. + + Returns: + The result from execute_command (result dict or Exception). + """ + if sort_by is None: + sort_by = {"_id": 1} + + collection.insert_many(docs) + + pipeline: list[dict[str, Any]] = [ + { + "$setWindowFields": { + "partitionBy": partition_by, + "sortBy": sort_by, + "output": { + "result": { + operator: "$value", + "window": window, + } + }, + } + }, + ] + + if extra_stages: + pipeline.extend(extra_stages) + + return execute_command( + collection, + { + "aggregate": collection.name, + "pipeline": pipeline, + "cursor": {}, + }, + ) diff --git a/documentdb_tests/framework/test_format_validator.py b/documentdb_tests/framework/test_format_validator.py index 4d224c66b..4a769dabf 100644 --- a/documentdb_tests/framework/test_format_validator.py +++ b/documentdb_tests/framework/test_format_validator.py @@ -33,6 +33,7 @@ def validate_test_format(file_path: str) -> list[str]: "execute_expression", "execute_expression_with_insert", "execute_admin_with_retry_command", + "run_window_operator", ] )