Skip to content

Tolerate one-ULP BF16 variation in correctness tests - #16

Open
morluto wants to merge 1 commit into
MoonshotAI:masterfrom
morluto:codex/numerical-tests
Open

Tolerate one-ULP BF16 variation in correctness tests#16
morluto wants to merge 1 commit into
MoonshotAI:masterfrom
morluto:codex/numerical-tests

Conversation

@morluto

@morluto morluto commented Jul 27, 2026

Copy link
Copy Markdown

TL;DR

Replace bitwise BF16 kernel/reference assertions with a one-ULP comparison while keeping FP32 state checks exact.

Relates to #12.

Motivation

Issue #12 reports that the forward test passes with CUDA 12.9 on B200 but fails with CUDA 13 because torch.equal observes a small BF16 output difference.

The kernel and Torch reference intentionally reproduce BF16 intermediate rounding. Requiring identical BF16 bits across compiler versions makes the test more restrictive than the numerical behavior it is intended to validate.

Comparison contract

Values being compared Result
Identical BF16 values Pass
Adjacent finite BF16 values Pass
Difference greater than one BF16 ULP Fail
Non-matching NaN or infinity Fail
Non-BF16 tensors, including FP32 state Require exact equality

Changes

  • Add a BF16 comparator based on distance between representable values.
  • Process large tensors in bounded chunks instead of allocating another full-size temporary.
  • Apply the comparator to fixed-length, variable-length, batched, and long-sequence tests.
  • Preserve exact comparisons for FP32 state tensors.
  • Update the README to describe the BF16 comparison accurately.
  • Add focused tests for adjacent values, two-ULP differences, non-finite values, and FP32 exactness.

Suggested review order

  1. tests/numerics.py — comparison contract and bounded-memory implementation.
  2. tests/test_numerics.py — edge-case coverage.
  3. tests/test_fwd.py and tests/test_fwd_full.py — integration at existing assertion sites.
  4. README.md — user-facing description.

The main question is whether one representable BF16 value is the appropriate boundary. Differences beyond that boundary still fail.

Validation

  • python -m pytest -q tests/test_numerics.py — 4 passed.
  • Exhaustively checked ordering and adjacency across all 65,280 finite BF16 encodings.
  • python -m py_compile tests/numerics.py tests/test_numerics.py tests/test_fwd.py tests/test_fwd_full.py
  • git diff --check origin/master...HEAD

@morluto
morluto marked this pull request as ready for review July 27, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant