Release v0.2.1#9
Merged
Merged
Conversation
Make the generated events files use the expected BIDS filename and column order. Add small tests so the filename and column order do not regress. Note: the BIDS Validator still reports `TSV_COLUMN_HEADER_DUPLICATE` for headerless `*_motion.tsv` files. These files are headerless by design, as required by the BIDS motion specification. This appears to be a validator false positive and will be tracked separately upstream.
Document the expected optional events CSV column order and explain that the input name column is exported as BIDS trial_type. Also note that events outputs use the configured task name in the BIDS filename.
Avoid creating invalid intervals when timestamp arrays contain trailing zeros. Compute the validation window from recording onset/offset and add a regression test to ensure start_time/end_time stay valid.
fix(validation): handle trailing-zero timestamps in sampling-rate check
fix(bids): clean up events export
* fix: use timeSinceStartup for quality flag boundaries
Root cause: the per-system clock (e.g. Node_HandLeft_Time) drops to 0
during hardware dropouts while timeSinceStartup keeps ticking. Three
layers broke:
1. QualityFlag.__post_init__ crashed: _find_contiguous_segments picked
the zero timestamp as a segment boundary, producing start_time > end_time.
2. apply_quality_masking silently did nothing: dropout rows had
timestamp=0, so (0.0 >= 630565.4) was False, nothing was NaN'd.
3. report.py np.interp was converting per-system times to global times
with a zeroed x-axis, producing garbage display values.
Fix: all validation checks pass timeSinceStartup to QualityFlag.from_mask.
Masking compares flags against timeSinceStartup. Report subtracts global
onset directly (no interp needed since flags are already in global time).
Changes:
- checks/hands_tracking_loss.py: use timeSinceStartup; error-log if missing
- checks/eyes_closed.py: same for FACE flags and EYES propagation; add logger
- checks/sampling_rate.py: same; add module-level logger
- preprocessing/stream_preprocessing.py: apply_quality_masking compares
against timeSinceStartup; error-log and return unmasked if missing;
update pipeline docstring
- visualization/report.py: remove np.interp block; drop unused numpy import;
update module and method docstrings
- tests/conftest.py: add timeSinceStartup to all stream builder helpers
- tests: add timeSinceStartup to inline DataFrames; update data_cols
filters; add dropout regression tests (divergent clocks, missing column)
* fix: format tests
Renames two internal utility functions for clarity:
find_recording_onset_index -> find_first_nonzero_index
find_recording_offset_index -> find_last_nonzero_index
All call sites updated (session.py, stream_preprocessing.py,
sampling_rate.py) along with their docstrings.
Adds find_internal_zero_blocks (utils/__init__.py):
Detects contiguous zero/NaN blocks strictly inside the recording
window (between onset and offset). Leading/trailing edge zeros
(device spin-up/down) are excluded.
Adds ClockDropoutCheck (validation/checks/clock_dropout.py):
Runs on every stream. Calls find_internal_zero_blocks on the
per-system timestamp column, then passes timeSinceStartup to
QualityFlag.from_mask so flag boundaries are in the global clock
space -- consistent with all other checks and apply_quality_masking.
Flags use mask=True / target_columns=[] to request full-row NaN
masking of dropout segments (latency would be invalid for those rows).
Registers clock_dropout in pipeline_config.yaml.
Tests:
- TestFindFirstNonzeroIndex (10 cases)
- TestFindInternalZeroBlocks (18 cases, including inf, NaN,
negatives, mixed blocks, integer dtype, boundary positions)
- TestClockDropoutCheck (17 cases across 5 test classes)
- TestFindLastNonzeroIndex renamed from TestFindRecordingOffsetIndex
578 tests pass.
… at session root)
…les_dir) instead of globs
…#6) * fix: validate shared custom column metadata and fix input file formatting - Enforce strict metadata match when custom classes share a BIDS column (e.g. Task/Trial) - Fix fallback terminology to use 'CustomTables sidecar json' - Sync pipeline logic and tests with updated input file formatting * style: fix trailing whitespace errors * style: apply ruff auto-formatting
-Deleted legacy files in test_data/ because their lowercase naming convention and outdated JSON structures are fully incompatible with the new schema parsing. -Replaced with fully compliant Demo Data to ensure test suites and pipeline inputs match the finalized metadata formats.
Add three new 2026-06-08 demo sessions (Binary, Maze, Museum) using the per-session *_CustomTables layout; remove the superseded BinaryChoice/001 session.
Derive the custom-tables folder per session as <session_id>_CustomTables (error-log when absent) and drop the now-unusable custom_tables_dir config option; point session_mappings at the three new demo sessions. Read custom-class CSVs as text so boolean/integer event columns keep their source tokens (false/true, 0) instead of being upcast to 0.0/1.0 during the events merge. Adds regression tests.
Clean regeneration so sourcedata, reports, and layout are consistent and the events typing fix is reflected in events.tsv.
- Read __version__ from package metadata via importlib.metadata instead of a hardcoded string, so pyproject.toml is the single source of truth and __init__.py can no longer drift (it was stuck at 0.1.0 while pyproject already said 0.1.1). - Bump version 0.1.1 -> 0.2.0 in pyproject.toml; regenerate uv.lock. - environment.yml: install the package non-editable (. instead of -e .) for the minimal/user env; editable -e stays in environment-dev.yml, which is the correct split between consumer and developer installs.
… in BIDS Rename detected_* fields to schema_* (they are compile-time CSV schema column counts, not live device measurements) and add raw device/OS fields: manufacturers_model_name_raw, software_versions_raw, horizon_os_version, device_serial_number. The reader falls back to legacy detected_* keys so existing sessions still parse. Fold horizon_os_version and software_versions_raw into the BIDS SoftwareVersions string (BIDS has no dedicated OS field; type is string), skipping editor/n-a/unknown sentinels, and emit DeviceSerialNumber only when known (omitted rather than empty, per BIDS convention). schema_* counts remain in sourcedata only. Update fixtures for the renamed fields; add tests for SoftwareVersions folding and DeviceSerialNumber gating.
Replace the 2026.06.08 Binary/Maze/Museum demo sessions with the re-recorded 2026.06.10 sessions and repoint config session_mappings to match. Regenerate sourcedata, motion/channels/events sidecars, scans, participants, and HTML reports. Sidecars now carry the richer SoftwareVersions (Unity + OVR Plugin + Horizon OS + Android) and the renamed schema_* metadata, and omit DeviceSerialNumber when unknown.
- SessionMetadata now captures every scalar "*version*" key from SessionMetadata.json into a generic software_versions dict; the vendor-named fields (unity_version, ovrplugin_version, horizon_os_version, software_versions_raw) and the unused manufacturers_model_name_raw are removed - HTML report header lists all captured versions (incl. OS) as a device-agnostic "Software versions" list - BIDS SoftwareVersions is built from a declarative _KNOWN_VERSION_FORMATS table with generic fallthrough for unknown engines; the OVR wrapper version is dropped only when it duplicates the runtime version; Quest demo output stays byte-identical - centralize the timeSinceStartup column name as GLOBAL_CLOCK_COLUMN (ResXR format contract, previously ~30 scattered literals) - warn when fixedDeltaTime is missing instead of silently defaulting to 0.02 s - drop the "VR headset" wording from the fallback task description - document the engine-agnostic version handling in the README
- Added 'readme_text' field to BIDSConfig to allow customizing dataset README contents. - Configured a default 'authors' array and 'readme_text' in pipeline_config.yaml. - Added generation of a mandatory README file at the root of both raw and derivative BIDS datasets. - Handled empty authors arrays by entirely omitting the Authors key in dataset_description.json to resolve BIDS validation errors.
Regenerated the BIDS dataset tracking data and metadata using the updated pipeline to fix missing authors and README validation warnings.
- eyes_closed now runs on face-only sessions (previously required an EYES stream and was skipped entirely) - report timeline reflects the merged native + custom-table events, matching events.tsv - events.json no longer lists the BIDS n/a token (or NaN) as a name level - a missing optional CustomTables folder is logged at debug, not error Each change has a regression test.
pandas to_csv defaults lineterminator to os.linesep (CRLF on Windows), so the generated dataset's line endings depended on the OS. Force LF in every writer (CSV lineterminator, JSON/text/HTML newline) so datasets are byte-reproducible across Windows and Linux. Add a .gitattributes text-normalization rule and an LF regression test.
- Keep a Changelog covering 0.1.0-0.2.0 with a Breaking Changes section - correct README config docs (missing_values, validation settings nesting, input patterns, programmatic example, clock_dropout, readme_text) - fix repository URLs (ResXR/ResXR -> ResXR/resxr-python-pipeline) - ship CHANGELOG.md in the sdist (MANIFEST.in)
# Conflicts: # pyproject.toml # src/resxr/io/writers.py # src/resxr/pipeline.py # src/resxr/validation/checks/sampling_rate.py # tests/integration/test_pipeline_flow.py # tests/io/test_writers.py # tests/validation/checks/test_sampling_rate.py
Bump version to 0.2.1
NoaTAU
approved these changes
Jun 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This release (v0.2.1) addresses configuration-related bugs and ensures the pipeline behaves exactly as documented.
Bug Fixes & Changes
report.output_dirsetting inpipeline_config.yaml. Previously, this setting was ignored and the HTML report was always forced into the BIDSsession folder.
python -m resxr) swallowedexceptions and incorrectly exited with a
0success code. It now accurately propagates exit codes (1for failure,130for keyboard interrupt) to match the behavior of theresxrconsole script.column_groupsandcheck_column_groupsblocks from the default shippedpipeline_config.yaml, as they were completely unused by the pipeline's built-in validation checks.Testing Performed
report.output_diraccurately routes HTML reports.python -m resxrreturnsexit 1on missing config andexit 130on Ctrl-C.ruff formatandruff check; codebase is clean.