feat: seed population/profile-prior baselines for new users#7
Open
shoshiiiin wants to merge 27 commits into
Open
feat: seed population/profile-prior baselines for new users#7shoshiiiin wants to merge 27 commits into
shoshiiiin wants to merge 27 commits into
Conversation
Analytics accuracy fixes
analytics: VO2max (Uth-Sørensen), Banister fitness/fatigue/form, Fost…
… R10+0x33 IMU, 8-step confirm, calibration gain)
…backend) The pure pedometer math (pedometer/calcSteps + locked params + calibration gain) now lives in steps.ts like every other metric, exported from index.ts, with a unit test (rest → 0, clean ~1.8 Hz walk → plausible count, gain applied). Backend keeps only the R2/decoder I/O runner. ALGORITHMS.md row 27 updated to reflect the split.
Detect every consolidated sleep period in a window (reusing the Cole-Kripke + HR-dip scorer), each with its own breakdown; longest flagged is_main. Naps are not special-cased — just shorter sleeps. Adds SleepPeriod/SleepPeriodsValue types, index export, and tests. calcSleep (v1) untouched.
feat: multi-period sleep (calcSleepPeriods) — naps as shorter sleeps
…Strap#4) calcSleep / calcSleepPeriods are anchored on resting HR (Cole–Kripke + HR-dip). For a brand-new user with no baseline, resting_hr is null/0 and `0.95 * null` evaluates to 0, so every minute reads "awake" — silently garbage sleep. Now abstain (return empty) until a real resting-HR baseline exists. All 151 analytics tests pass. Co-authored-by: abdulsaheel <abdulsaheel@users.noreply.github.com>
…ndows) - maxHR fallback: Tanaka 208−0.7·age (JACC 2001) in util.ts AND baselines.ts — more accurate than 220−age. - ACWR: EWMA λ=2/(N+1), N=7/28 (Williams 2017) — fixes rolling-average coupling. - Freq-domain HRV: gate LF/HF to the Task Force 1996 window rule (HF ≥~60s, LF ≥~250s); below that LF/HF was spectral noise. - HR zones: kept %HRmax (NOT Karvonen %HRR) — validated on real data, %HRR adds no real accuracy on an age-predicted max and collapses light-day zones. 151/151 tests pass; validated locally on real user data.
…V windows, %HRmax zones rationale
- calcBaselines now trusts an observed daily peak as a measured HRmax only when it exceeds the Tanaka age floor (208-0.7*age); a quiet day's peak can't shrink the zone/strain denominator and inflate those metrics. Mirrors resolveMaxHr so baseline and per-day agree. +2 guard tests (152 pass). - regularity: relabel as sleep-timing regularity (circular variance of bed/ wake clock-times), explicitly NOT the Phillips 2017 epoch-agreement Sleep Regularity Index. Comments + README only; field/column names unchanged.
Robust cosinor HR rhythm + single bounded change-point per cycle for main-sleep onset/wake. Validated on dev account (wake/onset within ~2min of Cole-Kripke). Pure, deterministic, abstains on weak rhythm.
…select Single change-point picked mid-sleep HR bumps as wake; now wake = start of the last sustained elevation above the cosinor mesor (the morning rise that stays up). Cycle selection no longer requires bathyphase+8h to be past, which wrongly stepped back a night for early risers. Validated on real data: wake 08:22 vs reported 08:20, span 5.4h.
Every night: onset = strongest evening->sleep HR drop (change-point); wake = end of consolidated below-mesor run (mesor = cosinor midline, robust to high light-sleep HR + daytime). sleepEfficiency: WASO = sustained near-wake elevations only (0.70 floor->mesor, >=20min) so light sleep isn't mis-scored. Validated across dev week: 06-19 5.4h (vs reported ~5h), wake 08:21 vs 08:20.
Replaces sleepEfficiency. One HR-only stager classifies every minute awake/ light/deep/rem (light-dominant bands; awake = sustained >=20min near-wake only) and returns BOTH the per-minute hypnogram AND reconciled totals, so the graph and the stage card can never disagree (the reported contradiction).
…e-Kripke + cardiac/CPD + van Hees; detectWakeState + peekRecentState; 165 tests green
stageSleep classified sleep on HR level alone; on flat-HR nights REM-HR overlaps light so REM read 0%. REM is parasympathetic withdrawal -> reduced beat-to-beat variability (low RMSSD) + mildly elevated HR; deep/SWS is high RMSSD + lowest HR (verified corr(HR,RMSSD) ~ -0.4 on real RR). stageSleep now takes optional per-minute rr and, when enough asleep minutes carry RR, splits deep/rem on a smoothed per-minute RMSSD ratio to the night's asleep-RMSSD median (self-calibrating): deep = RMSSD>=1.15*med AND HR<=asleep HR median, rem = RMSSD<=0.88*med, else light. No RR -> legacy HR-only bands. Validated live (dev night 2026-06-20): light 66% / deep 13% / REM 21%, 0 flaps. +6 regression tests (RR-REM path + HR-only fallback); 171 pass.
The day-close trigger missed real wakes: the flat 2-of-3 majority was dominated by two motion voters that are blind to sedentary wakefulness (HR up, lying still) -> they outvoted the correct cardiac voter -> close never fired -> recovery/HRV/stress never computed. Verified on a real night: cardiac said awake 100% of the morning, motion 1%, 2-of-3 said asleep. Fix (keeps the pluggable registry): - add a 4th voter, hrvArousal (smoothed beat-to-beat RR-SD; the 2nd autonomic signal), and smooth the cardiac HR (margin trough+8) so REM spikes don't flip. - replace flat majority with a >=2 consensus: a 2-2 split counts as AWAKE, so the autonomic pair (cardiac+hrvArousal) can carry a quiet wake; motion can't veto it, and one voter alone still can't flip it. - bout-smooth the per-minute labels so intermittent agreement reads as one bout. Precision gates unchanged (main sleep >=90min, sustained trailing wake). Validated on real data: wake_ts 08:56 IST (true ~09:00), no false night fires. +5 regression tests incl. quiet-sedentary-wake; 176 pass.
calcSpo2Index — confidence-gated (intra-night ratio stability + sample count); validated on 4 user datasets (ratio more stable than red-alone on clean signal). RELATIVE tier; never an absolute %. 184 tests pass.
…sleep cycles stageHypnogram — ONE source for the hypnogram + breakdown (replaces the stageSleep/ reconcile machinery): calcSleep's Cole-Kripke+HR-dip owns asleep/awake, the same HR-percentile bands as estimateStages own deep/light/rem, bout-smoothed once. This is the v1 method that worked, made per-minute and consistent (graph==breakdown, no flapping). sleepAwakeMask gains an RR tiebreaker for the HR-dip override's blind spot: 'HR>1.15*rhr => awake' also caught REM (REM HR runs ~15% above the sleeping floor) and on a calm wrist with a near-dead activity signal there's no movement to separate them. Fix: a high-HR minute whose smoothed RMSSD < 0.90x the night's asleep median is REM (parasympathetic withdrawal) => stays asleep, not awake. Validated: awake 69->22min, REM 10%->23%, no false mid-sleep awake bouts. cycles.ts — detectSleepCycles: ultradian NREM<->REM cycles via Rosenblum 2024's fractal- cycle algorithm (findpeaks 20min/0.9z) run on smoothed z-RMSSD (HRV is the EEG fractal- slope analog). +cycles/RR-REM tests; 188 pass.
- New calcCycle(starts, today): median cycle length, next-period + ovulation (next-14, Wilcox 2000) + fertile window, phase, confidence that collapses when overdue. Pure + honest (abstains with no logs). - Exported CycleValue/CyclePhase; +13 unit tests (201 pass).
feat: sleep/wake ensemble (wake.ts) — pluggable voters
…stlessness, daytime HRV, SpO2 desaturation, cycle-gated illness/anomaly - har.ts: Mannini 2013 wrist HAR — 15Hz Butterworth + SMV, FFT (dominant/cadence/peakiness), db10 wavelet energies; threshold classifier (ESTIMATE); segmentWorkout (smoothing + RLE phases, graceful activity switches) - detectSessions: 2-min sustained; motion-based type via per-minute act_class + segmentWorkout; segments + detected_type; HR-heuristic fallback for flash bouts - restlessness.ts: nocturnal movement fragmentation (bouts/mobility/continuity) from per-minute activity - hrv.ts: calcDaytimeHrv (waking ultradian RMSSD timeline) - spo2.ts: calcDesaturation (ODI-style overnight relative-desat screen) - illness.ts: respiratory rate as 4th Mahalanobis feature; cycle-phase gating - readiness.ts: calcAnomaly cycle-phase gating - 232 tests pass; db10 coefficients validated via orthonormality
Flash-drained nights (R24) carry no actigraphy, so per-minute activity is ~0 and Cole-Kripke is inert — calcSleep runs HR-only in practice. The fixed 1.15*rhr awake-override mis-fired against resting_hr, which is a 5th-percentile sleep FLOOR: normal/REM sleep HR runs well above it, so the override flagged whole nights awake and collapsed real sleep to ~1 minute (seen in prod on v3). Anchor the awake/asleep cutoff to the window's own depressed-HR level (10th percentile of worn HR, floored at resting_hr) so detection no longer hinges on the fragile RHR floor. Add an absolute 1.5*rhr backstop so the opposite failure mode — a flat, elevated, motion-inert sedentary-awake window — can't be manufactured into a full night. Regression tests cover both modes; 234 pass. The durable fix remains restoring R24 actigraphy (accel -> activity); noted.
fix(sleep): window-relative HR-dip detection + absolute awake backstop
Add seedBaselines(profile) returning low-confidence population/profile priors (resting HR by sex, Tanaka max HR, age-declining RMSSD, sleeping HR, resp rate, Baevsky SI). A brand-new user's strain/HR-zones then use a personal resting/max HR from day 1, and the EWMA autonomic baselines warm-start from physiological norms instead of cold, self-referential seeding on night 1. Measured data overwrites the priors as it accrues; the >=5-night gated scores (recovery, stress) still require real nights and are never fabricated by these priors. Includes 14 unit tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
seedBaselines(profile)— low-confidence population/profile priors for a user with no measured history yet:Why
On a brand-new user the
baselinesrow is empty, so everything falls back to flat constants (RHR 60, etc.). With these priors, strain/HR-zones use a personal resting/max HR from day 1, and the EWMA-rolled autonomic baselines warm-start from physiological norms instead of cold, self-referential seeding on night 1.Importantly, this does not fabricate the gated scores: recovery/stress still require ≥5 real nights (Plews/Baevsky) and are untouched. Measured data overwrites the priors as it accrues (the rolling-median
calcBaselines, the nightly EWMA rolls).The backend wiring (an
INSERT OR IGNOREof these priors inprocessUser) is a separate PR on the backend repo and depends on this export.Tests
14 new unit tests under
§11b seedBaselines(sex/age priors, Tanaka, sleep-need age bands, no-profile defaults). Full suite:npm test→ 248 passed, 0 failed.