diff --git a/README.md b/README.md index affb2da..883ca2d 100644 --- a/README.md +++ b/README.md @@ -33,18 +33,31 @@ When a batch lands, `ingest.ts` does four things in order: borrowed from the protocol package) and the IMU-bearing R10 are the ones that carry real signal. 3. **Saves the raw bytes to R2** under `raw/{you}/{device}/{when}-{first}-{last}.txt`, - one frame per line. This is the bit I never throw away. Minute rollups get pruned after - 90 days; the raw frames stay so they can be re-decoded forever. + one frame per line — the system of record for re-decoding. Retention is short and + layered: raw R2 objects expire at **14 days** (an R2 bucket lifecycle rule, matching + `RAW_RETENTION_DAYS` — the re-decode horizon), the per-minute `minute` table and the + device `events` table are pruned at **10 days**, and the *derived* tables (`daily`, + `sleep`, `baselines`) are **kept permanently** — that's what the long-window trend + metrics read from, so nothing needs re-decoding past 10 days. 4. **Rolls everything into minutes.** `rollup.ts` buckets the decoded samples by `floor(ts/60)*60` and writes them to the `minute` table. The `minute` table is the one clever bit worth understanding before you touch anything. It doesn't store an average heart rate, it stores the running pieces: `hr_sum`, `hr_n`, `act_sum`, `act_n`, plus min/max. The upsert adds the new pieces onto whatever's already -there. The reason is that uploads aren't clean. The phone retries, batches overlap, the -same frame shows up twice. If I stored averages I'd corrupt them on every double-send. -Storing sums means re-uploading the exact same data converges to the exact same answer. -Idempotency for free. Don't break this. +there. The reason is that uploads aren't clean. The phone retries and batches overlap, so +a day's frames arrive split across many POSTs in no particular order. Storing running sums +makes the minute totals **order- and partition-independent**: chop the same frames into any +set of batches, in any order, and the minute rollups come out identical. If I stored +averages instead, the split would corrupt them. + +One honest caveat, because it's easy to misread this as "duplicate-proof": the upsert is +*additive*, so the **same frame delivered in two separate POSTs is counted twice.** This is +not idempotent against duplicate sends on its own — it relies on the client deduping frames +before they ever hit here. The app does exactly that: `raw_records` has the frame hex as a +primary key (`INSERT OR IGNORE`) and only deletes a record after a 200, so each frame is +POSTed once. Keep that contract; if you ever ingest from a client without it, dedup frames +server-side before the rollup. Don't break this. Once minutes are written, the user gets flagged dirty (or pushed onto a queue if you've got the paid plan), and that's where ingest stops. The heavy math is deliberately not on @@ -52,13 +65,21 @@ the request path. ## Where the metrics actually get computed -`analytics.ts` is the brain, and it runs on a cron . Two schedules, -both in `wrangler.toml`: +`analytics.ts` is the brain, and it runs on a cron. Two schedules, both in +`wrangler.toml`: -- **Every hour** (`runAnalytics` with a 3-day window): re-derive every dirty user. This - is the safety net, it catches anyone the queue missed. -- **Every night at 3:30** (2-day window): a fuller pass, plus respiratory rate for anyone - who slept recently, plus pruning minute rows older than 90 days. +- **Every 30 minutes** (`*/30`): a light sweep — re-derive every dirty user (their daily + numbers, sleep, incremental steps), and the moment a night actually finishes, kick off + that night's HRV. Cheap work only; the heavy R2 re-decodes are fanned out onto a queue, + one bounded `(user, day)` unit per consumer invocation. +- **Every night at 3:30** (`30 3`): the backstop — re-decode HRV and respiratory rate + only for the recent nights still *missing* them (so a night is never decoded twice, and + a night the wake-time run left empty gets retried), true up steps, and prune minute rows + past their retention. Anything already computed is skipped. + +HRV is real now: the beat-to-beat R-R intervals live in the 1 Hz (V24) records, so +`biometrics.ts` re-decodes them from the raw bytes in R2 (off the request path) to drive +recovery, readiness, and HRV-based stress. `processUser` is where it happens for one person. It reads their minutes, pulls their baselines, and calls into the [analytics package](https://github.com/OpenStrap/analytics) @@ -69,6 +90,12 @@ signal, the coach plan, stress, nocturnal heart. The results land in `daily`, `s actually move day to day instead of collapsing into one flat value, which was a real bug early on. +The cross-day metrics that need long windows — training load (EWMA acute:chronic over +7/28 days), Banister fitness/fatigue/form, Foster monotony, sleep regularity — are +**seeded from the permanent `daily`/`sleep` tables**, not just the few days being +recomputed. We never re-decode old days (raw is gone after 14 days); we reuse the +already-derived rows, so a real 28-day ACWR or 14-night regularity actually exists. + Every number comes back wrapped: a value, a unit, a confidence between 0 and 1, a tier, and a label. If the inputs aren't there, the value is `null` and the confidence is `0`. I would rather show you a dash than make something up. The whole project falls apart the @@ -84,6 +111,9 @@ moment it starts inventing numbers, so it doesn't. | `decode.ts` | Hex frames into decoded samples | | `rollup.ts` | Decoded samples into per-minute buckets | | `analytics.ts` | The cron brain, `processUser` / `runAnalytics` | +| `queue.ts` | The queue consumer: one bounded `(user, job, day)` unit per invocation, and the wake-time HRV trigger | +| `biometrics.ts` | HRV (RMSSD/SDNN/LF-HF), recovery, stress, relative temp/SpO₂ — re-decoded from the R-R intervals in R2 | +| `steps_imu.ts` | Step counting from the wrist accelerometer (incremental + nightly true-up) | | `query.ts` | The read endpoints: today, sleep, strain, trends, chart, history | | `daydetail.ts` | Single-day drill-downs (the strain curve, the hypnogram, the stress band) | | `history.ts` | Range aggregation and the calendar heatmap | @@ -119,7 +149,7 @@ Admin stuff for when you run your own: `/admin/run-analytics`, `/admin/run-resp` ## The database Thirteen tables. The ones you'll care about: `minute` (the running-sum rollups, pruned at -90 days), `daily` and `sleep` and `sessions` (the derived output, mostly JSON columns for +10 days), `daily` and `sleep` and `sessions` (the derived output, mostly JSON columns for the structured bits like coach plans and HR zones), `baselines` (your resting HR, max HR, sleep need, the anchors everything else is measured against), and the auth trio (`users`, `otps`, `refresh_tokens`). Full DDL is in `src/db/schema.sql`, it's commented, go read it. diff --git a/src/analytics.ts b/src/analytics.ts index 2774487..b1f0caa 100644 --- a/src/analytics.ts +++ b/src/analytics.ts @@ -7,14 +7,16 @@ // Missing input → null + confidence 0; never fabricated. import { - calcRestingHR, calcStrain, calcHrZones, calcCalories, calcSleep, + calcRestingHR, calcStrain, calcHrZones, calcCalories, calcSleep, calcSleepPeriods, + stageHypnogram, calcSleepRegularity, detectSessions, calcLoad, calcFitnessTrend, calcVo2Max, calcFitnessModel, calcMonotony, - calcAnomaly, calcBaselines, buildCoach, + calcAnomaly, calcBaselines, seedBaselines, buildCoach, calcCycle, calcRestlessness, calcSleepStress, calcNocturnalHeart, buildNotifications, type Minute, type Profile, type Baseline, type DayHistory, type DailyStrain, type NightSummary, type SleepValue, type Metric, type Driver, } from 'openstrap-analytics' +import { readMinutes } from './minute_store' const DAY = 86400 @@ -31,6 +33,7 @@ interface MinuteRow { activity: number | null steps: number | null wrist_on: number | null + act_class?: string | null } const toMinute = (r: MinuteRow): Minute => ({ @@ -42,6 +45,7 @@ const toMinute = (r: MinuteRow): Minute => ({ activity: r.activity ?? 0, steps: r.steps ?? 0, wrist_on: !!r.wrist_on, + ...(r.act_class ? { act_class: r.act_class as Minute['act_class'] } : {}), }) // Build a per-metric flag entry from any Metric. @@ -63,7 +67,7 @@ async function loadProfile(db: D1Database, userId: string): Promise { type LoadedBaseline = Baseline & { sleeping_hr: number | null; resp_rate: number | null } -async function loadBaseline(db: D1Database, userId: string): Promise { +export async function loadBaseline(db: D1Database, userId: string): Promise { const b = await db.prepare( 'SELECT resting_hr, max_hr, sleep_need_min, skin_temp, chronic_strain, sleeping_hr, resp_rate FROM baselines WHERE user_id = ?', ).bind(userId).first() @@ -78,12 +82,11 @@ async function loadBaseline(db: D1Database, userId: string): Promise { - const { results } = await db.prepare( - 'SELECT ts_min, hr_avg, hr_min, hr_max, hr_n, activity, steps, wrist_on FROM minute ' + - 'WHERE user_id = ? AND ts_min >= ? AND ts_min < ? ORDER BY ts_min ASC', - ).bind(userId, from, to).all() - return (results ?? []).map(toMinute) +export async function loadMinutes(db: D1Database, userId: string, from: number, to: number): Promise { + // Day-packed store. processUser/wake windows are within HOT_DAYS → all in D1 + // minute_day (RAW_BUCKET omitted = no R2 fallback needed for hot reads). + const recs = await readMinutes({ DB: db }, userId, from, to) + return recs.map(toMinute) } // Sleep search window for the night that WAKES on `dateDayStart`: @@ -92,6 +95,45 @@ function sleepSearchWindow(dateDayStart: number): { from: number; to: number } { return { from: dateDayStart - 6 * 3600, to: dateDayStart + 12 * 3600 } } +// Precompute the intra-day cumulative-strain curve + day HR stats so /day/strain +// is a PURE READ (no live recompute on the endpoint). Uses the SAME resolved max +// HR + sex coefficients calcStrain used, so the curve's final point equals the +// stored daily.strain. Curve is downsampled to keep the JSON small. +function dayStrainDetail( + dayMin: Minute[], rhr: number, maxHr: number, sex?: 'm' | 'f', +): { curve: string; hrMax: number | null; hrMin: number | null; hrAvg: number | null } { + const [k, b] = sex === 'f' ? [0.86, 1.67] : [0.64, 1.92] + const denom = Math.max(1, maxHr - rhr) + const scale = (trimp: number) => Math.min(21, Math.log(trimp + 1) / Math.log(1.5)) + const sorted = [...dayMin].sort((a, c) => a.ts - c.ts) + let trimp = 0 + let hrMax = 0, hrMin = 0, hrSum = 0, hrN = 0 + const pts: { t: number; v: number }[] = [] + for (const m of sorted) { + const hr = m.hr_avg + if (m.wrist_on && hr > 0) { + const ratio = Math.max(0, Math.min(1, (hr - rhr) / denom)) + trimp += ratio * k * Math.exp(b * ratio) + hrMax = Math.max(hrMax, hr) + hrMin = hrMin === 0 ? hr : Math.min(hrMin, hr) + hrSum += hr; hrN++ + } + pts.push({ t: m.ts, v: Math.round(scale(trimp) * 10) / 10 }) + } + const MAXP = 120 + let curvePts = pts + if (pts.length > MAXP) { + const step = pts.length / MAXP + curvePts = Array.from({ length: MAXP }, (_, i) => pts[Math.min(pts.length - 1, Math.floor(i * step))]) + } + return { + curve: JSON.stringify(curvePts), + hrMax: hrMax || null, + hrMin: hrMin || null, + hrAvg: hrN ? Math.round(hrSum / hrN) : null, + } +} + interface DayBuf { date: string dayStart: number @@ -102,12 +144,16 @@ interface DayBuf { calories: ReturnType sleep: Metric wearMin: number - steps: number + daySteps: number // SUM of this day's AN-2554 minute.steps sleepStress: string // calcSleepStress JSON (nocturnal arousal) nocturnal: string // calcNocturnalHeart JSON sleepingHr: number | null // this night's sleeping-HR avg (for baseline) nocturnalElevated: boolean // overnight HR notably above baseline mainDrivers: Record // strain/rhr/sleep/zones driver graph + strainCurve: string // precomputed cumulative-strain curve (JSON) for /day/strain + hrMax: number | null + hrMin: number | null + hrAvg: number | null } /** @@ -135,6 +181,20 @@ export async function processUser( const lastDayStart = oldestDayStart + (dTo - 1) * DAY const profile = await loadProfile(db, userId) + + // Seed population/profile-prior baselines on the first-ever run (INSERT OR IGNORE + // → only when no row exists). Gives strain/HR-zones a personal resting/max HR from + // day 1 and warm-starts the EWMA autonomic baselines (RMSSD/sleeping-HR/SI) from + // physiological norms instead of cold. Real measured data overwrites these: the + // calcBaselines medians below, the nightly EWMA rolls, and the ≥5-night gated + // scores (recovery/stress) — which still need real nights and are never faked. + const seed = seedBaselines(profile) + await db.prepare( + 'INSERT OR IGNORE INTO baselines (user_id, resting_hr, max_hr, sleep_need_min, hrv_rmssd, sleeping_hr, resp_rate, hrv_si, updated_at) ' + + 'VALUES (?,?,?,?,?,?,?,?,?)', + ).bind(userId, seed.resting_hr, seed.max_hr, seed.sleep_need_min, seed.hrv_rmssd, + seed.sleeping_hr, seed.resp_rate, seed.hrv_si, now).run() + let baseline = await loadBaseline(db, userId) // HRV recovery (computed in biometrics.ts from RR) is read back here to drive @@ -146,12 +206,18 @@ export async function processUser( ).bind(userId).all<{ date: string; recovery: number }>() const recoveryByDate = new Map((recRows ?? []).map((r) => [r.date, r.recovery])) - // ── Pass 1: recompute baselines from prior derived history (last 30 days). ── + // ── Pass 1: recompute baselines from prior derived history. ── + // 90 days (not 30): baselines use the most-recent 30, but the trailing-window + // metrics (ACWR/monotony/RHR-trend, and SRI from sleep) need a longer seed — + // EWMA chronic (τ=28) wants ~3× that to converge. These are permanent derived + // rows and D1 reads are effectively free at this scale, so the longer read is + // ~$0. We NEVER re-decode old days (minute data is pruned at 10d); we just + // reuse what was already computed. Forward-only, no backfill. const { results: histRows } = await db.prepare( - 'SELECT date, resting_hr, strain, hr_zones FROM daily WHERE user_id = ? ORDER BY date DESC LIMIT 30', + 'SELECT date, resting_hr, strain, hr_zones, hr_max FROM daily WHERE user_id = ? ORDER BY date DESC LIMIT 90', ).bind(userId).all() const { results: sleepHistRows } = await db.prepare( - 'SELECT date, duration_min FROM sleep WHERE user_id = ? ORDER BY date DESC LIMIT 30', + 'SELECT date, duration_min, onset_ts, wake_ts FROM sleep WHERE user_id = ? ORDER BY date DESC LIMIT 90', ).bind(userId).all() const sleepByDate = new Map() for (const s of sleepHistRows ?? []) if (s.duration_min != null) sleepByDate.set(s.date, s.duration_min) @@ -166,12 +232,18 @@ export async function processUser( resting_hr: d.resting_hr ?? undefined, daily_strain: d.strain ?? undefined, sleep_duration_min: sleepByDate.get(d.date), + // The day's observed HR peak. calcBaselines only trusts it as a measured + // HRmax when it clears the age-predicted floor (a genuine effort); otherwise + // it falls back to Tanaka. Without this, the measured-max path was dead and + // everyone ran on the age formula. Forward-only, reuses the daily row. + session_hr_max: d.hr_max ?? undefined, zone_min, } as DayHistory }) if (history.length > 0) { - const bl = calcBaselines(history, profile) + // Baselines stay a 30-day metric even though we now fetch 90 for the windows. + const bl = calcBaselines(history.slice(-30), profile) baseline = { resting_hr: bl.resting_hr ?? baseline.resting_hr, max_hr: bl.max_hr ?? baseline.max_hr, @@ -210,6 +282,12 @@ export async function processUser( const arr = byDay.get(k) if (arr) arr.push(m); else byDay.set(k, [m]) } + // Per-minute RR for the window → the REM tiebreaker in stageHypnogram, so the STORED + // sleep duration/efficiency/stages match what /day/sleep shows (Today == Sleep). + const rrByMin = new Map() + for (const rec of await readMinutes({ DB: db }, userId, firstDayStart - DAY, lastDayStart + DAY)) { + if (rec.rr && rec.rr.length) rrByMin.set(rec.ts_min, rec.rr) + } const dayMinutes = (dayStart: number) => byDay.get(dayStart) ?? [] // Sleep window spans the previous evening → this noon; gather the two days. const sleepMinutes = (dayStart: number, from: number, to: number) => { @@ -221,6 +299,56 @@ export async function processUser( return out } + // ── Seed the trailing-window series from DURABLE derived history, so the + // cross-day metrics in Pass 3 (ACWR/monotony/RHR-trend/SRI) see their true + // 28/14/7-day windows instead of just the ~3-day recompute window. We reuse + // the daily/sleep rows already fetched above (no extra query, no re-decode). + // Gaps INSIDE the active span are genuine rest days → strain 0; days before + // the user's earliest derived row are NOT fabricated (else a young account + // gets a tiny chronic and a bogus high ACWR). + const SEED_DAYS = 84 // ≈3× the 28-day chronic time constant + let seedLen = 0 + { + const dayStrain = new Map() + const dayRhr = new Map() + for (const d of histRows ?? []) { + if (d.strain != null) dayStrain.set(d.date, d.strain) + if (d.resting_hr != null) dayRhr.set(d.date, d.resting_hr) + } + const dayWake = new Map() + for (const s of sleepHistRows ?? []) dayWake.set(s.date, { onset: s.onset_ts ?? null, wake: s.wake_ts ?? null }) + + // Earliest derived day within the seed window — start the continuous series + // there (don't 0-fill pre-account days). + const windowStart = firstDayStart - SEED_DAYS * DAY + let seedStart = firstDayStart + for (let d = windowStart; d < firstDayStart; d += DAY) { + const k = dayKey(d) + if (dayStrain.has(k) || dayRhr.has(k) || dayWake.has(k)) { seedStart = d; break } + } + for (let d = seedStart; d < firstDayStart; d += DAY) { + const k = dayKey(d) + dailyStrains.push({ ts: d, strain: dayStrain.get(k) ?? 0 }) // intra-span gap = rest day + rhrSeries.push(dayRhr.get(k) ?? null) + hrrSeries.push(null) // HRR isn't stored per-day → fitness-trend HRR stays sparse + const w = dayWake.get(k) + nightSummaries.push({ onset_ts: w?.onset ?? null, wake_ts: w?.wake ?? null }) + seedLen++ + } + } + + // Cycle-phase context (gated on track_cycle) so a luteal RHR/temp rise doesn't + // false-fire the anomaly signal. Loaded once; phase computed per-day below. + let cycleStarts: string[] = [] + const trackRow = await db.prepare('SELECT track_cycle FROM users WHERE id = ?') + .bind(userId).first<{ track_cycle: number | null }>() + if (trackRow?.track_cycle) { + const { results } = await db.prepare( + "SELECT date FROM cycle_log WHERE user_id = ? AND kind = 'start' ORDER BY date", + ).bind(userId).all<{ date: string }>() + cycleStarts = (results ?? []).map((r) => r.date) + } + for (let dayStart = firstDayStart; dayStart <= lastDayStart; dayStart += DAY) { const date = dayKey(dayStart) const dayMin = dayMinutes(dayStart) @@ -229,6 +357,21 @@ export async function processUser( const sw = sleepSearchWindow(dayStart) const sleepMin = sleepMinutes(dayStart, sw.from, sw.to) const sleep = calcSleep(sleepMin, baseline) + // RR-aware single-source staging: calcSleep finds the boundary, stageHypnogram + // (same as /day/sleep, incl. the REM tiebreaker) sets the STORED duration/awake/ + // efficiency/stages — so the Today summary can't disagree with the Sleep screen. + let sleepDuration = sleep.duration_min + let sleepEff = sleep.efficiency + let sleepStages = sleep.stages + if (sleep.onset_ts != null && sleep.wake_ts != null) { + const hyp = stageHypnogram(sleepMin, sleep.onset_ts, sleep.wake_ts, baseline, rrByMin) + if (hyp) { + const inBed = hyp.asleep_min + hyp.awake_min + sleepDuration = hyp.asleep_min + sleepEff = inBed > 0 ? Math.round((hyp.asleep_min / inBed) * 10000) / 10000 : sleepEff + sleepStages = { light_min: hyp.light_min, deep_min: hyp.deep_min, rem_min: hyp.rem_min } + } + } nightSummaries.push({ onset_ts: sleep.onset_ts, wake_ts: sleep.wake_ts }) // SRI for THIS night = regularity over a trailing ~2-week window ending here. // Needs ≥3 valid nights or it returns conf 0 → store null (matches §6). @@ -246,6 +389,8 @@ export async function processUser( const zones = calcHrZones(dayMin, baseline, profile) const calories = calcCalories(dayMin, profile, baseline.resting_hr, zones.max_hr_used) dailyStrains.push({ ts: dayStart, strain: strain.score }) + // Precompute the strain curve + HR stats here (cron) so /day/strain just reads. + const strainDetail = dayStrainDetail(dayMin, baseline.resting_hr, strain.max_hr_used, profile?.sex) // -- Sessions for this day. -- const sessions = detectSessions(dayMin, baseline, profile) @@ -259,24 +404,45 @@ export async function processUser( // user-owned and must survive a re-derive. Deleted tombstones are KEPT so a // user-deleted auto session isn't resurrected (its row's status stays // 'deleted'; the re-insert below only updates non-status fields via ON CONFLICT). + // ALSO preserve any auto session the user CONFIRMED/CORRECTED (the calibration + // ledger): exclude it from the DELETE and skip an overlapping re-detection so a + // baseline-drift start_ts shift can't insert a duplicate that re-overwrites the + // user's label. (Without this, every re-derive wiped user corrections.) + const { results: keepRev } = await db.prepare( + "SELECT start_ts, end_ts FROM sessions WHERE user_id = ? AND start_ts < ? AND end_ts > ? AND status != 'deleted' AND type_source IN ('confirmed','corrected')", + ).bind(userId, dayStart + DAY, dayStart).all<{ start_ts: number; end_ts: number }>() + const reviewed = (keepRev ?? []) as { start_ts: number; end_ts: number }[] + const overlapsReviewed = (s: { start_ts: number; end_ts: number }) => + reviewed.some((k) => s.start_ts < k.end_ts && s.end_ts > k.start_ts) statements.push( - db.prepare("DELETE FROM sessions WHERE user_id = ? AND start_ts >= ? AND start_ts < ? AND (source IS NULL OR source = 'auto') AND status != 'deleted'") + db.prepare("DELETE FROM sessions WHERE user_id = ? AND start_ts >= ? AND start_ts < ? AND (source IS NULL OR source = 'auto') AND COALESCE(type_source,'model') NOT IN ('confirmed','corrected') AND status != 'deleted'") .bind(userId, dayStart, dayStart + DAY), ) for (const s of sessions) { + if (overlapsReviewed(s)) continue const sid = `${userId}:${s.start_ts}` statements.push(db.prepare( - 'INSERT INTO sessions (user_id, id, start_ts, end_ts, type, avg_hr, max_hr, strain, calories, hrr60, zones, confidence, status, source) ' + - "VALUES (?,?,?,?,?,?,?,?,?,?,?,?,'done','auto') ON CONFLICT(user_id, id) DO UPDATE SET " + - 'end_ts=excluded.end_ts, type=excluded.type, avg_hr=excluded.avg_hr, max_hr=excluded.max_hr, ' + - 'strain=excluded.strain, calories=excluded.calories, hrr60=excluded.hrr60, zones=excluded.zones, confidence=excluded.confidence', + 'INSERT INTO sessions (user_id, id, start_ts, end_ts, type, avg_hr, max_hr, strain, calories, hrr60, zones, confidence, status, source, segments, detected_type, type_confidence, type_source) ' + + "VALUES (?,?,?,?,?,?,?,?,?,?,?,?,'done','auto',?,?,?,'model') ON CONFLICT(user_id, id) DO UPDATE SET " + + 'end_ts=excluded.end_ts, avg_hr=excluded.avg_hr, max_hr=excluded.max_hr, ' + + 'strain=excluded.strain, calories=excluded.calories, hrr60=excluded.hrr60, zones=excluded.zones, confidence=excluded.confidence, ' + + 'segments=excluded.segments, detected_type=excluded.detected_type, ' + + // Preserve a user-confirmed/corrected type across re-derivation; only the model fields refresh. + 'type_confidence=CASE WHEN sessions.type_source IN (\'confirmed\',\'corrected\') THEN sessions.type_confidence ELSE excluded.type_confidence END, ' + + 'type=CASE WHEN sessions.type_source IN (\'confirmed\',\'corrected\') THEN sessions.type ELSE excluded.type END', ).bind(userId, sid, s.start_ts, s.end_ts, s.type, Math.round(s.avg_hr), Math.round(s.max_hr), - s.strain, s.kcal, s.hrr60 == null ? null : Math.round(s.hrr60), JSON.stringify(s.zones), s.confidence)) + s.strain, s.kcal, s.hrr60 == null ? null : Math.round(s.hrr60), JSON.stringify(s.zones), s.confidence, + s.segments ? JSON.stringify(s.segments) : null, s.detected_type ?? s.type, s.type_confidence)) } - // -- Wear time (worn minutes) + daily steps (sum of detected per-minute). -- + // -- Wear time (worn minutes). -- const wearMin = dayMin.filter((m) => m.wrist_on).length - const steps = Math.round(dayMin.reduce((s, m) => s + (m.steps || 0), 0)) + // -- Steps = SUM of this day's per-minute AN-2554 counts (computed at ingest, + // stored in minute.steps). Folded into the daily row here at the close so past + // days have a permanent total after their minutes prune. NO R2, no recompute — + // steps_imu.ts is removed; AN-2554 is the only pedometer. Today's live total is + // served on-read by summing minute.steps (query.ts/daydetail.ts). -- + const daySteps = dayMin.reduce((a, m) => a + (m.steps ?? 0), 0) // -- Nocturnal heart (sleeping-HR dynamics over the main sleep period). -- const sleepWorn = (sleep.onset_ts && sleep.wake_ts) @@ -287,6 +453,9 @@ export async function processUser( // -- Sleep stress / nocturnal arousal (HR surges + motion in sleep; NOT HRV; // the HRV-based daytime stress is computed in biometrics.ts from RR). -- const sleepStress = calcSleepStress(sleepWorn, baseline) + // -- Restlessness (movement fragmentation; distinct from arousal). Folded into the + // sleep_stress JSON so no new column / upsert change is needed. -- + const restlessness = calcRestlessness(sleepWorn) // -- Main-metric drivers (the cross-metric "what affected this" graph). -- const mainDrivers: Record = { @@ -318,18 +487,46 @@ export async function processUser( 'efficiency=excluded.efficiency, light_min=excluded.light_min, deep_min=excluded.deep_min, ' + 'rem_min=excluded.rem_min, regularity=excluded.regularity, confidence=excluded.confidence, ' + 'flags=excluded.flags, updated_at=excluded.updated_at', - ).bind(userId, date, sleep.onset_ts, sleep.wake_ts, sleep.duration_min, sleep.efficiency, - sleep.stages?.light_min ?? null, sleep.stages?.deep_min ?? null, sleep.stages?.rem_min ?? null, + ).bind(userId, date, sleep.onset_ts, sleep.wake_ts, sleepDuration, sleepEff, + sleepStages?.light_min ?? null, sleepStages?.deep_min ?? null, sleepStages?.rem_min ?? null, regularityForSleep, sleep.confidence, sleepFlags, now)) sleepN++ + // -- Sleep v2 (multi-period; naps = shorter sleeps). ADDITIVE — the v1 write + // above is untouched. The window extends to THIS day's evening (18:00) so + // daytime naps are captured, without pulling in the NEXT night's onset + // (which belongs to date+1, same attribution convention as the night). -- + const periodsWin = sleepMinutes(dayStart, sw.from, dayStart + 18 * 3600) + const sleepV2 = calcSleepPeriods(periodsWin, baseline) + statements.push( + db.prepare('DELETE FROM sleep_periods WHERE user_id = ? AND date = ?').bind(userId, date), + ) + for (const p of sleepV2.periods) { + const pid = `${userId}:${p.onset_ts}` + statements.push(db.prepare( + 'INSERT INTO sleep_periods (user_id, id, date, onset_ts, wake_ts, duration_min, in_bed_min, efficiency, light_min, deep_min, rem_min, is_main, confidence, updated_at) ' + + 'VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?) ON CONFLICT(user_id, id) DO UPDATE SET ' + + 'date=excluded.date, wake_ts=excluded.wake_ts, duration_min=excluded.duration_min, in_bed_min=excluded.in_bed_min, ' + + 'efficiency=excluded.efficiency, light_min=excluded.light_min, deep_min=excluded.deep_min, rem_min=excluded.rem_min, ' + + 'is_main=excluded.is_main, confidence=excluded.confidence, updated_at=excluded.updated_at', + ).bind(userId, pid, date, p.onset_ts, p.wake_ts, p.duration_min, p.in_bed_min, p.efficiency, + p.stages?.light_min ?? null, p.stages?.deep_min ?? null, p.stages?.rem_min ?? null, + p.is_main ? 1 : 0, p.confidence, now)) + } + dayBuffer.push({ - date, dayStart, idx: dayBuffer.length, rhr, strain, zones, calories, sleep, wearMin, steps, - sleepStress: JSON.stringify(sleepStress), + // idx points into the parallel arrays, which now START with `seedLen` + // seeded history days — so trailing slices in Pass 3 reach into real history. + date, dayStart, idx: seedLen + dayBuffer.length, rhr, strain, zones, calories, sleep, wearMin, daySteps, + sleepStress: JSON.stringify({ ...sleepStress, restlessness }), nocturnal: JSON.stringify(nocturnal), sleepingHr: nocturnal.sleeping_hr_avg, nocturnalElevated: nocturnal.elevated, mainDrivers, + strainCurve: strainDetail.curve, + hrMax: strainDetail.hrMax, + hrMin: strainDetail.hrMin, + hrAvg: strainDetail.hrAvg, }) } @@ -337,8 +534,9 @@ export async function processUser( // trailing windows that end at that day (so ACWR/fitness/SRI/anomaly vary // across the history instead of collapsing to a single value). ── for (const buf of dayBuffer) { - const { date, idx, rhr, strain, zones, calories, sleep, wearMin, steps, - sleepStress, nocturnal, nocturnalElevated, mainDrivers } = buf + const { date, idx, rhr, strain, zones, calories, sleep, wearMin, daySteps, + sleepStress, nocturnal, nocturnalElevated, mainDrivers, + strainCurve, hrMax, hrMin, hrAvg } = buf // Trailing windows ending at this day (inclusive). const sriNights = nightSummaries.slice(Math.max(0, idx - 13), idx + 1) // ~2 weeks @@ -372,12 +570,13 @@ export async function processUser( // Readiness + HRV CV/irregular are owned by biometrics.ts (HRV is fresh there). const nocDip = (() => { try { return JSON.parse(nocturnal)?.dip_pct ?? null } catch { return null } })() + const cyclePhase = cycleStarts.length ? calcCycle(cycleStarts, date).phase : null const anomaly = calcAnomaly({ recent_rhr: recentRhr, skin_temp: baseline.skin_temp ?? null, - sleep_efficiency: sleep.efficiency || null, + sleep_efficiency: sleep.efficiency != null ? sleep.efficiency : null, // keep a real 0% (not coerced to "no data") baseline_sleep_efficiency: null, - }, baseline) + }, baseline, { cyclePhase }) const flags = JSON.stringify({ strain: flag(strain, 'Strain'), @@ -385,7 +584,7 @@ export async function processUser( recovery: { c: recovery != null ? 0.8 : 0, tier: 'HIGH', label: 'Recovery (HRV)' }, calories: flag(calories, calories.label), zones: flag(zones, zones.max_hr_source === 'age' ? 'HR zones (estimated max HR)' : 'HR zones'), - steps: { c: steps > 0 ? 0.5 : 0, tier: 'ESTIMATE', label: 'Steps (est.)' }, + steps: { c: wearMin > 0 ? 0.5 : 0, tier: 'ESTIMATE', label: 'Steps (est.)' }, load: flag(load, `Load — ${load.band}`), fitness: flag(fitness, `Fitness trend — ${fitness.direction}`), anomaly: flag(anomaly, anomaly.note), @@ -443,7 +642,7 @@ export async function processUser( sleep_last_min: sleep.duration_min || null, sleep_need_min: needFloored, sleep_debt_min: sleepDebt, - sleep_efficiency: sleep.efficiency || null, + sleep_efficiency: sleep.efficiency != null ? sleep.efficiency : null, // keep a real 0% (not coerced to "no data") sri: sri.sri, fitness_direction: fitness.direction, anomaly: bodyAlert ? JSON.parse(bodyAlert) : null, @@ -461,8 +660,9 @@ export async function processUser( // It writes `drivers` (main metrics) via COALESCE-free set but biometrics // read-merges, and on the hourly path (no biometrics) main drivers stand alone. statements.push(db.prepare( - 'INSERT INTO daily (user_id, date, strain, resting_hr, calories, wear_min, steps, hr_zones, acwr, fitness_trend, anomaly, coach, nocturnal, sleep_stress, drivers, vo2max, fitness, fatigue, form, monotony, nocturnal_dip_pct, confidence, flags, updated_at) ' + - 'VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) ON CONFLICT(user_id, date) DO UPDATE SET ' + + // `steps` = SUM of this day's AN-2554 minute counts (the only pedometer now). + 'INSERT INTO daily (user_id, date, strain, resting_hr, calories, wear_min, steps, hr_zones, acwr, fitness_trend, anomaly, coach, nocturnal, sleep_stress, drivers, vo2max, fitness, fatigue, form, monotony, nocturnal_dip_pct, strain_curve, hr_max, hr_min, hr_avg, confidence, flags, updated_at) ' + + 'VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) ON CONFLICT(user_id, date) DO UPDATE SET ' + 'strain=excluded.strain, resting_hr=excluded.resting_hr, ' + 'calories=excluded.calories, wear_min=excluded.wear_min, steps=excluded.steps, hr_zones=excluded.hr_zones, ' + 'acwr=excluded.acwr, fitness_trend=excluded.fitness_trend, anomaly=excluded.anomaly, coach=excluded.coach, ' + @@ -470,12 +670,14 @@ export async function processUser( 'drivers=json_patch(COALESCE(daily.drivers,\'{}\'), excluded.drivers), ' + 'vo2max=excluded.vo2max, fitness=excluded.fitness, fatigue=excluded.fatigue, form=excluded.form, ' + 'monotony=excluded.monotony, nocturnal_dip_pct=excluded.nocturnal_dip_pct, ' + + 'strain_curve=excluded.strain_curve, hr_max=excluded.hr_max, hr_min=excluded.hr_min, hr_avg=excluded.hr_avg, ' + 'confidence=excluded.confidence, flags=excluded.flags, updated_at=excluded.updated_at', ).bind(userId, date, strain.score, rhr.resting_hr == null ? null : Math.round(rhr.resting_hr), calories.kcal, - wearMin, steps, JSON.stringify(zones), load.acwr, fitness.direction, + wearMin, daySteps, JSON.stringify(zones), load.acwr, fitness.direction, bodyAlert, JSON.stringify(coach), nocturnal, sleepStress, driversJson, vo2.vo2max, fitModel.fitness, fitModel.fatigue, fitModel.form, monotony.monotony, nocDip, + strainCurve, hrMax, hrMin, hrAvg, confidence, flags, now)) dailyN++ diff --git a/src/biometrics.ts b/src/biometrics.ts deleted file mode 100644 index c46886b..0000000 --- a/src/biometrics.ts +++ /dev/null @@ -1,259 +0,0 @@ -// biometrics.ts — HRV + relative skin-temp + relative SpO₂, re-decoded from the -// raw V24 records already in R2. These fields ride in every 1 Hz historical record -// (rr intervals @18/19, raw red ADC @64, raw temp ADC @68) — validated on 127,971 -// of our own records (RR 99.7% physiological; |g|≈1.012). No new capture needed. -// -// HRV is the real one: RMSSD over artifact-filtered beat-to-beat intervals during -// the resting/sleep window — the same substrate WHOOP builds recovery on. SpO₂ and -// skin temp are RAW ADCs (the band never sends a finished %/°C), so we only ever -// express them as a deviation from the user's own baseline — relative, never absolute. - -import { parse_r24 } from 'openstrap-protocol/ts/records' -import { - timeDomainHrv, freqDomainHrv, baevskyStressIndex, - calcRecovery, calcStress, calcIllness, - calcHrvStability, calcIrregular, calcReadinessIndex, -} from 'openstrap-analytics' - -const DAY = 86400 - -interface BioEnv { DB: D1Database; RAW_BUCKET: R2Bucket } - -const hexToBytes = (hex: string): Uint8Array => { - const clean = hex.trim() - const m = clean.match(/.{1,2}/g) - if (!m) return new Uint8Array(0) - return new Uint8Array(m.map((b) => parseInt(b, 16))) -} - -const median = (a: number[]): number => { - if (!a.length) return 0 - const s = [...a].sort((x, y) => x - y) - const m = Math.floor(s.length / 2) - return s.length % 2 ? s[m] : (s[m - 1] + s[m]) / 2 -} - -// One decoded V24 record's biometrics, or null if it isn't a usable V24. -interface BioSample { ts: number; hr: number; rr: number[]; temp: number; spo2: number } -function decodeV24(hex: string): BioSample | null { - const b = hexToBytes(hex) - if (b.length < 89 || b[0] !== 0x2f || b[1] !== 24) return null - const r = parse_r24(b) - if (!r || !(r.ts_epoch > 0)) return null - return { - ts: r.ts_epoch, - hr: r.hr, - rr: r.rr_intervals_ms.filter((x) => x >= 300 && x <= 2000), // physiological - temp: r.skin_temp_raw, - spo2: r.spo2_red_raw, - } -} - -/** - * RMSSD (ms) over a time-ordered RR stream, with successive-difference artifact - * rejection (drop jumps > 200 ms — ectopics / missed beats). The standard - * short-term HRV index; what nocturnal recovery is built on. - */ -export function rmssd(samples: BioSample[]): { rmssd: number | null; nBeats: number } { - const seq: number[] = [] - for (const s of [...samples].sort((a, b) => a.ts - b.ts)) for (const x of s.rr) seq.push(x) - if (seq.length < 30) return { rmssd: null, nBeats: seq.length } - let sumSq = 0, n = 0 - for (let i = 1; i < seq.length; i++) { - const d = seq[i] - seq[i - 1] - if (Math.abs(d) <= 200) { sumSq += d * d; n++ } - } - if (n < 20) return { rmssd: null, nBeats: seq.length } - return { rmssd: Math.round(Math.sqrt(sumSq / n) * 10) / 10, nBeats: seq.length } -} - -async function rawKeysInWindow(bucket: R2Bucket, userId: string, from: number, to: number): Promise { - const out: string[] = [] - let cursor: string | undefined - do { - const listing = await bucket.list({ prefix: `raw/${userId}/`, cursor, limit: 1000 }) - for (const o of listing.objects) { - const m = o.key.match(/-(\d+)-(\d+)\.txt$/) - if (!m) { out.push(o.key); continue } - const minTs = parseInt(m[1]), maxTs = parseInt(m[2]) - if (maxTs >= from && minTs <= to) out.push(o.key) - } - cursor = listing.truncated ? listing.cursor : undefined - } while (cursor) - return out -} - -// Read + decode resting V24 records in [from,to]. "Resting" = HR in a calm band -// so HRV reflects parasympathetic tone, not exercise. Bounded to `maxObjects` -// most-recent objects — a Worker request has a tight CPU/subrequest budget, and -// each object holds ~50–90 records, so ~20 objects ≈ a few hundred beats, which -// is more than enough for a stable RMSSD. -async function restingSamples(env: BioEnv, userId: string, from: number, to: number, maxObjects: number): Promise { - const keys = await rawKeysInWindow(env.RAW_BUCKET, userId, from, to) - // Newest first by the maxTs embedded in the key (…--.txt). - const keyTs = (k: string) => { const m = k.match(/-(\d+)\.txt$/); return m ? parseInt(m[1]) : 0 } - keys.sort((a, b) => keyTs(b) - keyTs(a)) - const out: BioSample[] = [] - for (const key of keys.slice(0, maxObjects)) { - const obj = await env.RAW_BUCKET.get(key) - if (!obj) continue - const text = await obj.text() - for (const line of text.split('\n')) { - const s = decodeV24(line) - if (s && s.ts >= from && s.ts <= to && s.hr > 0 && s.hr <= 70) out.push(s) - } - } - return out -} - -interface NightBio { - date: string - rmssd: number | null; sdnn: number | null; lfhf: number | null; si: number | null - resp: number | null; respConf: number; nBeats: number - temp: number | null; spo2: number | null - rr: number[] // time-ordered RR stream (ms) for stress scoring -} - -/** Collect the time-ordered RR stream over a resting window + full HRV indices. */ -async function computeDay(env: BioEnv, userId: string, date: string, from: number, to: number, maxObjects: number): Promise { - const samples = (await restingSamples(env, userId, from, to, maxObjects)).sort((a, b) => a.ts - b.ts) - const rr: number[] = [] - for (const s of samples) for (const x of s.rr) rr.push(x) - const td = timeDomainHrv(rr) - const fd = freqDomainHrv(rr) - const si = baevskyStressIndex(rr) - const temps = samples.map((s) => s.temp).filter((t) => t > 0) - const spo2s = samples.map((s) => s.spo2).filter((s) => s > 0) - return { - date, - rmssd: td.rmssd, sdnn: td.sdnn, lfhf: fd.lf_hf, si: si.si, - resp: fd.resp_rate, respConf: fd.resp_conf, nBeats: td.n_beats, - temp: temps.length ? median(temps) : null, - spo2: spo2s.length ? median(spo2s) : null, - rr, - } -} - -interface DailyHistRow { date: string; resting_hr: number | null; hrv_rmssd: number | null; hrv_si: number | null; skin_temp_idx: number | null } - -/** - * runBiometrics — for the last `days` days, re-decode resting RR from R2 and - * compute the FULL HRV suite (RMSSD/SDNN/LF-HF/Baevsky-SI/resp), then derive the - * HRV-based metrics that need beat-to-beat data: RECOVERY (Plews lnRMSSD z-score), - * STRESS (Baevsky SI personal-relative), and the multivariate ILLNESS signal - * (Mahalanobis). Writes those + their drivers to the daily row. Heavy (R2 reads) → - * nightly cron / admin only, never inline ingest. - */ -export async function runBiometrics(env: BioEnv, userId: string, days = 3): Promise<{ days: number; computed: number }> { - const now = Math.floor(Date.now() / 1000) - - const since = new Date((now - days * DAY) * 1000).toISOString().slice(0, 10) - const { results: sleeps } = await env.DB.prepare( - 'SELECT date, onset_ts, wake_ts, duration_min FROM sleep WHERE user_id = ? AND date >= ? AND onset_ts IS NOT NULL AND wake_ts IS NOT NULL', - ).bind(userId, since).all<{ date: string; onset_ts: number; wake_ts: number; duration_min: number | null }>() - const sleepByDate = new Map((sleeps ?? []).map((s) => [s.date, s])) - - // Sleep-need baseline (for the composite Readiness sleep component). - const baseRow = await env.DB.prepare('SELECT sleep_need_min FROM baselines WHERE user_id = ?') - .bind(userId).first<{ sleep_need_min: number | null }>() - const sleepNeedMin = baseRow?.sleep_need_min ?? null - - // Prior history (for Plews baseline, personal SI baseline, illness covariance). - const { results: hist } = await env.DB.prepare( - 'SELECT date, resting_hr, hrv_rmssd, hrv_si, skin_temp_idx FROM daily WHERE user_id = ? ORDER BY date DESC LIMIT 60', - ).bind(userId).all() - const histRows = (hist ?? []) - const rhrByDate = new Map(histRows.map((h) => [h.date, h.resting_hr])) - const rmssdHist = histRows.map((h) => h.hrv_rmssd).filter((x): x is number => x != null) - const siHist = histRows.map((h) => h.hrv_si).filter((x): x is number => x != null) - const rhrHist = histRows.map((h) => h.resting_hr).filter((x): x is number => x != null) - const tempHist = histRows.map((h) => h.skin_temp_idx).filter((x): x is number => x != null) - - // Per-request work budget (Worker CPU/subrequest limit). Newest day first. - const PER_DAY = 12 - const TOTAL_OBJECTS = 24 - let spent = 0 - const out: NightBio[] = [] - for (let d = 0; d < days; d++) { - const dayStart = Math.floor((now - d * DAY) / DAY) * DAY - const date = new Date(dayStart * 1000).toISOString().slice(0, 10) - const budget = Math.min(PER_DAY, TOTAL_OBJECTS - spent) - if (budget <= 0) { out.push({ date, rmssd: null, sdnn: null, lfhf: null, si: null, resp: null, respConf: 0, nBeats: 0, temp: null, spo2: null, rr: [] }); continue } - const sl = sleepByDate.get(date) - const from = sl ? sl.onset_ts : dayStart - const to = sl ? sl.wake_ts + 60 : dayStart + DAY - out.push(await computeDay(env, userId, date, from, to, budget)) - spent += budget - } - - // Baselines: rolling medians over what we just computed + whatever's stored. - const allRmssd = [...out.map((o) => o.rmssd).filter((x): x is number => x != null), ...rmssdHist] - const allSi = [...out.map((o) => o.si).filter((x): x is number => x != null), ...siHist] - const temps = out.map((o) => o.temp).filter((x): x is number => x != null) - const spo2s = out.map((o) => o.spo2).filter((x): x is number => x != null) - const blHrv = allRmssd.length ? median(allRmssd) : null - const blSi = allSi.length ? median(allSi) : null - const blTemp = temps.length ? median(temps) : null - const blSpo2 = spo2s.length ? median(spo2s) : null - if (blHrv != null || blTemp != null || blSpo2 != null || blSi != null) { - await env.DB.prepare( - 'UPDATE baselines SET hrv_rmssd = COALESCE(?, hrv_rmssd), hrv_si = COALESCE(?, hrv_si), skin_temp_raw = COALESCE(?, skin_temp_raw), spo2_raw = COALESCE(?, spo2_raw) WHERE user_id = ?', - ).bind(blHrv, blSi, blTemp, blSpo2, userId).run() - } - - let computed = 0 - for (const o of out) { - const conf = o.rmssd == null ? 0 : Math.round(Math.min(1, o.nBeats / 500) * 1000) / 1000 - const tempIdx = (o.temp != null && blTemp != null) ? Math.round((o.temp - blTemp) * 10) / 10 : null - const spo2Idx = (o.spo2 != null && blSpo2 != null) ? Math.round((o.spo2 - blSpo2) * 10) / 10 : null - - // HRV-based metrics (all published algorithms; null when no usable RR). - const recovery = calcRecovery(o.rmssd, rmssdHist, { date: o.date }) - const stress = calcStress(o.rr, siHist, { date: o.date }) - const illness = calcIllness( - { resting_hr: rhrByDate.get(o.date) ?? null, rmssd: o.rmssd, skin_temp: tempIdx }, - { resting_hr: rhrHist, rmssd: rmssdHist, skin_temp: tempHist }, - ) - - // HRV stability (CV of recent nightly RMSSD) + irregular-rhythm screen (RR). - const hrvStab = calcHrvStability([o.rmssd, ...rmssdHist].filter((x): x is number => x != null).slice(0, 14)) - const irregular = calcIrregular(o.rr) - - // Composite Readiness — recovery (HRV) ∩ sleep vs need ∩ nocturnal dip ∩ arousal. - // Read this day's dip + sleep-stress (written by processUser) alongside drivers. - await env.DB.prepare('INSERT OR IGNORE INTO daily(user_id, date) VALUES(?, ?)').bind(userId, o.date).run() - const existing = await env.DB.prepare('SELECT drivers, sleep_stress, nocturnal_dip_pct FROM daily WHERE user_id = ? AND date = ?') - .bind(userId, o.date).first<{ drivers: string | null; sleep_stress: string | null; nocturnal_dip_pct: number | null }>() - let drivers: Record = {} - try { drivers = existing?.drivers ? JSON.parse(existing.drivers) : {} } catch { drivers = {} } - if (recovery.drivers) drivers.recovery = recovery.drivers - if (stress.drivers) drivers.stress = stress.drivers - if (illness.drivers) drivers.illness = illness.drivers - - const sleepStressScore = (() => { try { return existing?.sleep_stress ? JSON.parse(existing.sleep_stress)?.score ?? null : null } catch { return null } })() - const readiness = calcReadinessIndex({ - recovery: recovery.score, - sleepDurationMin: sleepByDate.get(o.date)?.duration_min ?? null, - sleepNeedMin, - dipPct: existing?.nocturnal_dip_pct ?? null, - sleepStress: sleepStressScore, - }) - if (readiness.drivers) drivers.readiness = readiness.drivers - - await env.DB.prepare( - 'UPDATE daily SET recovery = ?, hrv_rmssd = ?, hrv_conf = ?, hrv_sdnn = ?, hrv_lfhf = ?, hrv_si = ?, ' + - 'hrv_cv = ?, irregular = ?, readiness = ?, ' + - 'resp_rate = COALESCE(?, resp_rate), resp_conf = COALESCE(?, resp_conf), ' + - 'skin_temp_idx = ?, spo2_idx = ?, stress = ?, illness = ?, drivers = ?, updated_at = ? ' + - 'WHERE user_id = ? AND date = ?', - ).bind( - recovery.score, o.rmssd, conf, o.sdnn, o.lfhf, o.si, - hrvStab.cv, JSON.stringify(irregular), readiness.score, - o.respConf >= 0.3 ? o.resp : null, o.respConf >= 0.3 ? o.respConf : null, - tempIdx, spo2Idx, JSON.stringify(stress), JSON.stringify(illness), JSON.stringify(drivers), now, - userId, o.date, - ).run() - if (o.rmssd != null) computed++ - } - return { days, computed } -} diff --git a/src/biometrics_minute.ts b/src/biometrics_minute.ts new file mode 100644 index 0000000..e2b01d8 --- /dev/null +++ b/src/biometrics_minute.ts @@ -0,0 +1,191 @@ +// biometrics_minute.ts — [v2] HRV + recovery from minute.rr (D1), NOT R2. +// +// The demand-driven engine stores per-minute RR at ingest (minute.rr), so the +// headline HRV stack (RMSSD/SDNN/LF-HF, Plews recovery, Baevsky stress, illness, +// irregular screen) is computed from D1 — no R2 list/get, no re-decode. Skin-temp +// and SpO₂ now ride the same minute blob: ingest aggregates per-minute red/IR/temp +// ADCs (wrist-on), and the close path derives a RELATIVE skin-temp index + a red/IR +// SpO₂ index (confidence-gated) vs rolling personal baselines — still zero extra R2. +// +// Mirrors runBiometrics' write contract exactly (same daily columns, same +// null-safe COALESCE / json_patch drivers merge) so /today and the trend caches +// see an identical shape. + +import { + timeDomainHrv, freqDomainHrv, baevskyStressIndex, + calcRecovery, calcStress, calcIllness, calcHrvStability, calcIrregular, calcReadinessIndex, + calcSpo2Index, calcDesaturation, calcCycle, median, +} from 'openstrap-analytics' +import type { CyclePhase } from 'openstrap-analytics' +import { readMinutes } from './minute_store' +import { respFromMinuteGreen } from './resp' + +// [v3] RAW_BUCKET + MINUTE_SOURCE: RR comes via loadDayRr (D1 minute.rr by default, +// R2-decoded series when MINUTE_SOURCE='r2'). +interface BioEnv { DB: D1Database; RAW_BUCKET?: R2Bucket; MINUTE_SOURCE?: string } + +interface DailyHistRow { date: string; resting_hr: number | null; hrv_rmssd: number | null; hrv_si: number | null; skin_temp_idx: number | null; resp_rate: number | null } + +/** + * Compute + persist HRV/recovery for ONE physiological night, sourced from D1 + * minute.rr over [from,to] (sleep window). Idempotent & null-safe: a night with + * no usable RR leaves the stored row untouched (never regresses a good night). + */ +export async function runBiometricsMinute( + env: BioEnv, userId: string, date: string, from: number, to: number, +): Promise<{ computed: boolean }> { + const now = Math.floor(Date.now() / 1000) + + // 1. Read the sleep window's minutes ONCE (D1 hot + R2 sealed) → RR stream + optical + // aggregates. Single read keeps the close path cheap (no extra R2). + const recs = await readMinutes(env, userId, from, to) + const rr: number[] = [] + const spo2Ratios: number[] = [] + const tempVals: number[] = [] + for (const m of recs) { + for (const v of m.rr ?? []) if (v >= 300 && v <= 2000) rr.push(v) + // Optical: per-minute mean red/IR ratio + mean temp (wrist-on samples only at ingest). + if (m.opt_n && m.opt_n > 0 && m.ir_sum && m.ir_sum > 0) { + spo2Ratios.push((m.red_sum ?? 0) / m.ir_sum) + tempVals.push((m.temp_sum ?? 0) / m.opt_n) + } + } + + const td = timeDomainHrv(rr) + if (td.rmssd == null) return { computed: false } // no usable HRV → never write null + + const fd = freqDomainHrv(rr) + const si = baevskyStressIndex(rr) + + // PPG-resp from the stored R21 green RIIV proxy (D1, replaces the R2 re-decode). + // Present only when the night had a live optical session; else null → RSA-from-RR + // (fd.resp_rate) covers the night. Prefer PPG when it's confident. + const ppgResp = respFromMinuteGreen(recs) + const respRate = ppgResp.confidence >= 0.5 ? ppgResp.resp_rate + : (fd.resp_conf >= 0.3 ? fd.resp_rate : null) + const respConf = ppgResp.confidence >= 0.5 ? ppgResp.confidence + : (fd.resp_conf >= 0.3 ? fd.resp_conf : null) + + // 2. Prior history for Plews baseline / personal SI / illness covariance + sleep-need. + const { results: hist } = await env.DB.prepare( + 'SELECT date, resting_hr, hrv_rmssd, hrv_si, skin_temp_idx, resp_rate FROM daily WHERE user_id = ? ORDER BY date DESC LIMIT 60', + ).bind(userId).all() + const histRows = hist ?? [] + const rmssdHist = histRows.map((h) => h.hrv_rmssd).filter((x): x is number => x != null) + const siHist = histRows.map((h) => h.hrv_si).filter((x): x is number => x != null) + const rhrHist = histRows.map((h) => h.resting_hr).filter((x): x is number => x != null) + const tempHist = histRows.map((h) => h.skin_temp_idx).filter((x): x is number => x != null) + const respHist = histRows.map((h) => h.resp_rate).filter((x): x is number => x != null) + const rhrByDate = new Map(histRows.map((h) => [h.date, h.resting_hr])) + + // Cycle-phase context (only for users tracking their cycle) so a luteal-phase + // temp/RHR rise isn't mistaken for illness. One cheap gated read; null otherwise. + let cyclePhase: CyclePhase | null = null + const trackRow = await env.DB.prepare('SELECT track_cycle FROM users WHERE id = ?') + .bind(userId).first<{ track_cycle: number | null }>() + if (trackRow?.track_cycle) { + const { results: starts } = await env.DB.prepare( + "SELECT date FROM cycle_log WHERE user_id = ? AND kind = 'start' ORDER BY date", + ).bind(userId).all<{ date: string }>() + const startDates = (starts ?? []).map((s) => s.date) + if (startDates.length) cyclePhase = calcCycle(startDates, date).phase + } + + const baseRow = await env.DB.prepare( + // spo2_raw holds the rolling baseline red/IR RATIO (v2 uses the ratio, not raw red); + // skin_temp_raw holds the rolling baseline temp ADC. + 'SELECT sleep_need_min, spo2_raw, skin_temp_raw FROM baselines WHERE user_id = ?', + ).bind(userId).first<{ sleep_need_min: number | null; spo2_raw: number | null; skin_temp_raw: number | null }>() + const sleepNeedMin = baseRow?.sleep_need_min ?? null + const sleepRow = await env.DB.prepare( + 'SELECT duration_min FROM sleep WHERE user_id = ? AND date = ?', + ).bind(userId, date).first<{ duration_min: number | null }>() + + // 3a. Optical metrics (RELATIVE): red/IR SpO₂ index + skin-temp index, vs rolling + // personal baselines. Confidence-gated in analytics (noisy nights → low conf/null). + const baseRatio = baseRow?.spo2_raw ?? null + const baseTemp = baseRow?.skin_temp_raw ?? null + const spo2 = calcSpo2Index(spo2Ratios, baseRatio) + const nightTemp = tempVals.length >= 30 ? median(tempVals) : null + const tempIdx = (nightTemp != null && baseTemp != null) ? Math.round((nightTemp - baseTemp) * 10) / 10 : null + // EWMA-roll the baselines (α=0.1) so they adapt slowly; seed on first valid night. + const newBaseRatio = spo2.night_ratio != null + ? (baseRatio != null ? Math.round((baseRatio * 0.9 + spo2.night_ratio * 0.1) * 10000) / 10000 : spo2.night_ratio) + : null + const newBaseTemp = nightTemp != null + ? (baseTemp != null ? Math.round((baseTemp * 0.9 + nightTemp * 0.1) * 10) / 10 : Math.round(nightTemp * 10) / 10) + : null + + // 3b. Derived HRV metrics (published; SpO₂/temp now sourced from minute optical aggregates). + const conf = Math.round(Math.min(1, td.n_beats / 500) * 1000) / 1000 + const recovery = calcRecovery(td.rmssd, rmssdHist, { date }) + const stress = calcStress(rr, siHist, { date }) + const illness = calcIllness( + { resting_hr: rhrByDate.get(date) ?? null, rmssd: td.rmssd, skin_temp: tempIdx, resp_rate: respRate }, + { resting_hr: rhrHist, rmssd: rmssdHist, skin_temp: tempHist, resp_rate: respHist }, + { cyclePhase }, + ) + const hrvStab = calcHrvStability([td.rmssd, ...rmssdHist].filter((x): x is number => x != null).slice(0, 14)) + const irregular = calcIrregular(rr) + + // 4. Composite Readiness (reads this night's dip + sleep-stress written by processUser). + await env.DB.prepare('INSERT OR IGNORE INTO daily(user_id, date) VALUES(?, ?)').bind(userId, date).run() + const existing = await env.DB.prepare( + 'SELECT sleep_stress, nocturnal_dip_pct FROM daily WHERE user_id = ? AND date = ?', + ).bind(userId, date).first<{ sleep_stress: string | null; nocturnal_dip_pct: number | null }>() + const sleepStressScore = (() => { try { return existing?.sleep_stress ? JSON.parse(existing.sleep_stress)?.score ?? null : null } catch { return null } })() + const readiness = calcReadinessIndex({ + recovery: recovery.score, + sleepDurationMin: sleepRow?.duration_min ?? null, + sleepNeedMin, + dipPct: existing?.nocturnal_dip_pct ?? null, + sleepStress: sleepStressScore, + }) + + const bioDrivers: Record = {} + if (recovery.drivers) bioDrivers.recovery = recovery.drivers + if (stress.drivers) bioDrivers.stress = stress.drivers + if (illness.drivers) bioDrivers.illness = illness.drivers + if (readiness.drivers) bioDrivers.readiness = readiness.drivers + + if (spo2.drivers) bioDrivers.spo2 = spo2.drivers + + // Overnight SpO₂ desaturation screen (ODI-style, RELATIVE) from the same red/IR ratios. + // Folded into drivers so no new daily column / UPDATE change is needed. + const desat = calcDesaturation(spo2Ratios, baseRatio) + if (desat.confidence > 0 && desat.events > 0) { + bioDrivers.desaturation = { events: desat.events, odi: desat.odi, deepest_pct: desat.deepest_pct, note: desat.note } + } + + // 5. Persist (same null-safe COALESCE + json_patch contract as runBiometrics). SpO₂/temp + // now sourced from minute optical aggregates → spo2_idx / skin_temp_idx (RELATIVE). + await env.DB.prepare( + 'UPDATE daily SET recovery = ?, hrv_rmssd = COALESCE(?, hrv_rmssd), hrv_conf = ?, ' + + 'hrv_sdnn = COALESCE(?, hrv_sdnn), hrv_lfhf = COALESCE(?, hrv_lfhf), hrv_si = COALESCE(?, hrv_si), ' + + 'hrv_cv = COALESCE(?, hrv_cv), irregular = ?, readiness = ?, ' + + 'resp_rate = COALESCE(?, resp_rate), resp_conf = COALESCE(?, resp_conf), ' + + 'spo2_idx = COALESCE(?, spo2_idx), skin_temp_idx = COALESCE(?, skin_temp_idx), ' + + 'stress = ?, illness = ?, drivers = json_patch(COALESCE(drivers, \'{}\'), ?), updated_at = ? ' + + 'WHERE user_id = ? AND date = ?', + ).bind( + recovery.score, td.rmssd, conf, td.sdnn, fd.lf_hf, si.si, + hrvStab.cv, JSON.stringify(irregular), readiness.score, + respRate, respConf, + // SpO₂ index only when confidence is meaningful (noisy nights → keep prior, don't regress). + spo2.index != null && spo2.confidence >= 0.3 ? spo2.index : null, tempIdx, + JSON.stringify(stress), JSON.stringify(illness), JSON.stringify(bioDrivers), now, + userId, date, + ).run() + + // 5b. Roll the optical baselines (EWMA). Only when we got a fresh night value; null-safe. + if (newBaseRatio != null || newBaseTemp != null) { + await env.DB.prepare( + 'UPDATE baselines SET spo2_raw = COALESCE(?, spo2_raw), skin_temp_raw = COALESCE(?, skin_temp_raw), updated_at = ? WHERE user_id = ?', + ).bind(newBaseRatio, newBaseTemp, now, userId).run() + // Ensure a baselines row exists (first-ever night for a brand-new user). + await env.DB.prepare( + 'INSERT OR IGNORE INTO baselines(user_id, spo2_raw, skin_temp_raw, updated_at) VALUES(?,?,?,?)', + ).bind(userId, newBaseRatio, newBaseTemp, now).run() + } + return { computed: true } +} diff --git a/src/cache.ts b/src/cache.ts new file mode 100644 index 0000000..18ccbff --- /dev/null +++ b/src/cache.ts @@ -0,0 +1,37 @@ +// cache.ts — [feat/wake-trigger] simple TTL read-cache for Tier 1/2 on-read metrics. +// +// NO watermark. Invalidation is purely time-based: "today" gets a short TTL (~60s, +// the 1-min rollup granularity, so it stays near-live without recomputing on every +// request), a completed past day is effectively immutable until its minutes prune. +// This sidesteps the v2 day_version watermark that re-fired every minute as 1 Hz +// data rolled in. Tier 3/4 are NOT cached here — they're served from `daily`. + +/** Serve `key` from read_cache if fresh (< ttlSec old); else compute, store, return. */ +export async function cached( + db: D1Database, userId: string, key: string, ttlSec: number, compute: () => Promise, +): Promise { + const now = Math.floor(Date.now() / 1000) + const row = await db.prepare( + 'SELECT payload, computed_at FROM read_cache WHERE user_id = ? AND key = ?', + ).bind(userId, key).first<{ payload: string; computed_at: number }>() + if (row && now - row.computed_at < ttlSec) { + try { return JSON.parse(row.payload) as T } catch { /* corrupt → recompute */ } + } + const val = await compute() + await db.prepare( + 'INSERT INTO read_cache (user_id, key, payload, computed_at) VALUES (?,?,?,?) ' + + 'ON CONFLICT(user_id, key) DO UPDATE SET payload = excluded.payload, computed_at = excluded.computed_at', + ).bind(userId, key, JSON.stringify(val), now).run() + return val +} + +/** TTL by date: today (UTC) is live (60s); a past day is immutable-until-prune. */ +export function ttlForDate(ymd: string, nowSec = Math.floor(Date.now() / 1000)): number { + const today = new Date(nowSec * 1000).toISOString().slice(0, 10) + return ymd >= today ? 60 : 86400 +} + +/** Invalidate a day's cached Tier-2 entries (called by the day-close). */ +export async function invalidateDay(db: D1Database, userId: string, ymd: string): Promise { + await db.prepare("DELETE FROM read_cache WHERE user_id = ? AND key LIKE ?").bind(userId, `%:${ymd}`).run() +} diff --git a/src/coach.ts b/src/coach.ts new file mode 100644 index 0000000..6ac9873 --- /dev/null +++ b/src/coach.ts @@ -0,0 +1,144 @@ +// coach.ts — self-hosted, OpenAI-compatible LLM endpoint for the in-app AI Coach, +// backed by Cloudflare Workers AI (the `AI` binding). No external provider and no +// OpenAI key: it runs on your own Worker within Cloudflare's free AI allocation. +// +// The app's AI Coach is a BYOK, agentic tool-caller that POSTs OpenAI-shaped +// `/chat/completions` (model, messages, tools, tool_choice) and reads back +// `choices[0].message` (content + tool_calls). Point its base URL at +// `/coach/v1` and set its API key to COACH_KEY. We translate that contract +// to `env.AI.run(...)` and map Workers AI's `{ response, tool_calls }` back to the +// OpenAI shape the coach expects. + +export interface AiBinding { + run(model: string, input: Record): Promise +} + +export interface CoachEnv { + AI: AiBinding + COACH_KEY?: string +} + +interface OpenAIToolCall { + id: string + type: 'function' + function: { name: string; arguments: string } +} + +interface OpenAIMessage { + role: 'assistant' + content: string | null + tool_calls?: OpenAIToolCall[] +} + +// Llama 3.1 8B (fast) is the default: ~0.4s/call and it reliably CONVERGES the +// agentic tool loop (answers in prose after a tool result). The 70B fp8 model is +// available but tends to re-call tools instead of converging here, so it's slower +// in practice for the multi-step coach. The client may pin any of these. +const DEFAULT_MODEL = '@cf/meta/llama-3.1-8b-instruct-fast' + +const COACH_MODELS = [ + '@cf/meta/llama-3.1-8b-instruct-fast', + '@cf/meta/llama-3.3-70b-instruct-fp8-fast', + '@hf/nousresearch/hermes-2-pro-mistral-7b', +] + +// Constant-time-ish bearer check against the configured COACH_KEY. Without a +// configured key the endpoint is closed (never open by default). +function authorized(c: any): boolean { + const want = c.env.COACH_KEY + if (!want) return false + const auth = c.req.header('Authorization') ?? '' + return auth.startsWith('Bearer ') && auth.slice(7) === want +} + +// Pass tools through in OpenAI shape (`{ type:'function', function:{...} }`). +// Workers AI's OpenAI-compatible models validate this exact shape (the unwrapped +// Cloudflare form `{name,description,parameters}` is rejected by some models, e.g. +// llama-3.1-8b-instruct-fast). The coach already sends the OpenAI shape, so forward +// it unchanged. +function normalizeTools(tools: unknown): unknown[] | undefined { + if (!Array.isArray(tools) || tools.length === 0) return undefined + return tools +} + +function extractText(out: any): string { + if (typeof out === 'string') return out + if (typeof out?.response === 'string') return out.response + if (typeof out?.result?.response === 'string') return out.result.response + return '' +} + +function mapToolCalls(out: any): OpenAIToolCall[] | undefined { + const raw = out?.tool_calls ?? out?.result?.tool_calls + if (!Array.isArray(raw) || raw.length === 0) return undefined + return raw.map((tc: any, i: number) => { + const name = tc?.name ?? tc?.function?.name ?? '' + const argsRaw = tc?.arguments ?? tc?.function?.arguments ?? {} + const args = typeof argsRaw === 'string' ? argsRaw : JSON.stringify(argsRaw ?? {}) + return { + id: tc?.id ?? `call_${Date.now()}_${i}`, + type: 'function' as const, + function: { name, arguments: args }, + } + }) +} + +// GET /coach/v1/models — the coach lists models from here (OpenAI /models shape). +export async function coachModels(c: any) { + if (!authorized(c)) return c.json({ error: { message: 'Unauthorized' } }, 401) + return c.json({ + object: 'list', + data: COACH_MODELS.map((id) => ({ id, object: 'model', owned_by: 'cloudflare' })), + }) +} + +// POST /coach/v1/chat/completions — OpenAI-compatible, backed by Workers AI. +export async function coachChatCompletions(c: any) { + if (!authorized(c)) return c.json({ error: { message: 'Unauthorized' } }, 401) + + let body: any + try { + body = await c.req.json() + } catch { + return c.json({ error: { message: 'Invalid JSON body' } }, 400) + } + + const model = typeof body?.model === 'string' && body.model ? body.model : DEFAULT_MODEL + const messages = Array.isArray(body?.messages) ? body.messages : [] + if (messages.length === 0) { + return c.json({ error: { message: 'messages[] is required' } }, 400) + } + + const input: Record = { + messages, + temperature: typeof body?.temperature === 'number' ? body.temperature : 0.3, + max_tokens: typeof body?.max_tokens === 'number' ? body.max_tokens : 1024, + } + const tools = normalizeTools(body?.tools) + if (tools) input.tools = tools + + let out: unknown + try { + out = await c.env.AI.run(model, input) + } catch (e: unknown) { + const msg = e instanceof Error ? e.message : 'Workers AI request failed' + return c.json({ error: { message: `Workers AI error: ${msg}` } }, 502) + } + + const toolCalls = mapToolCalls(out) + const message: OpenAIMessage = toolCalls + ? { role: 'assistant', content: extractText(out) || null, tool_calls: toolCalls } + : { role: 'assistant', content: extractText(out) } + + return c.json({ + id: `chatcmpl-${Date.now()}`, + object: 'chat.completion', + created: Math.floor(Date.now() / 1000), + model, + choices: [{ + index: 0, + message, + finish_reason: toolCalls ? 'tool_calls' : 'stop', + }], + }) +} diff --git a/src/cycle.ts b/src/cycle.ts new file mode 100644 index 0000000..1d7661a --- /dev/null +++ b/src/cycle.ts @@ -0,0 +1,83 @@ +// cycle.ts — menstrual cycle tracking endpoints. +// +// POST /cycle/log {date:'YYYY-MM-DD', kind?:'start'|'end'|'spotting', note?} → upsert +// DELETE /cycle/log?date=YYYY-MM-DD → remove +// GET /cycle → prediction + logs + biometric overlay +// +// The prediction (calcCycle) is LOG-ANCHORED — driven by logged period starts, +// never inferred from biometrics. The biometric overlay (skin-temp / RHR / HRV +// across the current cycle) is descriptive enrichment pulled from stored `daily`. +// All JWT, scoped by user_id. Honest: an ESTIMATE, not medical guidance. + +import type { Context } from 'hono' +import { calcCycle } from 'openstrap-analytics' + +type Ctx = Context<{ Bindings: { DB: D1Database }; Variables: { userId: string } }> + +const isDate = (s: unknown): s is string => + typeof s === 'string' && /^\d{4}-\d{2}-\d{2}$/.test(s) +const today = () => new Date(Date.now()).toISOString().slice(0, 10) +const KINDS = new Set(['start', 'end', 'spotting']) + +/** Log (or update) a period event for a date. */ +export async function postCycleLog(c: Ctx) { + const userId = c.get('userId') + const body = await c.req.json<{ date?: string; kind?: string; note?: string }>() + if (!isDate(body.date)) return c.json({ error: 'date must be YYYY-MM-DD' }, 400) + const kind = KINDS.has(body.kind ?? '') ? (body.kind as string) : 'start' + const note = (body.note ?? '').toString().slice(0, 500) + await c.env.DB.prepare( + 'INSERT INTO cycle_log (user_id, date, kind, note, updated_at) VALUES (?,?,?,?,?) ' + + 'ON CONFLICT(user_id, date) DO UPDATE SET kind=excluded.kind, note=excluded.note, updated_at=excluded.updated_at', + ).bind(userId, body.date, kind, note, Math.floor(Date.now() / 1000)).run() + return c.json({ ok: true, date: body.date, kind, note }) +} + +/** Remove a logged event. */ +export async function deleteCycleLog(c: Ctx) { + const userId = c.get('userId') + const date = (c.req.query('date') || '').trim() + if (!isDate(date)) return c.json({ error: 'date=YYYY-MM-DD required' }, 400) + await c.env.DB.prepare('DELETE FROM cycle_log WHERE user_id = ? AND date = ?') + .bind(userId, date).run() + return c.json({ ok: true, deleted: true }) +} + +/** Current cycle position + prediction + logged events + biometric overlay. + * Only computed for users who explicitly opted in (users.track_cycle). */ +export async function getCycle(c: Ctx) { + const userId = c.get('userId') + + // Consent gate: no cycle computation unless the user turned tracking on. + const pref = await c.env.DB.prepare('SELECT track_cycle FROM users WHERE id = ?') + .bind(userId).first<{ track_cycle: number | null }>() + if (!pref || !pref.track_cycle) { + return c.json({ enabled: false, note: 'Cycle tracking is off. Enable it in your profile to start.' }) + } + + const logsRes = await c.env.DB.prepare( + 'SELECT date, kind, note FROM cycle_log WHERE user_id = ? ORDER BY date DESC LIMIT 60', + ).bind(userId).all<{ date: string; kind: string; note: string | null }>() + const logs = logsRes.results ?? [] + const starts = logs.filter((l) => l.kind === 'start').map((l) => l.date) + + const cycle = calcCycle(starts, today()) + + // Biometric overlay for the current cycle (descriptive). Pull stored daily + // signals from the cycle start onward — cheap point range on (user_id, date). + let overlay: Array<{ date: string; skin_temp_idx: number | null; resting_hr: number | null; hrv_rmssd: number | null }> = [] + if (cycle.last_start) { + const ov = await c.env.DB.prepare( + 'SELECT date, skin_temp_idx, resting_hr, hrv_rmssd FROM daily ' + + 'WHERE user_id = ? AND date >= ? ORDER BY date ASC', + ).bind(userId, cycle.last_start).all() + overlay = (ov.results ?? []).map((r: any) => ({ + date: r.date, + skin_temp_idx: r.skin_temp_idx ?? null, + resting_hr: r.resting_hr ?? null, + hrv_rmssd: r.hrv_rmssd ?? null, + })) + } + + return c.json({ ...cycle, logs, overlay }) +} diff --git a/src/daydetail.ts b/src/daydetail.ts index 0d68ab0..0a2300e 100644 --- a/src/daydetail.ts +++ b/src/daydetail.ts @@ -8,21 +8,53 @@ // All JWT, scoped by user_id. import type { Context } from 'hono' +import { cached, ttlForDate } from './cache' +import { readMinutes } from './minute_store' +import { ensureTodayWorkouts } from './workouts' +import { stageHypnogram, detectSleepCycles, calcDaytimeHrv } from 'openstrap-analytics' -type Ctx = Context<{ Bindings: { DB: D1Database }; Variables: { userId: string } }> +type Ctx = Context<{ Bindings: { DB: D1Database; RAW_BUCKET?: R2Bucket }; Variables: { userId: string } }> const DAY = 86400 const dayStartOf = (date: string) => Math.floor(Date.parse(`${date}T00:00:00Z`) / 1000) + +// Per-minute hypnogram (BETA) via analytics `stageHypnogram` — the v1 method: +// calcSleep's Cole-Kripke + HR-dip mask owns asleep/awake, the SAME HR-percentile bands +// as estimateStages own deep/light/rem within sleep, bout-smoothed once. ONE source for +// both the graph and the totals (so they can't disagree), no second stager, no RR, no +// per-night knob — exactly what worked in v1. +interface NightStaging { hypnogram: { t: number; stage: string }[]; totals: { light_min: number; deep_min: number; rem_min: number } | null; awake_min?: number; asleep_min?: number } +function stageNight( + mins: { ts_min: number; hr_avg?: number | null; hr_min?: number | null; hr_max?: number | null; activity?: number | null; wrist_on?: number | null; rr?: number[] | null }[], + onset: number, wake: number, rhr: number, +): NightStaging { + const ms = mins.map((m) => ({ + ts: m.ts_min, hr_avg: m.hr_avg ?? 0, hr_min: m.hr_min ?? 0, hr_max: m.hr_max ?? 0, + hr_n: (m.hr_avg ?? 0) > 0 ? 60 : 0, activity: m.activity ?? 0, steps: 0, wrist_on: !!m.wrist_on, + })) + // Per-minute RR → the REM tiebreaker in stageHypnogram (high HR + low RMSSD = REM, not awake). + const rrByMin = new Map() + for (const m of mins) if (m.rr && m.rr.length) rrByMin.set(m.ts_min, m.rr) + const h = stageHypnogram(ms, onset, wake, { resting_hr: rhr, max_hr: 0, sleep_need_min: 480 }, rrByMin) + if (h) { + return { + hypnogram: h.hypnogram.map((x) => ({ t: x.t, stage: x.stage })), + totals: { light_min: h.light_min, deep_min: h.deep_min, rem_min: h.rem_min }, + awake_min: h.awake_min, + asleep_min: h.asleep_min, + } + } + // No resting-HR baseline → staging can't run; caller keeps the stored breakdown. + return { hypnogram: [], totals: null } +} const clamp = (v: number, lo: number, hi: number) => Math.max(lo, Math.min(hi, v)) -interface Min { ts_min: number; hr_avg: number | null; hr_min: number | null; hr_max: number | null; activity: number | null; wrist_on: number | null } +interface Min { ts_min: number; hr_avg: number | null; hr_min: number | null; hr_max: number | null; activity: number | null; wrist_on: number | null; rr: number[] } async function loadMinutes(c: Ctx, from: number, to: number): Promise { - const { results } = await c.env.DB.prepare( - 'SELECT ts_min, hr_avg, hr_min, hr_max, activity, wrist_on FROM minute ' + - 'WHERE user_id = ? AND ts_min >= ? AND ts_min < ? ORDER BY ts_min ASC', - ).bind(c.get('userId'), from, to).all() - return results ?? [] + // Tiered read: D1 for hot days, R2 fallback for sealed days (see minute_store). + const rows = await readMinutes(c.env, c.get('userId'), from, to) + return rows.map((m) => ({ ts_min: m.ts_min, hr_avg: m.hr_avg, hr_min: m.hr_min, hr_max: m.hr_max, activity: m.activity, wrist_on: m.wrist_on, rr: m.rr ?? [] })) } async function loadHr(c: Ctx): Promise<{ rhr: number; maxHr: number }> { @@ -44,56 +76,51 @@ function downsample(arr: T[], cap = 300): T[] { } // ── /day/strain ────────────────────────────────────────────────────────────── +// PURE READ: serve the snapshot the analytics cron precomputed (daily row + +// sessions). No live recompute on the endpoint — strain, curve, zones and HR +// stats all come from `daily`, so /day/strain can never diverge from /today and +// the read is fast. Freshness is the cron's job (ingest → dirty → cron). export async function getDayStrain(c: Ctx) { const date = (c.req.query('date') || '').trim() if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) return c.json({ error: 'date=YYYY-MM-DD required' }, 400) const start = dayStartOf(date) - const mins = await loadMinutes(c, start, start + DAY) - const { rhr, maxHr } = await loadHr(c) - const denom = Math.max(1, maxHr - rhr) - - let trimp = 0 - const curve: { t: number; v: number }[] = [] - const zones = [0, 0, 0, 0, 0] - let hrMax = 0, hrMinNonZero = 0, hrSum = 0, hrN = 0 - for (const m of mins) { - const hr = m.hr_avg ?? 0 - if (hr <= 0) { curve.push({ t: m.ts_min, v: round1(strainScale(trimp)) }); continue } - const ratio = clamp((hr - rhr) / denom, 0, 1) - trimp += ratio * 0.64 * Math.exp(1.92 * ratio) - curve.push({ t: m.ts_min, v: round1(strainScale(trimp)) }) - const pct = (hr / maxHr) * 100 - if (pct >= 90) zones[4]++ - else if (pct >= 80) zones[3]++ - else if (pct >= 70) zones[2]++ - else if (pct >= 60) zones[1]++ - else if (pct >= 50) zones[0]++ - hrMax = Math.max(hrMax, hr) - hrMinNonZero = hrMinNonZero === 0 ? hr : Math.min(hrMinNonZero, hr) - hrSum += hr; hrN++ + const isToday = date === new Date().toISOString().slice(0, 10) + // For today, refresh auto-detected sessions on read (throttled) so the day's + // workout list isn't stale until the next wake-close. + if (isToday) await ensureTodayWorkouts(c.env.DB, c.get('userId')) + // Steps (AN-2554): live SUM of today's minute.steps; past days use the stored + // daily.steps (persisted by processUser at close). Zero R2 — hot day blob. + let liveSteps: number | null = null + if (isToday) { + const mins = await readMinutes(c.env, c.get('userId'), start, start + DAY) + liveSteps = mins.reduce((a, m) => a + (m.steps ?? 0), 0) } - const { results: sessions } = await c.env.DB.prepare( - 'SELECT id, start_ts, end_ts, type, avg_hr, max_hr, strain, calories, hrr60, zones FROM sessions ' + - "WHERE user_id = ? AND start_ts >= ? AND start_ts < ? AND status != 'deleted' ORDER BY start_ts ASC", - ).bind(c.get('userId'), start, start + DAY).all() - - // Training-load + day totals from the derived `daily` row (acwr/fitness/cals/steps + drivers). const dr = await c.env.DB.prepare( - 'SELECT acwr, fitness_trend, calories, steps, drivers, vo2max, fitness, fatigue, form, monotony FROM daily WHERE user_id = ? AND date = ?', + 'SELECT strain, hr_zones, wear_min, strain_curve, hr_max, hr_min, hr_avg, acwr, fitness_trend, ' + + 'calories, steps, drivers, vo2max, fitness, fatigue, form, monotony FROM daily WHERE user_id = ? AND date = ?', ).bind(c.get('userId'), date).first() + + const z = dr?.hr_zones ? safe(dr.hr_zones) : null const acwr = dr?.acwr ?? null const band = acwr == null ? null : acwr < 0.8 ? 'detraining' : acwr <= 1.3 ? 'optimal' : acwr <= 1.5 ? 'caution' : 'high-risk' + const { results: sessions } = await c.env.DB.prepare( + 'SELECT id, start_ts, end_ts, type, avg_hr, max_hr, strain, calories, hrr60, zones FROM sessions ' + + "WHERE user_id = ? AND start_ts >= ? AND start_ts < ? AND status != 'deleted' ORDER BY start_ts ASC", + ).bind(c.get('userId'), start, start + DAY).all() + return c.json({ date, - strain: round1(strainScale(trimp)), - curve: downsample(curve), - zones: { z1: zones[0], z2: zones[1], z3: zones[2], z4: zones[3], z5: zones[4] }, - hr: { max: hrMax || null, min: hrMinNonZero || null, avg: hrN ? Math.round(hrSum / hrN) : null }, - max_hr_used: maxHr, - worn_min: mins.filter((m) => m.wrist_on).length, + strain: dr?.strain ?? 0, + curve: dr?.strain_curve ? safe(dr.strain_curve) : [], + zones: z + ? { z1: z.zone1_min ?? 0, z2: z.zone2_min ?? 0, z3: z.zone3_min ?? 0, z4: z.zone4_min ?? 0, z5: z.zone5_min ?? 0 } + : { z1: 0, z2: 0, z3: 0, z4: 0, z5: 0 }, + hr: { max: dr?.hr_max ?? null, min: dr?.hr_min ?? null, avg: dr?.hr_avg ?? null }, + max_hr_used: z?.max_hr_used ?? null, + worn_min: dr?.wear_min ?? 0, // Training load (ACWR band) + fitness trend + day energy/steps. load: acwr == null ? null : { acwr: Math.round(acwr * 100) / 100, band }, fitness_trend: dr?.fitness_trend ?? null, @@ -103,7 +130,7 @@ export async function getDayStrain(c: Ctx) { ? { fitness: dr?.fitness ?? null, fatigue: dr?.fatigue ?? null, form: dr?.form ?? null } : null, monotony: dr?.monotony ?? null, calories: dr?.calories ?? null, - steps: dr?.steps ?? null, + steps: liveSteps ?? dr?.steps ?? null, drivers: dr?.drivers ? safe(dr.drivers) : null, sessions: (sessions ?? []).map((s: any) => ({ ...s, zones: s.zones ? safe(s.zones) : null, @@ -125,6 +152,8 @@ const safe = (s: any) => { try { return JSON.parse(s) } catch { return null } } export async function getDayWear(c: Ctx) { const date = (c.req.query('date') || '').trim() if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) return c.json({ error: 'date=YYYY-MM-DD required' }, 400) + const userId = c.get('userId') + const payload = await cached(c.env.DB, userId, `daywear:${date}`, ttlForDate(date), async () => { const start = dayStartOf(date) const mins = await loadMinutes(c, start, start + DAY) @@ -163,10 +192,10 @@ export async function getDayWear(c: Ctx) { // Prefer the derived daily.wear_min when present (same source), else the live count. const dr = await c.env.DB.prepare( 'SELECT wear_min FROM daily WHERE user_id = ? AND date = ?', - ).bind(c.get('userId'), date).first<{ wear_min: number | null }>() + ).bind(userId, date).first<{ wear_min: number | null }>() const wearMin = dr?.wear_min != null ? Math.round(dr.wear_min) : wornMin - return c.json({ + return { date, worn_min: wearMin, coverage_pct: Math.round((wearMin / 1440) * 100), @@ -176,10 +205,67 @@ export async function getDayWear(c: Ctx) { segments, // number of separate on-wrist stretches longest_off_min: longestGap, // longest off-wrist gap inside the worn window tier: 'AUTH', // straight from the device's wrist sensor + } }) + return c.json(payload) } // ── /day/sleep ─────────────────────────────────────────────────────────────── +// GET /day/v2/sleep?date= — every sleep period for the date (one card each; +// naps = shorter sleeps). Additive companion to GET /day/sleep (single-period). +export async function getDaySleepV2(c: Ctx) { + const date = (c.req.query('date') || '').trim() + if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) return c.json({ error: 'date=YYYY-MM-DD required' }, 400) + const userId = c.get('userId') + const payload = await cached(c.env.DB, userId, `daysleepv2:${date}`, ttlForDate(date), async () => { + const { results } = await c.env.DB.prepare( + 'SELECT * FROM sleep_periods WHERE user_id = ? AND date = ? ORDER BY onset_ts ASC', + ).bind(userId, date).all() + const baseRow = await c.env.DB.prepare('SELECT sleep_need_min, resting_hr FROM baselines WHERE user_id = ?') + .bind(userId).first() + const need = (baseRow?.sleep_need_min && baseRow.sleep_need_min >= 180) ? baseRow.sleep_need_min : 480 + const rhr = baseRow?.resting_hr && baseRow.resting_hr > 0 ? baseRow.resting_hr : 55 + const rows = results ?? [] + // ONE minute read spanning all periods; slice per period for its own hypnogram so + // every nap renders the same banded timeline as the main sleep (computed on-read; + // no per-nap storage). Naps are already counted in total_asleep_min. + let mins: Min[] = [] + const onsets = rows.map((r: any) => r.onset_ts).filter((x: any) => x) + const wakes = rows.map((r: any) => r.wake_ts).filter((x: any) => x) + if (onsets.length && wakes.length) { + mins = await loadMinutes(c, Math.min(...onsets), Math.max(...wakes) + 60) + } + const periods = rows.map((r: any) => { + const pm = mins.filter((m) => m.ts_min >= r.onset_ts && m.ts_min <= (r.wake_ts ?? r.onset_ts)) + const ng = pm.length ? stageNight(pm, r.onset_ts, r.wake_ts ?? r.onset_ts, rhr) : { hypnogram: [], totals: null } + const storedStages = (r.light_min != null || r.deep_min != null || r.rem_min != null) + ? { light_min: r.light_min, deep_min: r.deep_min, rem_min: r.rem_min } : null + return { + id: r.id, + onset_ts: r.onset_ts, + wake_ts: r.wake_ts, + duration_min: r.duration_min, + in_bed_min: r.in_bed_min, + efficiency: r.efficiency, + stages: ng.totals ?? storedStages, // same stageSleep source as the hypnogram → consistent + is_main: !!r.is_main, + confidence: r.confidence, + hypnogram: downsample(ng.hypnogram, 240), + } + }) + return { + date, + has_sleep: periods.length > 0, + need_min: need, + total_asleep_min: periods.reduce((a: number, p: any) => a + (p.duration_min || 0), 0), + naps: periods.filter((p: any) => !p.is_main).length, + periods, + stages_beta: true, + } + }) + return c.json(payload) +} + export async function getDaySleep(c: Ctx) { const date = (c.req.query('date') || '').trim() if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) return c.json({ error: 'date=YYYY-MM-DD required' }, 400) @@ -193,62 +279,66 @@ export async function getDaySleep(c: Ctx) { return c.json({ date, has_sleep: false, need_min: need }) } const rhr = baseline?.resting_hr && baseline.resting_hr > 0 ? baseline.resting_hr : 55 - const mins = await loadMinutes(c, row.onset_ts, row.wake_ts + 60) - - // Per-epoch hypnogram (BETA, consistent with the summary's beta stages): - // off-wrist / HR > 1.15*RHR → awake; very low activity + low HR → deep; - // low activity + higher HR (REM-ish) → rem; else light. - const hypnogram: { t: number; stage: string }[] = [] - for (const m of mins) { - const hr = m.hr_avg ?? 0 - const act = m.activity ?? 0 - let stage: string - if (hr <= 0 || hr > 1.15 * rhr || act > 0.25) stage = 'awake' - else if (act < 0.01 && hr < rhr + 4) stage = 'deep' - else if (hr > rhr + 5) stage = 'rem' - else stage = 'light' - hypnogram.push({ t: m.ts_min, stage }) - } - - // Sleep-debt over the last 7 real nights (incl. this one). - const { results: recent } = await c.env.DB.prepare( - 'SELECT duration_min FROM sleep WHERE user_id = ? AND date <= ? ORDER BY date DESC LIMIT 7', - ).bind(c.get('userId'), date).all<{ duration_min: number | null }>() - let debt = 0 - for (const r of recent ?? []) { - const d = r.duration_min ?? 0 - if (d >= 120) debt += Math.max(0, need - d) - } - - // Nocturnal-heart summary + gated respiratory rate for this date (stored on daily). - const dailyRow = await c.env.DB.prepare( - 'SELECT nocturnal, resp_rate, resp_conf FROM daily WHERE user_id = ? AND date = ?', - ).bind(c.get('userId'), date).first() - const nocturnal = dailyRow?.nocturnal ? safe(dailyRow.nocturnal) : null - const resp = (dailyRow?.resp_rate != null && (dailyRow?.resp_conf ?? 0) >= 0.5) - ? { value: Math.round(dailyRow.resp_rate * 10) / 10, confidence: dailyRow.resp_conf } - : null - - const inBed = Math.round((row.wake_ts - row.onset_ts) / 60) - const asleep = row.duration_min ?? 0 - return c.json({ - date, - has_sleep: true, - nocturnal, - resp, - onset_ts: row.onset_ts, - wake_ts: row.wake_ts, - in_bed_min: inBed, - duration_min: asleep, - awake_min: Math.max(0, inBed - asleep), - efficiency: row.efficiency, - need_min: need, - debt_min: Math.round(debt), - regularity: row.regularity, - stages: { light_min: row.light_min, deep_min: row.deep_min, rem_min: row.rem_min }, - stages_beta: true, - hypnogram: downsample(hypnogram, 240), + const userId = c.get('userId') + // Compute ONCE, then serve from the TTL read-cache (no per-read recompute): a past + // day is immutable (cached until prune), "today" refreshes ≤60s, and close_day's + // invalidateDay clears it so the next read after a close rebuilds it once. + const payload = await cached(c.env.DB, userId, `daysleep:${date}`, ttlForDate(date), async () => { + // WIDE window (prior evening → wake) so the staging window has full context. + const mins = await loadMinutes(c, row.onset_ts - 16 * 3600, row.wake_ts + 3600) + // ONE source (analytics stageHypnogram — v1 Cole-Kripke method) → hypnogram + totals. + const ng = stageNight(mins, row.onset_ts, row.wake_ts, rhr) + // Ultradian sleep cycles (Rosenblum 2024 fractal-cycle method, on smoothed z-RMSSD). + const cyc = detectSleepCycles(mins.map((m) => ({ ts: m.ts_min, rr: m.rr })), row.onset_ts, row.wake_ts) + + // Sleep-debt over the last 7 real nights (incl. this one). + const { results: recent } = await c.env.DB.prepare( + 'SELECT duration_min FROM sleep WHERE user_id = ? AND date <= ? ORDER BY date DESC LIMIT 7', + ).bind(userId, date).all<{ duration_min: number | null }>() + let debt = 0 + for (const r of recent ?? []) { const d = r.duration_min ?? 0; if (d >= 120) debt += Math.max(0, need - d) } + + // Nocturnal-heart summary + gated respiratory rate for this date (stored on daily). + const dailyRow = await c.env.DB.prepare( + 'SELECT nocturnal, resp_rate, resp_conf FROM daily WHERE user_id = ? AND date = ?', + ).bind(userId, date).first() + const nocturnal = dailyRow?.nocturnal ? safe(dailyRow.nocturnal) : null + const resp = (dailyRow?.resp_rate != null && (dailyRow?.resp_conf ?? 0) >= 0.5) + ? { value: Math.round(dailyRow.resp_rate * 10) / 10, confidence: dailyRow.resp_conf } : null + + const inBed = Math.round((row.wake_ts - row.onset_ts) / 60) + // Single source: duration/awake/stages all come from the one hypnogram so the + // breakdown and the graph can't disagree. Fall back to the stored row only if + // staging abstained (no resting-HR baseline). + const asleep = ng.asleep_min ?? row.duration_min ?? 0 + const awakeMin = ng.awake_min ?? Math.max(0, inBed - asleep) + const inBedMin = ng.asleep_min != null ? asleep + awakeMin : inBed + const efficiency = inBedMin > 0 && ng.asleep_min != null ? Math.round((asleep / inBedMin) * 10000) / 10000 : row.efficiency + return { + date, + has_sleep: true, + nocturnal, + resp, + onset_ts: row.onset_ts, + wake_ts: row.wake_ts, + in_bed_min: inBedMin, + duration_min: asleep, + awake_min: awakeMin, + efficiency, + need_min: need, + debt_min: Math.round(debt), + regularity: row.regularity, + stages: ng.totals ?? { light_min: row.light_min, deep_min: row.deep_min, rem_min: row.rem_min }, + stages_beta: true, + hypnogram: downsample(ng.hypnogram, 240), + // Ultradian cycles (NREM↔REM), fractal-cycle method on HRV. Beta. + cycles: cyc.cycles, + cycles_mean_min: cyc.mean_duration_min, + cycle_series: downsample(cyc.series, 240), + cycles_beta: true, + } }) + return c.json(payload) } // ── /day/stress ────────────────────────────────────────────────────────────── @@ -258,10 +348,11 @@ export async function getDaySleep(c: Ctx) { export async function getDayStress(c: Ctx) { const date = (c.req.query('date') || '').trim() if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) return c.json({ error: 'date=YYYY-MM-DD required' }, 400) + const userId = c.get('userId') as string + const payload = await cached(c.env.DB, userId, `daystress:${date}`, ttlForDate(date), async () => { const start = dayStartOf(date) const mins = await loadMinutes(c, start, start + DAY) - const userId = c.get('userId') as string const row = await c.env.DB.prepare( 'SELECT stress, sleep_stress, drivers FROM daily WHERE user_id = ? AND date = ?', ).bind(userId, date).first<{ stress: string | null; sleep_stress: string | null; drivers: string | null }>() @@ -273,19 +364,47 @@ export async function getDayStress(c: Ctx) { // Factual HR timeline (bpm) — context, not a stress band. const hr = mins.map((m) => ({ t: m.ts_min, v: m.hr_avg ?? 0 })) - return c.json({ + return { date, stress, // {score, si, lf_hf, rmssd, level, drivers} sleep_stress: sleepStress, // {score, arousal_events, restless_min, events[...]} drivers: drivers?.stress ?? null, hr: downsample(hr, 240), + } }) + return c.json(payload) +} + +// ── /day/hrv ─────────────────────────────────────────────────────────────────── +// Daytime (waking) HRV timeline — the ultradian RMSSD rhythm from minute.rr OUTSIDE +// the main sleep window. Complements nocturnal recovery: a daytime-stress curve. +export async function getDayHrv(c: Ctx) { + const date = (c.req.query('date') || '').trim() + if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) return c.json({ error: 'date=YYYY-MM-DD required' }, 400) + const userId = c.get('userId') as string + const payload = await cached(c.env.DB, userId, `dayhrv:${date}`, ttlForDate(date), async () => { + const start = dayStartOf(date) + const mins = await loadMinutes(c, start, start + DAY) + // Exclude this date's main sleep window so it's genuinely DAYTIME HRV. + const sleep = await c.env.DB.prepare( + 'SELECT onset_ts, wake_ts FROM sleep WHERE user_id = ? AND date = ?', + ).bind(userId, date).first<{ onset_ts: number | null; wake_ts: number | null }>() + const inSleep = (t: number) => sleep?.onset_ts != null && sleep?.wake_ts != null && t >= sleep.onset_ts && t <= sleep.wake_ts + const byMinute = mins + .filter((m) => m.rr && m.rr.length && !inSleep(m.ts_min)) + .map((m) => ({ ts: m.ts_min, rr: m.rr as number[] })) + const hrv = calcDaytimeHrv(byMinute) + return { date, daytime_hrv: hrv } // {rmssd_median, series[{ts,rmssd}], lowest_ts, n_windows, confidence, tier} + }) + return c.json(payload) } // ── /day/timeline ──────────────────────────────────────────────────────────── export async function getDayTimeline(c: Ctx) { const date = (c.req.query('date') || '').trim() if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) return c.json({ error: 'date=YYYY-MM-DD required' }, 400) + const userId = c.get('userId') + const payload = await cached(c.env.DB, userId, `daytimeline:${date}`, ttlForDate(date), async () => { const start = dayStartOf(date) const end = start + DAY const mins = await loadMinutes(c, start, end) @@ -311,9 +430,9 @@ export async function getDayTimeline(c: Ctx) { ).bind(c.get('userId'), start, end).all() const { results: events } = await c.env.DB.prepare( 'SELECT event_id, ts FROM events WHERE user_id = ? AND ts >= ? AND ts < ? ORDER BY ts ASC', - ).bind(c.get('userId'), start, end).all() + ).bind(userId, start, end).all() - return c.json({ + return { date, day_start: start, hr: downsample(hr), @@ -325,7 +444,9 @@ export async function getDayTimeline(c: Ctx) { peak_hr: peak.v ? peak : null, low_hr: low.v ? low : null, }, + } }) + return c.json(payload) } // ── /day/heart ───────────────────────────────────────────────────────────── @@ -335,18 +456,19 @@ export async function getDayTimeline(c: Ctx) { export async function getDayHeart(c: Ctx) { const date = (c.req.query('date') || '').trim() if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) return c.json({ error: 'date=YYYY-MM-DD required' }, 400) + const userId = c.get('userId') + const payload = await cached(c.env.DB, userId, `dayheart:${date}`, ttlForDate(date), async () => { const start = dayStartOf(date) const mins = await loadMinutes(c, start, start + DAY) - const userId = c.get('userId') const d = await c.env.DB.prepare( - 'SELECT resting_hr, recovery, readiness, hrv_rmssd, hrv_sdnn, hrv_lfhf, hrv_conf, hrv_cv, irregular, hr_zones, nocturnal, stress, illness, drivers, resp_rate, resp_conf, spo2_idx FROM daily WHERE user_id = ? AND date = ?', + 'SELECT resting_hr, recovery, readiness, hrv_rmssd, hrv_sdnn, hrv_lfhf, hrv_conf, hrv_cv, irregular, hr_zones, nocturnal, stress, illness, drivers, resp_rate, resp_conf, spo2_idx, skin_temp_idx FROM daily WHERE user_id = ? AND date = ?', ).bind(userId, date).first() const base = await c.env.DB.prepare('SELECT resting_hr, hrv_rmssd FROM baselines WHERE user_id = ?') .bind(userId).first() const parse = (s: string | null) => { try { return s ? JSON.parse(s) : null } catch { return null } } const worn = mins.filter((m) => m.wrist_on && (m.hr_avg ?? 0) > 0) const hrs = worn.map((m) => m.hr_avg as number) - return c.json({ + return { date, hr: downsample(mins.map((m) => ({ t: m.ts_min, v: m.hr_avg ?? 0 })), 240), avg_hr: hrs.length ? Math.round(hrs.reduce((a, b) => a + b, 0) / hrs.length) : null, @@ -367,8 +489,11 @@ export async function getDayHeart(c: Ctx) { resp: (d?.resp_rate != null && (d?.resp_conf ?? 0) >= 0.3) ? { value: d.resp_rate, confidence: d.resp_conf } : null, spo2: d?.spo2_idx != null ? { value: d.spo2_idx } : null, + skin_temp: d?.skin_temp_idx != null ? { value: d.skin_temp_idx } : null, drivers: parse(d?.drivers ?? null), + } }) + return c.json(payload) } // ── /day/lungs ───────────────────────────────────────────────────────────── diff --git a/src/dayseries.ts b/src/dayseries.ts new file mode 100644 index 0000000..f06b1cd --- /dev/null +++ b/src/dayseries.ts @@ -0,0 +1,19 @@ +// dayseries.ts — RR loader for the HRV-from-minute path. +// +// [feat/wake-trigger] RR now rides the day-packed minute blob (minute_store) as a +// plain number[] per minute — no separate column, no per-blob decode. Reads via the +// tiered store (D1 hot + R2 sealed), so HRV at the wake-close still costs zero extra +// R2 for hot nights. Returns RR (ms) per minute over [from, to]. + +import { readMinutes, type StoreEnv } from './minute_store' + +/** RR (ms) per minute over [from, to] from the day-packed store. Empty if none. */ +export async function loadDayRr( + env: StoreEnv, userId: string, from: number, to: number, +): Promise> { + const map = new Map() + for (const m of await readMinutes(env, userId, from, to)) { + if (m.rr && m.rr.length) map.set(m.ts_min, m.rr) + } + return map +} diff --git a/src/db/migrate_v10_bio_trigger.sql b/src/db/migrate_v10_bio_trigger.sql new file mode 100644 index 0000000..0bc2559 --- /dev/null +++ b/src/db/migrate_v10_bio_trigger.sql @@ -0,0 +1,4 @@ +-- v10 — event-driven biometrics. Track the last sleep-date for which the sweep +-- already fired biometrics, so "a fresh night just finished" triggers recovery/HRV +-- once (not every sweep). The nightly cron stays as the catch-all. Additive column. +ALTER TABLE analytics_cursor ADD COLUMN bio_last_date TEXT; diff --git a/src/db/migrate_v11_minute_rr.sql b/src/db/migrate_v11_minute_rr.sql new file mode 100644 index 0000000..4115e6d --- /dev/null +++ b/src/db/migrate_v11_minute_rr.sql @@ -0,0 +1,3 @@ +-- [feat/wake-trigger] v11 — beat-to-beat RR per minute, so HRV/recovery (Tier 4) +-- computes from D1 at the wake-close with ZERO R2 re-decode. Additive & safe. +ALTER TABLE minute ADD COLUMN rr BLOB; diff --git a/src/db/migrate_v12_read_cache.sql b/src/db/migrate_v12_read_cache.sql new file mode 100644 index 0000000..a73d3a0 --- /dev/null +++ b/src/db/migrate_v12_read_cache.sql @@ -0,0 +1,8 @@ +-- [feat/wake-trigger] v12 — TTL read-cache for Tier 1/2 on-read metrics (no watermark). +CREATE TABLE IF NOT EXISTS read_cache ( + user_id TEXT NOT NULL, + key TEXT NOT NULL, -- e.g. 'today:strain:2026-06-19' + payload TEXT NOT NULL, -- JSON + computed_at INTEGER NOT NULL, + PRIMARY KEY(user_id, key) +) WITHOUT ROWID; diff --git a/src/db/migrate_v13_cursor_wake.sql b/src/db/migrate_v13_cursor_wake.sql new file mode 100644 index 0000000..e114336 --- /dev/null +++ b/src/db/migrate_v13_cursor_wake.sql @@ -0,0 +1,6 @@ +-- [feat/wake-trigger] v13 — incremental sleep/wake state machine columns on the cursor. +-- The */N cron's ONLY job reads these: skip awake-and-closed users (cursor-only), +-- peek recent minutes for the asleep ones, fire close_day once per physiological day. +ALTER TABLE analytics_cursor ADD COLUMN sleep_phase TEXT; -- 'awake' | 'asleep' | NULL +ALTER TABLE analytics_cursor ADD COLUMN phase_since INTEGER; -- unix s of last transition +ALTER TABLE analytics_cursor ADD COLUMN last_close_date TEXT; -- YYYY-MM-DD of last day-close diff --git a/src/db/migrate_v14_minute_day.sql b/src/db/migrate_v14_minute_day.sql new file mode 100644 index 0000000..2a328a4 --- /dev/null +++ b/src/db/migrate_v14_minute_day.sql @@ -0,0 +1,11 @@ +-- [feat/wake-trigger] v14 — day-packed minute store: ONE row per (user, ymd), value = +-- gzipped JSON array of per-minute records. Replaces the row-per-minute `minute` table +-- as the hot store (ingest RMWs one row/day instead of ~1,440). Sealed days move to R2. +-- (The legacy `minute` table is left in place but unused; it stays empty on fresh DBs.) +CREATE TABLE IF NOT EXISTS minute_day ( + user_id TEXT NOT NULL, + ymd TEXT NOT NULL, -- 'YYYY-MM-DD' (UTC day of the minute) + blob BLOB NOT NULL, -- gzipped JSON MinuteRec[] (bound param; << 2MB cap) + updated_at INTEGER NOT NULL, + PRIMARY KEY(user_id, ymd) +) WITHOUT ROWID; diff --git a/src/db/migrate_v15_cycle.sql b/src/db/migrate_v15_cycle.sql new file mode 100644 index 0000000..36af5f0 --- /dev/null +++ b/src/db/migrate_v15_cycle.sql @@ -0,0 +1,10 @@ +-- v15 — menstrual cycle log (user-logged period events; anchor for calcCycle). +-- Additive & safe on existing DBs. +CREATE TABLE IF NOT EXISTS cycle_log( + user_id TEXT NOT NULL, + date TEXT NOT NULL, -- YYYY-MM-DD + kind TEXT NOT NULL, -- 'start' | 'end' | 'spotting' + note TEXT, + updated_at INTEGER, + PRIMARY KEY(user_id, date) +); diff --git a/src/db/migrate_v16_track_cycle.sql b/src/db/migrate_v16_track_cycle.sql new file mode 100644 index 0000000..83eecb3 --- /dev/null +++ b/src/db/migrate_v16_track_cycle.sql @@ -0,0 +1,4 @@ +-- v16 — explicit opt-in for menstrual cycle tracking. Cycle data is only computed +-- and shown when the user turns this on themselves (consent, not inferred from sex). +-- Additive & safe on existing DBs. +ALTER TABLE users ADD COLUMN track_cycle INTEGER DEFAULT 0; diff --git a/src/db/migrate_v17_workout_typing.sql b/src/db/migrate_v17_workout_typing.sql new file mode 100644 index 0000000..36df789 --- /dev/null +++ b/src/db/migrate_v17_workout_typing.sql @@ -0,0 +1,14 @@ +-- migrate_v17 — workout typing + calibration ledger. +-- +-- Auto-detected workouts now carry a motion-based TYPE (Mannini HAR classifier, run at +-- ingest on the live high-rate accel; the per-minute class rides minute_day.act_class) +-- plus a graceful PHASE breakdown for multi-activity sessions. We also record what the +-- model said (detected_type) vs what the user confirmed/corrected (type_source) so the +-- app can show how often the classifier was right and we know when it needs retraining. +-- +-- Additive; safe to re-run (column-exists errors are ignored by the migrate runner). + +ALTER TABLE sessions ADD COLUMN segments TEXT; -- JSON [{start_ts,end_ts,type,confidence}] phases +ALTER TABLE sessions ADD COLUMN detected_type TEXT; -- the classifier's call at detection (immutable record) +ALTER TABLE sessions ADD COLUMN type_confidence REAL; -- ESTIMATE confidence in the type +ALTER TABLE sessions ADD COLUMN type_source TEXT; -- 'model' | 'confirmed' | 'corrected' diff --git a/src/db/migrate_v6_sleep_periods.sql b/src/db/migrate_v6_sleep_periods.sql new file mode 100644 index 0000000..b423fc0 --- /dev/null +++ b/src/db/migrate_v6_sleep_periods.sql @@ -0,0 +1,15 @@ +-- v6 — Sleep v2 (multi-period). Naps are not a special case: every consolidated +-- sleep period is a first-class row with its own breakdown. Purely ADDITIVE — the +-- v1 `sleep` table and its endpoints are untouched. id = `${user_id}:${onset_ts}`. +CREATE TABLE IF NOT EXISTS sleep_periods( + user_id TEXT, id TEXT, + date TEXT, -- the sleep "day" this period was derived under + onset_ts INTEGER, wake_ts INTEGER, + duration_min REAL, in_bed_min REAL, efficiency REAL, + light_min REAL, deep_min REAL, rem_min REAL, + is_main INTEGER, -- 1 = longest period of the day (UI hint only) + confidence REAL, updated_at INTEGER, + PRIMARY KEY(user_id, id) +); +CREATE INDEX IF NOT EXISTS idx_sleep_periods_date ON sleep_periods(user_id, date); +CREATE INDEX IF NOT EXISTS idx_sleep_periods_onset ON sleep_periods(user_id, onset_ts); diff --git a/src/db/migrate_v7_step_goal.sql b/src/db/migrate_v7_step_goal.sql new file mode 100644 index 0000000..fe50614 --- /dev/null +++ b/src/db/migrate_v7_step_goal.sql @@ -0,0 +1,3 @@ +-- v7 — user-settable daily step goal. Additive nullable column; NULL means the +-- client uses its own default (8000). Non-breaking: existing rows read as NULL. +ALTER TABLE users ADD COLUMN step_goal INTEGER; diff --git a/src/db/migrate_v8_strain_curve.sql b/src/db/migrate_v8_strain_curve.sql new file mode 100644 index 0000000..3122082 --- /dev/null +++ b/src/db/migrate_v8_strain_curve.sql @@ -0,0 +1,7 @@ +-- v8 — precompute the intra-day strain curve + day HR stats in the cron so +-- /day/strain becomes a PURE READ (no live recompute on the endpoint). Additive +-- nullable columns; existing rows read as NULL until the next analytics run. +ALTER TABLE daily ADD COLUMN strain_curve TEXT; +ALTER TABLE daily ADD COLUMN hr_max INTEGER; +ALTER TABLE daily ADD COLUMN hr_min INTEGER; +ALTER TABLE daily ADD COLUMN hr_avg INTEGER; diff --git a/src/db/migrate_v9_steps_cursor.sql b/src/db/migrate_v9_steps_cursor.sql new file mode 100644 index 0000000..3505ee2 --- /dev/null +++ b/src/db/migrate_v9_steps_cursor.sql @@ -0,0 +1,6 @@ +-- v9 — incremental steps cursor. Lets the 30-min sweep recompute steps over only +-- the newly-SETTLED minutes since the last run (a few R2 objects) instead of +-- re-reading 2 days every time — keeping AN-2554 accuracy at ~free R2 cost. +-- Additive nullable columns on the existing analytics_cursor row. +ALTER TABLE analytics_cursor ADD COLUMN steps_cursor_ts INTEGER; -- last settled minute counted (unix s) +ALTER TABLE analytics_cursor ADD COLUMN steps_cursor_day TEXT; -- UTC day the accumulator is for diff --git a/src/db/schema.sql b/src/db/schema.sql index 9d5b48e..8830d29 100644 --- a/src/db/schema.sql +++ b/src/db/schema.sql @@ -11,6 +11,8 @@ CREATE TABLE IF NOT EXISTS users( height_cm REAL, weight_kg REAL, sex TEXT, -- 'm' | 'f' | NULL (NULL → sex-neutral calories) + step_goal INTEGER, -- user's daily step goal; NULL → client default (8000) + track_cycle INTEGER DEFAULT 0, -- explicit opt-in to menstrual cycle tracking (0/1) created_at INTEGER NOT NULL ); @@ -28,10 +30,37 @@ CREATE TABLE IF NOT EXISTS refresh_tokens( ); CREATE INDEX IF NOT EXISTS idx_refresh_user ON refresh_tokens(user_id); --- ── TIMESERIES ROLLUP (≤1440 rows/day/user; pruned to R2 after 90 days) ─────── --- hr_sum / act_sum / act_n are running aggregates kept so the ingest upsert can --- merge new samples into the stored minute EXACTLY (deterministic idempotency). --- hr_avg / activity are the derived display values (kept in sync on every write). +-- ── TIMESERIES (day-packed) ─────────────────────────────────────────────────── +-- [feat/wake-trigger] minute_day is the HOT minute store: ONE row per (user, ymd), +-- value = gzipped JSON MinuteRec[] (one entry per touched minute; RR rides the blob +-- as number[]). Ingest read-merge-writes ~1 row/day instead of ~1,440. Days older +-- than the hot window are sealed to gzipped R2 objects and the D1 row dropped; the +-- 10-day prune drops ~1 row/day. See minute_store.ts. (migrate_v14) +CREATE TABLE IF NOT EXISTS minute_day( + user_id TEXT NOT NULL, + ymd TEXT NOT NULL, -- 'YYYY-MM-DD' (UTC day of the minute) + blob BLOB NOT NULL, -- gzipped JSON MinuteRec[] (bound param; << 2MB cap) + updated_at INTEGER NOT NULL, + PRIMARY KEY(user_id, ymd) +) WITHOUT ROWID; +-- seal/prune scan by day across all users (WHERE ymd < cutoff) — index the bare ymd. +CREATE INDEX IF NOT EXISTS idx_minute_day_ymd ON minute_day(ymd); + +-- TTL read-cache for Tier 1/2 on-read metrics (no watermark; time-based only). +-- key e.g. 'today:strain:2026-06-19'; today→60s, past days immutable-until-prune. +-- See cache.ts. (migrate_v12) +CREATE TABLE IF NOT EXISTS read_cache( + user_id TEXT NOT NULL, + key TEXT NOT NULL, + payload TEXT NOT NULL, -- JSON + computed_at INTEGER NOT NULL, + PRIMARY KEY(user_id, key) +) WITHOUT ROWID; + +-- ── LEGACY ROLLUP (deprecated; empty on fresh DBs) ──────────────────────────── +-- The pre-v14 row-per-minute store. Superseded by minute_day above; left in place so +-- old deployments keep working, but ingest no longer writes here. Safe to drop once +-- no instance predates v14. CREATE TABLE IF NOT EXISTS minute( user_id TEXT NOT NULL, ts_min INTEGER NOT NULL, -- unix sec floored to minute @@ -69,6 +98,16 @@ CREATE TABLE IF NOT EXISTS journal( PRIMARY KEY(user_id, date) ); +-- Menstrual cycle log — user-logged period events (the anchor for calcCycle). +CREATE TABLE IF NOT EXISTS cycle_log( + user_id TEXT NOT NULL, + date TEXT NOT NULL, -- YYYY-MM-DD + kind TEXT NOT NULL, -- 'start' | 'end' | 'spotting' + note TEXT, + updated_at INTEGER, + PRIMARY KEY(user_id, date) +); + -- ── DERIVED (permanent, tiny) ───────────────────────────────────────────────── CREATE TABLE IF NOT EXISTS daily( user_id TEXT, date TEXT, @@ -92,6 +131,8 @@ CREATE TABLE IF NOT EXISTS daily( hrv_cv REAL, -- coefficient of variation of nightly RMSSD (%) nocturnal_dip_pct REAL, -- nocturnal HR dip (fraction) — trendable copy irregular TEXT, -- irregular-rhythm SCREEN (JSON, Poincaré) — not a diagnosis + strain_curve TEXT, -- precomputed intra-day cumulative-strain curve (JSON) so /day/strain is a pure read + hr_max INTEGER, hr_min INTEGER, hr_avg INTEGER, -- precomputed day HR stats (pure-read /day/strain) -- NOTE: `readiness` (above) is REPURPOSED for the composite Readiness index -- (HRV ∩ sleep ∩ dip ∩ arousal), written by biometrics.ts. confidence REAL, flags TEXT, updated_at INTEGER, @@ -106,14 +147,33 @@ CREATE TABLE IF NOT EXISTS sleep( PRIMARY KEY(user_id, date) ); +-- Sleep v2 (multi-period; naps = shorter sleeps). Additive; v1 `sleep` above is +-- the single main-period table and stays as-is. See migrate_v6_sleep_periods.sql. +CREATE TABLE IF NOT EXISTS sleep_periods( + user_id TEXT, id TEXT, + date TEXT, + onset_ts INTEGER, wake_ts INTEGER, + duration_min REAL, in_bed_min REAL, efficiency REAL, + light_min REAL, deep_min REAL, rem_min REAL, + is_main INTEGER, + confidence REAL, updated_at INTEGER, + PRIMARY KEY(user_id, id) +); +CREATE INDEX IF NOT EXISTS idx_sleep_periods_date ON sleep_periods(user_id, date); +CREATE INDEX IF NOT EXISTS idx_sleep_periods_onset ON sleep_periods(user_id, onset_ts); + CREATE TABLE IF NOT EXISTS sessions( user_id TEXT, id TEXT, start_ts INTEGER, end_ts INTEGER, type TEXT, avg_hr INTEGER, max_hr INTEGER, strain REAL, calories REAL, hrr60 INTEGER, zones TEXT, confidence REAL, status TEXT, -- 'live' | 'done' - source TEXT, -- 'manual' (user started) | 'auto' (detected) + source TEXT, -- 'manual' (user started) | 'auto' (minute backstop) | 'auto_live' (live-stream detected) title TEXT, -- optional user label + segments TEXT, -- JSON [{start_ts,end_ts,type,confidence}] phases (multi-activity workouts) + detected_type TEXT, -- the HAR classifier's call at detection (calibration ledger) + type_confidence REAL, -- ESTIMATE confidence in the workout type + type_source TEXT, -- 'model' | 'confirmed' | 'corrected' PRIMARY KEY(user_id, id) ); CREATE INDEX IF NOT EXISTS idx_sessions_user ON sessions(user_id, start_ts); @@ -145,7 +205,16 @@ CREATE TABLE IF NOT EXISTS analytics_cursor( user_id TEXT PRIMARY KEY, last_min_ts INTEGER DEFAULT 0, dirty INTEGER DEFAULT 1, - last_run INTEGER DEFAULT 0 + last_run INTEGER DEFAULT 0, + steps_cursor_ts INTEGER, -- incremental steps: last settled minute counted (unix s) + steps_cursor_day TEXT, -- UTC day the steps accumulator is for + bio_last_date TEXT, -- event-driven biometrics: last sleep-date already triggered + -- [feat/wake-trigger] incremental sleep/wake state machine (migrate_v13). The */N + -- cron reads these: skip awake-and-closed users, peek the asleep ones, fire close_day + -- once per physiological day. + sleep_phase TEXT, -- 'awake' | 'asleep' | NULL + phase_since INTEGER, -- unix s of last transition + last_close_date TEXT -- YYYY-MM-DD of last day-close ); -- Per-user ingest rate-limit token bucket (RESILIENCE §7). @@ -173,3 +242,38 @@ CREATE TABLE IF NOT EXISTS app_config( updated_at INTEGER ); INSERT OR IGNORE INTO app_config (id) VALUES (1); + +-- ── STRAVA (bidirectional sync) ─────────────────────────────────────────────── +-- OAuth tokens per user. access_token is short-lived; refresh via refresh_token. +CREATE TABLE IF NOT EXISTS strava_tokens( + user_id TEXT PRIMARY KEY, + athlete_id INTEGER, + access_token TEXT NOT NULL, + refresh_token TEXT NOT NULL, + expires_at INTEGER NOT NULL, -- unix seconds; refresh when now >= this + scope TEXT, + connected_at INTEGER +); +-- Activities pulled from Strava (Strava → Whoop), kept for the in-app overlay. +CREATE TABLE IF NOT EXISTS strava_activities( + user_id TEXT NOT NULL, + activity_id INTEGER NOT NULL, -- Strava's activity id + start_ts INTEGER, -- unix seconds (UTC) + elapsed_sec INTEGER, + type TEXT, + name TEXT, + distance_m REAL, + avg_hr REAL, max_hr REAL, + raw TEXT, -- the JSON summary, for later enrichment + pulled_at INTEGER, + PRIMARY KEY(user_id, activity_id) +); +CREATE INDEX IF NOT EXISTS idx_strava_act_user ON strava_activities(user_id, start_ts); +-- Which OpenStrap sessions we've pushed to Strava (Whoop → Strava dedupe). +CREATE TABLE IF NOT EXISTS strava_pushed( + user_id TEXT NOT NULL, + session_key TEXT NOT NULL, + activity_id INTEGER, + pushed_at INTEGER, + PRIMARY KEY(user_id, session_key) +); diff --git a/src/decode.ts b/src/decode.ts deleted file mode 100644 index 77f5dae..0000000 --- a/src/decode.ts +++ /dev/null @@ -1,206 +0,0 @@ -// decode.ts — verified WHOOP record decoders, mirroring the reference client / PROTOCOL.md. -// -// Each decoded record emits a DecodedSample: -// { ts, hr, activity, steps_inc, wrist_on, rec_type } -// where `activity` is the actigraphy signal = stddev of |accel(g)| over the -// 100-sample IMU window (R10 only; 0 for HR-only records). -// -// Offsets (PROTOCOL.md): -// R10 (rec_type 10, pkt 0x2F live/0x2B): ts@7, hr@17, counter@3, -// accel arrays @85/285/485, gyro @688/888/1088, scale ÷4096 (4096 LSB/g). -// 0x28 (live compact HR): ts@2 (u32 LE), hr@8 (u8), wrist via hr>0. -// 0x2B: same layout as R10 (live R10). -// R24 (rec_type 24): header ts@7, counter@3; spo2@72, skin_temp@70/4, resting_hr@88 (RELATIVE-only, not surfaced here). -// 0x33: live IMU stream — RAW-ONLY, no sample emitted (low decode confidence). -// -// NEVER decode HRV / RR-intervals. (R17 is BANNED.) - -import { parse_r24 } from 'openstrap-protocol/ts/records' - -export interface DecodedSample { - ts: number // unix seconds - hr: number // bpm (0 = off-wrist / no reading) - activity: number // motion magnitude (stddev of |accel(g)|), 0 if no IMU - steps_inc: number // steps detected in this record's IMU window (R10 only) - wrist_on: boolean // worn proxy (hr>0; authoritative wear is WRIST_ON/OFF events) - rec_type: number // 10 | 24 | 28 -} - -export const hexToBytes = (hex: string): Uint8Array => - new Uint8Array(hex.trim().match(/.{1,2}/g)!.map((b) => parseInt(b, 16))) - -/** One IMU frame's accel as ordered magnitude samples (g) + its time + sub-order. */ -export interface ImuFrame { ts: number; idx: number; mags: number[] } - -// frameAccel — decode one IMU frame's accelerometer into ordered |accel|(g) -// samples. Handles BOTH channels the strap streams: -// • 0x33 live IMU stream — ts@4, sub-frame idx@14, 10 accel samples -// (X[0:10],Y[10:20],Z[20:30]) from offset 24, scale 1/4096. -// • R10 (rec 0x0A) — ts@7, 100 accel samples @85/285/485, scale 1/4096. -// Returns null if it isn't an accel-bearing frame. Used by the backend steps -// runner (steps_imu.ts) to rebuild the signal for the AN-2554 pedometer that -// now lives in openstrap-analytics (calcSteps). Kept here with the other IMU -// decoders (see r10Motion) so all byte-offset knowledge stays in one place. -export function frameAccel(hex: string): ImuFrame | null { - let b: Uint8Array - try { b = hexToBytes(hex) } catch { return null } - if (b.length < 32) return null - const view = new DataView(b.buffer, b.byteOffset, b.byteLength) - const pkt = b[0], rec = b[1] - // 0x33 IMU stream: 10 accel samples (X,Y,Z) from offset 24. - if (pkt === 0x33 && b.length >= 84) { - const ts = view.getUint32(4, true) - const idx = view.getUint16(14, true) - const mags: number[] = [] - for (let i = 0; i < 10; i++) { - const x = view.getInt16(24 + 2 * i, true) - const y = view.getInt16(24 + 2 * (10 + i), true) - const z = view.getInt16(24 + 2 * (20 + i), true) - mags.push(Math.sqrt(x * x + y * y + z * z) / 4096) - } - return ts > 0 ? { ts, idx, mags } : null - } - // R10: rec 0x0A, ts@7, accel X@85/Y@285/Z@485 (100 int16 each). - if (rec === 0x0a && b.length >= 685) { - const ts = view.getUint32(7, true) - const mags: number[] = [] - for (let i = 0; i < 100; i++) { - const x = view.getInt16(85 + 2 * i, true) - const y = view.getInt16(285 + 2 * i, true) - const z = view.getInt16(485 + 2 * i, true) - mags.push(Math.sqrt(x * x + y * y + z * z) / 4096) - } - return ts > 0 ? { ts, idx: 0, mags } : null - } - return null -} - -// Decode the R10 IMU arrays into (activity, steps) over the 100-sample window. -// activity = stddev of per-sample |accel|(g) — actigraphy intensity. -// steps = count of GAIT CYCLES only when the window is genuinely rhythmic. -// -// The band has no pedometer field — steps are estimated from wrist IMU (ESTIMATE -// tier). The naive "count every peak" approach over-counts badly: any arm gesture, -// typing, or vehicle bump clears a fixed threshold. Instead we require RHYTHM: -// walking produces a periodic acceleration signal, so we (1) detrend the magnitude -// to remove gravity + slow drift, (2) measure how periodic it is via normalized -// autocorrelation over plausible step lags, and (3) only count steps when that -// periodicity is strong AND the limb is actually moving. Steps in the window = -// number of gait cycles = n / (dominant lag). Non-rhythmic motion → 0 steps. -// (Same autocorrelation idea we use for respiratory rate; standard wrist pedometry.) -function r10Motion(view: DataView, len: number): { activity: number; steps: number } { - if (len < 685) return { activity: 0, steps: 0 } - const ACC = 1 / 4096 - const arr = (off: number): number[] => { - const out: number[] = [] - for (let i = 0; i < 100; i++) { - const o = off + 2 * i - if (o + 2 <= len) out.push(view.getInt16(o, true)) - } - return out - } - const ax = arr(85), ay = arr(285), az = arr(485) // accel X/Y/Z - const n = Math.min(ax.length, ay.length, az.length) - if (n === 0) return { activity: 0, steps: 0 } - const mags: number[] = [] - for (let i = 0; i < n; i++) { - mags.push(Math.hypot(ax[i] * ACC, ay[i] * ACC, az[i] * ACC)) - } - const mean = mags.reduce((s, v) => s + v, 0) / n - const variance = mags.reduce((s, v) => s + (v - mean) ** 2, 0) / n - const std = Math.sqrt(variance) - const activity = Math.round(std * 1000) / 1000 - - // Limb must actually be oscillating — quiet wrist (typing/holding) → no steps. - const ACTIVITY_FLOOR = 0.05 // g RMS of the detrended signal - if (std < ACTIVITY_FLOOR || n < 24) return { activity, steps: 0 } - - // Detrend: remove a centered moving average (gravity + slow drift), leaving the - // gait oscillation around 0. - const W = 9 - const x: number[] = new Array(n); - for (let i = 0; i < n; i++) { - let s = 0, c = 0 - for (let j = Math.max(0, i - W); j <= Math.min(n - 1, i + W); j++) { s += mags[j]; c++ } - x[i] = mags[i] - s / c - } - const x0 = x.reduce((s, v) => s + v, 0) / n - let denom = 0 - for (let i = 0; i < n; i++) denom += (x[i] - x0) ** 2 - if (denom <= 1e-9) return { activity, steps: 0 } - - // Normalized autocorrelation over plausible step lags. Cadence ~1.4–3.0 steps/s; - // with a ~25 Hz IMU window that's ~8–18 samples/step. We scan a generous band - // and rely on the periodicity strength to confirm gait. - const MIN_LAG = 7, MAX_LAG = 40 - let bestLag = 0, bestR = 0 - for (let lag = MIN_LAG; lag <= Math.min(MAX_LAG, n - 1); lag++) { - let num = 0 - for (let i = 0; i < n - lag; i++) num += (x[i] - x0) * (x[i + lag] - x0) - const r = num / denom - if (r > bestR) { bestR = r; bestLag = lag } - } - - // Strong, sustained periodicity ⇒ walking/running; count the gait cycles. - const RHYTHM_THRESH = 0.45 - if (bestLag === 0 || bestR < RHYTHM_THRESH) return { activity, steps: 0 } - const steps = Math.round(n / bestLag) - return { activity, steps } -} - -/** - * Decode one hex record into a DecodedSample, or null if it carries no - * surfaceable sample (0x33 IMU stream, malformed, or unknown type). - */ -export function decodeRecord(hex: string): DecodedSample | null { - let b: Uint8Array - try { - b = hexToBytes(hex) - } catch { - return null - } - if (b.length < 4) return null - const view = new DataView(b.buffer, b.byteOffset, b.byteLength) - const pktType = b[0] - const recType = b[1] - - // 0x28 — live compact HR: ts@2 (u32 LE), hr@8 (u8). NO RR-intervals. - if (pktType === 0x28) { - if (b.length < 9) return null - const ts = view.getUint32(2, true) - const hr = b[8] - return { ts, hr, activity: 0, steps_inc: 0, wrist_on: hr > 0, rec_type: 28 } - } - - // 0x33 — live IMU stream: raw-only (kept in R2, no sample emitted). - if (pktType === 0x33) return null - - if (b.length < 18) return null - - // R24 — type-24 historical telemetry. - if (recType === 24) { - const d = parse_r24(b) - if (!d) return null - return { ts: d.ts_epoch, hr: d.hr, activity: 0, steps_inc: 0, wrist_on: d.hr > 0, rec_type: 24 } - } - - // R10 / 0x2B — ts@7, hr@17, IMU arrays → activity. - if (recType === 10) { - const ts = view.getUint32(7, true) - const hr = b[17] - const m = r10Motion(view, b.length) - return { ts, hr, activity: m.activity, steps_inc: m.steps, wrist_on: hr > 0, rec_type: 10 } - } - - return null -} - -/** Decode a batch of hex records, returning all surfaceable samples. */ -export function decodeBatch(records: string[]): DecodedSample[] { - const out: DecodedSample[] = [] - for (const hex of records) { - const s = decodeRecord(hex) - if (s) out.push(s) - } - return out -} diff --git a/src/index.ts b/src/index.ts index b91a68a..35a5e25 100644 --- a/src/index.ts +++ b/src/index.ts @@ -4,27 +4,35 @@ import { } from './auth' import { runAnalytics, processUser } from './analytics' import { ingestBatch, ingestEvents } from './ingest' -import { handleQueueBatch, type AnalyticsMessage } from './queue' -import { getToday, getSleep, getStrain, getSessions, getTrends, getChart } from './query' +import { handleQueueBatch, type AnalyticsMessage, type AnalyticsJob } from './queue' +import { runWakeLadder, retryStaleCloses } from './wake_cron' +import { sealOldDays, pruneMinuteDays } from './minute_store' +import { getToday, getSleep, getSleepV2, getStrain, getSessions, getTrends, getChart } from './query' import { getHistory } from './history' import { postJournal, getJournal, getJournalInsights } from './journal' -import { getDayStrain, getDaySleep, getDayTimeline, getDayStress, getDayHeart, getDayLungs, getDayWear } from './daydetail' +import { postCycleLog, deleteCycleLog, getCycle } from './cycle' +import { postSpotCheck } from './spotcheck' +import { getDayStrain, getDaySleep, getDaySleepV2, getDayTimeline, getDayStress, getDayHeart, getDayLungs, getDayWear, getDayHrv } from './daydetail' import { getTrend } from './trend' -import { workoutStart, workoutEnd, listWorkouts, getWorkout, deleteWorkout, autoCloseStaleWorkouts } from './workouts' +import { workoutStart, workoutEnd, listWorkouts, getWorkout, deleteWorkout, autoCloseStaleWorkouts, setWorkoutType, sweepWorkoutDetection } from './workouts' import { getRecords } from './records' import { getNotifications, markNotificationsRead } from './notifications' -import { runRespRate } from './resp' -import { runBiometrics } from './biometrics' -import { runStepsImu } from './steps_imu' import { getAppStatus, adminGetConfig, adminSetConfig } from './appconfig' import { seedInit, seedMinutes, seedAnalytics } from './seed' +import { coachChatCompletions, coachModels, type AiBinding } from './coach' +import { stravaConnect, stravaCallback, stravaStatus, stravaDisconnect, stravaSync, stravaActivities, syncAllStrava } from './strava' type Bindings = { DB: D1Database RAW_BUCKET: R2Bucket + RATE_LIMITER?: { limit(opts: { key: string }): Promise<{ success: boolean }> } ANALYTICS_Q?: Queue JWT_SECRET: string ADMIN_TOKEN?: string + AI: AiBinding // Cloudflare Workers AI (self-hosted AI Coach) + COACH_KEY?: string // bearer the in-app AI Coach uses for /coach/v1/* + STRAVA_CLIENT_ID?: string // Strava API app credentials (bidirectional sync) + STRAVA_CLIENT_SECRET?: string BREVO_API_KEY?: string RESEND_API_KEY?: string EMAIL_FROM?: string @@ -39,6 +47,11 @@ const REFRESH_TTL = 30 * 24 * 60 * 60 // 30d const OTP_TTL = 10 * 60 // 10m const OTP_MAX_ATTEMPTS = 5 const DAY = 86400 +// Per-minute rollups are kept this long. The app shows minute-level detail +// (hypnogram, 24h timelines, wear, workout HR curve) for the last 7 days; this +3 +// buffer avoids a boundary race at the edge of that window. Derived metrics +// (daily/sleep/sessions) are permanent — pruning minutes never affects them. +const MINUTE_RETENTION_DAYS = 10 const app = new Hono<{ Bindings: Bindings; Variables: Vars }>() @@ -60,10 +73,27 @@ const requireAdmin = async (c: any, next: any) => { await next() } +// ---------- self-hosted AI Coach (OpenAI-compatible, Workers AI) ---------- +// Auth is the COACH_KEY bearer, checked inside the handlers (not requireJwt) so the +// app's BYOK AI-Coach client can point its base URL at `/coach/v1`. +app.post('/coach/v1/chat/completions', coachChatCompletions) +app.get('/coach/v1/models', coachModels) + +// ---------- Strava (bidirectional sync) ---------- +// /connect, /status, /disconnect need the user JWT; /callback is Strava's public +// redirect target (auth is the signed `state` it round-trips). +app.get('/strava/connect', requireJwt, stravaConnect) +app.get('/strava/callback', stravaCallback) +app.get('/strava/status', requireJwt, stravaStatus) +app.post('/strava/disconnect', requireJwt, stravaDisconnect) +app.get('/strava/sync', requireJwt, stravaSync) +app.get('/strava/activities', requireJwt, stravaActivities) + app.use('/ingest/*', requireJwt) app.use('/profile', requireJwt) app.use('/today', requireJwt) app.use('/sleep', requireJwt) +app.use('/sleep/v2', requireJwt) app.use('/strain', requireJwt) app.use('/sessions', requireJwt) app.use('/trends', requireJwt) @@ -71,6 +101,9 @@ app.use('/chart', requireJwt) app.use('/history', requireJwt) app.use('/journal', requireJwt) app.use('/journal/*', requireJwt) +app.use('/cycle', requireJwt) +app.use('/cycle/*', requireJwt) +app.use('/spotcheck', requireJwt) app.use('/day/*', requireJwt) app.use('/trend/*', requireJwt) app.use('/workout/*', requireJwt) @@ -191,22 +224,27 @@ app.post('/auth/refresh', async (c) => { app.get('/profile', async (c) => { const user = await c.env.DB.prepare( - 'SELECT id, email, name, age, height_cm, weight_kg, sex, created_at FROM users WHERE id = ?', + 'SELECT id, email, name, age, height_cm, weight_kg, sex, step_goal, track_cycle, created_at FROM users WHERE id = ?', ).bind(c.get('userId')).first() if (!user) return c.json({ error: 'Not found' }, 404) return c.json(user) }) app.patch('/profile', async (c) => { - const { name, age, height_cm, weight_kg, sex } = - await c.req.json<{ name?: string; age?: number; height_cm?: number; weight_kg?: number; sex?: string }>() + const { name, age, height_cm, weight_kg, sex, step_goal, track_cycle } = + await c.req.json<{ name?: string; age?: number; height_cm?: number; weight_kg?: number; sex?: string; step_goal?: number; track_cycle?: boolean | number }>() const sexVal = sex === 'm' || sex === 'f' ? sex : null + // step_goal: clamp to a sane range when provided; null leaves it unchanged. + const goalVal = (typeof step_goal === 'number' && isFinite(step_goal)) + ? Math.max(1000, Math.min(50000, Math.round(step_goal))) : null + // track_cycle: explicit opt-in toggle (null leaves it unchanged). + const trackVal = track_cycle === undefined ? null : (track_cycle ? 1 : 0) await c.env.DB.prepare( 'UPDATE users SET name=COALESCE(?,name), age=COALESCE(?,age), height_cm=COALESCE(?,height_cm), ' + - 'weight_kg=COALESCE(?,weight_kg), sex=COALESCE(?,sex) WHERE id = ?', - ).bind(name ?? null, age ?? null, height_cm ?? null, weight_kg ?? null, sexVal, c.get('userId')).run() + 'weight_kg=COALESCE(?,weight_kg), sex=COALESCE(?,sex), step_goal=COALESCE(?,step_goal), track_cycle=COALESCE(?,track_cycle) WHERE id = ?', + ).bind(name ?? null, age ?? null, height_cm ?? null, weight_kg ?? null, sexVal, goalVal, trackVal, c.get('userId')).run() const user = await c.env.DB.prepare( - 'SELECT id, email, name, age, height_cm, weight_kg, sex, created_at FROM users WHERE id = ?', + 'SELECT id, email, name, age, height_cm, weight_kg, sex, step_goal, track_cycle, created_at FROM users WHERE id = ?', ).bind(c.get('userId')).first() return c.json(user) }) @@ -218,6 +256,7 @@ app.post('/ingest/events', ingestEvents) // ========================= QUERY ========================= app.get('/today', getToday) app.get('/sleep', getSleep) +app.get('/sleep/v2', getSleepV2) app.get('/strain', getStrain) app.get('/sessions', getSessions) app.get('/trends', getTrends) @@ -226,18 +265,25 @@ app.get('/history', getHistory) app.post('/journal', postJournal) app.get('/journal', getJournal) app.get('/journal/insights', getJournalInsights) +app.post('/cycle/log', postCycleLog) +app.delete('/cycle/log', deleteCycleLog) +app.get('/cycle', getCycle) +app.post('/spotcheck', postSpotCheck) app.get('/day/strain', getDayStrain) app.get('/day/sleep', getDaySleep) +app.get('/day/v2/sleep', getDaySleepV2) app.get('/day/timeline', getDayTimeline) app.get('/day/stress', getDayStress) app.get('/day/heart', getDayHeart) app.get('/day/lungs', getDayLungs) +app.get('/day/hrv', getDayHrv) app.get('/day/wear', getDayWear) app.get('/trend/:metric', getTrend) app.post('/workout/start', workoutStart) app.post('/workout/end', workoutEnd) app.get('/workouts', listWorkouts) app.get('/workout/:id', getWorkout) +app.post('/workout/:id/type', setWorkoutType) app.delete('/workout/:id', deleteWorkout) app.get('/records', getRecords) app.get('/notifications', getNotifications) @@ -249,43 +295,22 @@ app.post('/notifications/read', markNotificationsRead) app.get('/admin/config', adminGetConfig) app.post('/admin/config', adminSetConfig) +// [drop-r2] re-derive analytics from D1 (no R2). HRV/resp/optical land at the wake-close +// (close_day → runBiometricsMinute); to recompute them for a user, enqueue close_day via +// /admin/enqueue. This endpoint just re-runs the D1 daily/sleep/strain derivation. app.post('/admin/run-analytics', async (c) => { - const body = await c.req.json<{ user_id?: string; days?: number; bio?: boolean }>().catch(() => ({} as any)) + const body = await c.req.json<{ user_id?: string; days?: number }>().catch(() => ({} as any)) const days = body.days ?? 3 if (body.user_id) { - // Full re-derive sequence so HRV recovery feeds the coach: - // 1. processUser — minute metrics (strain/RHR/sleep/sessions...) + daily rows - // 2. runBiometrics — HRV recovery/stress/illness from RR (needs daily RHR) - // 3. processUser — coach picks up the recovery written in step 2 const r1 = await processUser(c.env.DB, body.user_id, { historyDays: days }) - let bio: any = null - if (body.bio !== false) { - bio = await runBiometrics(c.env, body.user_id, days) - await processUser(c.env.DB, body.user_id, { historyDays: days }) - } - return c.json({ ok: true, ...r1, bio }) + return c.json({ ok: true, ...r1 }) } const res = await runAnalytics(c.env.DB, { historyDays: days }) return c.json({ ok: true, ...res }) }) -// Respiratory rate from PPG (R21 re-decoded from R2). Heavy (R2 reads) → admin / -// cron only. Stores resp_rate/resp_conf on daily; gated (conf ≥ 0.5) before surfaced. -app.post('/admin/run-resp', async (c) => { - const body = await c.req.json<{ user_id: string; days?: number }>().catch(() => ({} as any)) - if (!body.user_id) return c.json({ error: 'user_id required' }, 400) - const res = await runRespRate(c.env, body.user_id, body.days ?? 3) - return c.json({ ok: true, ...res }) -}) - -// HRV (RMSSD) + relative skin-temp / SpO₂ from the V24 RR/ADC bytes, re-decoded -// from R2. Heavy (R2 reads) → admin / cron only. Writes daily.hrv_rmssd etc. -app.post('/admin/run-biometrics', async (c) => { - const body = await c.req.json<{ user_id: string; days?: number }>().catch(() => ({} as any)) - if (!body.user_id) return c.json({ error: 'user_id required' }, 400) - const res = await runBiometrics(c.env, body.user_id, body.days ?? 3) - return c.json({ ok: true, ...res }) -}) +// (Steps are AN-2554-only now: counted at ingest into minute.steps, summed into +// daily.steps by processUser, served live on-read. No R2 step job / admin endpoint.) // Phased to stay under the free-plan per-request subrequest cap. The shell // orchestrates: init → minutes (chunked) → analytics. `now` is pinned across @@ -338,11 +363,32 @@ app.post('/admin/issue-token', async (c) => { return c.json({ access_jwt: access, refresh_token: refresh, user_id: user.id }) }) +// Enqueue an analytics job for a user (ops + verification). The consumer runs it +// in its own bounded invocation. 404 if Queues isn't bound. +app.post('/admin/enqueue', async (c) => { + const body = await c.req.json<{ user_id: string; job?: AnalyticsJob; day?: string; onset_ts?: number; wake_ts?: number }>().catch(() => ({} as any)) + if (!body.user_id) return c.json({ error: 'user_id required' }, 400) + if (!c.env.ANALYTICS_Q) return c.json({ error: 'queue not bound' }, 404) + const msg = { + user_id: body.user_id, job: body.job ?? 'sweep', + ...(body.day ? { day: body.day } : {}), + // Forward the sleep window so a close_day job can be fired for verification + // (the wake_cron supplies these in production). + ...(body.onset_ts ? { onset_ts: body.onset_ts } : {}), + ...(body.wake_ts ? { wake_ts: body.wake_ts } : {}), + } + await c.env.ANALYTICS_Q.send(msg) + return c.json({ ok: true, enqueued: msg }) +}) + +// Clean up the legacy raw/ namespace ONLY. Per-POST raw archival was removed; existing +// raw/ objects also self-expire via the bucket lifecycle rule. This is prefix-scoped so +// it can NEVER touch the sealed minute/ blobs (the hot/seal cold tier we still use). app.post('/admin/wipe-raw', async (c) => { let deleted = 0 let cursor: string | undefined do { - const listing = await c.env.RAW_BUCKET.list({ cursor, limit: 1000 }) + const listing = await c.env.RAW_BUCKET.list({ prefix: 'raw/', cursor, limit: 1000 }) const keys = listing.objects.map((o) => o.key) if (keys.length > 0) { await c.env.RAW_BUCKET.delete(keys) @@ -353,71 +399,53 @@ app.post('/admin/wipe-raw', async (c) => { return c.json({ deleted }) }) -// Prune minute rows older than 90 days (raw stays in R2). +// Prune minute rows older than retention (sealed minute blobs stay in R2). app.post('/admin/prune', async (c) => { - const cutoff = Math.floor(Date.now() / 1000) - 90 * DAY - const res = await c.env.DB.prepare('DELETE FROM minute WHERE ts_min < ?').bind(cutoff).run() - return c.json({ ok: true, deleted: res.meta?.changes ?? 0, cutoff }) + const now = Math.floor(Date.now() / 1000) + await pruneMinuteDays(c.env, now, MINUTE_RETENTION_DAYS) // day-packed minute_day rows + const ev = await c.env.DB.prepare('DELETE FROM events WHERE ts < ?').bind(now - MINUTE_RETENTION_DAYS * DAY).run() + return c.json({ ok: true, events_deleted: ev.meta?.changes ?? 0 }) }) export default { fetch: app.fetch, - // Queue consumer (no-op if Queues disabled / binding absent — export is safe). + // Queue consumer — one bounded (user, job) unit per invocation. async queue(batch: MessageBatch, env: Bindings): Promise { await handleQueueBatch(batch, env) }, - // Crons: "7 * * * *" hourly safety sweep; "30 3 * * *" nightly re-derive+prune. + // [feat/wake-trigger] The frequent cron does ONE thing: the isUserAwake ladder + // (wake_cron.runWakeLadder) — detect each user's real wake and fire ONE close_day + // per physiological day. Cheap enough for */10 (awake-and-closed users = a cursor + // read; asleep = a 30-min peek; the heavy ensemble runs once, at the wake). All + // derivation lives in the close_day QUEUE job, never here. A SEPARATE nightly tick + // does maintenance only: prune aged minute/events + a retry-net for missed closes. async scheduled(event: ScheduledEvent, env: Bindings, ctx: ExecutionContext): Promise { - if (event.cron === '30 3 * * *') { - ctx.waitUntil((async () => { - // Full re-derive yesterday for all users that have any minute data, then prune. - await runAnalytics(env.DB, { historyDays: 2 }) - // Respiratory rate from PPG for users with a recent sleep row (R2 re-decode; - // null on nights without live PPG — gate handles it). Bounded per night. - try { - const since = new Date(Date.now() - 2 * DAY * 1000).toISOString().slice(0, 10) - const { results: users } = await env.DB.prepare( - 'SELECT DISTINCT user_id FROM sleep WHERE date >= ? AND onset_ts IS NOT NULL', - ).bind(since).all<{ user_id: string }>() - for (const u of users ?? []) await runRespRate(env, u.user_id, 2) - } catch (e) { console.error('resp cron failed', e) } - // HRV + relative skin-temp/SpO₂ from the V24 RR/ADC bytes (R2 re-decode). - try { - const since = new Date(Date.now() - 3 * DAY * 1000).toISOString().slice(0, 10) - const { results: users } = await env.DB.prepare( - 'SELECT DISTINCT user_id FROM daily WHERE date >= ?', - ).bind(since).all<{ user_id: string }>() - for (const u of users ?? []) await runBiometrics(env, u.user_id, 3) - } catch (e) { console.error('biometrics cron failed', e) } - // Steps from the wrist IMU (R10 + 0x33 re-decode → AN-2554 pedometer). - try { - const since = new Date(Date.now() - 2 * DAY * 1000).toISOString().slice(0, 10) - const { results: users } = await env.DB.prepare( - 'SELECT DISTINCT user_id FROM daily WHERE date >= ?', - ).bind(since).all<{ user_id: string }>() - for (const u of users ?? []) await runStepsImu(env, u.user_id, 2) - } catch (e) { console.error('steps cron failed', e) } - const cutoff = Math.floor(Date.now() / 1000) - 90 * DAY - await env.DB.prepare('DELETE FROM minute WHERE ts_min < ?').bind(cutoff).run() - })()) - } else { - ctx.waitUntil((async () => { - // Hourly safety net: process any dirty users, then close stale workouts. - await runAnalytics(env.DB, { historyDays: 3 }) - await autoCloseStaleWorkouts(env.DB) - // Steps LAST so the IMU-derived value (AN-2554) is authoritative over the - // minute-based one analytics writes. Refresh today + yesterday for users - // active in the last 2h (bounded R2 reads). - try { - const since = Math.floor(Date.now() / 1000) - 2 * 3600 - const { results: users } = await env.DB.prepare( - 'SELECT DISTINCT user_id FROM minute WHERE ts_min >= ?', - ).bind(since).all<{ user_id: string }>() - for (const u of users ?? []) await runStepsImu(env, u.user_id, 2) - } catch (e) { console.error('steps hourly cron failed', e) } - })()) - } + ctx.waitUntil((async () => { + // EVERY tick — wake detection ONLY (the cron's sole job). Auto-workout detection + // and forgotten-live-workout close moved ON-READ (workouts.ensureTodayWorkouts); + // the nightly tick keeps autoCloseStaleWorkouts as a safety net for users who + // never open the app. + try { await runWakeLadder(env) } catch (e) { console.error('wake ladder failed', e) } + // #D incremental workout detection: re-derive TODAY's auto-workouts for users who + // ingested since their last close, so workouts surface ~one tick after they end + // even with the app closed. Throttled per-user + bounded; no hot-path cost. + try { await sweepWorkoutDetection(env) } catch (e) { console.error('wkt sweep failed', e) } + // Nightly maintenance ONLY (separate from detection): seal + retention + retry-net. + if (event.cron === '30 3 * * *') { + try { await autoCloseStaleWorkouts(env.DB) } catch (e) { console.error('autoclose failed', e) } + // Seal days older than the hot window to gzipped R2 objects and drop them from + // D1 (D1-hot / R2-sealed tiering — cuts D1 storage + per-row prune-deletes). + try { await sealOldDays(env) } catch (e) { console.error('seal failed', e) } + // Backstop: clear any minute_day/events still unsealed past retention (e.g. no bucket). + const nowS = Math.floor(Date.now() / 1000) + await pruneMinuteDays(env, nowS, MINUTE_RETENTION_DAYS) + await env.DB.prepare('DELETE FROM events WHERE ts < ?').bind(nowS - MINUTE_RETENTION_DAYS * DAY).run() + try { await retryStaleCloses(env) } catch (e) { console.error('retry-net failed', e) } + // Strava: pull new activities (incl. Wahoo rides) + push non-duplicate workouts. + try { await syncAllStrava(env) } catch (e) { console.error('strava sync failed', e) } + } + })()) }, } diff --git a/src/ingest.ts b/src/ingest.ts index 3df7204..e6894f4 100644 --- a/src/ingest.ts +++ b/src/ingest.ts @@ -2,60 +2,45 @@ // Target <50ms CPU; NEVER runs analytics inline. import type { Context } from 'hono' -import { decodeBatch, hexToBytes } from './decode' +import { decodeBatch, hexToBytes } from 'openstrap-protocol/ts/live' import { rollupMinutes } from './rollup' - -// ── Rate limiting: per-user token bucket in a D1 row (RESILIENCE §7). ── -// Refill RATE tokens/sec, cap BURST. One ingest = one token. Over budget → 429. -const RL_BURST = 30 // allow short bursts (bulk reconnect upload) -const RL_REFILL = 0.5 // 0.5 tokens/sec ≈ 30 batches/min sustained - -export async function checkRateLimit(db: D1Database, userId: string): Promise { - const now = Math.floor(Date.now() / 1000) - const row = await db.prepare( - 'SELECT tokens, updated_at FROM rate_limit WHERE user_id = ?', - ).bind(userId).first<{ tokens: number; updated_at: number }>() - - let tokens = RL_BURST - if (row) { - const elapsed = Math.max(0, now - row.updated_at) - tokens = Math.min(RL_BURST, row.tokens + elapsed * RL_REFILL) - } - if (tokens < 1) { - // Persist the (refilled-but-still-empty) state so the clock keeps ticking. - await db.prepare( - 'INSERT INTO rate_limit (user_id, tokens, updated_at) VALUES (?,?,?) ' + - 'ON CONFLICT(user_id) DO UPDATE SET tokens=excluded.tokens, updated_at=excluded.updated_at', - ).bind(userId, tokens, now).run() - return false - } - tokens -= 1 - await db.prepare( - 'INSERT INTO rate_limit (user_id, tokens, updated_at) VALUES (?,?,?) ' + - 'ON CONFLICT(user_id) DO UPDATE SET tokens=excluded.tokens, updated_at=excluded.updated_at', - ).bind(userId, tokens, now).run() - return true -} +import { perMinuteSignals } from './ingest_signals' +import { writeBatch } from './minute_store' + +// ── Rate limiting: native Workers Rate-Limiting binding (in-memory at the edge, +// per-colo). Replaces the old D1 token-bucket, which cost 1 D1 read + 1 D1 WRITE on +// EVERY ingest POST to maintain a bucket that — at 60s batching — never actually +// throttled legitimate traffic (it fully refilled between posts). The binding has no +// D1 I/O and no separate billing. Limit kept equivalent to the old 30/min (limit 30 / +// 60s, configured as `simple` in wrangler). Per-colo + approximate, which is exactly +// right for abuse/burst protection. Guarded so the worker still runs if the binding +// isn't configured on a given environment (then ingest is unthrottled — the edge +// already self-throttles to ~1 POST/min). +interface RateLimiter { limit(opts: { key: string }): Promise<{ success: boolean }> } interface IngestEnv { DB: D1Database RAW_BUCKET: R2Bucket ANALYTICS_Q?: Queue + RATE_LIMITER?: RateLimiter } -async function enqueueOrMarkDirty(env: IngestEnv, userId: string, upto: number) { - if (env.ANALYTICS_Q) { - try { - await env.ANALYTICS_Q.send({ user_id: userId, upto }) - return - } catch (e) { - console.error('queue send failed; falling back to dirty flag', e) - } - } - // Fallback: mark dirty for the cron sweep. +// Mark the user dirty so the NEXT 30-min cron sweep enqueues exactly ONE analytics +// job for them (deduped). We deliberately do NOT enqueue per ingest batch: at scale +// that storms the queue (~240 msgs/user/day) and re-derives the same day hundreds of +// times. A cheap dirty flag here + the cron as the single enqueue point makes it one +// sweep per user per 30 min. Freshness is bounded by the sweep interval (fine — and +// recovery still fires promptly on wake via the sleep-detection trigger). +async function markUserDirty(env: IngestEnv, userId: string) { + // Cost: `dirty` stays 1 from the first ingest until the wake-close clears it, so an + // unconditional upsert bills a D1 ROW WRITE on every batch (~1,440/user/day) to set a + // flag that's already 1. The conditional DO UPDATE writes a row ONLY on the real 0→1 + // transition (first ingest of a new cycle / after a close) — steady-state batches hit + // the WHERE=false path and write 0 rows (just a cheap PK read). ~1 dirty-write per + // wake-cycle instead of per-POST, cutting ingest D1 writes ~1/3. Same observable state. await env.DB.prepare( 'INSERT INTO analytics_cursor (user_id, last_min_ts, dirty) VALUES (?,0,1) ' + - 'ON CONFLICT(user_id) DO UPDATE SET dirty = 1', + 'ON CONFLICT(user_id) DO UPDATE SET dirty = 1 WHERE analytics_cursor.dirty = 0', ).bind(userId).run() } @@ -65,63 +50,33 @@ export async function ingestBatch(c: Context<{ Bindings: IngestEnv; Variables: { const { device_id, records } = await c.req.json<{ device_id: string; records: string[] }>() if (!device_id || !Array.isArray(records)) return c.json({ error: 'Invalid payload' }, 400) - // 1. Rate limit. - if (!(await checkRateLimit(c.env.DB, userId))) { - return c.json({ error: 'Rate limit exceeded' }, 429) + // 1. Rate limit (edge binding; no D1). Skipped if the binding isn't configured. + if (c.env.RATE_LIMITER) { + const { success } = await c.env.RATE_LIMITER.limit({ key: userId }) + if (!success) return c.json({ error: 'Rate limit exceeded' }, 429) } // 2. Decode. const samples = decodeBatch(records) - // 3. R2 put the whole batch raw (re-decodable system of record). - let rawKey: string | null = null - if (records.length > 0) { - const tss = samples.map((s) => s.ts).filter((t) => t > 0) - const minTs = tss.length ? Math.min(...tss) : 0 - const maxTs = tss.length ? Math.max(...tss) : 0 - rawKey = `raw/${userId}/${device_id}/${Date.now()}-${minTs}-${maxTs}.txt` - try { - await c.env.RAW_BUCKET.put(rawKey, records.join('\n')) - } catch (e) { - console.error('R2 put failed', e) - rawKey = null - } - } - - // 4. Rollup → minute upsert (one batch). Merge deterministically with stored. + // 3. (Raw R2 archive removed.) Decoders are validated, and every surfaced signal — + // HR/RR/HRV, steps, SpO₂/temp, and PPG-resp (via the R21 green RIIV proxy) — is now + // derived at ingest and stored in the D1 minute blob, so there is nothing to + // re-decode from raw later. RAW_BUCKET is retained ONLY for the hot/seal tier + // (sealOldDays moves aged minute_day blobs to R2); no per-POST raw object is written. + + // 4. Rollup → day-packed minute store (ONE row per day, read-merge-write). This is + // the cost lever: ~1 row written/day instead of ~1,440. The merge inside writeBatch + // mirrors the old per-minute ON CONFLICT exactly (additive sums, CASE hr_min, MAX + // hr_max, steps add, rr longer-wins). Idempotent IFF the edge dedupes by hex (it does). + // RR (ms, R24 + live 0x28/R10) rides the blob as number[] — HRV folds into the + // wake-close with zero R2. (Storage: gzipped blob via bound param — well under D1's + // 2 MB value cap; serialized outside the 100 KB SQL-statement limit.) const buckets = rollupMinutes(samples) let minutesWritten = 0 - let maxTsMin = 0 if (buckets.length > 0) { - // The upsert folds new running sums into stored aggregates and recomputes - // the display columns (hr_avg, activity, hr_min/max) from the merged totals. - const stmt = c.env.DB.prepare( - 'INSERT INTO minute (user_id, ts_min, hr_avg, hr_min, hr_max, hr_n, hr_sum, activity, act_sum, act_n, steps, wrist_on) ' + - 'VALUES (?,?,?,?,?,?,?,?,?,?,?,?) ' + - 'ON CONFLICT(user_id, ts_min) DO UPDATE SET ' + - 'hr_sum = minute.hr_sum + excluded.hr_sum, ' + - 'hr_n = minute.hr_n + excluded.hr_n, ' + - 'hr_min = CASE WHEN minute.hr_min = 0 THEN excluded.hr_min ' + - ' WHEN excluded.hr_min = 0 THEN minute.hr_min ' + - ' ELSE MIN(minute.hr_min, excluded.hr_min) END, ' + - 'hr_max = MAX(minute.hr_max, excluded.hr_max), ' + - 'hr_avg = CASE WHEN (minute.hr_n + excluded.hr_n) > 0 ' + - ' THEN (minute.hr_sum + excluded.hr_sum) / (minute.hr_n + excluded.hr_n) ELSE 0 END, ' + - 'act_sum = minute.act_sum + excluded.act_sum, ' + - 'act_n = minute.act_n + excluded.act_n, ' + - 'activity = CASE WHEN (minute.act_n + excluded.act_n) > 0 ' + - ' THEN (minute.act_sum + excluded.act_sum) / (minute.act_n + excluded.act_n) ELSE 0 END, ' + - 'steps = minute.steps + excluded.steps, ' + - 'wrist_on = MAX(minute.wrist_on, excluded.wrist_on)', - ) - const batch = buckets.map((b) => { - if (b.ts_min > maxTsMin) maxTsMin = b.ts_min - const hr_avg = b.hr_n > 0 ? Math.round(b.hr_sum / b.hr_n) : 0 - const activity = b.act_n > 0 ? b.act_sum / b.act_n : 0 - return stmt.bind(userId, b.ts_min, hr_avg, b.hr_min, b.hr_max, b.hr_n, b.hr_sum, - activity, b.act_sum, b.act_n, b.steps, b.wrist_on) - }) - await c.env.DB.batch(batch) + const sig = perMinuteSignals(records) + await writeBatch(c.env, userId, buckets, sig, Math.floor(Date.now() / 1000)) minutesWritten = buckets.length } @@ -130,18 +85,17 @@ export async function ingestBatch(c: Context<{ Bindings: IngestEnv; Variables: { // /ingest/events route handles event-only payloads. Here we skip. // 6. Enqueue analytics (or mark dirty). Never run inline. - if (minutesWritten > 0) await enqueueOrMarkDirty(c.env, userId, maxTsMin || Math.floor(Date.now() / 1000)) + if (minutesWritten > 0) await markUserDirty(c.env, userId) - // 7. Respond. `received` = records persisted raw to R2 (the re-decodable system of - // record); `decoded` = records that yielded a surfaceable sample; `minutes_written` - // = per-minute rollups touched. The client shows `received` so the count is honest - // (a 2xx means we stored them all), not 0. + // 7. Respond. `received` = records accepted (a 2xx means we decoded + rolled them into + // the D1 minute store); `decoded` = records that yielded a surfaceable sample; + // `minutes_written` = per-minute rollups touched. The client shows `received` so the + // count is honest, not 0. return c.json({ ok: true, received: records.length, decoded: samples.length, minutes_written: minutesWritten, - raw_key: rawKey, }) } diff --git a/src/ingest_signals.ts b/src/ingest_signals.ts new file mode 100644 index 0000000..93ae5d2 --- /dev/null +++ b/src/ingest_signals.ts @@ -0,0 +1,175 @@ +// ingest_signals.ts — [v2] per-minute STEPS (AN-2554 pedometer) + RR, computed at +// ingest from the batch's raw frames, so the heavy jobs never re-read R2. +// +// For each minute the batch touches: +// • steps = calcSteps over that minute's accelerometer magnitude signal. +// • rr = beat-to-beat intervals (ms) from the minute's R24 (historical) AND +// live (0x28 compact HR + R10) records — all hard-gated to 300–2000 ms. +// +// R24 DETECTION mirrors decodeRecord EXACTLY: recType = b[1] === 24 (NOT gated on +// packet type) — so historical R24 RR is never silently dropped. +// +// IDEMPOTENCY: both merge into the minute row with "keep the fuller value" (steps = +// MAX, rr = longer blob), so a re-uploaded batch can't double-count and a fuller +// batch wins. + +import { calcSteps, cleanRr, extractHarFeaturesFromSmv, classifyActivityWindow } from 'openstrap-analytics' +import { frameAccel, hexToBytes, realtimeRr } from 'openstrap-protocol/ts/live' +import { parse_r24 } from 'openstrap-protocol/ts/records' + +const HAR_FS = 100 // live IMU ≈ 100 Hz (R10 100 samples/s, 0x33 stream) +const HAR_WIN = 400 // 4 s Mannini window +const HAR_MIN = 128 // need ≥~1.3 s for a defensible spectrum + +/** Classify a minute's magnitude (SMV) signal into a dominant HAR class via the + * Mannini feature extractor + threshold classifier. Returns undefined when there's + * too little high-rate accel (e.g. flash 1 Hz minutes carry none). */ +function classifyMinute(sig: number[]): string | undefined { + if (sig.length < HAR_MIN) return undefined + // Defensive: a HAR bug must NEVER break the ingest hot path (raw-first / never lose + // data). Any failure → no class label for the minute, ingest proceeds unaffected. + try { + const counts: Record = {} + for (let i = 0; i + HAR_MIN <= sig.length; i += HAR_WIN) { + const w = sig.slice(i, Math.min(i + HAR_WIN, sig.length)) + const { cls } = classifyActivityWindow(extractHarFeaturesFromSmv(w, HAR_FS)) + counts[cls] = (counts[cls] ?? 0) + 1 + } + let best: string | undefined, bestN = -1 + for (const k of Object.keys(counts)) if (counts[k] > bestN) { bestN = counts[k]; best = k } + return best + } catch { + return undefined + } +} + +export interface MinuteSignal { + steps: number + rr: number[] + // Optical aggregates from wrist-on R24 (running sums + count). RELATIVE raw ADCs; + // the close path turns red/IR → SpO₂ index and temp → skin-temp index. + opt_n?: number + red_sum?: number + ir_sum?: number + temp_sum?: number + // PPG RIIV proxy: per-second mean of the R21 green channel, time-ordered. The ONLY + // value estimateResp consumes — so storing it lets resp compute from D1 at the close + // with no raw R2. Present only during live optical sessions (R21 is live-only). + green?: number[] + // Dominant HAR activity class for the minute, from the live high-rate accel (Mannini + // classifier). One tiny label — drives workout typing. Live minutes only. + act_class?: string +} + +interface AccelFrame { idx: number; ts: number; mags: number[] } +interface OpticalAcc { n: number; red: number; ir: number; temp: number } + +// One R21 record's green PPG channel → per-second mean (RIIV proxy). Mirrors +// resp.ts decodeR21Green: rec_type 21, ts@7 (u32 LE), channel A @20 (100×u16 LE). +function r21GreenMean(b: Uint8Array): { ts: number; mean: number } | null { + if (b.length < 620 || b[1] !== 21) return null + const view = new DataView(b.buffer, b.byteOffset, b.byteLength) + const ts = view.getUint32(7, true) + if (!(ts > 0)) return null + let sum = 0, n = 0 + for (let i = 0; i < 100; i++) { + const o = 20 + 2 * i + if (o + 2 <= b.length) { sum += view.getUint16(o, true); n++ } + } + return n >= 50 ? { ts, mean: sum / n } : null +} + +/** Build per-minute {steps, rr, optical} from a batch of hex records. Pure; no I/O. */ +export function perMinuteSignals(records: string[]): Map { + const accelByMin = new Map>() + const rrByMin = new Map() + const optByMin = new Map() + const greenByMin = new Map() + + for (const hex of records) { + let b: Uint8Array + try { b = hexToBytes(hex) } catch { continue } + if (b.length < 2) continue + + // R21 PPG (rec_type 21): per-second mean green → RIIV proxy for resp (D1, no R2). + const g = r21GreenMean(b) + if (g) { + const m = Math.floor(g.ts / 60) * 60 + const arr = greenByMin.get(m) ?? [] + arr.push({ ts: g.ts, v: g.mean }) + greenByMin.set(m, arr) + continue + } + + // R24 (recType @ b[1] === 24): RR intervals via the protocol decoder. + if (b[1] === 24 && b.length >= 89) { + const r = parse_r24(b) + if (r && r.ts_epoch > 0) { + const m = Math.floor(r.ts_epoch / 60) * 60 + const arr = rrByMin.get(m) ?? [] + for (const v of r.rr_intervals_ms) arr.push(v) // raw; gated by analytics cleanRr below + rrByMin.set(m, arr) + // Optical: wrist-on only (hr>0 — off-wrist optical is meaningless). Sum red/IR/temp + // raw ADCs + count → per-minute means at the close, baseline-relative index there. + if (r.hr > 0 && r.spo2_ir_raw > 0) { + const o = optByMin.get(m) ?? { n: 0, red: 0, ir: 0, temp: 0 } + o.n += 1; o.red += r.spo2_red_raw; o.ir += r.spo2_ir_raw; o.temp += r.skin_temp_raw + optByMin.set(m, o) + } + } + continue + } + + // LIVE RR (un-banned: unit confirmed ms, cross-validated vs noop/Strand). 0x28 + // compact HR + R10 carry beat-to-beat intervals; gated by analytics cleanRr below, + // so an unvalidated 0x28 offset can only drop values, never store a bogus interval. + const rr = realtimeRr(hex) + if (rr) { + const m = Math.floor(rr.ts / 60) * 60 + const arr = rrByMin.get(m) ?? [] + for (const v of rr.rr_ms) arr.push(v) // raw; cleaned below + rrByMin.set(m, arr) + } + + // Accel-bearing frames (0x33 IMU stream, R10) → magnitude samples for the pedometer. + const f = frameAccel(hex) + if (f && f.ts > 0) { + const m = Math.floor(f.ts / 60) * 60 + let mm = accelByMin.get(m) + if (!mm) { mm = new Map(); accelByMin.set(m, mm) } + mm.set(`${f.ts}:${f.idx}`, { ts: f.ts, idx: f.idx, mags: f.mags }) + } + } + + const out = new Map() + const minutes = new Set([...accelByMin.keys(), ...rrByMin.keys(), ...optByMin.keys(), ...greenByMin.keys()]) + for (const m of minutes) { + let steps = 0 + let act_class: string | undefined + const frames = accelByMin.get(m) + if (frames && frames.size > 0) { + const ordered = [...frames.values()].sort((a, b) => a.ts - b.ts || a.idx - b.idx) + const sig: number[] = [] + for (const fr of ordered) for (const v of fr.mags) sig.push(v) + steps = calcSteps([sig]) + // Classify the minute's motion at ingest (the ONLY place the high-rate accel + // exists) → store just the dominant class label, never the raw samples. + act_class = classifyMinute(sig) + } + // PPG green: time-ordered per-second means for the minute (RIIV series → resp at close). + const gArr = greenByMin.get(m) + const green = gArr && gArr.length + ? gArr.sort((a, b) => a.ts - b.ts).map((x) => x.v) + : undefined + // Single library gate (300–2000 ms + ectopic |Δ|>200ms drop) — logic lives in + // analytics, not duplicated here. + const o = optByMin.get(m) + out.set(m, { + steps, rr: cleanRr(rrByMin.get(m) ?? []), + ...(o ? { opt_n: o.n, red_sum: o.red, ir_sum: o.ir, temp_sum: o.temp } : {}), + ...(green ? { green } : {}), + ...(act_class ? { act_class } : {}), + }) + } + return out +} diff --git a/src/minute_store.ts b/src/minute_store.ts new file mode 100644 index 0000000..2402c41 --- /dev/null +++ b/src/minute_store.ts @@ -0,0 +1,218 @@ +// minute_store.ts — [feat/wake-trigger] THE single minute-storage layer. +// +// Storage model: ONE row per (user, ymd) in `minute_day`, value = a gzipped JSON +// array of MinuteRec (one entry per touched minute, keyed by ts_min). This replaces +// the old row-per-minute `minute` table — so ingest writes ~1 row/day instead of +// ~1,440, and the 10-day prune deletes ~1 row/day instead of ~1,440 (the big D1 +// write-cost cut). Reads unpack the blob to the same MinuteRec[] every caller used. +// +// Tiering: hot days live in D1 `minute_day`; days older than HOT_DAYS are sealed — +// the SAME gzipped blob is moved to a gzipped R2 object and the D1 row dropped. +// readDay/readMinutes fall back to R2 for sealed days. +// +// D1 limits respected: blob is gzipped (~150–250 KB worst-case day, << the 2 MB +// value cap) and ALWAYS written via a bound parameter (the 100 KB SQL-statement +// limit is for SQL text, not bound-param values). +// +// Concurrency: writeBatch is read-merge-write, NOT atomic. Safe because a user has +// ONE band syncing serially (the edge uploader awaits each batch). Two concurrent +// writers for the same (user, ymd) could lose an update — not a real scenario here. + +import type { MinuteBucket } from './rollup' + +export const HOT_DAYS = 3 +const DAY = 86400 +const ymd = (ts: number): string => new Date(ts * 1000).toISOString().slice(0, 10) +const dayStart = (d: string): number => Math.floor(Date.parse(`${d}T00:00:00Z`) / 1000) +const objKey = (userId: string, date: string): string => `minute/${userId}/${date}.json.gz` + +/** One minute's stored aggregate (running sums kept so merges stay exact). */ +export interface MinuteRec { + ts_min: number + hr_avg: number; hr_min: number; hr_max: number; hr_n: number; hr_sum: number + activity: number; act_sum: number; act_n: number + steps: number; wrist_on: number + rr: number[] + // Optical aggregates (wrist-on R24 only) — running sums + count so merges stay + // exact and re-uploads can't double-count (edge dedupes by hex). RELATIVE raw ADCs; + // SpO₂/°C are derived in the close path, never on-band. Optional (older blobs lack them). + opt_n?: number; red_sum?: number; ir_sum?: number; temp_sum?: number + // PPG RIIV proxy: per-second mean of the R21 green channel (the only value + // estimateResp consumes). Present only during live optical sessions (R21 is + // live-stream-only), so usually empty. Replaces the R2 raw store for resp — + // resp is computed from this series at the wake-close. Optional (older blobs lack it). + green?: number[] + // Dominant HAR activity class for this minute, classified at ingest from the live + // high-rate accel (Mannini, see openstrap-analytics/har). One tiny enum string — NOT + // the raw samples. Present only for live-streamed minutes (flash R24 is 1 Hz → none). + // Feeds workout typing + segmentation in detectSessions. Optional (older blobs lack it). + act_class?: string +} + +export interface StoreEnv { DB: D1Database; RAW_BUCKET?: R2Bucket } + +// ── gzip via Workers CompressionStream ──────────────────────────────────────── +async function gzip(s: string): Promise { + const cs = new CompressionStream('gzip') + const w = cs.writable.getWriter(); void w.write(new TextEncoder().encode(s)); void w.close() + return new Uint8Array(await new Response(cs.readable).arrayBuffer()) +} +async function gunzip(b: ArrayBuffer): Promise { + const ds = new DecompressionStream('gzip') + const w = ds.writable.getWriter(); void w.write(new Uint8Array(b)); void w.close() + return new TextDecoder().decode(await new Response(ds.readable).arrayBuffer()) +} + +function zeroRec(ts_min: number): MinuteRec { + return { ts_min, hr_avg: 0, hr_min: 0, hr_max: 0, hr_n: 0, hr_sum: 0, activity: 0, act_sum: 0, act_n: 0, steps: 0, wrist_on: 0, rr: [] } +} + +// pack/unpack — pure, lossless JSON. +export function packDay(recs: MinuteRec[]): MinuteRec[] { return recs } +function serialize(map: Map): string { + return JSON.stringify([...map.values()].sort((a, b) => a.ts_min - b.ts_min)) +} +function deserialize(json: string): Map { + const out = new Map() + try { const arr = JSON.parse(json); if (Array.isArray(arr)) for (const r of arr) if (r && typeof r.ts_min === 'number') out.set(r.ts_min, r as MinuteRec) } catch { /* corrupt → empty */ } + return out +} + +/** Read ONE day's minutes as a map: D1 minute_day (hot) → else R2 sealed → empty. */ +export async function readDay(env: StoreEnv, userId: string, date: string): Promise> { + const row = await env.DB.prepare('SELECT blob FROM minute_day WHERE user_id = ? AND ymd = ?') + .bind(userId, date).first<{ blob: ArrayBuffer | null }>() + if (row?.blob) return deserialize(await gunzip(row.blob)) + if (env.RAW_BUCKET) { + const obj = await env.RAW_BUCKET.get(objKey(userId, date)) + if (obj) { try { return deserialize(await gunzip(await obj.arrayBuffer())) } catch { /* fall through */ } } + } + return new Map() +} + +/** Write ONE day's map back to D1 minute_day (gzip + bound param). */ +async function writeDay(env: StoreEnv, userId: string, date: string, map: Map, now: number): Promise { + if (map.size === 0) return + const gz = await gzip(serialize(map)) + await env.DB.prepare( + 'INSERT INTO minute_day (user_id, ymd, blob, updated_at) VALUES (?,?,?,?) ' + + 'ON CONFLICT(user_id, ymd) DO UPDATE SET blob = excluded.blob, updated_at = excluded.updated_at', + ).bind(userId, date, gz, now).run() +} + +/** Read minutes over [from, to): unpack each day in range (D1 hot + R2 sealed). */ +export async function readMinutes(env: StoreEnv, userId: string, from: number, to: number): Promise { + const out: MinuteRec[] = [] + for (let t = Math.floor(from / DAY) * DAY; t < to; t += DAY) { + const day = await readDay(env, userId, ymd(t)) + for (const r of day.values()) if (r.ts_min >= from && r.ts_min < to) out.push(r) + } + out.sort((a, b) => a.ts_min - b.ts_min) + return out +} + +/** Latest minute at/after `sinceTs` (for "current HR"): scans today's + sinceTs's day. */ +export async function latestMinute(env: StoreEnv, userId: string, sinceTs: number, now: number): Promise { + let best: MinuteRec | null = null + for (let t = Math.floor(sinceTs / DAY) * DAY; t <= now; t += DAY) { + for (const r of (await readDay(env, userId, ymd(t))).values()) { + if (r.ts_min >= sinceTs && (!best || r.ts_min > best.ts_min)) best = r + } + } + return best +} + +/** + * Fold a batch's per-minute buckets (+ steps/rr signals) into the day blob(s), + * RMW per touched day. Merge mirrors the old `minute` ON CONFLICT EXACTLY: + * hr/act sums + counts add, hr_min CASE-guarded for 0, hr_max MAX, steps add, + * rr keeps the longer (fuller) array, wrist_on MAX; hr_avg/activity recomputed. + * Idempotent for re-uploaded identical batches IFF the edge dedupes by hex (it does). + */ +export async function writeBatch( + env: StoreEnv, userId: string, + buckets: MinuteBucket[], + signals: Map, + now: number, +): Promise { + if (buckets.length === 0) return 0 + // group buckets by ymd + const byDay = new Map() + for (const b of buckets) { const d = ymd(b.ts_min); (byDay.get(d) ?? byDay.set(d, []).get(d)!).push(b) } + let daysWritten = 0 + for (const [date, dayBuckets] of byDay) { + const map = await readDay(env, userId, date) // hot (D1) or, rare re-drain, sealed (R2) + for (const b of dayBuckets) { + const rec = map.get(b.ts_min) ?? zeroRec(b.ts_min) + rec.hr_sum += b.hr_sum + rec.hr_n += b.hr_n + rec.hr_min = rec.hr_min === 0 ? b.hr_min : (b.hr_min === 0 ? rec.hr_min : Math.min(rec.hr_min, b.hr_min)) + rec.hr_max = Math.max(rec.hr_max, b.hr_max) + rec.act_sum += b.act_sum + rec.act_n += b.act_n + rec.wrist_on = Math.max(rec.wrist_on, b.wrist_on) + // Steps: accumulate the AN-2554 per-minute count from the ingest signals — the + // SINGLE step source now (the per-record r10Motion heuristic and the R2 steps_imu + // recompute are both gone). Additive across batches; idempotent because the edge + // dedupes by raw hex, so a minute's frames are never counted twice. + const sig = signals.get(b.ts_min) + rec.steps += sig?.steps ?? 0 + const newRr = sig?.rr ?? [] + if (newRr.length >= rec.rr.length) rec.rr = newRr + // PPG green RIIV proxy: same "keep the fuller array" idempotency as rr. + const newGreen = sig?.green ?? [] + if (newGreen.length >= (rec.green?.length ?? 0)) rec.green = newGreen + // HAR activity class (live minutes only): keep when present. Recompute on re-upload + // is deterministic, so a re-drained batch yields the same label. + if (sig?.act_class) rec.act_class = sig.act_class + // Optical: additive sums + count (same idempotency basis as steps — edge hex-dedup). + if (sig?.opt_n) { + rec.opt_n = (rec.opt_n ?? 0) + sig.opt_n + rec.red_sum = (rec.red_sum ?? 0) + (sig.red_sum ?? 0) + rec.ir_sum = (rec.ir_sum ?? 0) + (sig.ir_sum ?? 0) + rec.temp_sum = (rec.temp_sum ?? 0) + (sig.temp_sum ?? 0) + } + rec.hr_avg = rec.hr_n > 0 ? Math.round(rec.hr_sum / rec.hr_n) : 0 + rec.activity = rec.act_n > 0 ? rec.act_sum / rec.act_n : 0 + map.set(b.ts_min, rec) + } + await writeDay(env, userId, date, map, now) + daysWritten++ + } + return daysWritten +} + +/** Seal days older than HOT_DAYS: move the D1 day blob → gzipped R2 object, drop the + * D1 row (the blob is already in the seal format). Bounded per call. */ +export async function sealOldDays(env: StoreEnv, now = Math.floor(Date.now() / 1000), limit = 1000): Promise<{ sealed: number }> { + if (!env.RAW_BUCKET) return { sealed: 0 } + const cutoff = ymd(now - HOT_DAYS * DAY) + const { results } = await env.DB.prepare( + 'SELECT user_id, ymd, blob FROM minute_day WHERE ymd < ? LIMIT ?', + ).bind(cutoff, limit).all<{ user_id: string; ymd: string; blob: ArrayBuffer }>() + let sealed = 0 + for (const r of results ?? []) { + try { + await env.RAW_BUCKET.put(objKey(r.user_id, r.ymd), r.blob, { httpMetadata: { contentEncoding: 'gzip' } }) + await env.DB.prepare('DELETE FROM minute_day WHERE user_id = ? AND ymd = ?').bind(r.user_id, r.ymd).run() + sealed++ + } catch (e) { console.error('sealDay failed', r.user_id, r.ymd, e) } + } + return { sealed } +} + +/** Seed/admin: write MinuteRec[] straight into the day-packed store (groups by ymd). */ +export async function putDay(env: StoreEnv, userId: string, recs: MinuteRec[], now: number): Promise { + const byDay = new Map>() + for (const r of recs) { + const d = ymd(r.ts_min) + let m = byDay.get(d); if (!m) { m = new Map(); byDay.set(d, m) } + m.set(r.ts_min, r) + } + for (const [d, m] of byDay) await writeDay(env, userId, d, m, now) +} + +/** Backstop retention delete: drop any minute_day rows older than `days` still in D1. */ +export async function pruneMinuteDays(env: StoreEnv, now: number, days: number): Promise { + await env.DB.prepare('DELETE FROM minute_day WHERE ymd < ?').bind(ymd(now - days * DAY)).run() +} diff --git a/src/query.ts b/src/query.ts index a989583..d30db67 100644 --- a/src/query.ts +++ b/src/query.ts @@ -4,8 +4,10 @@ import type { Context } from 'hono' import { calcFitnessTrend, type DayHistory } from 'openstrap-analytics' +import { readMinutes, latestMinute } from './minute_store' +import { ensureTodayWorkouts } from './workouts' -type Ctx = Context<{ Bindings: { DB: D1Database }; Variables: { userId: string } }> +type Ctx = Context<{ Bindings: { DB: D1Database; RAW_BUCKET?: R2Bucket }; Variables: { userId: string } }> const nowSec = () => Math.floor(Date.now() / 1000) const todayDate = () => new Date().toISOString().slice(0, 10) @@ -38,11 +40,20 @@ export async function getToday(c: Ctx) { .bind(userId).first() const baseline = await c.env.DB.prepare('SELECT * FROM baselines WHERE user_id = ?') .bind(userId).first() + // User's daily step goal (null → client default). Carried at top level so the + // Today step ring can render progress without a second call. + const u = await c.env.DB.prepare('SELECT step_goal FROM users WHERE id = ?') + .bind(userId).first() + + // Live status: most recent minute (HR + worn) in the last 10 minutes (day-packed store). + const live = await latestMinute(c.env, userId, nowSec() - 600, nowSec()) - // Live status: most recent minute (HR + worn) in the last 10 minutes. - const live = await c.env.DB.prepare( - 'SELECT ts_min, hr_avg, wrist_on FROM minute WHERE user_id = ? AND ts_min >= ? ORDER BY ts_min DESC LIMIT 1', - ).bind(userId, nowSec() - 600).first() + // Steps (AN-2554): live total = SUM of today's per-minute counts (counted at ingest + // into minute.steps). close_day persists the same SUM to daily.steps for past days; + // for TODAY we sum live so the ring updates during the day. Zero R2 (hot day blob). + const dayStart = Math.floor(Date.parse(`${date}T00:00:00Z`) / 1000) + const todayMins = await readMinutes(c.env, userId, dayStart, dayStart + 86400) + const liveSteps = todayMins.reduce((a, m) => a + (m.steps ?? 0), 0) const df = parseFlags(daily?.flags) const sf = parseFlags(sleep?.flags) @@ -60,6 +71,8 @@ export async function getToday(c: Ctx) { return c.json({ date, + // User's daily step goal (null → client falls back to its own default). + step_goal: u?.step_goal ?? null, // Deterministic coach plan + strain target + readiness contributors + summary. coach: daily?.coach ? safeParse(daily.coach) : null, // HRV stress (Baevsky SI + LF/HF, personal-relative) — includes its drivers. @@ -114,7 +127,7 @@ export async function getToday(c: Ctx) { fitness: { value: daily.fitness ?? null, unit: '', confidence: daily.fitness != null ? 0.6 : 0, tier: 'ESTIMATE', label: 'Fitness' }, form: { value: daily.form ?? null, unit: '', confidence: daily.form != null ? 0.6 : 0, tier: 'ESTIMATE', label: 'Form' }, calories: metric(daily.calories, 'kcal', 'Active calories (est.)', df, 'calories'), - steps: metric(daily.steps, 'steps', 'Steps (est.)', df, 'steps'), + steps: metric(liveSteps, 'steps', 'Steps (est.)', df, 'steps'), // Wear is a direct count of worn minutes — full confidence when present // (otherwise the UI hides any metric with null/0 confidence). wear_min: { ...metric(daily.wear_min, 'min', 'Worn', df, 'wear'), confidence: daily.wear_min != null ? 1 : 0, tier: 'AUTH' }, @@ -191,10 +204,55 @@ export async function getSleep(c: Ctx) { return c.json(rows) } +// GET /sleep/v2?from&to — multi-period sleep (naps = shorter sleeps), grouped by +// date. Additive companion to GET /sleep (which stays the single-period shape). +export async function getSleepV2(c: Ctx) { + const userId = c.get('userId') + const from = c.req.query('from'), to = c.req.query('to') + let sql = 'SELECT * FROM sleep_periods WHERE user_id = ?' + const binds: any[] = [userId] + if (from) { sql += ' AND date >= ?'; binds.push(from) } + if (to) { sql += ' AND date <= ?'; binds.push(to) } + sql += ' ORDER BY date DESC, onset_ts ASC LIMIT 400' + const { results } = await c.env.DB.prepare(sql).bind(...binds).all() + const baseline = await c.env.DB.prepare('SELECT sleep_need_min FROM baselines WHERE user_id = ?') + .bind(userId).first() + const needMin = (baseline?.sleep_need_min && baseline.sleep_need_min >= 180) ? baseline.sleep_need_min : 480 + const byDate = new Map() + for (const r of results ?? []) { + const p = sleepPeriodRow(r) + const arr = byDate.get(r.date); if (arr) arr.push(p); else byDate.set(r.date, [p]) + } + const days = [...byDate.entries()].map(([date, periods]) => ({ + date, + periods, + period_count: periods.length, + total_asleep_min: periods.reduce((a: number, p: any) => a + (p.duration_min || 0), 0), + })) + return c.json({ days, need_min: needMin, stages_beta: true }) +} + +// Shape one sleep_periods row into the API period object. +function sleepPeriodRow(r: any) { + return { + id: r.id, + onset_ts: r.onset_ts, + wake_ts: r.wake_ts, + duration_min: r.duration_min, + in_bed_min: r.in_bed_min, + efficiency: r.efficiency, + stages: (r.light_min != null || r.deep_min != null || r.rem_min != null) + ? { light_min: r.light_min, deep_min: r.deep_min, rem_min: r.rem_min } : null, + is_main: !!r.is_main, + confidence: r.confidence, + } +} + export const getStrain = rangeRows('daily') // GET /sessions?from&to — auto-detected workouts (by start_ts unix range). export async function getSessions(c: Ctx) { + await ensureTodayWorkouts(c.env.DB, c.get('userId')) // on-read auto-detect + stale close (throttled) const from = parseInt(c.req.query('from') || '0') const to = parseInt(c.req.query('to') || String(nowSec())) const { results } = await c.env.DB.prepare( @@ -304,10 +362,8 @@ export async function getChart(c: Ctx) { ? 'steps' : 'hr_avg' - const { results } = await c.env.DB.prepare( - `SELECT ts_min, ${col} AS v, wrist_on FROM minute WHERE user_id = ? AND ts_min >= ? AND ts_min <= ? ORDER BY ts_min ASC`, - ).bind(userId, from, to).all<{ ts_min: number; v: number | null; wrist_on: number }>() - const rows = results ?? [] + const recs = await readMinutes(c.env, userId, from, to) + const rows = recs.map((r) => ({ ts_min: r.ts_min, v: (r as any)[col] as number | null, wrist_on: r.wrist_on })) const MAX = 500 let points: { ts: number; v: number | null }[] diff --git a/src/queue.ts b/src/queue.ts index ffb6ced..8812562 100644 --- a/src/queue.ts +++ b/src/queue.ts @@ -1,38 +1,97 @@ -// queue.ts — analytics queue consumer. Batched (≤20), dedup user_ids, per-user -// isolation (one user's failure never blocks others; retried → DLQ). +// queue.ts — analytics queue consumer. Each message is one (user, job) UNIT of +// work, so every consumer invocation does a single bounded job for a single user +// and stays well under the per-invocation subrequest cap (1000 on Paid). This is +// what lets the heavy R2-re-decode jobs (biometrics/resp/steps) scale: the cron +// just enqueues; the consumer fans them out, one bounded unit per invocation. import { processUser } from './analytics' +import { runBiometricsMinute } from './biometrics_minute' +import { invalidateDay } from './cache' + +// [drop-r2] All HRV/resp/optical now derive from the D1 minute store at the wake-close +// (runBiometricsMinute). The old R2 re-decode jobs ('biometrics'/'resp') and their +// fallback trigger are gone. Steps are AN-2554, counted at ingest into the minute blob. +export type AnalyticsJob = 'sweep' | 'close_day' export interface AnalyticsMessage { user_id: string upto?: number + job?: AnalyticsJob // default 'sweep' + day?: string // YYYY-MM-DD — per-(user,day) fan-out for heavy R2 jobs + onset_ts?: number // [wake-trigger] close_day: sleep onset (RR window start) + wake_ts?: number // [wake-trigger] close_day: wake (RR window end) } interface QueueEnv { DB: D1Database + RAW_BUCKET: R2Bucket + ANALYTICS_Q?: Queue +} + +// [free-tier] Close one physiological day for one user. Exported so the cron can run +// this INLINE when no queue is bound (Cloudflare Queues require Workers Paid). All work +// is D1 + (optional) R2 reads — well under the free plan's 1000 Cloudflare-subrequest +// cap for a single user. Mirrors the 'close_day' queue branch exactly. +export async function closeDay(env: QueueEnv, userId: string, day?: string, onset_ts?: number, wake_ts?: number): Promise { + // [wake-trigger] fired ONCE per physiological day when the user wakes. Derives + // the day (sleep/naps/strain/sessions/baselines/coach via processUser) and folds + // in HRV/recovery from D1 minute.rr — zero R2. Then invalidates the day's Tier-2 + // cache and clears the dirty flag (daytime ingests re-set it; the cron skips + // awake-and-closed users by last_close_date, so no churn). + // processUser derives the day AND folds daily.steps = SUM(minute.steps) (AN-2554 + // counted at ingest) — no separate step job. + await processUser(env.DB, userId, { historyDays: 3 }) + if (day && wake_ts) { + const from = onset_ts ?? (wake_ts - 8 * 3600) + try { await runBiometricsMinute({ DB: env.DB, RAW_BUCKET: env.RAW_BUCKET }, userId, day, from, wake_ts + 60) } catch (e) { console.error('biometrics_minute failed', userId, day, e) } + await invalidateDay(env.DB, userId, day) + } + await env.DB.prepare('UPDATE analytics_cursor SET dirty = 0 WHERE user_id = ?').bind(userId).run() +} + +// Run one bounded unit of work. Each branch is sized to fit in a single +// invocation's budget (one user, one job, a few days of R2 at most). +async function runJob(env: QueueEnv, userId: string, job: AnalyticsJob, day?: string, onset_ts?: number, wake_ts?: number): Promise { + switch (job) { + case 'close_day': + await closeDay(env, userId, day, onset_ts, wake_ts) + break + case 'sweep': + default: + // The frequent path: derive daily/sleep/strain (D1); steps folded in by processUser. + // HRV/resp/optical are derived at the wake-close (close_day → runBiometricsMinute), + // not here — no R2 re-decode jobs anymore. + await processUser(env.DB, userId, { historyDays: 3 }) + break + } } export async function handleQueueBatch( batch: MessageBatch, env: QueueEnv, ): Promise { - // Dedup user_ids within the batch (multiple ingests for one user → one run). - const byUser = new Map[]>() + // Dedup identical (user, job) units within the batch. + const groups = new Map[]>() for (const msg of batch.messages) { const uid = msg.body.user_id if (!uid) { msg.ack(); continue } - const arr = byUser.get(uid) ?? [] + const key = `${uid}::${msg.body.job ?? 'sweep'}::${msg.body.day ?? ''}` + const arr = groups.get(key) ?? [] arr.push(msg) - byUser.set(uid, arr) + groups.set(key, arr) } - for (const [userId, msgs] of byUser) { + for (const [, msgs] of groups) { + const uid = msgs[0].body.user_id + const job = msgs[0].body.job ?? 'sweep' + const day = msgs[0].body.day + const { onset_ts, wake_ts } = msgs[0].body try { - await processUser(env.DB, userId) + await runJob(env, uid, job, day, onset_ts, wake_ts) for (const m of msgs) m.ack() } catch (e) { - console.error('queue: processUser failed for', userId, e) - // Retry the whole group (Cloudflare re-delivers; max_retries → DLQ). + console.error('queue: job failed', uid, job, day, e) + // Retry the group (Cloudflare re-delivers; max_retries → DLQ). for (const m of msgs) m.retry() } } diff --git a/src/resp.ts b/src/resp.ts index 2cc2a3f..bbe7dcf 100644 --- a/src/resp.ts +++ b/src/resp.ts @@ -1,51 +1,24 @@ -// resp.ts — respiratory rate from the optical PPG (R21), re-decoded from the raw -// frames in R2. THE ONLY honest source of breaths/min on this hardware: the green -// PPG channel's slow amplitude/baseline modulation (RIIV) tracks breathing. +// resp.ts — respiratory rate from the optical PPG (R21) green channel's slow +// amplitude/baseline modulation (RIIV), which tracks breathing. THE only honest +// optical source of breaths/min on this hardware. // -// IMPORTANT REALITY: R21 (6-channel optical PPG) is only emitted during the LIVE -// realtime stream (SEND_R10_R11_REALTIME). Overnight, the band flashes R24 (1 Hz -// HR, NO PPG arrays), so on normal nights there is NO R21 in R2 and resp stays -// null. This pipeline computes a number ONLY when there is enough contiguous R21 -// data AND the respiratory periodicity is strong — otherwise confidence 0 and it -// is never surfaced (the /today + /day/sleep gate requires resp_conf ≥ 0.5). -// This mirrors the HRV stance: never fabricate. BETA until validated on real PPG. - -const DAY = 86400 - -interface RespEnv { DB: D1Database; RAW_BUCKET: R2Bucket } - -const hexToBytes = (hex: string): Uint8Array => { - const clean = hex.trim() - const m = clean.match(/.{1,2}/g) - if (!m) return new Uint8Array(0) - return new Uint8Array(m.map((b) => parseInt(b, 16))) -} - -// Decode one R21 record's green PPG channel (channel A @20, 100×u16 LE). Returns -// the record's epoch ts + the green samples, or null if it isn't a usable R21. -// Layout from the reference client — ts@7 (u32), channel A @20 (100 u16). -function decodeR21Green(hex: string): { ts: number; green: number[] } | null { - const b = hexToBytes(hex) - if (b.length < 620) return null - if (b[1] !== 21) return null // rec_type 21 - const view = new DataView(b.buffer, b.byteOffset, b.byteLength) - const ts = view.getUint32(7, true) - if (!(ts > 0)) return null - const green: number[] = [] - for (let i = 0; i < 100; i++) { - const o = 20 + 2 * i - if (o + 2 <= b.length) green.push(view.getUint16(o, true)) - } - return green.length >= 50 ? { ts, green } : null -} +// [drop-r2] The PPG waveform is no longer archived to R2. Instead, ingest stores the +// per-second mean-green RIIV proxy in the D1 minute blob (MinuteRec.green) — the ONLY +// value this algorithm consumes — and resp is computed from D1 at the wake-close via +// respFromMinuteGreen(). estimateResp (the validated autocorrelation core) is unchanged. +// +// REALITY: R21 is emitted ONLY during the live realtime stream; overnight the band +// flashes R24 (1 Hz HR, no PPG), so on normal nights there is no green series and resp +// stays null (RSA-from-RR resp, computed in biometrics_minute, covers those nights). +// Computes a number ONLY with enough contiguous signal AND strong periodicity; else +// confidence 0 and it's never surfaced (display gate requires resp_conf ≥ 0.5). const mean = (a: number[]) => a.reduce((s, v) => s + v, 0) / (a.length || 1) /** - * Estimate respiratory rate (breaths/min) from a window of R21 records via - * autocorrelation of the per-record green level (RIIV proxy). Returns - * {resp_rate, confidence}; confidence 0 when the data is insufficient or the - * periodicity is weak (→ never surfaced). Conservative by design. + * Estimate respiratory rate (breaths/min) from a window of per-record green levels + * via autocorrelation of the RIIV proxy. Returns {resp_rate, confidence}; confidence 0 + * when data is insufficient or periodicity is weak (→ never surfaced). Conservative. */ export function estimateResp(records: { ts: number; green: number[] }[]): { resp_rate: number | null; confidence: number } { if (records.length < 120) return { resp_rate: null, confidence: 0 } @@ -95,68 +68,19 @@ export function estimateResp(records: { ts: number; green: number[] }[]): { resp return { resp_rate: resp, confidence } } -// List R2 raw objects whose key time-range overlaps [from,to]. Keys look like -// raw///--.txt (see ingest.ts). -async function rawKeysInWindow(bucket: R2Bucket, userId: string, from: number, to: number): Promise { - const out: string[] = [] - let cursor: string | undefined - do { - const listing = await bucket.list({ prefix: `raw/${userId}/`, cursor, limit: 1000 }) - for (const o of listing.objects) { - const m = o.key.match(/-(\d+)-(\d+)\.txt$/) - if (!m) { out.push(o.key); continue } // unparseable → include (cheap safety) - const minTs = parseInt(m[1]), maxTs = parseInt(m[2]) - if (maxTs >= from && minTs <= to) out.push(o.key) - } - cursor = listing.truncated ? listing.cursor : undefined - } while (cursor) - return out -} - -/** Compute + store resp rate for one night. Idempotent. */ -async function computeNight(env: RespEnv, userId: string, date: string, from: number, to: number): Promise<{ resp_rate: number | null; confidence: number; r21: number }> { - const keys = await rawKeysInWindow(env.RAW_BUCKET, userId, from, to) - const records: { ts: number; green: number[] }[] = [] - for (const key of keys) { - const obj = await env.RAW_BUCKET.get(key) - if (!obj) continue - const text = await obj.text() - for (const line of text.split('\n')) { - const r = decodeR21Green(line) - if (r && r.ts >= from && r.ts <= to) records.push(r) - } - } - const { resp_rate, confidence } = estimateResp(records) - // Always write (null + 0 conf when absent) so the gate is authoritative. - await env.DB.prepare( - 'UPDATE daily SET resp_rate = ?, resp_conf = ? WHERE user_id = ? AND date = ?', - ).bind(resp_rate, confidence, userId, date).run() - return { resp_rate, confidence, r21: records.length } -} - /** - * runRespRate — for each of the last `days` nights with a sleep row, re-decode - * R21 from R2 and store resp_rate/resp_conf on daily. Also refreshes the resp - * baseline (median of nights with conf ≥ 0.5). Heavy (R2 reads) → nightly cron / - * admin only, NEVER inline ingest. + * PPG-resp from the D1 minute store. Flattens each minute's stored per-second green + * RIIV proxy into a time-ordered series and runs estimateResp over it. Synthesizes a + * 1 Hz timestamp per sample (ts_min + index) — estimateResp resamples by ts span, so + * the exact sub-minute placement is immaterial. Returns {null, 0} when no usable PPG + * (the normal-night case → resp stays whatever RSA-from-RR produced). */ -export async function runRespRate(env: RespEnv, userId: string, days = 3): Promise<{ nights: number; computed: number }> { - const since = new Date(Date.now() - days * DAY * 1000).toISOString().slice(0, 10) - const { results: nights } = await env.DB.prepare( - 'SELECT date, onset_ts, wake_ts FROM sleep WHERE user_id = ? AND date >= ? AND onset_ts IS NOT NULL AND wake_ts IS NOT NULL ORDER BY date DESC', - ).bind(userId, since).all<{ date: string; onset_ts: number; wake_ts: number }>() - - let computed = 0 - const valid: number[] = [] - for (const n of nights ?? []) { - const r = await computeNight(env, userId, n.date, n.onset_ts, n.wake_ts + 60) - if (r.resp_rate != null && r.confidence >= 0.5) { computed++; valid.push(r.resp_rate) } - } - if (valid.length >= 3) { - valid.sort((a, b) => a - b) - const med = valid[Math.floor(valid.length / 2)] - await env.DB.prepare('UPDATE baselines SET resp_rate = ? WHERE user_id = ?') - .bind(Math.round(med * 10) / 10, userId).run() +export function respFromMinuteGreen(recs: { ts_min: number; green?: number[] }[]): { resp_rate: number | null; confidence: number } { + const records: { ts: number; green: number[] }[] = [] + for (const m of recs) { + if (!m.green || !m.green.length) continue + for (let i = 0; i < m.green.length; i++) records.push({ ts: m.ts_min + i, green: [m.green[i]] }) } - return { nights: (nights ?? []).length, computed } + if (!records.length) return { resp_rate: null, confidence: 0 } + return estimateResp(records) } diff --git a/src/rollup.ts b/src/rollup.ts index 33a17a5..6442a52 100644 --- a/src/rollup.ts +++ b/src/rollup.ts @@ -9,7 +9,7 @@ // ingest). We store enough to recompute averages exactly: the upsert SQL // folds new (sum,n) into the stored running aggregate — see ingest.ts. -import type { DecodedSample } from './decode' +import type { DecodedSample } from 'openstrap-protocol/ts/live' export interface MinuteBucket { ts_min: number @@ -46,7 +46,8 @@ export function rollupMinutes(samples: DecodedSample[]): MinuteBucket[] { } bk.act_sum += s.activity bk.act_n += 1 - bk.steps += s.steps_inc + // steps no longer come from the per-record r10Motion heuristic (s.steps_inc) — + // ingest_signals' AN-2554 (calcSteps) is the sole step source, folded in writeBatch. if (s.wrist_on) bk.wrist_on = 1 } return [...buckets.values()].sort((a, b) => a.ts_min - b.ts_min) diff --git a/src/seed.ts b/src/seed.ts index 5c408c7..d8f3144 100644 --- a/src/seed.ts +++ b/src/seed.ts @@ -11,6 +11,7 @@ // orchestrates the minutes phases in chunks. Each phase is idempotent. import { processUser } from './analytics' +import { putDay } from './minute_store' import { uuid } from './auth' const DAY = 86400 @@ -319,7 +320,7 @@ async function findOrCreateUser(db: D1Database, email: string, now: number): Pro // phase=init — create/clear user. Returns user_id. export async function seedInit(db: D1Database, email: string, now: number): Promise<{ user_id: string }> { const userId = await findOrCreateUser(db, email, now) - for (const t of ['minute', 'daily', 'sleep', 'sessions', 'baselines', 'analytics_cursor']) { + for (const t of ['minute', 'minute_day', 'daily', 'sleep', 'sessions', 'baselines', 'analytics_cursor']) { await db.prepare(`DELETE FROM ${t} WHERE user_id = ?`).bind(userId).run() } return { user_id: userId } @@ -332,23 +333,21 @@ export async function seedMinutes( ): Promise<{ minutes: number }> { const todayStart = Math.floor(now / DAY) * DAY const firstDayStart = todayStart - (days - 1) * DAY - const insert = db.prepare( - 'INSERT OR REPLACE INTO minute (user_id, ts_min, hr_avg, hr_min, hr_max, hr_n, hr_sum, activity, act_sum, act_n, steps, wrist_on) ' + - 'VALUES (?,?,?,?,?,?,?,?,?,?,?,?)', - ) let minutes = 0 for (let d = dayFrom; d < dayTo && d < days; d++) { const dayStart = firstDayStart + d * DAY const rows = buildDay(dayStart, d, days) - const batch: D1PreparedStatement[] = [] - for (const m of rows) { - const hr_sum = m.hr_avg * m.hr_n + // Day-packed store (minute_day). Build MinuteRec[] (running sums for exact merges; rr empty). + const recs = rows.map((m) => { const act_n = 60 - const act_sum = m.activity * act_n - batch.push(insert.bind(userId, m.ts_min, m.hr_avg, m.hr_min, m.hr_max, m.hr_n, hr_sum, - m.activity, act_sum, act_n, m.steps, m.wrist_on)) - } - if (batch.length) { await db.batch(batch); minutes += batch.length } + return { + ts_min: m.ts_min, hr_avg: m.hr_avg, hr_min: m.hr_min, hr_max: m.hr_max, hr_n: m.hr_n, + hr_sum: m.hr_avg * m.hr_n, activity: m.activity, act_sum: m.activity * act_n, act_n, + steps: m.steps, wrist_on: m.wrist_on, rr: [] as number[], + } + }) + await putDay({ DB: db }, userId, recs, now) + minutes += recs.length } return { minutes } } diff --git a/src/spotcheck.ts b/src/spotcheck.ts new file mode 100644 index 0000000..ecccae1 --- /dev/null +++ b/src/spotcheck.ts @@ -0,0 +1,46 @@ +// spotcheck.ts — on-demand live HRV reading. +// +// POST /spotcheck { records: [hex, ...] } → { rmssd, sdnn, pnn50, mean_hr, n_beats, ok } +// +// The edge enables wrist-gated optical + realtime records for ~60s, collects the +// raw live frames, and posts them here. We decode beat-to-beat RR from each frame +// (protocol realtimeRr — the SAME decoder the ingest path uses) and compute HRV +// with the analytics time-domain method (cleanRr gate + RMSSD/SDNN/pNN50). +// +// Heavy lifting stays server-side; the client just collects bytes and renders. +// Stateless: nothing is written to D1/R2 (a spot check is ephemeral by design). + +import type { Context } from 'hono' +import { realtimeRr } from 'openstrap-protocol/ts/live' +import { timeDomainHrv } from 'openstrap-analytics' + +type Ctx = Context<{ Bindings: { DB: D1Database }; Variables: { userId: string } }> + +export async function postSpotCheck(c: Ctx) { + const body = await c.req.json<{ records?: unknown }>().catch(() => ({ records: [] })) + const recs = Array.isArray(body.records) + ? body.records.filter((h): h is string => typeof h === 'string').slice(0, 5000) + : [] + if (recs.length === 0) return c.json({ error: 'records[] required' }, 400) + + // Decode RR (ms) from every live frame that carries it (0x28 / R10). + const rr: number[] = [] + for (const hex of recs) { + try { + const r = realtimeRr(hex) + if (r && Array.isArray(r.rr_ms) && r.rr_ms.length) rr.push(...r.rr_ms) + } catch { /* skip undecodable frame */ } + } + + const hrv = timeDomainHrv(rr) + return c.json({ + n_frames: recs.length, + n_beats: hrv.n_beats, + rmssd: hrv.rmssd, + sdnn: hrv.sdnn, + pnn50: hrv.pnn50, + mean_hr: hrv.mean_hr, + // honest gate: timeDomainHrv needs ≥20 clean beats, else everything is null + ok: hrv.rmssd != null, + }) +} diff --git a/src/steps_imu.ts b/src/steps_imu.ts deleted file mode 100644 index 4e6f3bc..0000000 --- a/src/steps_imu.ts +++ /dev/null @@ -1,87 +0,0 @@ -// steps_imu.ts — the STEPS RUNNER (I/O only). The band has no pedometer field, so -// steps are derived from the wrist accelerometer. This file does the heavy, -// binding-dependent work — list + read the raw IMU frames from R2, dedup, order, -// group per minute — then hands the per-minute magnitude signals to the PURE -// AN-2554 pedometer in openstrap-analytics (calcSteps) and persists the result. -// -// Mirrors the runBiometrics / runRespRate pattern: decode (decode.ts) + math -// (analytics) live elsewhere; this owns only R2 reads and the daily.steps write. -// -// Heavy (R2 reads) → cron / admin only, NEVER inline ingest. Owns daily.steps -// (written AFTER analytics in the cron so it's authoritative). - -import { calcSteps } from 'openstrap-analytics' -import { frameAccel, type ImuFrame } from './decode' - -const DAY = 86400 - -interface StepsEnv { DB: D1Database; RAW_BUCKET: R2Bucket } - -async function rawKeysInWindow(bucket: R2Bucket, userId: string, from: number, to: number): Promise { - const out: string[] = [] - let cursor: string | undefined - do { - const listing = await bucket.list({ prefix: `raw/${userId}/`, cursor, limit: 1000 }) - for (const o of listing.objects) { - const m = o.key.match(/-(\d+)-(\d+)\.txt$/) - if (!m) { out.push(o.key); continue } - if (parseInt(m[2]) >= from && parseInt(m[1]) <= to) out.push(o.key) - } - cursor = listing.truncated ? listing.cursor : undefined - } while (cursor) - return out -} - -// Steps for one UTC day: assemble per-minute accel (ordered by ts,idx), then run -// the pure AN-2554 pedometer (analytics calcSteps) over the per-minute signals. -async function computeDaySteps(env: StepsEnv, userId: string, dayStart: number): Promise { - const from = dayStart, to = dayStart + DAY - const keys = await rawKeysInWindow(env.RAW_BUCKET, userId, from, to) - // DEDUP by (ts, idx): R2 upload windows overlap, so the same frame appears in - // multiple objects — without this, samples (and steps) are double-counted. - const seen = new Map() - for (const key of keys) { - const obj = await env.RAW_BUCKET.get(key) - if (!obj) continue - const text = await obj.text() - for (const line of text.split('\n')) { - if (!line) continue - const f = frameAccel(line) - if (!f || f.ts < from || f.ts >= to) continue - seen.set(`${f.ts}:${f.idx}`, f) - } - } - // Group frames per minute, ordered by (ts, idx), into one contiguous signal each. - const byMin = new Map() - for (const f of seen.values()) { - const m = Math.floor(f.ts / 60) * 60 - const arr = byMin.get(m); if (arr) arr.push(f); else byMin.set(m, [f]) - } - const minuteSignals: number[][] = [] - for (const frames of byMin.values()) { - frames.sort((a, b) => a.ts - b.ts || a.idx - b.idx) - const sig: number[] = [] - for (const f of frames) for (const v of f.mags) sig.push(v) - minuteSignals.push(sig) - } - return calcSteps(minuteSignals) -} - -/** - * runStepsImu — recompute steps for the last `days` UTC days from R2 IMU and store - * on daily.steps. Authoritative source of steps (analytics no longer needs to be). - */ -export async function runStepsImu(env: StepsEnv, userId: string, days = 1): Promise<{ days: number; total: number }> { - const now = Math.floor(Date.now() / 1000) - let grand = 0 - for (let d = 0; d < days; d++) { - const dayStart = Math.floor((now - d * DAY) / DAY) * DAY - const date = new Date(dayStart * 1000).toISOString().slice(0, 10) - const steps = await computeDaySteps(env, userId, dayStart) - await env.DB.prepare('INSERT OR IGNORE INTO daily(user_id, date) VALUES(?,?)').bind(userId, date).run() - await env.DB.prepare('UPDATE daily SET steps = ? WHERE user_id = ? AND date = ?') - .bind(steps, userId, date).run() - grand += steps - } - return { days, total: grand } -} diff --git a/src/strava.ts b/src/strava.ts new file mode 100644 index 0000000..e72c4ff --- /dev/null +++ b/src/strava.ts @@ -0,0 +1,296 @@ +// strava.ts — Strava OAuth + token management (foundation for bidirectional sync). +// +// Connect flow (no client secret ever touches the app): +// app → GET /strava/connect (JWT) → { url } authorize link with a short-lived +// signed `state` carrying the user id +// user authorizes in a browser → Strava redirects to +// GET /strava/callback?code&state → we exchange the code for tokens, store them +// per user, and show a "return to OpenStrap" page. +// +// Access tokens are short-lived (~6h); getStravaAccessToken() transparently +// refreshes via the stored refresh_token and persists the rotation. + +import { signJwt, verifyJwt } from './auth' + +const STRAVA_AUTH = 'https://www.strava.com/oauth/authorize' +const STRAVA_TOKEN = 'https://www.strava.com/oauth/token' +const STRAVA_API = 'https://www.strava.com/api/v3' +// Read all activities + write new ones (both directions). +const SCOPE = 'activity:read_all,activity:write' +const STATE_TTL = 600 // 10 min to complete the browser hop + +export interface StravaEnv { + DB: D1Database + JWT_SECRET: string + STRAVA_CLIENT_ID?: string + STRAVA_CLIENT_SECRET?: string +} + +function configured(env: StravaEnv): boolean { + return !!(env.STRAVA_CLIENT_ID && env.STRAVA_CLIENT_SECRET) +} + +function originOf(c: any): string { + const url = new URL(c.req.url) + return `${url.protocol}//${url.host}` +} + +function htmlPage(message: string, ok: boolean): string { + const accent = ok ? '#34d399' : '#f87171' + return `` + + `` + + `OpenStrap × Strava` + + `` + + `
${ok ? '✅' : '⚠️'}
` + + `

${message}

` + + `

You can close this tab and return to OpenStrap.

` + + `

OpenStrap × Strava

` +} + +// GET /strava/connect (requireJwt) → { url } for the app to open in a browser. +export async function stravaConnect(c: any) { + if (!configured(c.env)) return c.json({ error: 'Strava integration not configured on this backend.' }, 503) + const userId: string = c.get('userId') + const state = await signJwt({ sub: userId, typ: 'strava_state' }, c.env.JWT_SECRET, STATE_TTL) + const redirectUri = `${originOf(c)}/strava/callback` + const url = `${STRAVA_AUTH}?client_id=${encodeURIComponent(c.env.STRAVA_CLIENT_ID)}` + + `&response_type=code&redirect_uri=${encodeURIComponent(redirectUri)}` + + `&approval_prompt=auto&scope=${encodeURIComponent(SCOPE)}&state=${encodeURIComponent(state)}` + return c.json({ url }) +} + +// GET /strava/callback?code&state (public; Strava redirect target) → HTML page. +export async function stravaCallback(c: any) { + if (!configured(c.env)) return c.html(htmlPage('Strava is not configured.', false), 503) + const error = c.req.query('error') + if (error) return c.html(htmlPage('Strava authorization was declined.', false), 400) + const code = c.req.query('code') + const state = c.req.query('state') + if (!code || !state) return c.html(htmlPage('Missing authorization code.', false), 400) + + const payload = await verifyJwt(state, c.env.JWT_SECRET) + if (!payload || (payload as any).typ !== 'strava_state') { + return c.html(htmlPage('This link expired. Start the connection again from the app.', false), 400) + } + const userId = payload.sub + + const form = new URLSearchParams({ + client_id: String(c.env.STRAVA_CLIENT_ID), + client_secret: String(c.env.STRAVA_CLIENT_SECRET), + code, + grant_type: 'authorization_code', + }) + const resp = await fetch(STRAVA_TOKEN, { method: 'POST', body: form }) + if (!resp.ok) return c.html(htmlPage('Strava token exchange failed.', false), 502) + const tok = await resp.json() as { + access_token: string; refresh_token: string; expires_at: number + athlete?: { id?: number } + } + + await c.env.DB.prepare( + 'INSERT INTO strava_tokens (user_id, athlete_id, access_token, refresh_token, expires_at, scope, connected_at) ' + + 'VALUES (?,?,?,?,?,?,?) ON CONFLICT(user_id) DO UPDATE SET ' + + 'athlete_id=excluded.athlete_id, access_token=excluded.access_token, refresh_token=excluded.refresh_token, ' + + 'expires_at=excluded.expires_at, scope=excluded.scope', + ).bind( + userId, tok.athlete?.id ?? null, tok.access_token, tok.refresh_token, tok.expires_at, SCOPE, + Math.floor(Date.now() / 1000), + ).run() + + return c.html(htmlPage('Strava connected!', true)) +} + +// GET /strava/status (requireJwt) → { connected, athlete_id, scope } +export async function stravaStatus(c: any) { + const userId: string = c.get('userId') + const row = (await c.env.DB.prepare( + 'SELECT athlete_id, scope, connected_at FROM strava_tokens WHERE user_id = ?', + ).bind(userId).first()) as { athlete_id: number | null; scope: string | null; connected_at: number | null } | null + return c.json({ + connected: !!row, + athlete_id: row?.athlete_id ?? null, + scope: row?.scope ?? null, + connected_at: row?.connected_at ?? null, + }) +} + +// POST /strava/disconnect (requireJwt) → { ok } +export async function stravaDisconnect(c: any) { + const userId: string = c.get('userId') + await c.env.DB.prepare('DELETE FROM strava_tokens WHERE user_id = ?').bind(userId).run() + return c.json({ ok: true }) +} + +// ── Strava → Whoop : pull recent activities into strava_activities ──────────── +export async function stravaPull( + env: StravaEnv, userId: string, perPage = 30, +): Promise<{ pulled: number }> { + const token = await getStravaAccessToken(env, userId) + if (!token) return { pulled: 0 } + const r = await fetch(`${STRAVA_API}/athlete/activities?per_page=${perPage}`, { + headers: { Authorization: `Bearer ${token}` }, + }) + if (!r.ok) return { pulled: 0 } + const acts = (await r.json()) as Array> + const now = Math.floor(Date.now() / 1000) + let pulled = 0 + for (const a of acts) { + const startTs = typeof a.start_date === 'string' + ? Math.floor(new Date(a.start_date).getTime() / 1000) : null + await env.DB.prepare( + 'INSERT INTO strava_activities (user_id, activity_id, start_ts, elapsed_sec, type, name, distance_m, avg_hr, max_hr, raw, pulled_at) ' + + 'VALUES (?,?,?,?,?,?,?,?,?,?,?) ON CONFLICT(user_id, activity_id) DO UPDATE SET ' + + 'start_ts=excluded.start_ts, elapsed_sec=excluded.elapsed_sec, type=excluded.type, name=excluded.name, ' + + 'distance_m=excluded.distance_m, avg_hr=excluded.avg_hr, max_hr=excluded.max_hr, raw=excluded.raw, pulled_at=excluded.pulled_at', + ).bind( + userId, a.id, startTs, a.elapsed_time ?? null, (a.sport_type ?? a.type) ?? null, a.name ?? null, + a.distance ?? null, a.average_heartrate ?? null, a.max_heartrate ?? null, + JSON.stringify({ + id: a.id, name: a.name, type: a.sport_type ?? a.type, start_date: a.start_date, + moving_time: a.moving_time, elapsed_time: a.elapsed_time, distance: a.distance, + total_elevation_gain: a.total_elevation_gain, average_speed: a.average_speed, + average_heartrate: a.average_heartrate, max_heartrate: a.max_heartrate, + average_watts: a.average_watts, kilojoules: a.kilojoules, calories: a.calories, + }), now, + ).run() + pulled++ + } + return { pulled } +} + +// ── Whoop → Strava : push completed OpenStrap workouts as manual activities ──── +// Dedupe against pulled activities (a ride already on Strava via the bike computer +// must NOT be duplicated) and against strava_pushed (idempotent). HR/strain land in +// the description (manual activities don't accept HR streams without a file upload). +const STRAVA_SPORT: Record = { + run: 'Run', running: 'Run', walk: 'Walk', walking: 'Walk', hike: 'Hike', + ride: 'Ride', cycling: 'Ride', bike: 'Ride', + strength: 'WeightTraining', weights: 'WeightTraining', gym: 'Workout', + swim: 'Swim', yoga: 'Yoga', rowing: 'Rowing', elliptical: 'Elliptical', +} + +export async function stravaPush( + env: StravaEnv, userId: string, +): Promise<{ pushed: number; skipped: number }> { + const token = await getStravaAccessToken(env, userId) + if (!token) return { pushed: 0, skipped: 0 } + + const { results: sessions } = await env.DB.prepare( + "SELECT id, start_ts, end_ts, type, avg_hr, max_hr, strain, calories FROM sessions " + + "WHERE user_id = ? AND status = 'done' AND start_ts IS NOT NULL AND end_ts > start_ts " + + "ORDER BY start_ts DESC LIMIT 20", + ).bind(userId).all>() + + let pushed = 0, skipped = 0 + for (const s of sessions ?? []) { + const sid = String(s.id) + const startTs = Number(s.start_ts), endTs = Number(s.end_ts) + + const already = await env.DB.prepare( + 'SELECT 1 FROM strava_pushed WHERE user_id = ? AND session_key = ?', + ).bind(userId, sid).first() + if (already) { skipped++; continue } + + // Overlap with an existing Strava activity (e.g. a Wahoo-synced ride) → skip. + const overlap = await env.DB.prepare( + 'SELECT 1 FROM strava_activities WHERE user_id = ? AND start_ts BETWEEN ? AND ? LIMIT 1', + ).bind(userId, startTs - 1800, endTs + 1800).first() + if (overlap) { + await env.DB.prepare( + 'INSERT OR IGNORE INTO strava_pushed (user_id, session_key, activity_id, pushed_at) VALUES (?,?,?,?)', + ).bind(userId, sid, null, Math.floor(Date.now() / 1000)).run() + skipped++; continue + } + + const typeKey = String(s.type ?? '').toLowerCase() + const sportType = STRAVA_SPORT[typeKey] ?? 'Workout' + const descParts = [ + s.strain != null ? `Strain ${Number(s.strain).toFixed(1)}` : null, + s.avg_hr != null ? `Avg HR ${s.avg_hr}` : null, + s.max_hr != null ? `Max HR ${s.max_hr}` : null, + s.calories != null ? `${Math.round(Number(s.calories))} kcal` : null, + 'via OpenStrap', + ].filter(Boolean) + + const form = new URLSearchParams({ + name: `OpenStrap ${sportType}`, + sport_type: sportType, + start_date_local: new Date(startTs * 1000).toISOString(), + elapsed_time: String(Math.max(60, endTs - startTs)), + description: descParts.join(' · '), + }) + const r = await fetch(`${STRAVA_API}/activities`, { + method: 'POST', + headers: { Authorization: `Bearer ${token}` }, + body: form, + }) + if (!r.ok) { skipped++; continue } + const created = (await r.json()) as { id?: number } + await env.DB.prepare( + 'INSERT OR IGNORE INTO strava_pushed (user_id, session_key, activity_id, pushed_at) VALUES (?,?,?,?)', + ).bind(userId, sid, created.id ?? null, Math.floor(Date.now() / 1000)).run() + pushed++ + } + return { pushed, skipped } +} + +// Cron backstop: pull + push for every connected user (called from the nightly tick). +export async function syncAllStrava(env: StravaEnv): Promise { + const { results } = await env.DB.prepare( + 'SELECT user_id FROM strava_tokens LIMIT 1000', + ).all<{ user_id: string }>() + for (const u of results ?? []) { + try { + await stravaPull(env, u.user_id) + await stravaPush(env, u.user_id) + } catch (e) { + console.error('strava sync failed', u.user_id, e) + } + } +} + +// GET /strava/sync (requireJwt) → pull (Strava→Whoop) then push (Whoop→Strava). +export async function stravaSync(c: any) { + const userId: string = c.get('userId') + const pull = await stravaPull(c.env, userId) + const push = await stravaPush(c.env, userId) + return c.json({ ok: true, ...pull, ...push }) +} + +// GET /strava/activities (requireJwt) → the pulled activities (for the app overlay). +export async function stravaActivities(c: any) { + const userId: string = c.get('userId') + const { results } = await c.env.DB.prepare( + 'SELECT activity_id, start_ts, elapsed_sec, type, name, distance_m, avg_hr, max_hr ' + + 'FROM strava_activities WHERE user_id = ? ORDER BY start_ts DESC LIMIT 50', + ).bind(userId).all() + return c.json({ activities: results ?? [] }) +} + +// A valid access token for `userId`, refreshing + persisting if expired. Returns +// null if the user has not connected Strava or the refresh fails. +export async function getStravaAccessToken(env: StravaEnv, userId: string): Promise { + if (!configured(env)) return null + const row = await env.DB.prepare( + 'SELECT access_token, refresh_token, expires_at FROM strava_tokens WHERE user_id = ?', + ).bind(userId).first<{ access_token: string; refresh_token: string; expires_at: number }>() + if (!row) return null + + const now = Math.floor(Date.now() / 1000) + if (row.expires_at - 60 > now) return row.access_token // still valid (60s skew) + + const form = new URLSearchParams({ + client_id: String(env.STRAVA_CLIENT_ID), + client_secret: String(env.STRAVA_CLIENT_SECRET), + grant_type: 'refresh_token', + refresh_token: row.refresh_token, + }) + const resp = await fetch(STRAVA_TOKEN, { method: 'POST', body: form }) + if (!resp.ok) return null + const tok = await resp.json() as { access_token: string; refresh_token: string; expires_at: number } + await env.DB.prepare( + 'UPDATE strava_tokens SET access_token=?, refresh_token=?, expires_at=? WHERE user_id=?', + ).bind(tok.access_token, tok.refresh_token, tok.expires_at, userId).run() + return tok.access_token +} diff --git a/src/trend.ts b/src/trend.ts index 161d394..18765af 100644 --- a/src/trend.ts +++ b/src/trend.ts @@ -40,6 +40,7 @@ interface DayRow { nocturnal_dip_pct: number | null; acwr: number | null efficiency: number | null; deep_min: number | null; rem_min: number | null light_min: number | null; regularity: number | null // from sleep + skin_temp_idx: number | null; spo2_idx: number | null // RELATIVE deviations } const REGISTRY: Record = { @@ -68,12 +69,14 @@ const REGISTRY: Record = { deep: { pull: (r) => r.deep_min, unit: 'min', label: 'Deep sleep' }, rem: { pull: (r) => r.rem_min, unit: 'min', label: 'REM sleep' }, light: { pull: (r) => r.light_min, unit: 'min', label: 'Light sleep' }, - regularity: { pull: (r) => r.regularity, unit: '', label: 'Sleep regularity (SRI)' }, + regularity: { pull: (r) => r.regularity, unit: '', label: 'Sleep consistency' }, resp: { pull: (r) => r.resp_rate, unit: 'brpm', label: 'Respiratory rate' }, sleep: { pull: (r) => r.duration_min, unit: 'min', label: 'Sleep', target: (_r, need) => need, goalDir: 'min' }, stress: { pull: (r) => { try { return r.stress ? (JSON.parse(r.stress).score ?? null) : null } catch { return null } }, unit: '', label: 'Stress' }, + skin_temp: { pull: (r) => r.skin_temp_idx, unit: 'Δ', label: 'Skin temp vs baseline' }, + spo2: { pull: (r) => r.spo2_idx, unit: 'Δ', label: 'Blood-oxygen vs baseline' }, } interface Bucket { @@ -104,6 +107,7 @@ export async function getTrend(c: Ctx) { 'SELECT d.date AS date, d.strain, d.recovery, d.resting_hr, d.calories, d.steps, d.wear_min, ' + 'd.hrv_rmssd, d.resp_rate, d.stress, d.readiness, d.vo2max, d.fitness, d.fatigue, d.form, ' + 'd.monotony, d.hrv_sdnn, d.hrv_lfhf, d.hrv_si, d.hrv_cv, d.nocturnal_dip_pct, d.acwr, ' + + 'd.skin_temp_idx, d.spo2_idx, ' + 's.duration_min AS duration_min, s.efficiency AS efficiency, s.deep_min AS deep_min, ' + 's.rem_min AS rem_min, s.light_min AS light_min, s.regularity AS regularity ' + 'FROM daily d LEFT JOIN sleep s ON s.user_id = d.user_id AND s.date = d.date ' + diff --git a/src/wake_cron.ts b/src/wake_cron.ts new file mode 100644 index 0000000..3a97bfa --- /dev/null +++ b/src/wake_cron.ts @@ -0,0 +1,103 @@ +// wake_cron.ts — [feat/wake-trigger] the frequent cron's ONLY job: detect each +// user's real wake and fire close_day once per physiological day. Designed to be +// O(cheap) per tick so it's ~free even at */10: +// +// ladder step 1 awake & already closed today → cursor-only skip (no minute read) +// ladder step 2 else → cheap peek of the last ~30 min +// ladder step 3 peek says awake → ONE full ensemble over 36h, enqueue +// +// The expensive ensemble + 36h read runs at most ONCE per user per day (at the wake); +// every other tick is a tiny cursor/30-min read. No analytics, no prune here. + +import { detectWakeState, peekRecentState } from 'openstrap-analytics' +import { loadMinutes, loadBaseline } from './analytics' +import { loadDayRr } from './dayseries' +import { closeDay, type AnalyticsMessage } from './queue' + +interface WakeEnv { DB: D1Database; RAW_BUCKET?: R2Bucket; ANALYTICS_Q?: Queue } + +const DAY = 86400 +const ymd = (ts: number): string => new Date(ts * 1000).toISOString().slice(0, 10) + +export async function runWakeLadder( + env: WakeEnv, now = Math.floor(Date.now() / 1000), +): Promise<{ checked: number; closed: number }> { + const today = ymd(now) + // Candidates = users with fresh data (dirty). Awake-and-closed ones are filtered + // by step 1 below at ~zero cost; the close_day job clears dirty when it runs. + const { results } = await env.DB.prepare( + 'SELECT user_id, last_close_date FROM analytics_cursor WHERE dirty = 1 LIMIT 1000', + ).all<{ user_id: string; last_close_date: string | null }>() + + let checked = 0, closed = 0 + for (const u of results ?? []) { + // Step 1 — got this user's wake already today → skip (cursor-only, cheapest path). + if (u.last_close_date === today) continue + checked++ + + const baseline = await loadBaseline(env.DB, u.user_id) + + // Step 2 — cheap, liberal peek over the last ~30 min (high recall on purpose). + const recent = await loadMinutes(env.DB, u.user_id, now - 30 * 60, now + 60) + const peek = peekRecentState(recent, baseline) + if (peek !== 'awake') { + // still asleep/unknown → mark phase ONLY on transition (no-op write = 0 rows billed). + await env.DB.prepare( + "UPDATE analytics_cursor SET sleep_phase = 'asleep', phase_since = ? WHERE user_id = ? AND COALESCE(sleep_phase,'') <> 'asleep'", + ).bind(now, u.user_id).run() + continue + } + + // Step 3 — candidate wake confirmed by the cheap peek → run the FULL ensemble ONCE + // over a generous 36h window (so the circadian fit isn't starved). RR enables the + // cardiac arm; missing RR just drops that voter. + const win = await loadMinutes(env.DB, u.user_id, now - 36 * 3600, now + 60) + if (win.length < 60) continue + const rrByMin = await loadDayRr(env, u.user_id, now - 36 * 3600, now + 60) + const ws = detectWakeState({ minutes: win, baseline, rrByMin, now }) + if (!ws.wake_ts || ws.state !== 'awake') continue + + const dayLabel = ymd(ws.wake_ts) + if (dayLabel === u.last_close_date) continue // already closed this physiological day + + // Set last_close_date NOW to dedupe re-enqueue; close_day clears dirty when done. + await env.DB.prepare( + "INSERT INTO analytics_cursor (user_id, last_close_date, sleep_phase, phase_since) VALUES (?,?,'awake',?) " + + "ON CONFLICT(user_id) DO UPDATE SET last_close_date = excluded.last_close_date, sleep_phase = 'awake', phase_since = excluded.phase_since", + ).bind(u.user_id, dayLabel, ws.wake_ts).run() + + if (env.ANALYTICS_Q) { + await env.ANALYTICS_Q.send({ + user_id: u.user_id, job: 'close_day', day: dayLabel, + onset_ts: ws.onset_ts ?? undefined, wake_ts: ws.wake_ts, + }) + } else { + // [free-tier] No queue bound (Workers Free): derive the day INLINE. At most one + // user is closed per tick, so this stays well under the 1000 Cloudflare-subrequest + // cap. Without this the day would be marked closed but never actually computed. + try { await closeDay({ DB: env.DB, RAW_BUCKET: env.RAW_BUCKET as R2Bucket }, u.user_id, dayLabel, ws.onset_ts ?? undefined, ws.wake_ts) } + catch (e) { console.error('inline close_day failed', u.user_id, dayLabel, e) } + } + closed++ + } + return { checked, closed } +} + +// Retry-net for the nightly maintenance tick: users with fresh data whose last close +// is stale (>~28h) — re-attempt a close so a missed/failed wake self-heals. +export async function retryStaleCloses(env: WakeEnv, now = Math.floor(Date.now() / 1000)): Promise { + const cutoff = ymd(now - 1 * DAY) + const { results } = await env.DB.prepare( + "SELECT user_id FROM analytics_cursor WHERE dirty = 1 AND (last_close_date IS NULL OR last_close_date < ?) LIMIT 1000", + ).bind(cutoff).all<{ user_id: string }>() + let n = 0 + for (const u of results ?? []) { + if (env.ANALYTICS_Q) { await env.ANALYTICS_Q.send({ user_id: u.user_id, job: 'sweep' }); n++ } + else { + // [free-tier] No queue: re-derive inline as the nightly retry-net backstop. + try { await closeDay({ DB: env.DB, RAW_BUCKET: env.RAW_BUCKET as R2Bucket }, u.user_id) } catch (e) { console.error('inline retry close failed', u.user_id, e) } + n++ + } + } + return n +} diff --git a/src/workouts.ts b/src/workouts.ts index f7f014a..a619e5d 100644 --- a/src/workouts.ts +++ b/src/workouts.ts @@ -5,13 +5,17 @@ // blipped, the record is reconstructed from uploaded data on /workout/end. import type { Context } from 'hono' import { - calcStrain, calcHrZones, calcCalories, calcHrRecovery, + calcStrain, calcHrZones, calcCalories, calcHrRecovery, detectSessions, type Minute, type Profile, type Baseline, } from 'openstrap-analytics' +import { readMinutes } from './minute_store' +import { cached } from './cache' type Ctx = Context<{ Bindings: { DB: D1Database }; Variables: { userId: string } }> const nowSec = () => Math.floor(Date.now() / 1000) const DAY = 86400 +const startOfDayUtc = (ts: number) => Math.floor(ts / DAY) * DAY +const ymd = (ts: number) => new Date(ts * 1000).toISOString().slice(0, 10) const VALID_TYPES = ['run', 'cycle', 'strength', 'walk', 'swim', 'cardio', 'yoga', 'other'] @@ -34,13 +38,11 @@ async function loadBaseline(db: D1Database, userId: string): Promise { } } async function loadMinutes(db: D1Database, userId: string, from: number, to: number): Promise { - const { results } = await db.prepare( - 'SELECT ts_min, hr_avg, hr_min, hr_max, hr_n, activity, steps, wrist_on FROM minute ' + - 'WHERE user_id = ? AND ts_min >= ? AND ts_min <= ? ORDER BY ts_min ASC', - ).bind(userId, from, to).all() - return (results ?? []).map((r: any) => ({ - ts: r.ts_min, hr_avg: r.hr_avg ?? 0, hr_min: r.hr_min ?? 0, hr_max: r.hr_max ?? 0, - hr_n: r.hr_n ?? 0, activity: r.activity ?? 0, steps: r.steps ?? 0, wrist_on: !!r.wrist_on, + // Day-packed store. Only ever called at workout-end (recent/hot) → D1 minute_day. + const recs = await readMinutes({ DB: db }, userId, from, to) + return recs.map((r) => ({ + ts: r.ts_min, hr_avg: r.hr_avg, hr_min: r.hr_min, hr_max: r.hr_max, + hr_n: r.hr_n, activity: r.activity, steps: r.steps, wrist_on: !!r.wrist_on, })) } @@ -119,17 +121,22 @@ const RANGE_DAYS: Record = { week: 7, month: 35, quarter: 91, '7 // GET /workouts?range=week|month|quarter → list + training-volume summary. export async function listWorkouts(c: Ctx) { const userId = c.get('userId') + await ensureTodayWorkouts(c.env.DB, userId) // on-read auto-detect + stale-live close (throttled) const range = c.req.query('range') || 'month' const days = RANGE_DAYS[range] ?? 35 const from = nowSec() - days * DAY const { results } = await c.env.DB.prepare( - 'SELECT id, start_ts, end_ts, type, avg_hr, max_hr, strain, calories, hrr60, zones, status, source, title ' + + 'SELECT id, start_ts, end_ts, type, avg_hr, max_hr, strain, calories, hrr60, zones, status, source, title, ' + + 'segments, detected_type, type_confidence, type_source ' + "FROM sessions WHERE user_id = ? AND start_ts >= ? AND status != 'deleted' ORDER BY start_ts DESC", ).bind(userId, from).all() const rows = (results ?? []).map((r: any) => ({ ...r, duration_min: r.end_ts && r.start_ts ? Math.max(0, Math.round((r.end_ts - r.start_ts) / 60)) : 0, zones: r.zones ? safeParse(r.zones) : null, + segments: r.segments ? safeParse(r.segments) : null, + // distinguish in UI: detected (auto/auto_live) vs selected (manual) + detected: r.source === 'auto' || r.source === 'auto_live', })) // Training-volume summary (HONEST: time/type/zones/calories — no GPS distance, // no rep/weight; we don't fabricate what the band can't measure). @@ -149,6 +156,13 @@ export async function listWorkouts(c: Ctx) { } } const hardest = done.reduce((best, r) => (best == null || (r.strain ?? 0) > (best.strain ?? 0)) ? r : best, null) + // Calibration ledger: of the workouts the user confirmed/corrected, how often did the + // classifier already have it right? Surfaced so we know when the model needs retraining. + const acc = await c.env.DB.prepare( + "SELECT COUNT(*) AS total, SUM(CASE WHEN type = detected_type THEN 1 ELSE 0 END) AS correct " + + "FROM sessions WHERE user_id = ? AND detected_type IS NOT NULL AND type_source IN ('confirmed','corrected')", + ).bind(userId).first<{ total: number; correct: number }>() + const accTotal = acc?.total ?? 0 return c.json({ range, workouts: rows, @@ -159,10 +173,49 @@ export async function listWorkouts(c: Ctx) { by_type: byType, zone_min: zoneTotals, hardest: hardest ? { id: hardest.id, type: hardest.type, strain: hardest.strain } : null, + // model accuracy from user feedback (null until the user has confirmed/corrected any) + classifier: { reviewed: accTotal, correct: acc?.correct ?? 0, accuracy: accTotal > 0 ? Math.round(((acc!.correct) / accTotal) * 100) / 100 : null }, }, }) } +// POST /workout/:id/type { type } — user confirms or corrects an auto-detected type. +// Feeds the calibration ledger (was the model right?) and pins the type so re-derivation +// won't overwrite it. The accumulated (detected_type → corrected type) pairs are the +// labelled dataset that will train the real classifier. +export async function setWorkoutType(c: Ctx) { + const userId = c.get('userId') + const id = c.req.param('id') + const body = await c.req.json<{ type?: string }>().catch(() => ({} as any)) + if (!body.type || !VALID_TYPES.includes(body.type)) return c.json({ error: 'invalid type' }, 400) + const w = await c.env.DB.prepare('SELECT detected_type FROM sessions WHERE user_id = ? AND id = ?') + .bind(userId, id).first<{ detected_type: string | null }>() + if (!w) return c.json({ error: 'not found' }, 404) + const source = w.detected_type && w.detected_type === body.type ? 'confirmed' : 'corrected' + await c.env.DB.prepare( + 'UPDATE sessions SET type = ?, type_source = ?, type_confidence = 1.0 WHERE user_id = ? AND id = ?', + ).bind(body.type, source, userId, id).run() + return c.json({ ok: true, type: body.type, type_source: source }) +} + +/** + * sweepWorkoutDetection(env) — the #D incremental cron. Re-derive TODAY's auto-workouts + * for every user who's ingested since their last close (dirty=1), so a workout surfaces + * ~one tick (≤10 min) after it ends WITHOUT the app being opened. Reuses the throttled + * ensureTodayWorkouts (≤1 detect / 120 s / user) and is bounded per tick. The bout closes + * at its last data minute naturally (re-derivation over available minutes). + */ +export async function sweepWorkoutDetection(env: { DB: D1Database }, limit = 200): Promise { + const { results } = await env.DB.prepare( + 'SELECT user_id FROM analytics_cursor WHERE dirty = 1 LIMIT ?', + ).bind(limit).all<{ user_id: string }>() + let n = 0 + for (const r of results ?? []) { + try { await ensureTodayWorkouts(env.DB, r.user_id); n++ } catch (e) { console.error('wkt detect failed', r.user_id, e) } + } + return n +} + // GET /workout/:id → full breakdown + HR/activity timeline + derived effort // analytics (zone bands w/ bpm ranges + %, min HR, HR drift, time-to-peak, the // HR-recovery curve, steps/cadence, wrist coverage). Derived fields are computed @@ -277,11 +330,13 @@ function deriveEffort( // so a forgotten "stop" never inflates duration with idle time. Returns # closed. const QUIET_MIN = 12 const MAX_LIVE_HOURS = 6 -export async function autoCloseStaleWorkouts(db: D1Database): Promise { +export async function autoCloseStaleWorkouts(db: D1Database, userId?: string): Promise { const now = nowSec() - const { results } = await db.prepare( - "SELECT user_id, id, start_ts FROM sessions WHERE status = 'live'", - ).all() + // Optionally scope to ONE user (on-demand read path); else all (nightly safety net). + const stmt = userId + ? db.prepare("SELECT user_id, id, start_ts FROM sessions WHERE status = 'live' AND user_id = ?").bind(userId) + : db.prepare("SELECT user_id, id, start_ts FROM sessions WHERE status = 'live'") + const { results } = await stmt.all() let closed = 0 for (const s of results ?? []) { const baseline = await loadBaseline(db, s.user_id) @@ -314,3 +369,73 @@ export async function autoCloseStaleWorkouts(db: D1Database): Promise { } function safeParse(s: string) { try { return JSON.parse(s) } catch { return null } } + +// ── On-demand auto-workout detection (replaces the cron sweep) ──────────────── +// detectSessions used to run only inside the once-a-day close_day (via processUser), +// so an auto-detected workout didn't surface until the next wake. We now run it +// ON READ for TODAY, mirroring processUser Pass-2's session block EXACTLY (delete +// auto-non-deleted, insert 'done'/'auto'; manual/edited/deleted sessions untouched), +// so the on-read path and the day-close produce identical rows. +async function detectStoreDay(db: D1Database, userId: string, dayStart: number, baseline: Baseline, profile: Profile): Promise { + const recs = await readMinutes({ DB: db }, userId, dayStart, dayStart + DAY) + if (!recs.length) return + const dayMin: Minute[] = recs.map((r) => ({ + ts: r.ts_min, hr_avg: r.hr_avg, hr_min: r.hr_min, hr_max: r.hr_max, + hr_n: r.hr_n, activity: r.activity, steps: r.steps, wrist_on: !!r.wrist_on, + act_class: r.act_class as Minute['act_class'], + })) + const sessions = detectSessions(dayMin, baseline, profile) + // Reconcile: don't clobber the user's manual workouts, deleted tombstones, any + // already-typed live-detected sessions (source='auto_live'), OR any auto session the + // user CONFIRMED/CORRECTED (type_source in confirmed/corrected — the calibration + // ledger). Only the minute-detector's own un-reviewed 'auto' rows are re-derived. Then + // skip auto bouts that overlap any preserved session so a live-streamed or + // user-labelled workout isn't double-counted (and a baseline-drift start_ts shift can't + // spawn a duplicate that re-overwrites the user's label). + const { results: keep } = await db.prepare( + "SELECT start_ts, end_ts FROM sessions WHERE user_id = ? AND start_ts < ? AND end_ts > ? AND status != 'deleted' AND (source IN ('manual','auto_live') OR type_source IN ('confirmed','corrected'))", + ).bind(userId, dayStart + DAY, dayStart).all<{ start_ts: number; end_ts: number }>() + const covered = (keep ?? []) as { start_ts: number; end_ts: number }[] + const overlaps = (s: { start_ts: number; end_ts: number }) => + covered.some((k) => s.start_ts < k.end_ts && s.end_ts > k.start_ts) + const stmts: D1PreparedStatement[] = [ + db.prepare("DELETE FROM sessions WHERE user_id = ? AND start_ts >= ? AND start_ts < ? AND (source IS NULL OR source = 'auto') AND COALESCE(type_source,'model') NOT IN ('confirmed','corrected') AND status != 'deleted'") + .bind(userId, dayStart, dayStart + DAY), + ] + for (const s of sessions) { + if (overlaps(s)) continue // a manual/live-detected session already owns this window + const sid = `${userId}:${s.start_ts}` + stmts.push(db.prepare( + 'INSERT INTO sessions (user_id, id, start_ts, end_ts, type, avg_hr, max_hr, strain, calories, hrr60, zones, confidence, status, source, segments, detected_type, type_confidence, type_source) ' + + "VALUES (?,?,?,?,?,?,?,?,?,?,?,?,'done','auto',?,?,?,'model') ON CONFLICT(user_id, id) DO UPDATE SET " + + 'end_ts=excluded.end_ts, type=excluded.type, avg_hr=excluded.avg_hr, max_hr=excluded.max_hr, ' + + 'strain=excluded.strain, calories=excluded.calories, hrr60=excluded.hrr60, zones=excluded.zones, confidence=excluded.confidence, ' + + // Preserve a user-confirmed/corrected type across re-derivation; only refresh the model fields. + 'segments=excluded.segments, detected_type=excluded.detected_type, ' + + 'type_confidence=CASE WHEN sessions.type_source IN (\'confirmed\',\'corrected\') THEN sessions.type_confidence ELSE excluded.type_confidence END, ' + + 'type=CASE WHEN sessions.type_source IN (\'confirmed\',\'corrected\') THEN sessions.type ELSE excluded.type END', + ).bind(userId, sid, s.start_ts, s.end_ts, s.type, Math.round(s.avg_hr), Math.round(s.max_hr), + s.strain, s.kcal, s.hrr60 == null ? null : Math.round(s.hrr60), JSON.stringify(s.zones), s.confidence, + s.segments ? JSON.stringify(s.segments) : null, s.detected_type ?? s.type, s.type_confidence)) + } + await db.batch(stmts) +} + +/** + * On-read freshness for workouts (the wake-trigger replacement for the cron sweep). + * Closes this user's forgotten LIVE workouts and (re)detects TODAY's auto sessions. + * THROTTLED via read_cache (~120s) so a burst of reads costs one detect, not N — and + * it only ever runs for users actively opening the app. Call from any workout- + * surfacing read endpoint. The nightly cron still runs autoCloseStaleWorkouts as a + * safety net for users who never open the app; close_day still re-derives at wake. + */ +export async function ensureTodayWorkouts(db: D1Database, userId: string): Promise { + const today = ymd(nowSec()) + await cached(db, userId, `wkscan:${today}`, 120, async () => { + await autoCloseStaleWorkouts(db, userId) + const baseline = await loadBaseline(db, userId) + const profile = await loadProfile(db, userId) + await detectStoreDay(db, userId, startOfDayUtc(nowSec()), baseline, profile) + return { scanned: nowSec() } + }) +} diff --git a/wrangler.toml b/wrangler.toml index 2c17360..96fdede 100644 --- a/wrangler.toml +++ b/wrangler.toml @@ -12,28 +12,38 @@ database_id = "REPLACE_WITH_YOUR_D1_DATABASE_ID" binding = "RAW_BUCKET" bucket_name = "openstrap-raw" -# Analytics queue (producer + consumer). DISABLED on this account: Cloudflare -# Queues require the Workers Paid plan, which isn't enabled here (queue create -# returns "specified queue settings are invalid"). The code path is fully -# preserved — ingest.ts falls back to setting analytics_cursor.dirty=1 and the -# hourly cron ("7 * * * *") + /admin/run-analytics process the work. To enable: -# upgrade to Workers Paid, `wrangler queues create openstrap-analytics` + -# `... -dlq`, then uncomment these blocks. The queue() consumer export in -# index.ts is harmless to keep exported while the binding is absent. +# Workers AI — powers the self-hosted, OpenAI-compatible AI Coach (src/coach.ts). +# No external key; runs within Cloudflare's free daily AI allocation. +[ai] +binding = "AI" + +# Analytics queue (producer + consumer). Cloudflare Queues require the Workers PAID +# plan ($5/mo). DISABLED here so this deploys on the FREE plan. With no ANALYTICS_Q +# binding, the cron derives each day INLINE instead of enqueuing (see wake_cron.ts / +# queue.ts closeDay) — fine for a personal single-user backend (one close per tick, +# well under the free plan's 1000 Cloudflare-subrequest cap). +# +# To re-enable on Workers Paid: uncomment the four blocks below and +# npx wrangler queues create openstrap-analytics +# npx wrangler queues create openstrap-analytics-dlq +# # [[queues.producers]] # binding = "ANALYTICS_Q" # queue = "openstrap-analytics" # # [[queues.consumers]] # queue = "openstrap-analytics" -# max_batch_size = 20 -# max_batch_timeout = 30 +# max_batch_size = 1 +# max_batch_timeout = 10 # max_retries = 3 # dead_letter_queue = "openstrap-analytics-dlq" -# Crons: hourly safety sweep (enqueue dirty users) + nightly re-derive/prune. +# Crons: 30-min sweep (re-derive dirty users → keeps "today" fresh; pure-read +# endpoints serve this snapshot) + nightly full re-derive/prune. Any non-nightly +# cron string falls through to the sweep branch in scheduled(); only "30 3 * * *" +# triggers the nightly path. (Future: tighten the sweep to */15 if needed.) [triggers] -crons = ["7 * * * *", "30 3 * * *"] +crons = ["*/30 * * * *", "30 3 * * *"] [vars] EMAIL_FROM_NAME = "OpenStrap"