Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 108 additions & 68 deletions .github/workflows/windows-beta.yml
Original file line number Diff line number Diff line change
@@ -1,83 +1,123 @@
name: Windows beta

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

name: Windows beta
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
permissions:
contents: read

jobs:
build-and-package:
runs-on: windows-latest
timeout-minutes: 45

steps:
- name: Check out repository
uses: actions/checkout@v6

- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: 22
cache: npm

- name: Set up .NET
uses: actions/setup-dotnet@v5
with:
dotnet-version: 8.0.x

- name: Restore JavaScript dependencies
run: npm ci

- name: Audit dependencies
run: npm audit --audit-level=high

- name: Typecheck
run: npm run typecheck

- name: Run renderer and main-process tests
run: npm test

- name: Build and test Windows helper
run: npm run test:helper

- name: Publish self-contained Windows helper
run: npm run helper:build

- name: Build Electron application
run: npm run build

- name: Run Electron overlay smoke tests
run: npx playwright test

- name: Evaluate Milestone 4 retrieval offline
run: npm run eval:m4

actions: read

jobs:
build-and-package:
runs-on: windows-latest
timeout-minutes: 45

steps:
- name: Check out repository
uses: actions/checkout@v6

- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: 22
cache: npm

- name: Set up .NET
uses: actions/setup-dotnet@v5
with:
dotnet-version: 8.0.x

- name: Restore JavaScript dependencies
run: npm ci

- name: Audit dependencies
run: npm audit --audit-level=high

- name: Typecheck
run: npm run typecheck

- name: Run renderer and main-process tests
run: npm test

- name: Build and test Windows helper
run: npm run test:helper

- name: Publish self-contained Windows helper
run: npm run helper:build

- name: Build Electron application
run: npm run build

- name: Run Electron overlay smoke tests
run: npx playwright test

- name: Evaluate Milestone 4 retrieval offline
run: npm run eval:m4

- name: Validate Milestone 6 live-campaign budget offline
run: npm run eval:m6:preflight

- name: Create unsigned NSIS installer
run: npx electron-builder --win nsis --publish never
env:
CSC_IDENTITY_AUTO_DISCOVERY: "false"

- name: Verify packaged Electron SQLite FTS5
run: npm run test:packaged-fts

- name: Evaluate Milestone 7 grounding offline
run: npm run eval:m7

- name: Create unsigned NSIS installer
run: npx electron-builder --win nsis --publish never
env:
CSC_IDENTITY_AUTO_DISCOVERY: "false"

- name: Verify packaged Electron SQLite FTS5
run: npm run test:packaged-fts

- name: Verify packaged Windows helper health
run: npm run test:packaged-helper

- name: Upload Windows installer and Milestone 4 report
uses: actions/upload-artifact@v4
- name: Locate previous successful main installer
id: previous-main
shell: pwsh
env:
GH_TOKEN: ${{ github.token }}
run: |
$runs = gh run list --repo "${{ github.repository }}" --workflow windows-beta.yml --branch main --status success --limit 20 --json databaseId,headSha,createdAt | ConvertFrom-Json
$baseline = $runs | Where-Object {
$_.databaseId -ne [long]"${{ github.run_id }}" -and $_.headSha -ne "${{ github.sha }}"
} | Sort-Object createdAt -Descending | Select-Object -First 1
if (-not $baseline) { throw 'A previous successful main Windows-beta installer from a different commit/run is required for the upgrade smoke test.' }
"run_id=$($baseline.databaseId)" | Out-File -FilePath $env:GITHUB_OUTPUT -Encoding utf8 -Append
"head_sha=$($baseline.headSha)" | Out-File -FilePath $env:GITHUB_OUTPUT -Encoding utf8 -Append

- name: Download previous successful main installer
uses: actions/download-artifact@v5
with:
github-token: ${{ github.token }}
repository: ${{ github.repository }}
run-id: ${{ steps.previous-main.outputs.run_id }}
name: PresenterAI-Windows-beta
path: |
path: artifacts/previous-main

- name: Verify clean install, upgrade, Delete All, persistence, and uninstall
shell: pwsh
run: |
$current = Get-ChildItem release -Filter 'PresenterAI-*-setup.exe' -File | Sort-Object LastWriteTimeUtc -Descending | Select-Object -First 1
$previous = Get-ChildItem artifacts/previous-main -Filter 'PresenterAI-*-setup.exe' -File -Recurse | Sort-Object LastWriteTimeUtc -Descending | Select-Object -First 1
if (-not $current -or -not $previous) { throw 'Current or previous PresenterAI installer artifact was not found.' }
node scripts/installer-smoke.mjs --current $current.FullName --previous $previous.FullName --require-previous --report artifacts/installer/installer-lifecycle-report.json

- name: Upload Windows installer and validation reports
uses: actions/upload-artifact@v4
with:
name: PresenterAI-Windows-beta
path: |
release/PresenterAI-*-setup.exe
artifacts/m4/m4-retrieval-report.json
artifacts/m7/m7-offline-report.json
artifacts/installer/installer-lifecycle-report.json
docs/validation/milestones-0-2.md
docs/validation/milestone-5.md
docs/validation/milestone-6.md
if-no-files-found: error
docs/validation/milestone-7.md
docs/validation/milestone-8.md
docs/validation/plan-alignment.md
if-no-files-found: error
64 changes: 64 additions & 0 deletions docs/validation/milestone-7.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Milestone 7 conversation context and grounding record

Status: **SOURCE COMPLETE / OFFLINE GREEN — formal acceptance still depends on accepted M6 evidence and a separately authorized live Luna evaluation.**

No OpenAI request, API credential, or billable evaluation is used by this branch gate.

## Implemented invariants

- Typed and audio questions use the same immutable prepared-answer path. A prepared snapshot contains the validated current question, context revision, expanded retrieval query, selected bounded chunks, five-turn background, and bounded project summary.
- Conversation memory retains five questions and locally derived answer summaries in memory only. Questions are limited to 1,000 Unicode code points, summaries to 600, and the project summary to 4,000.
- Referential retrieval may append only the immediately preceding reviewer question. It never uses a prior model answer as evidence, and the 4,000-code-point query limit reserves space for that antecedent.
- Clearing a session increments its revision. An already prepared late answer may render, but it cannot repopulate cleared context or replace the exact chunks/background already disclosed in the outbound preview.
- Project summaries and conversation summaries are labeled reference/background only. Retrieved document chunks remain the only project-specific evidence.
- Structured responses report `document-supported`, `general-technical`, or `unsupported-project-claim` plus `none`, `missing`, `insufficient`, or `conflicting` evidence status.
- Document-supported answers require unique citations from the exact selected chunks. General explanations cannot carry citations or project-warning state. Unsupported project claims require an explicit warning and evidence issue.
- Contradictory project evidence requires two supplied citations, an unsupported/conflicting support state, and an explicit conflict warning. Forged and duplicate citations remain malformed output.
- One Responses request is used per answer with `store:false`; PresenterAI creates no OpenAI Conversation and uses no `previous_response_id`.

## Offline corpus and evaluator

The committed `m7-offline-v2` corpus contains exactly 50 synthetic cases:

- 20 follow-ups: 15 supported and five unsupported.
- 15 additional unsupported/adversarial project claims.
- Five supported factual questions.
- Five skeptical challenges.
- Five contradictory-source questions.

`npm run eval:m7` runs the production conversation and prepared-answer path against an in-memory production `RetrievalIndex` backed by SQLite FTS5, then applies the same schema, citation, support, and grounding validators used by production. Supported follow-up evidence is keyed to the prior reviewer question, so retrieval must actually use the antecedent. Document-supported cases carry synthetic semantic anchors so a generic answer cannot count as resolving a follow-up. Unsupported checks reject numeric and named declarative project fabrications; challenge checks require acknowledgement, evidence-based defence, and limitation language; contradiction checks reject silently choosing either supplied side.

This is an offline invariant/retrieval-orchestration evaluation, not a model-quality claim and not a replacement for the accepted M4 FTS recall corpus. The final branch result and generated redacted JSON path must be recorded only after the complete gate runs.

| Offline gate | Required | Result |
|---|---:|---:|
| Corpus cases | 50 | 50 |
| Referential follow-ups containing the prior reviewer question | 20/20 | 20/20 |
| Production SQLite FTS5 prepared selections matching bounded evidence | 50/50 | 50/50 |
| Grounding/citation invariants | 50/50 | 50/50 |
| Semantic anchor, unsupported-safety, challenge, and conflict checks | 50/50 | 50/50 |
| Failed case IDs | 0 | 0 |

## Live gate (not run)

`npm run eval:m7:live -- --budget-usd=N` is opt-in, refuses CI, requires an explicit positive budget, disables SDK retries, checks the conservative maximum before every request, and writes only redacted metrics. It does not persist prompts, model answers, reasoning content, or credentials.

Formal M7 acceptance remains blocked until M6 is accepted and the user separately authorizes a live budget. The unchanged promotion thresholds are:

- At least 18/20 follow-up resolutions.
- At least 19/20 unsupported warnings/refusals.
- Zero accepted invented project results, including named dataset/hardware/algorithm claims.
- Every citation was supplied in that request.
- Explicit neutral conflict handling for 5/5 contradiction cases.
- Structured challenge handling for 5/5 challenge cases.

Until that later gate passes, UI, documentation, commits, and release notes must say **source complete/offline green; formal acceptance pending**, never “M7 accepted.”

## Branch evidence

- Gate date: 2026-07-16; commit: the commit containing this record.
- `npm run eval:m7`: 50/50 passed, 20/20 contextual follow-ups, 50/50 production FTS selections, 50/50 semantic checks, 50/50 grounding checks, zero failed IDs.
- Redacted report: `artifacts/m7/m7-offline-report.json` (generated and ignored locally; uploaded by Windows CI).
- Regression gate: TypeScript checks passed; Vitest 259/259; .NET helper tests 29/29; Electron production build passed.
- No live M7 harness, OpenAI credential read, or OpenAI request occurred.
- Windows PR/post-merge CI and uploaded-report links are publication evidence and are recorded on GitHub rather than predeclared here.
Loading
Loading