fix(test): update check-orchestrator spec for 4-check baseline#225
Merged
operatoruplift merged 1 commit intomasterfrom Apr 28, 2026
Merged
fix(test): update check-orchestrator spec for 4-check baseline#225operatoruplift merged 1 commit intomasterfrom
operatoruplift merged 1 commit intomasterfrom
Conversation
The orchestrator test scaffolds fake check scripts and asserts the "X passed, Y failed, Z skipped" summary. Adding fabrication-rot as the 4th check in #224 shifted those counts and broke 3 of 5 tests: fail-fast: "0 passed, 1 failed, 2 not run" -> 3 not run --all: "1 passed, 2 failed" -> 2 passed, 2 failed mix: "1 passed, 1 failed, 1 skipped" -> 1 passed, 1 failed, 2 skipped Updated each scenario to include a fabrication-rot-check entry where the prior 3-script setups are now 4-script setups, and adjusted the expected counts. The skips-missing-scripts test now also asserts the new SKIP line for fabrication-rot. The orchestrator contract (run, skip-if-missing, fail-propagate, fail-fast) is unchanged; only the count of registered checks moves.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
2 tasks
operatoruplift
added a commit
that referenced
this pull request
Apr 28, 2026
…sion net (#226) CLAUDE.md still listed 3 grep-guards and only 4 dashboard pages covered by the honesty net. After this batch's PRs (#194-#225), the actual state is: - 4 grep-guards: copy-check, capability-check, trust-gate, fabrication-rot. The new one is described inline so a reader who runs into a fabrication-rot failure can jump to scripts/fabrication-rot-check.mjs for the rule + PR pointer. - dashboard-honesty.spec.ts now covers 7 surfaces (added /memory, /integrations, /agents/builder, /settings on top of the original /app, /notifications, /workflows). - request-id-runtime.spec.ts is at 17 probes including the new 410-Gone endpoints from #209 (/api/risk, /api/gold). Pure docs update. pnpm check passes (4/4).
3 tasks
operatoruplift
added a commit
that referenced
this pull request
Apr 28, 2026
) Update the "Honesty regression net" section to accurately describe what's locked in after the local-machine claim cleanups landed in PRs #225-#240: - PR range bumped from #147-#224 to #147-#240 - consumer-copy.spec.ts banned-phrase list documented in full, including the 8 retired local-machine claims added in #240 - fabrication-rot-check.mjs documented as 20 rules (was 10), with the new local-machine patterns enumerated alongside the original retired-fabrication catalog (Gold Agent, Webacy, x402, random- vector, expires-in-30-days, fake-install) This is documentation-only — no source changes. Verifies pnpm check 4/4 still passes after the description is brought up to date.
1 task
operatoruplift
added a commit
that referenced
this pull request
Apr 28, 2026
After PR #242 retired "Runs on your computer" from meta description, Open Graph, Twitter card, and the JSON-LD SoftwareApplication block, add explicit assertions to consumer-copy.spec.ts so a future PR cannot silently revert the change. OG metadata test: - meta description must NOT contain "Runs on your computer" (was the centerpiece fabrication PR #242 retired) - og:description meta tag asserted not to contain it either - twitter:description meta tag asserted not to contain it either JSON-LD test: - ld+json blob must NOT contain "Runs on your computer" - ld+json blob MUST contain "WebApplication" (the @type was changed from SoftwareApplication, since the web app is browser-based, not a Windows/macOS/Linux installer today) Both tests already validated "drafts your email" presence + banned sci-fi phrases; this extends them to also catch a regression of the local-machine fabrication that was the largest meta-level surface in the PR #225-#242 sweep. Verification: - pnpm exec playwright test tests/e2e/consumer-copy.spec.ts: 8/8 pass
2 tasks
operatoruplift
added a commit
that referenced
this pull request
Apr 28, 2026
…d state (#247) The repo README's headline claim contradicted the line two sections below it ("This is a Next.js 16 app deployed on Vercel"). Two internally-inconsistent claims about hosting on the same page is the exact pattern the homepage honesty sweep (#225-#246) was retiring. Three changes: 1. Top pitch paragraph: - "It runs on your computer (not a cloud), and every action waits for your tap." was replaced with "The web app pauses for your tap before any action and emits a signed receipt afterward. The desktop+Ollama build on the roadmap removes the provider hop entirely for users who need full on-device inference." - This matches the homepage Privacy Policy + Compliance + blog post copy that landed in the sweep. 2. "Pick any AI" bullet: - "Claude, ChatGPT, Gemini, Grok, or a model running on your laptop via Ollama" -> "Claude, ChatGPT, Gemini, Grok per turn (the desktop build adds Ollama on the roadmap)" - Ollama as a roadmap is honest; framing it as already-shipped was not. 3. CI checks section: - "grep-guards: copy-check, capability-check, trust-gate" -> adds fabrication-rot (the 4th guard, covering 20 anchored patterns of retired fabrications). - The honesty-regression description now lists local-machine claims (Runs on your computer, AES-256 encrypted local storage, Your agent ran locally, Encrypted on your computer) alongside LLM Council / Gold Agent / dev-jargon items. Verification: pnpm check 4/4 passes (20 fabrication-rot rules clean, no source-text changes that would trip a guard).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adding
fabrication-rotas the 4th grep-guard in #224 broke the orchestrator acceptance spec. The test scaffolds fake check scripts and asserts "X passed, Y failed, Z skipped" totals; those counts shifted with the new check.Three failing scenarios fixed:
Each scenario now includes a
fabrication-rot-checkentry where the prior 3-script setups are 4-script setups. The skips-missing-scripts test also asserts the new[fabrication-rot] SKIPline.The orchestrator contract (run, skip-if-missing, fail-propagate, fail-fast) is unchanged; only the count of registered checks moves.
Test plan
check-orchestrator.spec.tstests pass