perf: wire launch-to-interactive CI budget; verify updateForState deopt still needed#1492
Merged
Conversation
…pt still needed Measurement-driven perf pass for 1.1.49 release prep (PRD WS4.2). Launch-to-interactive budget (new) - The launch UI smoke already proves the menu bar is interactive but had no timing. Add launchToInteractiveMs to the smoke report, measured from the kernel process-start time (kinfo_proc.p_starttime) so it captures true launch latency including pre-main dyld/runtime setup. - Score it in performance-budget.rb via --launch-ui-smoke / --max-launch-interactive-ms and enforce in CI's Premium performance budget step (extends the existing budget; no parallel harness). build.sh scores it locally too. - Measured on Apple Silicon: warm ~285ms (meets PRD <400ms warm target), cold ~600ms, fresh-build cold ~900ms. CI ceiling set to 1500ms as a regression guard with headroom for a cold GitHub runner. updateForState forced-deopt (verified, kept) - Re-tested removing @_optimize(none) under the current toolchain (swiftlang-6.3.3.1.3 / Swift 6.3.3): the identical SimplifyCFG / BorrowUtils.swift:542 compiler crash still reproduces. Workaround still required. Confirmed it is not a hot path (fires only on discrete overlay state transitions, not per audio frame), so the deopt costs nothing. Refreshed the comment with the re-verification so nobody re-tests blindly. No behavior change. App size 88.3 MiB (budget 220), resources 2.0 MiB (budget 80), Home recent-captures 20.6ms@1k / 214.9ms@10k avg (budget 750) all unchanged & green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First CI run measured cold launch-to-interactive at 1420ms on the GitHub macos-26 runner — only 80ms (5%) under the initial 1500ms ceiling, which would flake under normal runner variance. Raise to 2500ms: still a firm regression guard (fails on a ~1.8x+ blow-up from the 1420ms cold baseline / ~285ms warm product number) but no longer flaky. A perf gate that flakes is worse than a loose one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…o flake) Two CI runs on identical commits measured cold launch-to-interactive at 1420ms and 2095ms — a ~48% run-to-run swing on the GitHub runner's single cold sample. At 2500ms the 2095ms sample sits only 16% under the ceiling, still flake-prone. Set 3000ms: the CI gate is explicitly a coarse catastrophic-regression guard (the real UX number is the ~285ms warm Apple-Silicon product measurement, not this noisy cold CI sample). Documented min-of-N smoke sampling as the future signal-sharpening follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measurement-driven performance pass for the 1.1.49 release, grounded in the existing Premium performance budget + Home recent-captures benchmark and the PRD WS4.2 targets. No speculative changes — every item below is backed by a real measurement on this Apple Silicon machine (Xcode 26.6, swiftlang-6.3.3.1.3).
What I measured (baseline, all green)
Home list transcript re-read (the #1399/#1465 concern): fully solved.
RecentMeetingsScanner.loadRecentserves rows fromRecentMeetingMetadataCacheon the warm path with zero transcript/summary reads (only the live audio attachment is re-resolved, intentionally). The residual 10k-capture cost is the inherent O(N) directorystat+sort to find the 5 newest — 215 ms, comfortably under the 750 ms budget. No change needed.Fix 1 — Launch-to-interactive is now measured and CI-enforced (PRD WS4.2)
The PRD asks for "app launch to interactive <400ms warm" plus "a perf test to CI that fails on regression." The launch UI smoke already proved the menu bar is functionally interactive (status item + popover + primary actions) but carried no timing, and CI only budgeted model-ready (≤30s), which is a different thing.
launchToInteractiveMsto the launch UI smoke report, measured from the kernel process-start time (kinfo_proc.p_starttime) so it captures true launch latency including pre-maindyld/runtime setup — not just time since the app delegate started.performance-budget.rb(--launch-ui-smoke/--max-launch-interactive-ms) and wire it into CI's existing Premium performance budget step (no parallel harness).build.shscores it on local builds too, guarded so a skipped smoke doesn't fail the budget.Fix 2 —
updateForStateforced-deopt: re-verified still required, keptThe
@_optimize(none)workaround was flagged as maybe-obsolete (it cited swiftc 6.3.2; the toolchain is now 6.3.3). I removed the attribute and rebuilt: the identical compiler crash still reproduces on swiftlang-6.3.3.1.3 —So the workaround stays. I also confirmed it is not a hot path:
updateForStatefires only on discrete overlay state transitions (FloatingOverlayController.pushStateToViews), never per audio frame — the per-frame meter path callsheaderView.updateWaveformLeveldirectly. So the deopt costs nothing measurable. Refreshed the comment to record the 6.3.3 re-verification so the next person doesn't re-test blindly.Verification
build.sh(-O -whole-module-optimization) builds clean with the workaround; crashes without it (attribution isolated in a separate rebuild).performance-budget.rbpass/fail/missing-field/bad-JSON paths unit-checked; over-budget and missing-field exit 1, under-budget exits 0.check-build-source-lists.pygreen;build/launch-ui-smoke.jsonconfirmed at the path CI reads.Scope: measured performance only. No visual/UX or speaker-matching changes (owned by separate threads).
🤖 Generated with Claude Code