-
Notifications
You must be signed in to change notification settings - Fork 1
docs(perf): #114 warm-deploy loop first-pass results #230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+87
−0
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| # Warm-Deploy Loop Results (FastLED/fbuild#114) | ||
|
|
||
| First-pass measurement of the warm **rebuild + deploy + monitor reconnect** | ||
| path against real ESP32-S3 hardware, against the 4 000 ms end-to-end budget | ||
| defined in #114. | ||
|
|
||
| ## TL;DR | ||
|
|
||
| - **5/5 consecutive iterations** of the full `deploy + monitor reattach + | ||
| first-byte-from-device` loop land at **3.585–3.697 s** — all inside the | ||
| 4 000 ms budget with ~300–400 ms of slack. | ||
| - Deploy-only warm path (`fbuild deploy` without `--monitor`) is **~2.6 s** | ||
| steady-state (~3.3 s on the first call after daemon spawn). | ||
| - Acceptance criterion from #114 (*three consecutive in-budget iterations*) | ||
| is met; loop spec (`LOOP.md`) is retired. | ||
|
|
||
| ## Methodology | ||
|
|
||
| - Host: Windows 10 Pro, x86_64-pc-windows-msvc. | ||
| - Binary: `target/x86_64-pc-windows-msvc/release/fbuild.exe` built from | ||
| `feat/114-warm-deploy-loop-first-pass`. | ||
| - Project: `tests/platform/esp32s3/` (Arduino, `ARDUINO_USB_CDC_ON_BOOT=1`, | ||
| blinking LED + `Serial.println("Hello from ESP32-S3!")` once per 1 s loop). | ||
| - Device: ESP32-S3-DevKitC-1, native USB-CDC on COM13 (`303a:1001`). | ||
| - Daemon: in-process via CLI auto-spawn, persistent across iterations. | ||
| - Pre-condition: cold deploy (full flash, 3m 16s) brings the device to the | ||
| exact image the warm path then verify-skips against. | ||
|
|
||
| ## Results | ||
|
|
||
| ### Deploy-only warm path (no monitor) | ||
|
|
||
| `fbuild deploy -e esp32s3 -p COM13` | ||
|
|
||
| | iter | wall-clock | server-side outcome | | ||
| |---:|---:|---| | ||
| | 1 | 3.257 s | `verify skipped, device already matched` (incl. daemon warm-up) | | ||
| | 2 | 2.642 s | `verify skipped, device already matched` | | ||
| | 3 | 2.640 s | `verify skipped, device already matched` | | ||
| | 4 | 2.615 s | `verify skipped, device already matched` | | ||
| | 5 | 2.568 s | `verify skipped, device already matched` | | ||
|
|
||
| ### Full loop (T1 + T2 + T3 + TTFB) | ||
|
|
||
| `fbuild deploy -e esp32s3 -p COM13 --monitor --halt-on-success "Hello from ESP32-S3" --timeout 5` | ||
|
|
||
| | iter | wall-clock | budget | margin | | ||
| |---:|---:|---:|---:| | ||
| | 1 | 3.587 s | 4.000 s | +413 ms | | ||
| | 2 | 3.590 s | 4.000 s | +410 ms | | ||
| | 3 | 3.697 s | 4.000 s | +303 ms | | ||
| | 4 | 3.585 s | 4.000 s | +415 ms | | ||
| | 5 | 3.605 s | 4.000 s | +395 ms | | ||
|
|
||
| Mean 3.613 s, max 3.697 s. The ~1 s overhead between the deploy-only path | ||
| and the full loop is dominated by the test sketch's `delay(500)` × 2 loop | ||
| period — TTFB is the next `Serial.println` cadence, not a property of the | ||
| deploy/reconnect path. A sketch emitting at boot would shave that ~1 s. | ||
|
|
||
| ## What got us here | ||
|
|
||
| Landed before this measurement: | ||
|
|
||
| - **#116** — `FBUILD_TRUST_DEVICE_HASH=1` opt-in trust-skip on verify-flash | ||
| (server cost: ~1.5 s → ~50 ms). | ||
| - **#118** — `ImageHashMemo` (skip SHA-256 of unchanged firmware) + | ||
| `DeviceManager::refresh_devices_if_stale(2s)` (~50 ms → ~1–2 ms server-side). | ||
| - **#120** — `DaemonWatchSetCache` removes the `fp-watches-collect` walk on | ||
| back-to-back warm builds. | ||
|
|
||
| The full loop runs cleanly inside the 4 s envelope without any phase | ||
| breaching its individual budget; no follow-up perf issues are filed against | ||
| #114. | ||
|
|
||
| ## Closing the loop | ||
|
|
||
| Acceptance criteria from #114: | ||
|
|
||
| - [x] LOOP.md committed (#113), and later untracked (#115); local-only | ||
| scratch, removed. | ||
| - [x] First pass of the loop against real ESP32-S3 hardware to confirm or | ||
| tune budgets — this document. | ||
| - [x] Follow-up issues for any phase that consistently breaches its | ||
| budget — none; all five iterations green. | ||
| - [x] T1–T3 land inside the total budget for three consecutive iterations — | ||
| five consecutive, lowest margin 303 ms. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Line 73: avoid accidental ATX heading syntax
#114.at start-of-line is parsed as a heading marker and triggers MD018. Keep it inline text (for example:issue#114.) so markdownlint stays clean.Suggested fix
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 73-73: No space after hash on atx style heading
(MD018, no-missing-space-atx)
🤖 Prompt for AI Agents