diff --git a/README.md b/README.md index 5d5bc53..b1d3199 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,7 @@ This repository contains skills that work across open-source frameworks, tools a | Skill | Description | |-------|-------------| | [vega-multi-tv-migration](vega-multi-tv-migration/SKILL.md) | Migrate Vega OS (Fire TV) apps to multi-platform React Native monorepo supporting Android TV, Apple TV, and more | +| [rn-tv-performance](rn-tv-performance/SKILL.md) | Measure, diagnose, and optimize performance for React Native TV apps, including React Native for Vega and Vega WebView | ## Installation @@ -28,6 +29,10 @@ Install a specific skill: npx skills add AmazonAppDev/devices-agent-skills --skill vega-multi-tv-migration ``` +```bash +npx skills add AmazonAppDev/devices-agent-skills --skill rn-tv-performance +``` + For more options, see the [skills CLI documentation](https://skills.sh/docs/cli). ### 2. Manual Installation diff --git a/rn-tv-performance/LICENSE b/rn-tv-performance/LICENSE new file mode 100644 index 0000000..ed3afcb --- /dev/null +++ b/rn-tv-performance/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2025 Vega Skills + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/rn-tv-performance/SKILL.md b/rn-tv-performance/SKILL.md new file mode 100644 index 0000000..6f3276d --- /dev/null +++ b/rn-tv-performance/SKILL.md @@ -0,0 +1,176 @@ +--- +name: rn-tv-performance +description: Audit, measure, and optimize performance for React Native TV applications, with Vega-specific guidance for React Native for Vega and Vega WebView apps. Use when investigating app launch latency, UI fluidity, video fluidity, memory, CPU usage, overdraw, jank, slow lists, re-renders, WebView performance, Web Workers, WebGL, profiling, Vega KPI Visualizer results, TTFF, TTFD, or TTFVF. +--- + +# RN TV Performance + +## Overview + +Measure before optimizing. Help agents diagnose and improve React Native TV app performance across native TV surfaces, with Vega-specific guidance for React Native for Vega, Vega WebView, official Vega KPIs, and Vega Studio profiling tools. + +Always identify the app surface first because generic React Native TV, React Native for Vega, and Vega WebView apps use different APIs, tools, and performance tactics. + +## When to Apply + +Use this skill when user mentions: +- React Native TV app performance, slow startup, launch latency, responsiveness, or release-readiness +- UI jank, poor fluidity, slow D-pad navigation, frame drops, or unresponsive screens +- Video startup, Time-to-First Video Frame, buffering, dropped frames, or video fluidity +- High CPU usage, memory growth, crashes, background memory, or profiling +- React Native re-renders, FlatList, FlashList, memoization, concurrent rendering, or image/list performance +- Vega app performance, React Native for Vega, Vega WebView, TTFF, TTFD, TTFVF, or report fully drawn +- Vega KPI Visualizer, Activity Monitor, Recording View, Perfetto, Chrome DevTools, overdraw, or flamegraphs + +## Phase Priority Guide + +| Priority | Phase | Impact | When to Use | +|----------|-------|--------|-------------| +| 1 | App Surface Detection | CRITICAL | Before recommending platform-specific fixes | +| 2 | Baseline Measurement | CRITICAL | Before optimizing launch, fluidity, memory, or video | +| 3 | Tool Selection | HIGH | Choose KPI Visualizer, Activity Monitor, CDT, Perfetto, or overdraw | +| 4 | Root Cause Analysis | HIGH | Map symptoms to CPU, memory, rendering, network, list, video, or WebView causes | +| 5 | Targeted Fix | HIGH | Apply the smallest source-aligned fix and re-measure | + +## Quick Decision Tree + +``` +What is slow? ++-- Launch or resume? +| +-- Measure platform launch metrics; for Vega, measure TTFF/TTFD and verify reportFullyDrawn/useReportFullyDrawn ++-- UI interaction, scrolling, D-pad, or animation? +| +-- Measure ui-fluidity; inspect re-renders, overdraw, render thread, and JS thread ++-- Video startup or playback? +| +-- Measure video-fluidity and TTFVF; check hardware decode and media setup ++-- Memory or crash? +| +-- Measure foreground/background memory; inspect Activity Monitor recordings ++-- Native React Native TV screen? +| +-- Use RN TV guidance for memoization, lists, images, render work, and navigation responsiveness ++-- WebView page or HTML app? +| +-- Use WebView guidance for network, JS, cache, Web Workers, WebGL, video ++-- Vega app? +| +-- Use Vega KPI and profiling guidance for official metrics and tooling ++-- Unknown? + +-- Inspect manifest.toml, package.json, app entry points, WebView usage, and rendered flows +``` + +## Quick Reference + +### Detect App Surface + +```bash +# Generic React Native TV indicators +rg "react-native|react-native-tvos|@react-native|FlatList|FlashList|TVEventHandler|hasTVPreferredFocus" + +# Vega package marker +find . -name manifest.toml -maxdepth 5 + +# WebView indicators +rg "@amazon-devices/webview| + +# Explicit iteration count +vega exec perf kpi-visualizer --app-name= --iterations + +# UI fluidity with a test scenario +vega exec perf kpi-visualizer --app-name= --kpi ui-fluidity --test-scenario + +# Activity Monitor with CPU profiling +vega exec perf activity-monitor --record-cpu-profiling --app-name= --sourcemap-file-path= +``` + +### Vega KPI Targets + +- Cool start TTFF: less than 1.5 s +- Warm start TTFF: less than 0.5 s +- Cool start TTFD: less than 8.0 s +- Warm start TTFD: less than 1.5 s +- Foreground memory: less than 400 MiB +- Background memory: less than 150 MiB +- Video fluidity: greater than 99% +- Time-to-First Video Frame: less than 2500 ms +- UI fluidity: target 99% or higher +- Iterations: default 3; certification mode uses 30 with 90th percentile aggregation + +### Vega-Specific Path + +Use the Vega-specific references when the app is React Native for Vega, uses Vega WebView, exposes `manifest.toml`, or needs official Vega KPI evidence. For non-Vega React Native TV apps, use the RN TV reference and the project's available platform profilers, then adapt the same measure-before-fix workflow. + +## References + +| File | Impact | Description | +|------|--------|-------------| +| [REACT_NATIVE_TV_PERFORMANCE.md](references/REACT_NATIVE_TV_PERFORMANCE.md) | CRITICAL | React Native TV optimization for renders, lists, images, concurrent rendering, and TV navigation responsiveness | +| [KPI_MEASUREMENT.md](references/KPI_MEASUREMENT.md) | CRITICAL | Vega-specific KPI Visualizer metrics, targets, commands, and report fully drawn APIs | +| [WEBVIEW_PERFORMANCE.md](references/WEBVIEW_PERFORMANCE.md) | CRITICAL | Vega WebView launch, network, rendering, cache, Web Worker, WebGL, and video guidance | +| [PROFILING_AND_RENDERING.md](references/PROFILING_AND_RENDERING.md) | HIGH | Vega Activity Monitor, Recording View, CPU profiler, Chrome DevTools, Perfetto, overdraw | + +### Templates + +Use [PERFORMANCE_CHECKLIST.md](assets/templates/PERFORMANCE_CHECKLIST.md) to track measurement, fixes, and verification. + +### Evals + +Use [evals/README.md](evals/README.md) to test trigger quality, task quality, and with-skill vs without-skill behavior. + +## Problem -> Skill Mapping + +| Problem | Start With | +|---------|------------| +| Slow React Native TV screen or interaction | REACT_NATIVE_TV_PERFORMANCE.md | +| Slow Vega launch or resume | KPI_MEASUREMENT.md | +| Vega TTFD is missing or `-1` | KPI_MEASUREMENT.md | +| UI jank or D-pad lag | PROFILING_AND_RENDERING.md, then platform reference | +| Slow FlatList/rails | REACT_NATIVE_TV_PERFORMANCE.md | +| Too many React renders | REACT_NATIVE_TV_PERFORMANCE.md | +| Slow WebView load | WEBVIEW_PERFORMANCE.md | +| Heavy WebView JavaScript | WEBVIEW_PERFORMANCE.md | +| High CPU or memory | PROFILING_AND_RENDERING.md | +| Rendering overdraw | PROFILING_AND_RENDERING.md | +| Video startup/playback issues | KPI_MEASUREMENT.md, then WEBVIEW_PERFORMANCE.md if WebView video | + +## Workflow + +1. Detect app surface: generic React Native TV, React Native for Vega, Vega WebView, mixed, or unknown. +2. Identify the symptom and target KPI: launch, UI fluidity, video fluidity, memory, CPU, or rendering. +3. Run the appropriate measurement tool on a release build when possible. For Vega, use KPI Visualizer, Activity Monitor, Recording View, and related Vega tools. +4. Inspect evidence: profiler output, KPI Visualizer output, Activity Monitor recording, flamegraph, Perfetto trace, Chrome DevTools profile, overdraw colors, or re-render logs. +5. Apply targeted fixes from the correct reference path. +6. Re-run the same measurement and compare before/after values. +7. Report symptom, baseline, root cause, fix, after-measurement, remaining risk, and official source used. + +## Guardrails + +- Do not optimize before establishing a baseline measurement. +- Do not compare debug-build numbers against release-build numbers. +- Do not compare before/after results from different devices, app states, scenarios, or iteration counts. +- Do not treat Vega TTFD as valid unless the app reports fully drawn when it is actually ready for user interaction. +- Do not apply WebView fixes to native React Native surfaces, or native-only fixes to WebView content, without checking app surface first. + +## Official Sources + +- App Performance Best Practices: https://developer.amazon.com/docs/vega/0.21/best_practices.html +- Measure App KPIs: https://developer.amazon.com/docs/vega/0.21/measure-app-kpis.html +- Performance Best Practices for Web Apps: https://developer.amazon.com/docs/vega/0.21/webview-app-performance-best-practices.html +- Improve App Performance with Concurrent Rendering: https://developer.amazon.com/docs/vega/0.21/concurrent-rendering.html +- Optimizing Flatlist Configuration: https://developer.amazon.com/docs/react-native-vega/0.72/optimizing-flatlist-configuration.html +- Monitor CPU Usage: https://developer.amazon.com/docs/vega/0.21/monitor-cpu-usage.html +- Use Chrome DevTools for App Profiling: https://developer.amazon.com/docs/vega/0.21/chrome-devtools.html +- Identify UI Rendering Issues: https://developer.amazon.com/docs/vega/0.21/ui-rendering.html +- Detect Overdraw: https://developer.amazon.com/docs/vega/0.21/detect-overdraw.html + +## Attribution + +Based on official React Native performance patterns plus official Vega, React Native for Vega, Vega WebView, and Vega Studio performance documentation. diff --git a/rn-tv-performance/assets/templates/PERFORMANCE_CHECKLIST.md b/rn-tv-performance/assets/templates/PERFORMANCE_CHECKLIST.md new file mode 100644 index 0000000..30997ef --- /dev/null +++ b/rn-tv-performance/assets/templates/PERFORMANCE_CHECKLIST.md @@ -0,0 +1,62 @@ +# Performance Checklist + +Use this checklist during React Native TV performance investigations. Use Vega-specific KPI and profiling checks when the app targets Vega. + +## Scope + +- [ ] Identified app surface: generic React Native TV, React Native for Vega, Vega WebView, mixed, or unknown. +- [ ] Identified symptom: launch, UI fluidity, video, CPU, memory, rendering, network, list, or WebGL. +- [ ] Identified target device and build variant. +- [ ] Confirmed measurements are from a repeatable scenario. +- [ ] Recorded KPI iteration count, using default 3 iterations for quick checks or 30 iterations for certification-mode evidence. + +## KPI Baseline + +- [ ] Recorded cool start TTFF. +- [ ] Recorded warm start TTFF if relevant. +- [ ] Recorded cool start TTFD and verified report fully drawn marker. +- [ ] Recorded warm start TTFD if relevant. +- [ ] Recorded UI fluidity for navigation or scrolling. +- [ ] Recorded video fluidity and TTFVF for playback. +- [ ] Recorded foreground memory. +- [ ] Recorded background memory. +- [ ] Compared P90 values from the same iteration count before and after the fix. + +## Native React Native TV + +- [ ] Checked unnecessary re-renders with React DevTools or why-did-you-render in development. +- [ ] Memoized expensive stable children where evidence supports it. +- [ ] Stabilized callbacks, objects, arrays, and render functions passed to list items. +- [ ] Tuned FlatList props: `initialNumToRender`, `maxToRenderPerBatch`, `windowSize`, `getItemLayout`, and `keyExtractor`. +- [ ] Considered FlashList when list performance remains poor. +- [ ] Used thumbnail/cropped images in lists and rails. +- [ ] Applied concurrent rendering for heavy non-urgent updates. + +## Vega WebView + +- [ ] Used native SplashScreen API where applicable. +- [ ] Implemented TTFD marker only after real readiness. +- [ ] Minimized startup JavaScript and render-blocking scripts. +- [ ] Optimized network requests, redirects, cache headers, and server response time. +- [ ] Deferred noncritical data and code. +- [ ] Avoided forced reflows and heavy scroll handlers. +- [ ] Used Web Workers only for suitable heavy work and bounded worker count. +- [ ] Optimized WebGL draw calls, texture sizes, shaders, object reuse, and blocking calls. +- [ ] Used hardware-supported video codecs and avoided multiple simultaneous video elements. + +## Profiling and Rendering + +- [ ] Used KPI Visualizer for the target KPI. +- [ ] Used Activity Monitor or Recording View for CPU/memory spikes. +- [ ] Inspected flamegraphs or call trees for hot functions. +- [ ] Inspected Perfetto traces for render thread jank. +- [ ] Checked overdraw with `SHOW_OVERDRAWN=true` where rendering cost is suspected. +- [ ] Used Chrome DevTools for debug/web profiling where appropriate. + +## Verification + +- [ ] Re-ran the same KPI command and scenario after fixes. +- [ ] Compared before/after numbers. +- [ ] Confirmed no regression in adjacent flows. +- [ ] Confirmed UI remains responsive with D-pad navigation. +- [ ] Documented source, fix, measurement, and remaining risk. diff --git a/rn-tv-performance/evals/README.md b/rn-tv-performance/evals/README.md new file mode 100644 index 0000000..9793ccd --- /dev/null +++ b/rn-tv-performance/evals/README.md @@ -0,0 +1,28 @@ +# RN TV Performance Evals + +Use these evals to measure whether `rn-tv-performance` improves performance investigations compared with a baseline run without the skill. + +## Files + +| File | Purpose | +|------|---------| +| [TRIGGER_PROMPTS.md](TRIGGER_PROMPTS.md) | Trigger and non-trigger prompts for description quality | +| [TASK_SCENARIOS.md](TASK_SCENARIOS.md) | Realistic performance diagnosis and fix scenarios | +| [SCORING_RUBRIC.md](SCORING_RUBRIC.md) | Quantitative and qualitative scoring criteria | +| [RUNBOOK.md](RUNBOOK.md) | Step-by-step instructions for running and comparing evals | + +## Success Targets + +- Trigger precision: at least 9/10 trigger prompts should use the skill. +- Non-trigger specificity: at least 8/10 non-trigger prompts should avoid the skill. +- Task pass rate: at least 80% of task scenarios score 4 or 5. +- Safety: zero outputs should recommend optimizations without measurement or mix WebView and native RN guidance incorrectly. + +## Required Behaviors + +High-quality outputs should: +- Detect app surface first: generic React Native TV, React Native for Vega, Vega WebView, mixed, or unknown. +- Choose a KPI and establish a baseline before recommending fixes. +- Use official Vega tools and targets: KPI Visualizer, Activity Monitor, Recording View, Chrome DevTools, Perfetto, overdraw, and report fully drawn APIs. +- Apply platform-specific fixes for native RN or WebView. +- Re-measure the same scenario after changes. diff --git a/rn-tv-performance/evals/RUNBOOK.md b/rn-tv-performance/evals/RUNBOOK.md new file mode 100644 index 0000000..bd6e6b3 --- /dev/null +++ b/rn-tv-performance/evals/RUNBOOK.md @@ -0,0 +1,68 @@ +# Eval Runbook + +Use this runbook to compare baseline agent behavior with behavior after installing or enabling `rn-tv-performance`. + +## Preparation + +1. Use the same model and environment for baseline and skill-enabled runs. +2. Use the same prompt text in both runs. +3. Record whether the skill was available, explicitly invoked, or automatically selected. +4. Capture response text, timing, and token usage if the runner exposes those metrics. + +## Trigger Eval + +1. Run each prompt in [TRIGGER_PROMPTS.md](TRIGGER_PROMPTS.md). +2. Mark expected trigger prompts as pass when the agent uses or clearly follows the skill. +3. Mark expected non-trigger prompts as pass when the agent avoids the skill and uses a more relevant workflow. +4. Calculate: + - Trigger precision: trigger passes / 10 + - Non-trigger specificity: non-trigger passes / 10 + +## Task Eval + +1. Run each prompt in [TASK_SCENARIOS.md](TASK_SCENARIOS.md) without the skill. +2. Run the same prompt with the skill available. +3. Score both outputs with [SCORING_RUBRIC.md](SCORING_RUBRIC.md). +4. Mark scenarios scoring 4 or 5 as pass. +5. Compare: + - Overall score delta + - Pass/fail delta + - Automatic failures + - Missing guidance patterns + +## Review Template + +Use this template for each scenario: + +```markdown +Prompt ID: +Skill available: yes/no +Skill selected: yes/no/unknown +Overall score: +App surface detection: +Measurement discipline: +Official tool usage: +Fix quality: +Verification: +Automatic failure: yes/no +Key strengths: +Key gaps: +Recommended skill change: +``` + +## Iteration Rules + +Update the skill when: +- Two or more scenarios miss the same required behavior. +- Any scenario recommends optimization before measurement. +- Any scenario mixes WebView and native React Native TV guidance incorrectly. +- Trigger precision is below 9/10. +- Non-trigger specificity is below 8/10. +- Task pass rate is below 80%. + +Update the description when: +- Trigger prompts are missed despite good skill content. +- Non-trigger prompts activate because the description is too broad. + +Update references or checklist when: +- The agent uses the skill but still misses official Vega KPI targets, iteration count, release-build guidance, or validation steps. diff --git a/rn-tv-performance/evals/SCORING_RUBRIC.md b/rn-tv-performance/evals/SCORING_RUBRIC.md new file mode 100644 index 0000000..0cb1bf4 --- /dev/null +++ b/rn-tv-performance/evals/SCORING_RUBRIC.md @@ -0,0 +1,64 @@ +# Scoring Rubric + +Score each task scenario from 1 to 5. A score of 4 or 5 is a pass. + +## Overall Score + +| Score | Meaning | +|-------|---------| +| 5 | Excellent: measured, platform-aware, source-aligned, actionable, and verified | +| 4 | Good: mostly correct with minor omissions | +| 3 | Partial: useful generic advice but misses one important Vega requirement | +| 2 | Poor: generic performance advice with little platform-specific measurement | +| 1 | Harmful: recommends changes without measurement or applies wrong platform guidance | + +## Criteria + +### 1. App Surface Detection + +- 5: Identifies native RN for Vega, WebView, mixed, or unknown before fixing. +- 3: Mentions app type but does not adapt the plan. +- 1: Applies WebView or RN-only guidance blindly. + +### 2. Measurement Discipline + +- 5: Establishes baseline KPI, command/tool, target, and repeatable scenario before fixes. +- 3: Mentions measurement but lacks target or repeatability. +- 1: Optimizes without measuring. + +### 3. Official Tool Usage + +- 5: Uses appropriate Vega tools: KPI Visualizer, Activity Monitor, Recording View, Chrome DevTools, Perfetto, or overdraw. +- 3: Uses some tools but misses the best one. +- 1: Uses only generic profiling advice. + +### 4. Fix Quality + +- 5: Gives concrete, platform-specific fixes tied to evidence. +- 3: Gives plausible but broad advice. +- 1: Gives unsafe, speculative, or unrelated fixes. + +### 5. Verification + +- 5: Re-runs the same KPI/scenario and compares before/after values. +- 3: Includes basic testing without KPI comparison. +- 1: No verification. + +## Automatic Failure Conditions + +Set overall score to 1 if the output: +- Recommends optimizations without any baseline measurement. +- Reports TTFD success without explaining `useReportFullyDrawn()` or real readiness. +- Mixes WebView APIs and native RN APIs incorrectly. +- Ignores UI fluidity targets for jank/D-pad performance. +- Treats debug-build profiling as final publish-readiness evidence. + +## Aggregate Metrics + +Track: +- Trigger precision. +- Non-trigger specificity. +- Task pass rate. +- Average score. +- Automatic failures. +- Most common missing behavior. diff --git a/rn-tv-performance/evals/TASK_SCENARIOS.md b/rn-tv-performance/evals/TASK_SCENARIOS.md new file mode 100644 index 0000000..b534e76 --- /dev/null +++ b/rn-tv-performance/evals/TASK_SCENARIOS.md @@ -0,0 +1,119 @@ +# Task Scenarios + +Run each scenario with and without `rn-tv-performance`. Score both outputs using [SCORING_RUBRIC.md](SCORING_RUBRIC.md). + +## Scenario 1: Launch KPI Baseline + +Prompt: + +```text +Our React Native TV app takes a long time to launch. Tell me how to establish a baseline before changing code, and explain what extra official KPI path to use if the app targets Vega. +``` + +Expected behavior: +- Starts with app surface and platform detection. +- Establishes repeatable launch measurements before fixes. +- Mentions cool and warm start where the platform exposes those metrics. +- For Vega, uses KPI Visualizer, TTFF/TTFD targets, `useReportFullyDrawn()`, and real readiness. +- Avoids premature code optimization. + +## Scenario 2: Missing TTFD + +Prompt: + +```text +KPI Visualizer reports TTFD as -1 for our Vega app. What is likely wrong and how should we fix and verify it? +``` + +Expected behavior: +- Identifies missing Report Fully Drawn marker. +- Recommends `@amazon-devices/kepler-performance-api` and `useReportFullyDrawn()`. +- Explains cool and warm launch readiness. +- Re-runs the same KPI command after the fix. + +## Scenario 3: Slow Native Rails + +Prompt: + +```text +A React Native TV home page has large horizontal content rails. D-pad movement stutters and memory grows while scrolling. Suggest a performance investigation and fix plan. +``` + +Expected behavior: +- Treats this as native React Native TV. +- Measures UI responsiveness/fluidity and memory using the best available platform tools. +- Checks re-renders, list configuration, image sizes, and render functions. +- Covers FlatList/FlashList tradeoffs. +- Adds Vega KPI Visualizer guidance if the target is Vega. +- Re-measures same scenario. + +## Scenario 4: WebView Startup + +Prompt: + +```text +A Vega WebView app shows a blank screen for several seconds before the home page appears. It loads several scripts and API calls at startup. What should we do? +``` + +Expected behavior: +- Treats this as WebView. +- Uses native SplashScreen and TTFD marker guidance. +- Optimizes requests, compression, caching, redirects, async/defer scripts, code splitting, and noncritical data. +- Measures with KPI Visualizer and Chrome DevTools. + +## Scenario 5: Web Workers + +Prompt: + +```text +Our Vega WebView app uses many Web Workers to preload images when users move quickly through a carousel. The UI freezes. Diagnose this. +``` + +Expected behavior: +- Explains worker floods and main-thread response overhead. +- Uses worker count guidance based on `navigator.hardwareConcurrency`. +- Recommends batching, transferable objects, pooling, termination, and caching. +- Measures UI fluidity before/after. + +## Scenario 6: WebGL and Video + +Prompt: + +```text +A Vega WebView app uses WebGL animations over video playback and video fluidity is below 99%. What should we inspect? +``` + +Expected behavior: +- Measures video fluidity and TTFVF. +- Checks hardware-supported codecs and video element usage. +- Avoids canvas video for WebGL. +- Reduces draw calls, texture sizes, shaders, and GPU contention. +- Re-tests playback scenario. + +## Scenario 7: CPU Hot Functions + +Prompt: + +```text +Activity Monitor shows CPU spikes during app launch. How should I use Vega tooling to find and prioritize fixes? +``` + +Expected behavior: +- Uses Activity Monitor, launch mode recording, Recording View, hot functions, flamegraphs, call tree. +- Mentions release variant for hot functions. +- Uses source maps where needed. +- Prioritizes high self/total CPU time in app code. + +## Scenario 8: Overdraw + +Prompt: + +```text +Our Vega app has jank on a dense detail screen with layered backgrounds and translucent overlays. How can we detect and fix overdraw? +``` + +Expected behavior: +- Uses `SHOW_OVERDRAWN=true`. +- Explains color meanings. +- Recommends removing unnecessary backgrounds, flattening view hierarchy, and reducing transparency. +- Re-runs UI fluidity or rendering validation. diff --git a/rn-tv-performance/evals/TRIGGER_PROMPTS.md b/rn-tv-performance/evals/TRIGGER_PROMPTS.md new file mode 100644 index 0000000..dfdc070 --- /dev/null +++ b/rn-tv-performance/evals/TRIGGER_PROMPTS.md @@ -0,0 +1,31 @@ +# Trigger and Non-Trigger Prompts + +Expected behavior: +- Trigger prompts should use `rn-tv-performance`. +- Non-trigger prompts should not use `rn-tv-performance` unless the user adds performance context. + +## Trigger Prompts + +1. "Our React Native TV app has slow cold startup. Help measure a baseline before optimizing." +2. "The KPI Visualizer shows UI fluidity below 99%. What should we inspect?" +3. "React Native TV FlatList rails feel sluggish with D-pad navigation." +4. "Vega WebView launch is slow because of network and JavaScript loading." +5. "How do I use Activity Monitor to find CPU hot functions in a Vega app?" +6. "Video playback drops frames in our Vega WebView app." +7. "TTFD is showing -1 in the Vega KPI Visualizer." +8. "Help optimize a React Native TV search screen with useTransition." +9. "Chrome DevTools shows long WebView tasks during page load on Vega." +10. "Detect and fix overdraw in a Vega TV app." + +## Non-Trigger Prompts + +1. "Migrate a Vega app to a multi-platform React Native monorepo." +2. "Audit VoiceView labels and D-pad accessibility for a Vega app." +3. "Create a GitHub Actions workflow for triage summaries." +4. "Update README installation instructions." +5. "Explain React Native accessibilityRole and aria-label." +6. "Set up Yarn workspaces for Android TV and Apple TV." +7. "Fix a TypeScript type error in a reducer." +8. "Add captions to a media player for accessibility." +9. "Create a release note for a Fire TV app." +10. "Review license text for a new skill." diff --git a/rn-tv-performance/references/KPI_MEASUREMENT.md b/rn-tv-performance/references/KPI_MEASUREMENT.md new file mode 100644 index 0000000..cb48c88 --- /dev/null +++ b/rn-tv-performance/references/KPI_MEASUREMENT.md @@ -0,0 +1,116 @@ +# KPI Measurement + +Use this reference before making performance changes. Establish a baseline, choose the right KPI, and re-run the same measurement after each fix. + +Primary source: https://developer.amazon.com/docs/vega/0.21/measure-app-kpis.html + +## Required Baseline + +Before optimizing, record: +- App surface: native React Native for Vega, Vega WebView, or mixed. +- Build variant. Prefer release for publish-readiness measurements. +- Device type and whether it is a physical device or virtual device. +- App name from the default interactive component in `manifest.toml`. +- Scenario: cool start, warm start, UI interaction, video playback, foreground memory, or background memory. +- KPI command and output. + +## Official KPI Targets + +| KPI | Scenario | Target | +|-----|----------|--------| +| TTFF | Cool start app launch | less than 1.5 s | +| TTFF | Warm start app launch | less than 0.5 s | +| TTFD | Cool start app launch | less than 8.0 s | +| TTFD | Warm start app launch | less than 1.5 s | +| Foreground Memory | App active | less than 400 MiB | +| Background Memory | App backgrounded | less than 150 MiB | +| Video Fluidity | Video playback | greater than 99% | +| TTFVF | Video playback startup | less than 2500 ms | +| UI Fluidity | UI interaction | 99% or higher | + +## Commands + +Default cool-start latency: + +```bash +vega exec perf kpi-visualizer --app-name= +``` + +Set an explicit iteration count: + +```bash +vega exec perf kpi-visualizer --app-name= --iterations +``` + +UI fluidity: + +```bash +vega exec perf kpi-visualizer --app-name= --kpi ui-fluidity --test-scenario +``` + +Video fluidity: + +```bash +vega exec perf kpi-visualizer --app-name= --kpi video-fluidity --test-scenario +``` + +Foreground memory: + +```bash +vega exec perf kpi-visualizer --app-name= --kpi foreground-memory --test-scenario +``` + +Supported scenarios include: +- `cool-start-latency` +- `warm-start-latency` +- `ui-fluidity` +- `foreground-memory` +- `background-memory` +- `video-fluidity` + +## Iterations and Aggregation + +Official Vega KPI Visualizer guidance says: +- The visualizer performs three iterations for selected KPIs by default. +- The visualizer window shows the P90 (90th percentile) value calculated from three test iterations. +- The CLI supports `--iterations ` to set how many times to run the test. +- Certification mode uses 30 iterations with 90th percentile aggregation. + +Use the default 3 iterations for quick local investigation. Use more iterations when results are noisy, when validating a risky optimization, or when preparing release/certification evidence. Keep the iteration count, device, build variant, scenario, and app state the same when comparing before/after results. + +## Fully Drawn Marker + +TTFD requires the app to report when it is ready for user interaction. + +Use `@amazon-devices/kepler-performance-api` and `useReportFullyDrawn()` when the app has completed the work required for user interaction. + +Important rules: +- Report after the first render only when required async work is complete. +- Report again for warm launch when the app returns from background to foreground and is ready. +- If TTFD shows `-1`, check whether the app failed to call the Report Fully Drawn API. +- Do not report too early just to improve the metric; the marker must match real readiness. + +## Micro KPIs + +Use micro KPIs to explain TTFD: +- JavaScript bundle load time. +- Network calls time. +- Other app-defined startup stages. + +When TTFD is high, split startup work into measured phases before optimizing. + +## Measurement Quality + +Good measurement: +- Uses repeatable app state. +- Records the iteration count. +- Uses a scenario that matches real user behavior. +- Uses the same command before and after the fix. +- Captures launch, foreground, background, and playback where relevant. +- Notes unavailable data and why it is missing. + +Avoid: +- Comparing debug builds to release builds. +- Comparing different devices or scenarios. +- Optimizing without a baseline. +- Treating one run as conclusive when results are noisy. diff --git a/rn-tv-performance/references/PROFILING_AND_RENDERING.md b/rn-tv-performance/references/PROFILING_AND_RENDERING.md new file mode 100644 index 0000000..4455b6e --- /dev/null +++ b/rn-tv-performance/references/PROFILING_AND_RENDERING.md @@ -0,0 +1,137 @@ +# Profiling and Rendering + +Use this reference to choose Vega Studio and platform tools for CPU, memory, rendering, and jank investigations. + +Primary sources: +- Monitor CPU Usage: https://developer.amazon.com/docs/vega/0.21/monitor-cpu-usage.html +- Chrome DevTools for App Profiling: https://developer.amazon.com/docs/vega/0.21/chrome-devtools.html +- Identify UI Rendering Issues: https://developer.amazon.com/docs/vega/0.21/ui-rendering.html +- Detect Overdraw: https://developer.amazon.com/docs/vega/0.21/detect-overdraw.html + +## Tool Selection + +Use: +- **Vega App KPI Visualizer** for TTFF, TTFD, UI fluidity, video fluidity, and memory KPIs. +- **Activity Monitor** for real-time CPU and memory graphs on connected devices. +- **Recording View** for timelines, traces, hot functions, flamegraphs, and call trees. +- **CPU Profiler on App Launch** for launch bottlenecks. +- **Chrome DevTools** for React Native for Vega debug profiling and WebView web profiling. +- **Perfetto traces** for render thread and system-level jank analysis. +- **Overdraw detection** for unnecessary GPU work from repeated pixel rendering. + +## Activity Monitor + +Use Activity Monitor to observe: +- CPU utilization over time. +- Memory usage over time. +- App crashes and restart gaps. +- Correlation between user actions and spikes. + +Important: +- Vega Virtual Device does not support Vega Studio Performance Tools. +- Release variant is required for some profiling data such as hot functions. +- CPU utilization of 400% means full usage across four cores. + +## Recording View + +Use Recording View to inspect: +- Timeline windows. +- Lifecycle traces: launch, foreground, background, crash. +- Thread states. +- Hot functions. +- Flamegraphs. +- Call tree sorted by total or self CPU time. + +Prioritize: +- Wide flamegraph nodes. +- High self CPU time in app code. +- Hot functions that align with KPI drops. +- JS thread, UI thread, or render thread contention. + +## CLI CPU Profiling + +Use CPU profiling with Activity Monitor when you need hot functions and source mapping. + +```bash +vega exec perf activity-monitor \ + --record-cpu-profiling \ + --app-name= \ + --sourcemap-file-path= +``` + +Generated output can include: +- CPU and memory utilization. +- Process state. +- Hot functions summary. +- Trace event file for Chrome DevTools. + +## UI Rendering Jank + +Jank means stuttering or lag in UI rendering and input response. + +To investigate: +1. Run a UI fluidity test in KPI Visualizer. +2. Locate drops in the fluidity graph. +3. Open the CPU profile graph and flamegraph near the drop. +4. Inspect Perfetto traces. +5. Examine `Toolkit/Render thread` for long or recurring events. + +Common causes: +- Frame size mismatch with display resolution. +- Frame overdraw. +- Heavy render thread work. +- Heavy JavaScript work during interactions. +- Expensive layout or image decode during navigation. + +## Overdraw Detection + +Overdraw means rendering the same pixel multiple times in one frame. + +Enable overdraw detection by configuring launch options with: + +```text +SHOW_OVERDRAWN=true +``` + +Color guide: +- True color: no overdraw. +- Blue: overdrawn 1 time. +- Green: overdrawn 2 times. +- Pink: overdrawn 3 times. +- Red: overdrawn 4 or more times. + +Fixes: +- Remove unnecessary backgrounds. +- Flatten the view hierarchy. +- Reduce transparency. +- Avoid stacked full-screen overlays when not needed. + +## Chrome DevTools + +Use Chrome DevTools: +- In Debug builds for React Native for Vega profiling. +- For WebView web content profiling with Performance and Network tabs. +- With source maps where available. + +Do not treat debug performance as publish-ready. Use release builds and Vega performance tools for final validation. + +## Custom Tracing + +React Native Systrace can add custom trace markers for suspected expensive code paths. + +Use sparingly: +- Traces can add overhead. +- Over-instrumentation can create false positives. +- Add markers only around code paths being investigated. + +## Report Format + +For each performance finding, include: +- Symptom. +- KPI and baseline. +- Tool used. +- Evidence file or observation. +- Suspected root cause. +- Code or asset fix. +- After-measurement. +- Remaining risk. diff --git a/rn-tv-performance/references/REACT_NATIVE_TV_PERFORMANCE.md b/rn-tv-performance/references/REACT_NATIVE_TV_PERFORMANCE.md new file mode 100644 index 0000000..16f5b12 --- /dev/null +++ b/rn-tv-performance/references/REACT_NATIVE_TV_PERFORMANCE.md @@ -0,0 +1,116 @@ +# React Native TV Performance + +Use this reference for native React Native TV app surfaces. Apply the general guidance to Android TV, Apple TV, and other React Native TV targets, then use the Vega notes when the app is React Native for Vega. + +Primary sources: +- App Performance Best Practices: https://developer.amazon.com/docs/vega/0.21/best_practices.html +- Concurrent Rendering: https://developer.amazon.com/docs/vega/0.21/concurrent-rendering.html +- Optimizing Flatlist Configuration: https://developer.amazon.com/docs/react-native-vega/0.72/optimizing-flatlist-configuration.html +- Investigate Component Re-rendering Issues: https://developer.amazon.com/docs/vega/0.21/investigate-component-re-render.html + +## First Checks + +Before changing code: +- Confirm this is a native React Native TV surface, not WebView content. +- Identify the user-visible symptom: launch, focus/D-pad lag, list scroll, rail navigation, animation, video, CPU, or memory. +- Measure the relevant KPI. +- Inspect React renders, JS thread, render thread, and memory before choosing a fix. + +## Re-render Optimization + +Use `memo`, `useCallback`, and `useMemo` when they reduce measured wasted renders. + +Good candidates: +- List items. +- Buttons and tiles. +- Expensive computed props. +- Components with stable props. +- Children of screens that use concurrent rendering. + +Avoid: +- Adding memoization everywhere without evidence. +- Omitting hook dependencies and creating stale closures. +- Passing inline functions, inline objects, or regenerated arrays into memoized children. + +Tools: +- React DevTools Profiler with "Record why each component rendered" when available. +- `why-did-you-render` in development only. Do not ship it in production. + +## Lists and Rails + +Prefer virtualized lists for large rows, rails, and catalogs. + +For `FlatList`: +- Tune `initialNumToRender` so the initial viewport is filled without extra work. +- Tune `maxToRenderPerBatch`; higher values reduce blanks but can block JavaScript event processing. +- Tune `windowSize`; larger values reduce blank areas but increase memory. +- Use `getItemLayout` when item size is fixed. +- Use `keyExtractor` or stable keys. +- Move `renderItem` out of JSX and wrap with `useCallback`. +- Keep item components basic and light. +- Use thumbnails/cropped assets for list items. + +For `FlashList`: +- Use FlashList as a performant alternative when it is supported by the project and target platform. +- Set `estimatedItemSize`. +- Account for item recycling; avoid item-local state that carries across recycled items. +- Remove unnecessary `key` props inside item components. +- Do not expect FlatList-only props such as `windowSize`, `getItemLayout`, or `maxToRenderPerBatch` to apply to FlashList. + +## Images + +Use right-sized image assets for TV surfaces. React Native for Vega includes native caching mechanisms in its `Image` implementation; other React Native TV targets may need project-specific image caching. + +Use: +- Cropped or thumbnail-sized images in lists and rails. +- Multiple asset sizes for different use cases. +- Detail-size images only on detail screens. + +Avoid: +- Full-resolution artwork in list tiles. +- Excessive image effects in scrollable surfaces. +- Re-decoding large assets during D-pad navigation. + +## Concurrent Rendering + +Use React 18 concurrent rendering when heavy updates block urgent interactions. + +Use `useTransition` when the component controls both urgent and non-urgent state, such as: +- Search input stays responsive while results update. +- Navigation stays responsive while a large screen update happens. +- Large list updates are lower priority than user input. + +Use `useDeferredValue` when expensive work comes from props or external values that the component does not directly control. + +Use `Suspense` and `lazy` for heavy components or screens that can be code-split. + +Always pair concurrent rendering with memoization for expensive children, otherwise parent renders can erase the benefit. + +## Vega Notes + +For React Native for Vega: +- Use official Vega KPI Visualizer targets and commands when release-readiness evidence is needed. +- Use Vega Activity Monitor, Recording View, and CPU profiler for CPU and memory issues. +- React Native for Vega includes native caching mechanisms in its `Image` implementation. +- Vega supports Shopify FlashList as a performant alternative. +- Re-test D-pad navigation and focus movement on the target Vega device after performance fixes. + +## CPU and Memory + +If CPU is high: +- Identify hot functions with Activity Monitor or CPU profiler. +- Look for repeated computations, unnecessary state updates, heavy effects, and large render trees. +- Reduce work on focus movement and D-pad handlers. + +If memory is high: +- Check image sizes, mounted list window size, cached data, background state, and retained references. +- Measure foreground and background memory separately. + +## Validation + +After a fix: +- Re-run the same KPI command and scenario. +- Check UI fluidity stays within the target for the platform. For Vega, target 99% or higher. +- Re-test D-pad navigation and focus movement on the target device. +- Confirm memory did not regress. +- Capture before/after values and the code path changed. diff --git a/rn-tv-performance/references/WEBVIEW_PERFORMANCE.md b/rn-tv-performance/references/WEBVIEW_PERFORMANCE.md new file mode 100644 index 0000000..38802be --- /dev/null +++ b/rn-tv-performance/references/WEBVIEW_PERFORMANCE.md @@ -0,0 +1,107 @@ +# Vega WebView Performance + +Use this reference for Vega apps that render HTML, JavaScript, CSS, media, WebGL, or hosted content inside Vega WebView. + +Primary sources: +- WebView performance best practices: https://developer.amazon.com/docs/vega/0.21/webview-app-performance-best-practices.html +- Web Worker best practices: https://developer.amazon.com/docs/vega/0.21/webview-web-workers-best-practices.html +- WebGL best practices: https://developer.amazon.com/docs/vega/0.21/webview-webgl-best-practices.html + +## First Checks + +Before fixing: +- Confirm WebView usage with `@amazon-devices/webview`, ``, `webview.html`, hosted HTML, or `ReactNativeWebView`. +- Measure TTFF, TTFD, UI fluidity, memory, video fluidity, or TTFVF as appropriate. +- Use Vega KPI Visualizer plus Chrome DevTools when possible. +- Separate native wrapper issues from web content issues. + +## Launch Performance + +Use the native SplashScreen API for immediate visual feedback at launch. + +For WebView TTFD: +- Use `useReportFullyDrawn()` after the web app has loaded and is ready for user interaction. +- In WebView, the `onLoad` callback can be a good marker only if the loaded URL represents the actual ready screen. +- Report fully drawn for warm starts when the app returns from background and is ready. +- Do not report fully drawn when the user cannot interact yet. + +## Network and Server Response + +Optimize startup and navigation by: +- Minimizing HTTP requests. +- Combining CSS/JavaScript files when it is logically safe. +- Removing unnecessary third-party dependencies. +- Enabling Gzip or Brotli for text assets. +- Using HTTP/2 or HTTP/3 where available. +- Reducing redirects. +- Using CDN caching and appropriate `Cache-Control` headers. +- Reducing TTFB through backend/API optimization. +- Combining API calls when appropriate and avoiding redundant requests. +- Deferring noncritical data and loading the first useful content first. + +## JavaScript and Rendering + +Keep startup JavaScript small: +- Execute only essential code during launch. +- Use `async` or `defer` for noncritical scripts. +- Use dynamic imports and code splitting. +- Avoid long tasks on the main thread. +- Move heavy computation off the main thread when appropriate. +- Debounce or throttle expensive input and scroll handlers. +- Use `requestAnimationFrame` for visual updates. + +For smooth rendering: +- Avoid synchronous layout reads/writes in scroll handlers. +- Avoid forced reflows from repeated `offsetTop`, `scrollHeight`, or layout property access. +- Render only visible or necessary content in large scrollable surfaces. +- Prefer transform-based animations over layout-changing properties such as `top`, `left`, or `width`. + +## Web Workers + +Use Web Workers for computationally heavy work that would block the main thread. + +Vega guidance: +- Restrict worker count based on device cores. +- Conservative formula: `navigator.hardwareConcurrency - 2`. +- Upper guideline: `(2 * navigator.hardwareConcurrency) + 1`. +- Reduce data passed between workers and the main thread. +- Use transferable objects for large data. +- Terminate unneeded workers. +- Reuse workers or worker pools. +- Cache expensive results. +- Avoid worker floods during fast navigation or image preloading. + +## WebGL + +Vega WebView supports WebGL 2.0 and WASM. It does not currently support WebGPU. + +Optimize WebGL by: +- Avoiding canvas video; use HTML video or web video player libraries instead. +- Reducing draw calls; keep below 500 draw calls per frame when possible. +- Drawing from `requestAnimationFrame`. +- Optimizing texture sizes, compressed formats, mipmaps, and texture atlases. +- Simplifying shaders and complex algorithms. +- Using LOD, frustum culling, or occlusion culling. +- Preallocating memory and reusing objects. +- Reducing state changes. +- Avoiding blocking calls like `getError()` and `getParameter()` on the main thread. + +## Video and Hardware Decode + +For media-heavy WebView apps: +- Use `video` elements with hardware-supported codecs. +- Prefer H.264, H.265, VP8, or VP9 where supported by the target device. +- Avoid software-only formats that increase CPU load. +- Avoid multiple simultaneous `video` elements. +- Reuse video elements where possible. +- For MSE, ensure `MediaSource.addSourceBuffer()` codec strings match hardware-supported formats. +- Use `MediaSource.isTypeSupported()` and `HTMLVideoElement.canPlayType()` before playback setup. +- Avoid heavy CSS/WebGL animations during playback. + +## Validation + +After a fix: +- Re-run the same KPI Visualizer command. +- Use Chrome DevTools Performance and Network tabs for detailed web hotspots. +- Test with realistic network and CPU restrictions when reproducing in Desktop Chrome. +- Confirm TTFF, TTFD, UI fluidity, memory, video fluidity, or TTFVF improved without regressing user-visible behavior.