Skip to content

test: split the Android platform test aggregation and share the scripted adb stub#1103

Merged
thymikee merged 4 commits into
mainfrom
claude/android-test-split
Jul 6, 2026
Merged

test: split the Android platform test aggregation and share the scripted adb stub#1103
thymikee merged 4 commits into
mainfrom
claude/android-test-split

Conversation

@thymikee

@thymikee thymikee commented Jul 4, 2026

Copy link
Copy Markdown
Member

Summary

Companion to #1102, for the Android offender AGENTS.md names directly ("platform index.test.ts … shrink opportunistically"). The 2,735-line src/platforms/android/__tests__/index.test.ts splits along its already well-factored source modules — every test moved verbatim, 92 tests before and after:

File Tests Mirrors
ui-hierarchy.test.ts 22 ui-hierarchy.ts
app-lifecycle-install.test.ts 13 app-lifecycle.ts (install/resolve/infer/component parsing)
app-lifecycle-open.test.ts 19 app-lifecycle.ts (open/close, deep links, launch args, TV, fallback resolve-activity)
input-actions.test.ts 11 input-actions.ts (type/fill/swipe/scroll/rotate)
settings.test.ts 14 settings.ts
notifications.test.ts / app-parsers.test.ts 2 / 1 notifications.ts / app-parsers.ts
appended to existing device-input-state.test.ts 10 device-input-state.ts (keyboard state/dismiss)

adb-mock consistency fix folded in: the monolith carried a local withMockedAdb fork because it needs scripted per-subcommand adb responses, which the shared arg-recorder helper can't express. That fork now lives in src/__tests__/test-utils/mocked-binaries.ts as withScriptedAdb, next to the recorder withMockedAdb, and hands each callback a fresh copy of the shared ANDROID_EMULATOR fixture (which the local fork had duplicated field-for-field).

The fresh copy is load-bearing, and the split caught a real latent hazard: the Android TV test mutated the callback's device (device.target = 'tv'). The old per-call object literal absorbed that silently; with a shared fixture the mutation leaked into the following test and flipped its launch to LEANBACK. The helper now clones per call and the TV test builds { ...device, target: 'tv' } instead of mutating.

Per the discussion: no new e2e scenario — the real-emulator Android workflow (settings replay + snapshot-helper smoke) stays the e2e layer, and these integration tests keep covering install/fill flows against scripted stubs.

Touched files: 10 (1 deleted, 7 created, device-input-state.test.ts extended, mocked-binaries.ts).

Validation

All 92 relocated tests pass in their new homes (including the previously-leaking LEANBACK/fallback pair, which now passes in both orderings), and the full suites are green: 349 test files / 3248 unit tests, smoke 13/13, check:quick (oxlint + tsgo) clean, fallow audit vs origin/main clean. Blocks were moved verbatim except the one withMockedAdbwithScriptedAdb rename and the TV-test mutation fix described above.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS


Generated by Claude Code

…ted adb stub

AGENTS.md names the platform index.test.ts aggregations as offenders to
shrink opportunistically; this splits the 2,735-line Android one along
its (already well-factored) source modules, every test moved verbatim
(92 tests before and after):

- ui-hierarchy.test.ts (22): parseUiHierarchy/androidUiNodes
- app-lifecycle-install.test.ts (13): install/resolve/infer/launch
  component parsing
- app-lifecycle-open.test.ts (19): open/close, deep links, launch args,
  TV category, fallback resolve-activity
- input-actions.test.ts (11): type/fill/swipe/scroll/rotate
- settings.test.ts (14): appearance/clear-app-state/fingerprint/
  permissions
- notifications.test.ts (2), app-parsers.test.ts (1)
- keyboard state/dismiss tests (10) appended to the existing
  device-input-state.test.ts

Consistency fix folded in: the file carried a local withMockedAdb fork
because it needs scripted per-subcommand adb responses, which the shared
arg-recorder helper cannot express. The fork now lives in
src/__tests__/test-utils/mocked-binaries.ts as withScriptedAdb next to
withMockedAdb, and hands each call a fresh copy of the shared
ANDROID_EMULATOR fixture.

The copy matters: the Android TV test mutated the callback's device
(device.target = 'tv'), which the old per-call object literal absorbed
silently. With a shared fixture that mutation leaked into the next test
and flipped its launch to LEANBACK. The helper now clones per call and
the TV test builds { ...device, target: 'tv' } instead of mutating.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.5 MB 1.5 MB 0 B
JS gzip 491.3 kB 491.3 kB 0 B
npm tarball 590.3 kB 590.3 kB +9 B
npm unpacked 2.1 MB 2.1 MB +54 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 25.9 ms 25.6 ms -0.3 ms
CLI --help 49.4 ms 49.4 ms +0.0 ms

Top changed chunks: no changes in the largest emitted chunks.

@thymikee

thymikee commented Jul 4, 2026

Copy link
Copy Markdown
Member Author

Blocking review finding: the split lets the scripted-adb tests run in parallel across files while they still mutate global process state (PATH and AGENT_DEVICE_TEST_ARGS_FILE). That makes the new files interfere with each other when run together.

Repro from the PR branch:

pnpm exec vitest run src/platforms/android/__tests__/app-lifecycle-install.test.ts src/platforms/android/__tests__/app-lifecycle-open.test.ts src/platforms/android/__tests__/app-parsers.test.ts src/platforms/android/__tests__/device-input-state.test.ts src/platforms/android/__tests__/input-actions.test.ts src/platforms/android/__tests__/notifications.test.ts src/platforms/android/__tests__/settings.test.ts src/platforms/android/__tests__/ui-hierarchy.test.ts

That run failed 10 tests with adb exit/timeouts. Running one of the failing files alone passes:

pnpm exec vitest run src/platforms/android/__tests__/app-lifecycle-open.test.ts

So the monolith split changed execution semantics: the old single file effectively serialized the env-mutating adb stubs, but the new file split allows concurrent mutation. Please make the scripted-adb helper/tests safe under Vitest's file parallelism, or explicitly serialize this group if that is the intended contract, then rerun the grouped command above.

Review follow-up for the android index.test.ts split: the monolith
implicitly serialized the env-mutating adb-stub tests (PATH,
AGENT_DEVICE_TEST_ARGS_FILE) in one worker, and the split let vitest
run them across parallel files. Make the contract explicit:

- new android-adb vitest project runs the six scripted-adb test files
  in a single fork (singleFork), keeping the pre-split execution
  semantics; ui-hierarchy and app-parsers stay in the parallel unit
  project (pure parsing, no env mutation)
- test/test:unit scripts run both projects
- the five slow-test ratchet pins that referenced index.test.ts keys
  now point at the split file names, so the pinned real-time offenders
  keep their exemption instead of failing at 2x budget under load; the
  reporter's own pinned-key fixture updated to match

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS

thymikee commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

Fixed in 5a5dcd8 — the serialization contract is now explicit, plus the split had silently invalidated the slow-test ratchet pins, which is likely what your 10 failures actually were:

  • Five PINNED_SLOW_UNIT_TESTS entries still keyed index.test.ts :: … (the fillAndroid/installAndroidApp real-time offenders), so after the split those tests lost their exemption and fail at 2× budget under parallel load. The pins now point at the split file names (and the reporter's own pinned-key fixture is updated).
  • The six env-mutating scripted-adb files (app-lifecycle-install, app-lifecycle-open, device-input-state, input-actions, notifications, settings) now run in a dedicated android-adb vitest project with singleFork, restoring the monolith's serialized execution semantics as an explicit contract; ui-hierarchy and app-parsers are pure parsing and stay in the parallel unit project. test/test:unit run both projects.

Reran your grouped command: 8 files / 94 tests pass, with the six serialized files visibly running under |android-adb| and the two pure ones under |unit|. Full test:unit (349 files / 3248 tests) + smoke + fallow also green.


Generated by Claude Code

@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-06 09:23 UTC

@thymikee

thymikee commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

Follow-up after the latest fix: the original grouped-command blocker is resolved for me too, but the new serialization contract is currently expressed through a removed Vitest 4 config path.

The grouped run now starts with:

DEPRECATED  `test.poolOptions` was removed in Vitest 4. All previous `poolOptions` are now top-level options.

This matters because the PR depends on serialization to preserve the old monolithic index.test.ts execution semantics, and vitest.config.ts adds that via poolOptions.forks.singleFork. Please switch the android-adb project to the supported Vitest 4 config surface, for example project-level fileParallelism: false with the default forks pool, then rerun the grouped command. After that I think this is ready.

@thymikee

thymikee commented Jul 6, 2026

Copy link
Copy Markdown
Member Author

Follow-up from the latest review: the Vitest 4 serialization issue is only part of the blocker.

  • vitest.config.ts: poolOptions.forks.singleFork is ignored on Vitest 4, so the intended android-adb serialization is not enforced. Please move this to supported Vitest 4 project-level config, such as maxWorkers: 1 / fileParallelism: false as appropriate.
  • Keep file isolation unless the Android app-resolution cache is explicitly cleared. With serialization plus isolate: false, androidAppResolutionCache in src/platforms/android/app-lifecycle.ts can leak between the split files; shuffled runs can fail depending on cache population/invalidation order.
  • The split also means vitest run --project unit skips the Android ADB files with exit 0. Please update docs/scripts or add a guard so maintainers do not accidentally treat that as full unit coverage.

Keeping this as not merge-ready until serialization and isolation/coverage semantics are explicit.

@thymikee

thymikee commented Jul 6, 2026

Copy link
Copy Markdown
Member Author

Addressed the remaining review blockers in e54d617.

Changes:

  • Replaced the ignored Vitest 4 poolOptions.forks.singleFork path with project-level fileParallelism: false, maxWorkers: 1, and explicit isolate: true for android-adb.
  • Renamed the broad project from unit to unit-core and updated test / test:unit to run unit-core + android-adb, so stale vitest run --project unit now fails instead of silently skipping the Android ADB files.

Validation:

  • pnpm exec vitest run src/platforms/android/__tests__/app-lifecycle-install.test.ts src/platforms/android/__tests__/app-lifecycle-open.test.ts src/platforms/android/__tests__/app-parsers.test.ts src/platforms/android/__tests__/device-input-state.test.ts src/platforms/android/__tests__/input-actions.test.ts src/platforms/android/__tests__/notifications.test.ts src/platforms/android/__tests__/settings.test.ts src/platforms/android/__tests__/ui-hierarchy.test.ts passed: 8 files / 94 tests.
  • pnpm exec vitest run --project unit now exits with No projects matched the filter "unit".
  • pnpm format passed.
  • pnpm check:tooling passed.
  • pnpm check:unit passed: 349 Vitest files / 3252 tests, then smoke tests passed with the expected live-web skip.

Remaining risk: none beyond normal CI confirmation.

@thymikee

thymikee commented Jul 6, 2026

Copy link
Copy Markdown
Member Author

Follow-up pushed in be8bd77.

Changes:

  • Updated AGENTS.md PR readiness guidance to stop naming stale --project unit and instead point agents to pnpm test:unit or direct vitest run --project unit-core --project android-adb.
  • No test split or serialization code changed.

Validation:

  • git diff --check passed.
  • Docs-only guidance change; no test gate rerun needed.

@thymikee

thymikee commented Jul 6, 2026

Copy link
Copy Markdown
Member Author

Review pass after be8bd77: no remaining actionable blockers found.

The Android ADB split now uses supported Vitest 4 serialization (fileParallelism: false, maxWorkers: 1) with explicit file isolation, test / test:unit run both unit-core and android-adb, and stale --project unit usage now fails instead of silently skipping the serialized files. The follow-up AGENTS.md guidance now points agents at pnpm test:unit or the explicit unit-core + android-adb projects.

Validation reported on the PR covers the grouped Android ADB run, stale-project guard, pnpm format, pnpm check:tooling, and pnpm check:unit; CI is green. Marking this ready for human maintainer judgment.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Jul 6, 2026
@thymikee thymikee merged commit f9721e7 into main Jul 6, 2026
22 checks passed
@thymikee thymikee deleted the claude/android-test-split branch July 6, 2026 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants