Skip to content

feat(ci): add devbox plugin integration testing#1241

Closed
abueide wants to merge 7 commits intomasterfrom
feat/rn-devbox-plugin
Closed

feat(ci): add devbox plugin integration testing#1241
abueide wants to merge 7 commits intomasterfrom
feat/rn-devbox-plugin

Conversation

@abueide
Copy link
Copy Markdown
Contributor

@abueide abueide commented Apr 29, 2026

Summary

Adds comprehensive CI testing for mobile-devtools devbox plugin integration in E2E examples.

Changes

1. New Workflow: devbox-plugin-integration.yml

Tests the devbox plugin integration across both E2E examples (E2E-compat and E2E-latest) on both platforms (Android and iOS).

Triggers:

  • Pull Requests to feat/rn-devbox-plugin when devbox files change
  • Weekly Schedule (Mondays 10am UTC) to catch upstream plugin changes
  • Manual Trigger with options to test specific examples/platforms
  • Workflow Call for use as release prerequisite

2. Test Coverage

Validation Job:

  • Validates devbox.json configuration
  • Checks mobile-devtools plugin is included
  • Runs on ubuntu-latest (fast, cheap)

Android Build Tests:

  • Tests E2E-compat Android build
  • Tests E2E-latest Android build
  • Verifies Android tools availability
  • Runs yarn install and Gradle build
  • Runs on ubuntu-latest

iOS Build Tests:

  • Tests E2E-compat iOS build
  • Tests E2E-latest iOS build
  • Verifies iOS tools availability
  • Runs yarn install, pod install, and Xcode build
  • Runs on macos-14 (required for Xcode)

Summary Job:

  • Aggregates all test results
  • Fails if any job fails
  • Provides clear feedback on which tests failed

3. Benefits

Early Detection - Catches breaking changes in mobile-devtools plugin before they affect developers

Example Parity - Ensures both E2E-compat and E2E-latest work with the plugin

Release Safety - Can be called as prerequisite to prevent releasing incompatible SDK

Clear Feedback - Matrix-style testing shows exactly which example/platform combination fails

Cost Optimization - Only runs iOS tests on macos-14 runners (expensive), uses ubuntu-latest where possible

Testing

Local Testing

# Tested E2E-compat
cd examples/E2E-compat
devbox version  # ✓ 0.17.2
devbox run install  # ✓ Yarn install successful
devbox run --list  # ✓ All scripts available

# Tested E2E-latest
cd examples/E2E-latest  
devbox run install  # ✓ Yarn install successful

CI Testing

This PR will trigger the workflow and validate:

  • ✓ Configuration validation
  • ✓ Android E2E-compat build
  • ✓ Android E2E-latest build
  • ✓ iOS E2E-compat build
  • ✓ iOS E2E-latest build

Integration

Use as Release Prerequisite

To use this as a release prerequisite, add to .github/workflows/release.yml:

jobs:
  devbox-integration:
    name: Devbox Plugin Integration
    uses: ./.github/workflows/devbox-plugin-integration.yml
    with:
      example: 'all'
  
  release-production:
    needs: [ci, devbox-integration]  # Add this dependency
    # ... rest of release job

Manual Testing

Developers can trigger manually via Actions tab:

  1. Go to Actions → Devbox Plugin Integration Tests
  2. Click "Run workflow"
  3. Select example (E2E-compat, E2E-latest, all)
  4. Select platform (android, ios, both)

Architecture Decisions

Separate Jobs per Example/Platform

  • Makes it clear which combination fails
  • Allows parallel execution
  • Reduces total runtime

Conditional Execution

  • Only runs relevant jobs based on trigger
  • Saves CI minutes on scheduled/manual runs

Pure Shell Mode

  • Uses devbox run --pure to ensure reproducible builds
  • Isolates from host environment

Node.js Caching

  • Caches yarn dependencies via actions/setup-node
  • Speeds up subsequent runs

Known Issues

Android SDK Nix Build Warning

  • Android SDK evaluation shows build failures on macOS
  • This is expected and doesn't block builds
  • Android tools are installed on first use
  • CI uses Linux runners which don't have this issue

Related

  • Branch: feat/rn-devbox-plugin
  • Mobile-devtools plugin: https://github.com/segment-integrations/mobile-devtools
  • Previous commits:
    • 8ea2ade5 - fix: update E2E examples for mobile-devtools devbox plugin compatibility
    • ab47da45 - feat: add mobile-devtools devbox plugin to E2E examples

🤖 Generated with Claude Code

abueide and others added 7 commits April 24, 2026 15:16
Added devbox.json configuration to both E2E-compat and E2E-latest examples
using the React Native plugin from segment-integrations/mobile-devtools (main branch).

This replaces the archived feat/mobile-devtools-e2e-integration branch with
a cleaner implementation based on the latest mobile-devtools plugin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixed three issues preventing devbox from working with the mobile-devtools plugin:

1. Added missing 'start:app' script to devbox.json in both E2E examples
2. Updated build.gradle to read SDK versions from environment variables
   (ANDROID_MAX_API, ANDROID_COMPILE_SDK, ANDROID_TARGET_SDK) instead of
   hardcoding compileSdkVersion=33
3. Added Android hash overrides for platform-tools 37.0.0 to fix Nix build
   failures

This allows the E2E examples to properly use the mobile-devtools React Native
plugin for Android SDK management and device configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit adds comprehensive CI testing for the mobile-devtools devbox
plugin integration in E2E examples.

**Workflow Triggers:**

1. **Pull Requests** - Runs on PRs to feat/rn-devbox-plugin branch when:
   - devbox.json or devbox.lock files change
   - workflow file itself changes

2. **Weekly Schedule** - Runs every Monday at 10am UTC to catch
   upstream plugin changes

3. **Manual Trigger** - Can be triggered manually with options to:
   - Test specific example (E2E-compat, E2E-latest, or all)
   - Test specific platform (android, ios, or both)

4. **Release Prerequisite** - Can be called from release workflow
   to ensure plugin compatibility before releases

**Test Coverage:**

- Validates devbox.json configuration
- Tests devbox shell initialization
- Verifies Android/iOS tools availability
- Tests yarn install
- Tests pod install (iOS)
- Tests Android builds
- Tests iOS builds
- Provides comprehensive summary of all results

**Architecture:**

- Separate jobs for each example × platform combination
- Conditional execution based on trigger type and inputs
- All jobs must pass for release to proceed
- Uses macos-14 runners for iOS (required for Xcode)
- Uses ubuntu-latest for Android (faster, cheaper)

**Benefits:**

- Catches breaking changes in mobile-devtools plugin early
- Ensures both examples work with latest plugin
- Prevents releasing SDK that's incompatible with devbox workflow
- Provides clear feedback on which platform/example fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add start:ios and start:android convenience scripts (aliases for start:sim/start:emu)
- Add export statements to build scripts to skip unnecessary platform setup
- Add README-devbox.md with usage examples and performance tips
- Document ANDROID_SKIP_SETUP and IOS_SKIP_SETUP environment variables

This fixes the issue where running iOS commands would try to evaluate
the Android SDK on macOS (which fails but doesn't block iOS development).

Usage:
  devbox run -e ANDROID_SKIP_SETUP=1 start:ios
  devbox run -e IOS_SKIP_SETUP=1 start:android

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add dev:ios script: installs deps, pods, builds, and deploys to simulator
- Add dev:android script: installs deps, builds, and deploys to emulator
- Update README-devbox.md to clarify script purposes
- Keep start:sim/start:emu for just launching simulator/emulator

Usage:
  devbox run dev:ios        # Full iOS workflow
  devbox run dev:android    # Full Android workflow
  devbox run start:sim      # Just launch simulator (no build)
  devbox run start:emu      # Just launch emulator (no build)

This fixes the confusion where start:ios only launched the simulator
but didn't build or deploy the app.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@abueide abueide closed this Apr 29, 2026
@abueide abueide deleted the feat/rn-devbox-plugin branch April 29, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant