Skip to content

fix(ci): validate all jniLibs ABIs and exclude armeabi-v7a from debug builds#173

Open
TimeToBuildBob wants to merge 2 commits into
ActivityWatch:masterfrom
TimeToBuildBob:fix/ci-abifilters-and-jnilibs-check
Open

fix(ci): validate all jniLibs ABIs and exclude armeabi-v7a from debug builds#173
TimeToBuildBob wants to merge 2 commits into
ActivityWatch:masterfrom
TimeToBuildBob:fix/ci-abifilters-and-jnilibs-check

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Problem

Master CI has been failing since 2026-07-02 with two distinct issues:

1. INSTALL_FAILED_NO_MATCHING_ABIS in E2E tests

Root cause chain:

  1. PR ci: upgrade deprecated GitHub Actions before Node.js 24 deadline (June 2nd) #160 upgraded actions/cache v3→v4. GitHub Actions cache v4 uses a different backend, so all old v3 caches were invalidated.
  2. Cache miss triggered a fresh Rust debug build for all 4 ABIs (arm7, arm8, x86, x86_64).
  3. Debug Rust binaries are very large (~160MB+ for armeabi-v7a). Building all 4 in debug mode appears to exhaust runner disk space, truncating armeabi-v7a/libaw_server.so mid-write (corrupt ELF: e_shoff=0x9d93a8c past EOF).
  4. The old jniLibs check only verified x86_64/libaw_server.so exists — so the corrupt arm7 file passed silently and was uploaded as the artifact.
  5. The E2E test job downloads the jniLibs, Gradle's mergeDebugNativeLibs tries to llvm-strip the corrupt arm7 .so, and fails with:
    section header table goes past the end of the file: e_shoff = 0x9d93a8c
    INSTALL_FAILED_NO_MATCHING_ABIS: Failed to extract native libraries, res=-113
    

2. age: no identity matched any of the recipients (not addressed here)

The KEY_FASTLANE_API signing secret no longer matches the encrypted .age file. This is a separate signing-infrastructure issue requiring secret rotation — not addressed in this PR.

Fixes

build.yml — Comprehensive jniLibs validity check

Replaces the x86_64-existence-only check with a Python script that validates all 4 ABIs for:

  • File presence
  • Valid ELF magic (\x7fELF)
  • Section header table offset within file bounds

This catches corruption at build-rust time with a clear diagnostic instead of a cryptic INSTALL_FAILED_NO_MATCHING_ABIS deep in the E2E logs.

mobile/build.gradle — Exclude legacy ABIs from debug builds

Adds abiFilters 'arm64-v8a', 'x86_64' to the debug build type.

  • The CI E2E emulator is x86_64. It never needs armeabi-v7a or x86 libs.
  • Physical devices since ~2014 use arm64-v8a; armeabi-v7a support is legacy.
  • This prevents a corrupt arm7 .so from ever blocking the E2E install, even if the CI check is bypassed in future.
  • Release builds are unaffected (no abiFilters on release).

Testing

The ELF validity check is straightforward Python using only stdlib (os, struct). The abiFilters change follows the standard Android Gradle Plugin pattern for per-variant ABI filtering.

The age signing failure means the full build-aab / build-apk jobs will still fail until the signing secret is rotated, but the test-e2e job should pass once the jniLibs cache is populated for the v4 backend (or after one successful fresh build where arm7 isn't corrupted).

… builds

Two CI failures on master since 2026-07-02:

1. `INSTALL_FAILED_NO_MATCHING_ABIS` in E2E tests: the actions/cache v3→v4
   upgrade in PR ActivityWatch#160 invalidated the jniLibs cache, forcing a fresh Rust
   debug build. Building all 4 ABIs in debug mode (~160MB+ per .so) exhausts
   runner disk space, leaving the armeabi-v7a .so with a corrupt ELF section
   header table (e_shoff=0x9d93a8c past EOF). The old x86_64-only check passes
   silently, the corrupt .so is uploaded, and the E2E build fails when
   llvm-strip tries to process it.

Fixes:
- `.github/workflows/build.yml`: replace the x86_64-only existence check with
  a Python-based ELF validity check for all 4 ABIs. This catches corruption
  at build-rust time with a clear diagnostic instead of a cryptic
  INSTALL_FAILED_NO_MATCHING_ABIS at E2E time.
- `mobile/build.gradle`: add `abiFilters 'arm64-v8a', 'x86_64'` to the debug
  build type. Debug APKs used for E2E testing on x86_64 emulators don't need
  armeabi-v7a or x86 libs. Filtering them out prevents a corrupt ARM7 .so from
  blocking the E2E install even if the CI check is bypassed in future.

Note: the `age: no identity matched any of the recipients` failure in the
`build-aab` job is a separate signing-infrastructure issue not addressed here.
@greptile-apps

greptile-apps Bot commented Jul 3, 2026

Copy link
Copy Markdown

Greptile Summary

This PR improves CI reliability after master broke on 2026-07-02 by replacing a single-file existence check with a Python ELF validator that inspects all 4 ABIs, and by limiting debug APK packaging to arm64-v8a and x86_64 via Gradle abiFilters.

  • scripts/check-jnilibs.py: New validator that reads a 64-byte ELF header per ABI, verifies the magic bytes, class field, and section-header-table offset against the actual file size — catching the truncated-arm7 corruption that previously passed silently.
  • .github/workflows/build.yml: Swaps the test -e x86_64/libaw_server.so one-liner for python3 scripts/check-jnilibs.py in three jobs (build-rust, build-apk, test), so a corrupt library fails fast at build time with a readable diagnostic instead of a cryptic emulator install error.
  • mobile/build.gradle: Adds ndk { abiFilters 'arm64-v8a', 'x86_64' } to the debug build type, preventing any armeabi-v7a or x86 library from being packaged into the debug APK even if a stale cache entry bypasses the ELF check in a future refactor.

Confidence Score: 5/5

Safe to merge — the ELF validator is stdlib-only Python with correct header parsing, and the abiFilters follows standard Android Gradle Plugin conventions without touching release builds.

Both changes are narrowly scoped: the Python script adds validation that was clearly missing, and the Gradle change is a standard per-build-type ABI filter. The ELF header parsing logic is correct for both 32-bit and 64-bit classes, the short-circuit accumulation pattern evaluates every ABI, and the truncation guard comes after the magic check. The abiFilters is release-build-safe. The one design tension — the check script still validates armeabi-v7a even though the debug APK no longer needs it — is a future-proofing gap rather than a current defect.

No files require special attention, though the alignment between ABIS in check-jnilibs.py and the debug abiFilters in mobile/build.gradle is worth revisiting if disk-space-driven arm7 corruption recurs.

Important Files Changed

Filename Overview
scripts/check-jnilibs.py New validator replacing the x86_64-existence-only check; correctly reads 64-byte ELF header, checks magic, validates e_shoff bounds for both 32-bit and 64-bit ELF classes, and checks all 4 ABIs. Minor design tension: ABIS includes armeabi-v7a, which debug builds now exclude via abiFilters, so a corrupt arm7 file can still block the pipeline even though it would never land in the debug APK.
.github/workflows/build.yml Replaces the single-file existence check with the new Python ELF validator in three jobs (build-rust, build-apk, test). The test-e2e job downloads the artifact but has no ELF check step, acceptable because the artifact is already validated before upload in build-rust.
mobile/build.gradle Adds ndk { abiFilters 'arm64-v8a', 'x86_64' } to the debug build type only, following the standard Android Gradle Plugin pattern. Release builds are unaffected. Defense-in-depth change that prevents a future stale corrupt armeabi-v7a from reaching the emulator APK install if the ELF check were bypassed.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant MR as make aw-server-rust
    participant ELF as check-jnilibs.py
    participant ART as upload-artifact
    participant E2E as test-e2e job
    participant GRADLE as Gradle assembleDebug

    Note over MR: build-rust job (all 4 ABIs)
    MR->>ELF: jniLibs produced
    alt All 4 ABIs valid ELF
        ELF->>ART: upload artifact
        ART->>E2E: download artifact
        E2E->>GRADLE: assemble debug APK
        Note over GRADLE: abiFilters excludes armeabi-v7a and x86
        GRADLE-->>E2E: debug APK with arm64-v8a + x86_64 only
    else Any ABI corrupt or missing
        ELF--xART: FAIL with clear CORRUPT diagnostic
        Note over E2E: No artifact uploaded
    end
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant MR as make aw-server-rust
    participant ELF as check-jnilibs.py
    participant ART as upload-artifact
    participant E2E as test-e2e job
    participant GRADLE as Gradle assembleDebug

    Note over MR: build-rust job (all 4 ABIs)
    MR->>ELF: jniLibs produced
    alt All 4 ABIs valid ELF
        ELF->>ART: upload artifact
        ART->>E2E: download artifact
        E2E->>GRADLE: assemble debug APK
        Note over GRADLE: abiFilters excludes armeabi-v7a and x86
        GRADLE-->>E2E: debug APK with arm64-v8a + x86_64 only
    else Any ABI corrupt or missing
        ELF--xART: FAIL with clear CORRUPT diagnostic
        Note over E2E: No artifact uploaded
    end
Loading

Reviews (2): Last reviewed commit: "fix(ci): reuse jniLibs ELF validation" | Re-trigger Greptile

Comment thread .github/workflows/build.yml Outdated
Comment on lines +99 to +103
if header[:4] != b'\x7fELF':
print(f'NOT_ELF {path}', file=sys.stderr)
ok = False
continue
e_class = header[4] # 1=32-bit, 2=64-bit

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The script reads 64 bytes but never checks that len(header) is actually 64 before calling struct.unpack. If a file begins with \x7fELF but is truncated before offset 36 (ELFCLASS32) or offset 48 (ELFCLASS64), the slice header[32:36] or header[40:48] is shorter than required, and struct.unpack raises struct.error with an unhandled exception. CI still fails, but the error is a Python traceback rather than the intended CORRUPT diagnostic.

Suggested change
if header[:4] != b'\x7fELF':
print(f'NOT_ELF {path}', file=sys.stderr)
ok = False
continue
e_class = header[4] # 1=32-bit, 2=64-bit
if len(header) < 64:
print(f'CORRUPT {path}: file too small for ELF header '
f'({len(header)} bytes)', file=sys.stderr)
ok = False
continue
if header[:4] != b'\x7fELF':
print(f'NOT_ELF {path}', file=sys.stderr)
ok = False
continue
e_class = header[4] # 1=32-bit, 2=64-bit

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

@greptileai review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant