Skip to content

Swift SDK: IdentityResolverSignIntegrationTests flakes on CI due to real-login-keychain writes #4026

Description

@Claudius-Maginificent

Expected Behavior

IdentityResolverSignIntegrationTests should pass reliably in CI regardless of the runner's keychain state.

Current Behavior

Two tests in IdentityResolverSignIntegrationTests intermittently fail on the self-hosted macOS CI runner (latte-ssh) with a keychain error thrown from WalletStorage.swift:95:

  • testResolverRejectsWrongPathBeforeSigningkeychainError(-60008)
  • testResolverSignsForConsistentBreadcrumbkeychainError(-60005)

Both tests call storage.storeMnemonic(mnemonic, for: walletId), which writes a real BIP-39 mnemonic to the host macOS login keychain via SecItemAdd — no kSecUseDataProtectionKeychain, no test-specific access group, no mock storage. On a headless/SSH CI session the login keychain write can fail nondeterministically. Two different OSStatus codes surfacing across the same run (-60008 vs -60005) is the signature of nondeterministic securityd state, not a deterministic logic error.

Confirmed as an environment flake, not a code defect:

  • Same tests pass on v4.1-dev runs on the same runner pool.
  • A clean rerun of the exact same CI job (run 28791155484, job 85376353259) passed with zero code changes (Swift SDK build / Swift SDK build + tests completed green in 9m19s).
  • The test file was introduced by feat: complete dashpay in platform wallet and swift example app #3841 (already on v4.1-dev) and is unrelated to whatever PR happens to trigger the run.

Possible Solution

Isolate these tests from the real login keychain, e.g.:

  • Use kSecUseDataProtectionKeychain: true with a dedicated test access group, or
  • Inject an in-memory/mock WalletStorage for these tests, or
  • throw XCTSkip(...) when the keychain is unavailable in a headless session, so CI success stops depending on runner keychain state.

Steps to Reproduce (for bugs)

  1. Run the Swift SDK CI job (Swift SDK build / Swift SDK build + tests) on the latte-ssh self-hosted runner enough times.
  2. Occasionally IdentityResolverSignIntegrationTests.testResolverRejectsWrongPathBeforeSigning and/or testResolverSignsForConsistentBreadcrumb fail with a keychainError thrown from WalletStorage.swift:95, while an immediate rerun with no code change passes.

Context

This flake blocked PR #3692's CI checks for reasons unrelated to that PR's own diff, costing review/merge time. Any other PR that happens to run the Swift SDK job on this runner is equally exposed.

Your Environment

🤖 Co-authored by Claudius the Magnificent AI Agent

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions