Skip to content

Upload Snapshots to Sentry using SnapshotPreview and SwiftSnapshotTesting libraries #780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cameroncooke wants to merge 23 commits into
base: main
Choose a base branch
from
Open

Upload Snapshots to Sentry using SnapshotPreview and SwiftSnapshotTesting libraries #780

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
c2b918e
Use SnapshotPreviews from `cameroncooke/snapshot-ci` branch
cameroncooke Apr 7, 2026
56fdd63
Upload workflow to support new SnapshotPreviews image exports
cameroncooke Apr 1, 2026
81230e5
fix: use absolute path for snapshot export dir to avoid test bootstra…
cameroncooke Apr 1, 2026
676b945
Update package reference
cameroncooke Apr 3, 2026
4b99f38
ci(ios): Hoist snapshot env vars to job level and boot iPad sim separ…
cameroncooke Apr 7, 2026
5f81e4c
ci(ios): Add pull_request trigger to snapshot upload workflow
cameroncooke Apr 7, 2026
1c63516
Disable iPad run
cameroncooke Apr 7, 2026
86400e6
feat(fastlane): Add lane to upload SnapshotPreviews snapshots to Sentry
cameroncooke Apr 7, 2026
1a07a3c
ci(ios): Cache DerivedData and split build/test phases for snapshots
cameroncooke Apr 7, 2026
5a524f3
ci(ios): Pin DD path, skip SPM updates, enable compilation caching
cameroncooke Apr 7, 2026
4693c5c
ci: trigger snapshot workflow to test warm cache
cameroncooke Apr 8, 2026
7832aa8
ci(ios): Remove redundant SPM cache step
cameroncooke Apr 8, 2026
7ffad21
ci(ios): Recombine build and test into single xcodebuild invocation
cameroncooke Apr 8, 2026
887a063
Update to use latest SnapshotPreviews branch commit
cameroncooke Apr 8, 2026
05b0f54
ci(ios): Replace -skipPackageUpdates with -skipPackagePluginValidation
cameroncooke Apr 8, 2026
bf734e6
ci(ios): Remove DD cache, restore original SPM cache
cameroncooke Apr 8, 2026
d363a7c
ci(ios): Pin arm64 arch in simulator destination to skip disambiguation
cameroncooke Apr 8, 2026
22a179e
ci(ios): Add swift-snapshot-testing workflow for Sentry uploads
cameroncooke Apr 15, 2026
10b9207
Revert SnapshotPreviews pin to main
cameroncooke Apr 16, 2026
73af48b
ci(ios): Re-enable iPad snapshots and restructure upload paths
cameroncooke Apr 17, 2026
0e0dc8f
ci(ios): Pin arm64 arch in iPad Sentry snapshot destination
cameroncooke Apr 17, 2026
a7053a1
ci(ios): Remove Sentry snapshot upload step
cameroncooke Apr 17, 2026
b8e01e3
ci(ios): Split snapshot tests into build-for-testing and test phases
cameroncooke Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .github/workflows/ios_emerge_upload_snapshots.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,3 @@ jobs:
--client-library swift-snapshot-testing \
--project-root . \
--debug
- name: Upload snapshots to Sentry
env:
SENTRY_SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_SENTRY_AUTH_TOKEN }}
run: bundle exec fastlane ios upload_sentry_snapshots
115 changes: 115 additions & 0 deletions .github/workflows/ios_sentry_upload_snapshots.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
name: Sentry iOS Upload (Snapshots)

on:
push:
branches: [main]
paths: [ios/**, .github/workflows/ios*]
pull_request:
branches: [main]
paths: [ios/**, .github/workflows/ios*]
Comment thread
cameroncooke marked this conversation as resolved.

jobs:
upload_sentry_snapshots:
runs-on: macos-26

defaults:
run:
working-directory: ./ios

env:
SNAPSHOT_UPLOAD_BASE_DIR: "${{ github.workspace }}/ios/snapshot-images"
DERIVED_DATA_PATH: "${{ github.workspace }}/DerivedData-snapshot-upload"
XCODE_RUNNING_FOR_PREVIEWS: 1

steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Set up Ruby env
uses: ruby/setup-ruby@v1
with:
ruby-version: 3.3.10
bundler-cache: true

- name: Setup gems
run: exec ../.github/scripts/ios/setup.sh

- name: Boot iPhone simulator
run: xcrun simctl boot "iPhone 17 Pro Max" || true

- name: Cache Swift Package Manager
uses: actions/cache@v4
with:
path: |
~/Library/Caches/org.swift.swiftpm
${{ env.DERIVED_DATA_PATH }}/SourcePackages
key: ${{ runner.os }}-spm-${{ hashFiles('**/Package.resolved') }}
restore-keys: ${{ runner.os }}-spm-

- name: Prepare snapshot export directories
run: |
mkdir -p "${SNAPSHOT_UPLOAD_BASE_DIR}/iphone"
mkdir -p "${SNAPSHOT_UPLOAD_BASE_DIR}/ipad"
mkdir -p "${DERIVED_DATA_PATH}"

- name: Build snapshot tests
run: |
set -o pipefail && xcodebuild build-for-testing \
-scheme HackerNews \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPhone 17 Pro Max,arch=arm64' \
-only-testing:HackerNewsTests/HackerNewsSnapshotTest \
-resultBundlePath ../SnapshotBuildForTesting.xcresult \
-derivedDataPath "${DERIVED_DATA_PATH}" \
-skipPackagePluginValidation \
ONLY_ACTIVE_ARCH=YES \
TARGETED_DEVICE_FAMILY="1,2" \
SUPPORTS_MACCATALYST=NO \
CODE_SIGNING_ALLOWED=NO \
COMPILATION_CACHING=YES \
EAGER_LINKING=YES \
FUSE_BUILD_SCRIPT_PHASES=YES \
| xcpretty

- name: Boot iPad simulator
run: xcrun simctl boot "iPad Air 11-inch (M4)" || true

- name: Generate snapshot images (iPhone)
env:
TEST_RUNNER_SNAPSHOTS_EXPORT_DIR: "${{ env.SNAPSHOT_UPLOAD_BASE_DIR }}/iphone"
run: |
set -o pipefail && xcodebuild test-without-building \
-scheme HackerNews \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPhone 17 Pro Max,arch=arm64' \
-only-testing:HackerNewsTests/HackerNewsSnapshotTest \
-resultBundlePath ../SnapshotResults-iphone.xcresult \
-derivedDataPath "${DERIVED_DATA_PATH}" \
| xcpretty

- name: Generate snapshot images (iPad)
env:
TEST_RUNNER_SNAPSHOTS_EXPORT_DIR: "${{ env.SNAPSHOT_UPLOAD_BASE_DIR }}/ipad"
run: |
set -o pipefail && xcodebuild test-without-building \
-scheme HackerNews \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPad Air 11-inch (M4),arch=arm64' \
-only-testing:HackerNewsTests/HackerNewsSnapshotTest \
-resultBundlePath ../SnapshotResults-ipad.xcresult \
-derivedDataPath "${DERIVED_DATA_PATH}" \
| xcpretty
Comment on lines +93 to +103
Copy link
Copy Markdown

@sentry sentry bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The iPad snapshot generation step lacks continue-on-error: true, making it less resilient to transient failures compared to similar steps in other workflows.
Severity: LOW

Suggested Fix

Add continue-on-error: true to the "Generate snapshot images (iPad)" step. This will align its behavior with other snapshot testing jobs and prevent the entire workflow from failing due to non-critical, transient test issues.

Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: .github/workflows/ios_sentry_upload_snapshots.yml#L92-L103

Potential issue: The "Generate snapshot images (iPad)" step in the
`ios_sentry_upload_snapshots.yml` workflow does not set `continue-on-error: true`. This
is inconsistent with similar snapshot testing steps in other workflows like
`ios_sentry_upload_swift_snapshots.yml` and `ios_emerge_upload_snapshots.yml`, which do
use this flag. Without it, the entire workflow could fail due to transient issues with
the iPad simulator or the test itself, reducing the overall reliability of the CI
pipeline. While a race condition with simulator boot-up is mitigated by a preceding
step, this lack of error handling introduces unnecessary brittleness.


- name: List generated snapshot files
run: |
echo "Generated snapshot files:"
find "${SNAPSHOT_UPLOAD_BASE_DIR}" -type f | sort
echo
echo "Total PNG images: $(find "${SNAPSHOT_UPLOAD_BASE_DIR}" -type f -name '*.png' | wc -l | tr -d ' ')"

- name: Upload snapshots to Sentry
env:
SENTRY_SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_SENTRY_AUTH_TOKEN }}
run: bundle exec fastlane ios upload_sentry_snapshots path:"${SNAPSHOT_UPLOAD_BASE_DIR}"
78 changes: 78 additions & 0 deletions .github/workflows/ios_sentry_upload_swift_snapshots.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Sentry iOS Upload (Swift Snapshot Testing)

on:
push:
branches: [main]
paths: [ios/**, .github/workflows/ios*]
pull_request:
branches: [main]
paths: [ios/**, .github/workflows/ios*]

jobs:
upload_sentry_swift_snapshots:
runs-on: macos-26

defaults:
run:
working-directory: ./ios

env:
XCODE_RUNNING_FOR_PREVIEWS: 1
TEST_RUNNER_SNAPSHOT_TESTING_RECORD: all

steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Set up Ruby env
uses: ruby/setup-ruby@v1
with:
ruby-version: 3.3.10
bundler-cache: true

- name: Setup gems
run: exec ../.github/scripts/ios/setup.sh

- name: Boot iPhone simulator
run: xcrun simctl boot "iPhone 17 Pro Max" || true

- name: Cache Swift Package Manager
uses: actions/cache@v4
with:
path: |
~/Library/Caches/org.swift.swiftpm
~/Library/Developer/Xcode/DerivedData/HackerNews-*/SourcePackages
key: ${{ runner.os }}-spm-${{ hashFiles('**/Package.resolved') }}
restore-keys: ${{ runner.os }}-spm-

- name: Generate snapshot images
continue-on-error: true
Comment thread
sentry[bot] marked this conversation as resolved.
run: |
set -o pipefail && xcodebuild test \
-scheme HackerNews \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPhone 17 Pro Max,arch=arm64' \
-only-testing:HackerNewsTests/SwiftSnapshotTest \
-resultBundlePath ../SnapshotResults-swift-snapshots.xcresult \
-skipPackagePluginValidation \
ONLY_ACTIVE_ARCH=YES \
TARGETED_DEVICE_FAMILY=1 \
SUPPORTS_MACCATALYST=NO \
CODE_SIGNING_ALLOWED=NO \
COMPILATION_CACHING=YES \
EAGER_LINKING=YES \
FUSE_BUILD_SCRIPT_PHASES=YES \
| xcpretty

- name: List generated images
run: |
echo "Generated snapshot images:"
ls -1 HackerNewsTests/__Snapshots__/SwiftSnapshotTest/ || echo "No snapshots found"
echo "Total: $(ls -1 HackerNewsTests/__Snapshots__/SwiftSnapshotTest/ 2>/dev/null | wc -l | tr -d ' ') images"

- name: Upload snapshots to Sentry
env:
SENTRY_SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_SENTRY_AUTH_TOKEN }}
run: bundle exec fastlane ios upload_sentry_snapshots_swift_snapshot_testing
4 changes: 2 additions & 2 deletions ios/HackerNews.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 0 additions & 9 deletions ios/HackerNewsTests/SwiftSnapshotTest.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,6 @@ import Common

final class SwiftSnapshotTest: XCTestCase {

override func invokeTest() {
// record: .all always records new snapshots and fails every test.
// Change to .missing to only record when no reference image exists,
// or remove this override entirely to compare against existing snapshots.
withSnapshotTesting(record: .all) {
super.invokeTest()
}
}

Copy link
Copy Markdown
Contributor

@NicoHinderling NicoHinderling Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

intentional?

Copy link
Copy Markdown
Author

@cameroncooke cameroncooke Apr 16, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NicoHinderling Yes, because Swift Snapshot Testing does diffing locally, we would only want to enable recording on CI workflows. We use an environmental variable in the GitHub workflow example TEST_RUNNER_SNAPSHOT_TESTING_RECORD which causes the library to record. This allows the user to run tests and catch regressions locally before pushing via the Pointfree library's own diff algo and on CI it will just be used to export images. It's also worth noting enabling recording causes XCTest failures which we ignore on CI but will run as real regression tests locally.

@MainActor func testPostListScreen() {
@State var appViewModel = AppViewModel(
bookmarkStore: FakeBookmarkDataStore(),
Expand Down
43 changes: 27 additions & 16 deletions ios/fastlane/Fastfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,24 +209,24 @@ platform :ios do
lane :export_thinned_build do |options|
xcarchive_path = lane_context[SharedValues::XCODEBUILD_ARCHIVE]
UI.message("Using archive: #{xcarchive_path}")

# Device type configuration for thinning, default is iPhone 16 Pro
device_type = options[:device_type] || 'iPhone17,1'
UI.message("Thinning for device type: #{device_type}")

# Export options for thinning
export_options = {
method: 'ad-hoc',
thinning: device_type, # This creates a thinned build for the specific device type
team_id: ENV['APPLE_TEAM_ID']
}

# Add provisioning profiles if available
app_identifier = CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier)
bundle_identifier = ENV['BUNDLE_ID'] || app_identifier
profile_name = 'HackerNews AdHoc Distribution'
widget_profile_name = 'HackerNews AdHoc HomeWidget'

export_options[:provisioningProfiles] = {
bundle_identifier => profile_name,
'com.emergetools.hackernews.HackerNewsHomeWidget' => widget_profile_name
Expand All @@ -240,10 +240,10 @@ platform :ios do
sh("#{xcsafe} -exportArchive -archivePath #{xcarchive_path} -exportOptionsPlist export_options.plist -exportPath ./build")

plist_data = Plist.parse_xml('./build/app-thinning.plist')

# Find the variant matching our device and OS version
target_ipa_path = nil

if plist_data && plist_data['variants']
plist_data['variants'].each do |variant|
path = variant[0]
Expand All @@ -256,43 +256,54 @@ platform :ios do
end
end
end

ipa_path = File.join('../', target_ipa_path)
UI.message("Using IPA: #{ipa_path}")

temp_dir = './build/temp_extraction'
FileUtils.rm_rf(temp_dir) if Dir.exist?(temp_dir)
FileUtils.mkdir_p(temp_dir)

# Extract the IPA
sh("cd #{temp_dir} && unzip -q '#{ipa_path}'")
payload_dir = File.join(temp_dir, 'Payload')
app_files = Dir.glob(File.join(payload_dir, '*.app'))

if app_files.empty?
UI.error("No .app file found in extracted IPA")
return
end

thinned_app_path = app_files.first
# Find the original .app in the XCArchive
original_app_pattern = File.join(xcarchive_path, 'Products', 'Applications', '*.app')
original_app_files = Dir.glob(original_app_pattern)

original_app_path = original_app_files.first

FileUtils.rm_rf(original_app_path)
FileUtils.cp_r(thinned_app_path, original_app_path)

# Clean up temporary directory
FileUtils.rm_rf(temp_dir)
end

UI.success("Successfully replaced app in XCArchive with thinned build for #{device_type}")
end

desc 'Upload SnapshotPreviews snapshots to Sentry'
lane :upload_sentry_snapshots do |options|
sentry_upload_snapshots(
path: options[:path],
app_id: 'com.emergetools.hackernews',
auth_token: ENV['SENTRY_SENTRY_AUTH_TOKEN'],
org_slug: 'sentry',
project_slug: 'hackernews-ios'
)
end

desc 'Upload swift-snapshot-testing snapshots to Sentry'
lane :upload_sentry_snapshots do
lane :upload_sentry_snapshots_swift_snapshot_testing do
sentry_upload_snapshots(
path: 'HackerNewsTests/__Snapshots__/SwiftSnapshotTest',
app_id: 'com.emergetools.hackernews',
Expand Down
Loading