Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions AnkiDroid/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
// Gradle plugin portal
alias(libs.plugins.tripletPlay)
alias(libs.plugins.android.application)
alias(libs.plugins.androidx.baselineprofile)
alias(libs.plugins.kotlin.parcelize)
alias(libs.plugins.kotlin.serialization)
alias(libs.plugins.keeper)
Expand All @@ -22,6 +23,14 @@ keeper {
}
}

baselineProfile {
// Write generated profiles to `src/main/baselineProfiles/` instead of the
// default per-variant `src/playRelease/generated/baselineProfiles/`. This
// ensures the profile is consumed by ALL variants (play, amazon, full) and
// ALL buildTypes (release, benchmark), not just `playRelease`.
mergeIntoMain = true
Comment thread
david-allison marked this conversation as resolved.
}

idea {
module {
downloadJavadoc = System.getenv("CI") != "true"
Expand Down Expand Up @@ -178,6 +187,13 @@ android {
resValue 'color', 'anki_foreground_icon_color_0', "#FF29B6F6"
resValue 'color', 'anki_foreground_icon_color_1', "#FF0288D1"
}

create('benchmark') {
initWith release
signingConfig signingConfigs.debug
matchingFallbacks = ['release']
debuggable false
}
}

/**
Expand Down Expand Up @@ -401,6 +417,7 @@ dependencies {
implementation project(":compat")
implementation project(":libanki")
implementation project(":vbpd")
"baselineProfile" project(":baselineprofile")

implementation libs.androidx.activity
implementation libs.androidx.annotation
Expand All @@ -413,6 +430,7 @@ dependencies {
implementation libs.androidx.lifecycle.process
implementation libs.androidx.media
implementation libs.androidx.preference.ktx
implementation libs.androidx.profileinstaller
implementation libs.androidx.recyclerview
implementation libs.androidx.sqlite.framework
implementation libs.androidx.swiperefreshlayout
Expand Down
21 changes: 21 additions & 0 deletions AnkiDroid/src/benchmark/AndroidManifest.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Benchmark-only manifest overlay. Merged into the benchmark buildType only.
Keeps <profileable> out of production release APKs.
-->
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools">

<application>
<!--
Allows Macrobenchmark / Perfetto to attach via adb shell.
Required by the :baselineprofile module to collect baseline
profiles and run StartupBenchmark. `android:shell="true"`
restricts attachment to shell-initiated profiling only.
-->
<profileable
android:shell="true"
tools:targetApi="29" />
</application>

</manifest>
14,406 changes: 14,406 additions & 0 deletions AnkiDroid/src/main/generated/baselineProfiles/baseline-prof.txt

Large diffs are not rendered by default.

54 changes: 54 additions & 0 deletions baselineprofile/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# `:baselineprofile`

This module generates AnkiDroid's **baseline profile** - a list of classes
and methods ART pre-compiles at install time so cold start is faster.

It has two classes:

- **`BaselineProfileGenerator`** captures a cold-start trace and writes
`baseline-prof.txt` into `:AnkiDroid`.
- **`StartupBenchmark`** measures cold-start time with vs without the
profile so you can see the impact.

For results to be valid for comparisons and real-world impact, they must
be run on real hardware devices ideally the same device over time and
representative of typical user hardware. So this is currently not
intended to be used in CI unless CI is connected to real devices (e.g.
Firebase Test Lab or similar).

## Requirements

- Physical Android device, API 28+ (emulators aren't trustworthy)
- USB debugging on, device plugged in

## Regenerate the profile

Run with `-PcustomSuffix` and `-PcustomName` so the benchmarked APK installs
alongside your real AnkiDroid (`com.ichi2.anki.bp`) instead of replacing it.
Without these flags the build targets `com.ichi2.anki`, the production app
ID — installing it would clobber your real collection or fail with a
signing-key mismatch.

```sh
./gradlew :AnkiDroid:generateBaselineProfile \
-PcustomSuffix=".bp" \
-PcustomName="AnkiDroid Baseline"
```

Takes a few minutes. The plugin installs a non-minified build, runs the
cold-start journey (launch app -> dismiss intro screen if present -> wait
for DeckPicker), and writes the result to `baseline-prof.txt`.

## When to regenerate

When the startup path changes meaningfully — new init work in
`AnkiDroidApp.onCreate`, a refactor to `IntentHandler` or `DeckPicker`,
a new dependency wired into `Application`. Otherwise, leave it alone;
the profile is device- and OS-agnostic, one commit covers everyone.
Comment on lines +42 to +47
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth adding an annotation for this case, to flag to developers that changes may be necessary

Is it worth enforcing that this is run before a public release?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not worth it IMO since developer will be concerned with their change, so a change to DeckPicker rarely touches the cold-start path meaningfully.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Is it worth enforcing that this is run before a public release?

I suspect this is a more interesting question

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very interesting -

  • the onCreate "you should regenerate baselines" thing is interesting, note that we just touched onCreate in order to set up widget reminders on boot, and I'd be shocked if in PRs similar to that one the person remembered. Bears some thought 🤔
  • when to run / run before release builds - would be interesting to perhaps check for a change in the version code of format "switching from alpha to beta" or "beta to public release" - if that digit in the versionCode changed, the android test release build run could re-run baselines and...do I don't know what with the information - similar to APK size check I suppose it could post a comment on the commit line with the information and a link to git blame where you could click and see the last commit with the last baseline commit message from it with baseline timing ? Just a brainstorm

Anyway, will research this a bit, and as a dev / CI feature will likely at least get it merged in some form even if imperfect now so we can start "feeling" it and see how it works, what we really want out of it

Copy link
Copy Markdown
Member

@mikehardy mikehardy Apr 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having read the documentation and this PR and these comments I think the ideal path forward is to connect to a test lab that is CI accessible but has real hardware devices and a free tier or is free for open source.

I have not looked extensively but Firebase Test Lab has some free runs per day on their free tier and I could create an AnkiDroid project there to take advantage of that, in the absence of some better idea. If it goes away, no big deal.

But assuming it stays we could select our idea of an "old phone" and a "new phone* (over time new phone becomes old phone and you pick new phone but you'll maintain comparables between each pair for a while to see if anything goes seriously wrong), and start collecting metrics over time (TBD: somehow? that was CI available in order to graph over time?) and we could run it on a schedule or on push to main but only if versionCode changed to capture change-over-time-for-releases ?

Not sure how to hook Firebase Test Lab up to CI but they sell that service so docs should be commercial-grade
Not sure how to persist state (performance over time each versionCode change) in a CI-accessible way but there must be some way ? Maybe even a wiki page with entries and a script-generated graph of performance over time, and the CI script edits it or something


## Config notes

See comments in [`build.gradle.kts`](build.gradle.kts) for the rationale
behind `minSdk`, `missingDimensionStrategy`, the `benchmark` buildType,
and `self-instrumenting`. The KDoc on each class explains the journey
and the measurement modes.
93 changes: 93 additions & 0 deletions baselineprofile/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import com.android.build.api.dsl.TestExtension

plugins {
alias(libs.plugins.android.test)
alias(libs.plugins.androidx.baselineprofile)
}

configure<TestExtension> {
namespace = "com.ichi2.anki.baselineprofile"

compileSdk =
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cross-linking with related PR for boilerplate-ness:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll rebase and see what boilerplate in baselineprofile/build.gradle.kts can be consolidated. Not blocking this PR. Or David can if this gets in first, all good for now

libs.versions.compileSdk
.get()
.toInt()

compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}

defaultConfig {
// Macrobenchmark + baseline profile generation require API 28+
// even though the app may support a lower version. If the app's
// minSdk ever rises above 28, follow it automatically.
minSdk =
maxOf(
28,
libs.versions.minSdk
.get()
.toInt(),
)
targetSdk =
libs.versions.targetSdk
.get()
.toInt()

testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"

// AnkiDroid defines three product flavors (play, amazon, full) in the
// `appStore` dimension. Rather than duplicate them here we target the
// `play` flavor via `missingDimensionStrategy`, which keeps this module
// flavor-free but still able to resolve `:AnkiDroid` variants.
Comment on lines +41 to +42
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what a coincidence, the documentation for macrobenchmark was also important, but flavorless

missingDimensionStrategy("appStore", "play")
Comment thread
david-allison marked this conversation as resolved.
}

buildTypes {
create("benchmark") {
isDebuggable = true
signingConfig = signingConfigs.getByName("debug")
matchingFallbacks += listOf("release")
}
}

targetProjectPath = ":AnkiDroid"

@Suppress("UnstableApiUsage")
experimentalProperties["android.experimental.self-instrumenting"] = true
}

baselineProfile {
useConnectedDevices = true
}

apply(from = "../lint.gradle")

dependencies {
implementation(libs.androidx.test.junit)
implementation(libs.androidx.espresso.core)
implementation(libs.androidx.uiautomator)
implementation(libs.androidx.benchmark.macro.junit4)
}

androidComponents {
onVariants { v ->
val artifactsLoader = v.artifacts.getBuiltArtifactsLoader()
// Pull the applicationId from the actual built APK instead of
// hardcoding `com.ichi2.anki`. Most of us build with something like
// `-PcustomSuffix=".bp"` so the benchmark APK installs next to our
// real AnkiDroid (`com.ichi2.anki.bp`) rather than replacing it.
// If we hardcoded the fallback, the benchmark would happily target
// the production app and nuke your actual collection.
v.instrumentationRunnerArguments.put(
"targetAppId",
v.testedApks.map {
artifactsLoader.load(it)?.applicationId
?: error(
"Could not resolve targetAppId from testedApks. " +
"Refusing to fall back to a default to avoid clobbering the production install.",
)
},
)
}
}
4 changes: 4 additions & 0 deletions baselineprofile/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest>

</manifest>
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
/*
* Copyright (c) 2026 Ashish Yadav <mailtoashish693@gmail.com>
*
* This program is free software; you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation; either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT ANY
* WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
* PARTICULAR PURPOSE. See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see <http://www.gnu.org/licenses/>.
*/

package com.ichi2.anki.baselineprofile

import androidx.benchmark.macro.junit4.BaselineProfileRule
import androidx.test.ext.junit.runners.AndroidJUnit4
import androidx.test.filters.LargeTest
import androidx.test.platform.app.InstrumentationRegistry
import androidx.test.uiautomator.By
import androidx.test.uiautomator.Until
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith

/**
* Generates a baseline profile that captures the critical user journey for AnkiDroid's
* cold start: launching the app and reaching the main [com.ichi2.anki.DeckPicker] screen.
*
* The captured launch path is:
* 1. [com.ichi2.anki.IntentHandler] — the MAIN/LAUNCHER activity declared in
* the manifest. `startActivityAndWait()` launches this.
* 2. [com.ichi2.anki.DeckPicker] — IntentHandler.onCreate() immediately
* creates an intent for DeckPicker and calls `launchDeckPickerIfNoOtherTasks()`.
* 3. The `device.wait` + `device.waitForIdle()` calls wait for DeckPicker to
* fully render before the trace window closes.
*
* On a fresh install, [com.ichi2.anki.IntroductionActivity] appears before
* DeckPicker. The journey handles this by tapping the "Get Started" button
* (resource id `get_started`) if it is present, so the profile captures the
* full path to DeckPicker regardless of device state.
*
* Run on a connected API 28+ physical device with:
* ```
* ./gradlew :AnkiDroid:generateBaselineProfile
* ```
* The generated profile is committed to
* `AnkiDroid/src/main/baselineProfiles/baseline-prof.txt` by the
* [androidx.baselineprofile] plugin and is automatically embedded in subsequent
* app builds.
*/
@LargeTest
@RunWith(AndroidJUnit4::class)
class BaselineProfileGenerator {
@get:Rule
val rule = BaselineProfileRule()

@Test
fun generate() {
val targetPackage =
InstrumentationRegistry.getArguments().getString("targetAppId")
?: throw IllegalStateException("targetAppId not passed as instrumentation runner arg")

rule.collect(packageName = targetPackage) {
pressHome()
// Launches IntentHandler (MAIN/LAUNCHER), which routes to DeckPicker.
startActivityAndWait()
Comment thread
david-allison marked this conversation as resolved.

// On a fresh install, IntroductionActivity appears before DeckPicker.
// Dismiss it by tapping "Get Started" so the profile always captures
// the full path through to DeckPicker, regardless of device state.
device.findObject(By.res(targetPackage, "get_started"))?.click()

// Wait for DeckPicker to fully render so all startup classes are captured.
device.wait(Until.hasObject(By.pkg(targetPackage).depth(0)), 5_000)
device.waitForIdle()
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
/*
* Copyright (c) 2026 Ashish Yadav <mailtoashish693@gmail.com>
*
* This program is free software; you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation; either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT ANY
* WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
* PARTICULAR PURPOSE. See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see <http://www.gnu.org/licenses/>.
*/

package com.ichi2.anki.baselineprofile

import androidx.benchmark.macro.BaselineProfileMode
import androidx.benchmark.macro.CompilationMode
import androidx.benchmark.macro.StartupMode
import androidx.benchmark.macro.StartupTimingMetric
import androidx.benchmark.macro.junit4.MacrobenchmarkRule
import androidx.test.ext.junit.runners.AndroidJUnit4
import androidx.test.filters.LargeTest
import androidx.test.platform.app.InstrumentationRegistry
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith

/**
* Measures AnkiDroid cold-startup time in two configurations so the
* before/after impact of the baseline profile is visible in a single run:
*
* - [startupCompilationNone] — [CompilationMode.None]: no AOT compilation,
* all code is JIT'd on the fly. This is the "before" baseline.
* - [startupCompilationBaselineProfile] — [CompilationMode.Partial] with
* [BaselineProfileMode.Require]: AOT-compiles exactly the methods listed
* in `baseline-prof.txt`. `Require` fails the test if the profile isn't
* installed, so any improvement is guaranteed to come from the profile.
*
* Both methods run [StartupMode.COLD] with 10 iterations. Compare the
* median of `timeToInitialDisplayMs` across the two methods.
*
* **This benchmark is for local, manual use only.** It requires a connected
* physical device, has no pass/fail assertions, and should not be included
* in CI test suites. Run with:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know these need to run on device but I work with Firebase a lot, and...they have a free plan which allows for a sufficient (IMHO) number of runs a day for this to run as a scheduled thing for graphs-over-time and/or the occasional manual_dispatch...first column is the "free" plan:

Image

* ```
* ./gradlew :baselineprofile:connectedBenchmarkBenchmarkAndroidTest
* ```
*/
@LargeTest
@RunWith(AndroidJUnit4::class)
Comment thread
david-allison marked this conversation as resolved.
class StartupBenchmark {
@get:Rule
val rule = MacrobenchmarkRule()

@Test
fun startupCompilationNone() = startup(CompilationMode.None())

@Test
fun startupCompilationBaselineProfile() = startup(CompilationMode.Partial(baselineProfileMode = BaselineProfileMode.Require))

private fun startup(mode: CompilationMode) {
val targetPackage =
InstrumentationRegistry.getArguments().getString("targetAppId")
?: throw IllegalStateException("targetAppId not passed as instrumentation runner arg")

rule.measureRepeated(
packageName = targetPackage,
metrics = listOf(StartupTimingMetric()),
iterations = 10,
startupMode = StartupMode.COLD,
compilationMode = mode,
) {
pressHome()
startActivityAndWait()
}
}
}
Loading
Loading