Perf: Fast Cache Approach - Exploring Possible Optimisations in the Token Acquisition Process

[Divyansh-jain2]

Description

When connecting with authentication=ActiveDirectoryServicePrincipal (and the other AAD principal flows that go through SQLServerMSAL4JUtils.getSqlFedAuthTokenPrincipal), every token acquisition — including cache hits — is currently serialized behind a global Semaphore(1) keyed on the principal. The semaphore exists to stop a burst of cache misses from stampeding the Entra ID token endpoint, but it also forces threads that only need to read an already-cached token to queue one-at-a-time behind whoever currently holds the permit.

Under load where many connections share the same service principal, this turns a cheap in-memory cache read into a serialization point: N concurrent connections that could all be satisfied from the MSAL cache instead drain through a single permit.

This change adds a lock-free silent fast-path that attempts a cache-only token read before taking the semaphore. Cache hits return immediately and in parallel; only cache misses fall through to the unchanged, semaphore-gated slow path that protects the token endpoint.

How it works

acquireTokenSilently(...) is MSAL's cache-only lookup — it checks the client application's in-memory token cache (rehydrated here from the persistent cache aspect) for a still-valid token for the requested scopes and returns it without contacting Entra ID. If nothing valid is cached, its future fails with an ExecutionException, which we treat as a miss and fall through to the existing gated path.

The gated slow path is unchanged and remains safe: the first caller through the gate refreshes the token and populates the shared cache; callers that were waiting on the gate then get that freshly-cached token because acquireToken(...) does its own cache lookup first (skipCache defaults to false), so they don't make a redundant AAD call.

Before:  [all connections] ──> Semaphore(1) ──> cache check / AAD ──> token
                                    ▲ cache hits queue here too

After:   cache hit  ──> return immediately (no semaphore)
         cache miss ──> Semaphore(1) ──> AAD refresh ──> token   (unchanged)

Net effect: cache hits are served concurrently; the token endpoint keeps the same stampede protection for misses.

Testing

Validated with a standalone single-service-principal Token Acquisiton benchmark, included for reference (not part of the build) at docs/performance/TokenAcquisitionTest.java.

What the harness does

• Spins up K worker threads that each open a brand-new JDBC connection to the same service principal, run SELECT 1, close it, and repeat in a tight loop for T seconds. Sharing one credential across all threads is the worst-case head-of-line / token-cache contention scenario, which isolates exactly what this change affects.
• All workers are released simultaneously via a CountDownLatch so they contend at once rather than ramping up.
• Runs a warm-up phase first (primes the MSAL token, TLS and DB caches, then rests) so the measured window consists of steady-state cache hits; warm-up metrics are discarded. A cold mode is also available to measure the cache-miss stampede.
• Registers a PerformanceLogCallback that captures the driver's internal TOKEN_ACQUISITION timing for every connection and reports count / avg / max, accumulated lock-free so the measurement adds negligible overhead.
• Reports throughput (connections/sec), avg/max end-to-end latency, and failures alongside the token-acquisition numbers.

How it's run

The same command is used twice, changing only which driver jar is on the classpath (before vs. after this change). The driver-org / fast-cache argument is just a label printed in the output.

# baseline: the current driver jar
java -cp "mssql-jdbc-<version>.jar;." TokenAcquisitionTest driver-org <K> <T> warm

# this change: the driver jar built from this branch
java -cp "mssql-jdbc-<version>.jar;." TokenAcquisitionTest fast-cache <K> <T> warm

The file will not compile inside this repo by design — it depends on a small, local, git-ignored Creds helper that the reader supplies with their own Azure SQL server FQDN, database name and service-principal id/secret. No credentials are embedded; it is kept purely so reviewers can see exactly how the numbers below were produced.

Results — token acquisition average latency (warm cache; lower is better):

Measured window	driver-org (baseline)	fast-cache	Change
10 s	87.3 ms	38.3 ms	−56.1%
60 s	97.2 ms	40.6 ms	−58.2%
300 s	95.5 ms	35.7 ms	−62.6%
600 s	93.8 ms	37.8 ms	−59.7%

The improvement reflects cache hits no longer queuing behind the single semaphore permit. Cache-miss behavior (the AAD refresh path) is unchanged.

Guidelines

• The code builds clean without any errors or warnings
• I have followed the coding style guidelines of this project
• Backwards compatible: no public API changes; miss path (AAD stampede protection) unchanged
• I have added tests / the change is covered by existing behavior (see note below)
• CHANGELOG.md updated (will add once the PR number is assigned)

[Divyansh-jain2]

@Divyansh-jain2 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

• (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

• (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree company="Microsoft"