Skip to content

[Platform] Add provider-level record/replay cassettes for test mocks#2129

Draft
wachterjohannes wants to merge 4 commits into
symfony:mainfrom
wachterjohannes:feature/platform-fake-recording
Draft

[Platform] Add provider-level record/replay cassettes for test mocks#2129
wachterjohannes wants to merge 4 commits into
symfony:mainfrom
wachterjohannes:feature/platform-fake-recording

Conversation

@wachterjohannes

@wachterjohannes wachterjohannes commented May 30, 2026

Copy link
Copy Markdown
Contributor
Q A
Bug fix? no
New feature? yes
Docs? yes
Issues -
License MIT

Builds on / lands after #2123 (the Mock test provider). Sibling of #2128 (HTTP-boundary cassettes); both are independent and stack on #2123.

Adds a provider-level record/replay tool for tests that only need a realistic, deterministic answer from a provider — without a network or API key.

Symfony\AI\Platform\Mock\Recording\RecordingProvider decorates a real ProviderInterface. By default the mode follows the cassette file:

  • Record — when the cassette file does not exist yet: invokes the real provider, serializes its finished ResultInterface into a JSON Cassette, and returns it.
  • Replay — when the cassette exists: rebuilds the result from the cassette and returns it; the real provider is never called, so it can be built with a placeholder key.

Delete the cassette to re-record. The mode can be forced via the record constructor argument (e.g. record: false in CI to fail loudly instead of issuing a real request when a cassette is missing).

use Symfony\AI\Platform\Bridge\Anthropic\Factory;
use Symfony\AI\Platform\Mock\Recording\Cassette;
use Symfony\AI\Platform\Mock\Recording\RecordingProvider;

$provider = new RecordingProvider(
    Factory::createProvider($_SERVER['ANTHROPIC_API_KEY'] ?? 'no-key-needed-on-replay'),
    new Cassette(__DIR__.'/fixtures/weather.json'),
);

$result = $provider->invoke('claude-3-7-sonnet-latest', $messages);

ResultSerializer supports text, structured output (ObjectResult), embeddings (VectorResult), tool calls (ToolCallResult) and text streams. Interactions are matched by a signature over (model, input, options) and consumed FIFO.

Scope note: on replay the recorded result is returned verbatim, so the bridge ResultConverter runs only at record time. To exercise bridge internals (converter/formatter) offline, use the HTTP-boundary cassette (Mock\Http\CassetteHttpClient, #2128) instead. Result metadata and token usage are not preserved in this first version.

Includes CHANGELOG entry, a docs subsection under Testing Tools, and offline tests (incl. an Anthropic record→replay demo). PHPUnit, PHPStan, php-cs-fixer and doctor-rst are green.

@wachterjohannes wachterjohannes force-pushed the feature/platform-fake-recording branch from 67693e1 to c3fc67c Compare May 30, 2026 20:25
@OskarStark

Copy link
Copy Markdown
Contributor

You should check Symfony repo there is an open pull request for cassettes like this IIRC

@wachterjohannes wachterjohannes force-pushed the feature/platform-fake-recording branch from c3fc67c to f0f7bc1 Compare June 1, 2026 06:26
@wachterjohannes wachterjohannes changed the title [Platform] Add provider-level record/replay cassettes for test fakes [Platform] Add provider-level record/replay cassettes for test mocks Jun 1, 2026
@wachterjohannes

Copy link
Copy Markdown
Contributor Author

You should check Symfony repo there is an open pull request for cassettes like this IIRC

@OskarStark you mean symfony/symfony#63781 (RecorderHttpClient, php‑vcr‑style HAR record/replay)? Good catch. That one records at the HTTP boundary, so it overlaps with the sibling #2128, not this PR.

This RecordingProvider sits a layer higher: it records the already‑converted ResultInterface returned by a ProviderInterface, so it's bridge‑agnostic, needs no http-client, and on replay it skips the converter entirely. Different trade‑off, complementary to a HTTP‑level recorder. I'll add a note to the PR cross‑referencing #63781 so the layering is clear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants