LLM TCK

ManagedCode.LlmTck is a deterministic Technology Compatibility Kit for LLM APIs. It gives tests a local provider-compatible server that can be hosted by Aspire, scripted with explicit scenarios, and called through regular HTTP or Microsoft.Extensions.AI.

The implemented OpenAI-compatible surface covers chat, streaming chat, models, embeddings, image generation, audio speech fixtures, bearer-token failures, scenario misses, and assertion summaries. Provider routes are explicitly namespaced by provider. For example, OpenAI-compatible routes live under /openai, Anthropic Messages lives under /anthropic, and Azure OpenAI lives under /azure-openai. The first implemented native non-OpenAI route is Anthropic Messages: POST /anthropic/v1/messages, including the anthropic-version header, x-api-key auth, text message responses, and Anthropic-named streaming events. When a bearer token is configured, both provider endpoints and control endpoints require it.

Packages

Package	NuGet	Description
`ManagedCode.LlmTck`		Provider-neutral scenario runtime and assertion state.
`ManagedCode.LlmTck.OpenAI`		OpenAI-compatible wire contracts.
`ManagedCode.LlmTck.AzureOpenAI`		Azure OpenAI compatibility profile and future wire contracts.
`ManagedCode.LlmTck.Foundry`		Microsoft Foundry compatibility profile and future wire contracts.
`ManagedCode.LlmTck.Anthropic`		Anthropic Messages compatibility profile and wire contracts.
`ManagedCode.LlmTck.Gemini`		Gemini compatibility profile and future wire contracts.
`ManagedCode.LlmTck.Groq`		Groq compatibility profile and future wire contracts.
`ManagedCode.LlmTck.Mistral`		Mistral compatibility profile and future wire contracts.
`ManagedCode.LlmTck.Ollama`		Ollama compatibility profile and future wire contracts.
`ManagedCode.LlmTck.Cohere`		Cohere compatibility profile and future wire contracts.
`ManagedCode.LlmTck.Bedrock`		Amazon Bedrock compatibility profile and future wire contracts.
`ManagedCode.LlmTck.OpenRouter`		OpenRouter compatibility profile and future wire contracts.
`ManagedCode.LlmTck.DeepSeek`		DeepSeek compatibility profile and future wire contracts.
`ManagedCode.LlmTck.Perplexity`		Perplexity compatibility profile and future wire contracts.
`ManagedCode.LlmTck.Hosting`		ASP.NET Core endpoint mapping.
`ManagedCode.LlmTck.Client`		Control client plus `IChatClient`, `IEmbeddingGenerator<string, Embedding<float>>`, and `IImageGenerator` implementations.
`ManagedCode.LlmTck.Aspire`		Aspire AppHost extension methods.

Provider Packages

Provider packages keep vendor-specific protocol and compatibility metadata out of the provider-neutral runtime. The package surface exists now so applications can select a provider family explicitly; provider-specific DTOs and endpoint mappings are added inside each package as support grows.

Package	Provider ID	Protocol family	Default endpoint shape
`ManagedCode.LlmTck.OpenAI`	`openai`	OpenAI	`/openai/v1`
`ManagedCode.LlmTck.AzureOpenAI`	`azure-openai`	Azure OpenAI	`/azure-openai/openai/deployments/{deployment}/chat/completions`
`ManagedCode.LlmTck.Foundry`	`microsoft-foundry`	Microsoft Foundry	`/microsoft-foundry/models/chat/completions`
`ManagedCode.LlmTck.Anthropic`	`anthropic`	Anthropic Messages	`/anthropic/v1/messages`
`ManagedCode.LlmTck.Gemini`	`gemini`	Gemini	`/gemini/v1beta/models/{model}:generateContent`
`ManagedCode.LlmTck.Groq`	`groq`	Groq	`/groq/openai/v1/chat/completions`
`ManagedCode.LlmTck.Mistral`	`mistral`	Mistral	`/mistral/v1/chat/completions`
`ManagedCode.LlmTck.Ollama`	`ollama`	Ollama	`/ollama/api/chat`
`ManagedCode.LlmTck.Cohere`	`cohere`	Cohere	`/cohere/v2/chat`
`ManagedCode.LlmTck.Bedrock`	`bedrock`	Amazon Bedrock	`/bedrock/model/{modelId}/converse`
`ManagedCode.LlmTck.OpenRouter`	`openrouter`	OpenRouter	`/openrouter/api/v1/chat/completions`
`ManagedCode.LlmTck.DeepSeek`	`deepseek`	DeepSeek	`/deepseek/v1/chat/completions`
`ManagedCode.LlmTck.Perplexity`	`perplexity`	Perplexity	`/perplexity/v1/sonar`

Each provider profile also carries a doc-backed ApiContract with official documentation links, retrieval date, API version or header requirements, documented operations, streaming support, and whether ManagedCode.LlmTck.Hosting currently maps the operation.

Use a provider profile when code needs a stable package-level declaration without starting a host:

using ManagedCode.LlmTck.AzureOpenAI;

var profile = AzureOpenAiCompatibility.Profile;
Console.WriteLine(profile.Id); // azure-openai

Azure OpenAI And Foundry SDKs

Azure OpenAI deployment routes use the deployment name as the TCK model id. API keys sent through the official SDK's api-key header are accepted anywhere a bearer token would be accepted.

using Azure.AI.OpenAI;
using OpenAI.Chat;
using OpenAI.Embeddings;
using System.ClientModel;

var azure = new AzureOpenAIClient(
    new Uri("http://localhost:5000/azure-openai/"),
    new ApiKeyCredential("test-key"));

ChatClient chat = azure.GetChatClient("gpt-4.1-mini");
EmbeddingClient embeddings = azure.GetEmbeddingClient("text-embedding-3-small");

var answer = await chat.CompleteChatAsync(
[
    new UserChatMessage("What is the invoice total?"),
]);

var vector = await embeddings.GenerateEmbeddingAsync("invoice");

Microsoft Foundry / Azure AI Inference clients call the /microsoft-foundry/chat/completions and /microsoft-foundry/embeddings routes. Set Model to the model id configured in LLM TCK.

using Azure;
using Azure.AI.Inference;

var endpoint = new Uri("http://localhost:5000/microsoft-foundry/");
var credential = new AzureKeyCredential("test-key");

var chat = new ChatCompletionsClient(endpoint, credential);
var embeddings = new EmbeddingsClient(endpoint, credential);

var completion = await chat.CompleteAsync(
    new ChatCompletionsOptions(
    [
        new ChatRequestUserMessage("What is the invoice total?"),
    ])
    {
        Model = "gpt-4.1-mini",
    });

var embedding = await embeddings.EmbedAsync(
    new EmbeddingsOptions(["invoice"])
    {
        Model = "text-embedding-3-small",
    });

Configuration Model

LLM TCK is configured with one LlmTckConfiguration.

You can apply that configuration in two places:

at host startup through builder.Services.AddLlmTck(options => ...)
at test runtime through LlmTckClient.ConfigureAsync(...) or POST /admin/llm-tck/configure

Runtime configuration is a replacement, not a merge. Applying a new configuration resets scenario positions and assertion events. The runtime snapshots the configuration, so later mutations to your builder/list objects do not change the running provider.

The default configuration advertises four model IDs:

Model ID	Kind
`llm-tck-chat`	Chat
`llm-tck-embedding`	Embedding
`llm-tck-image`	Image
`llm-tck-audio`	Audio

Model IDs and model kinds are enforced. A chat request to an embedding model, or an embedding request to an unknown model, returns 404 with llm_tck_unknown_model.

ASP.NET Core Startup

The simplest host config is one chat scenario:

using ManagedCode.LlmTck.Hosting;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddLlmTck(options =>
{
    options.AddChatScenario(
        "blue-whale",
        scenario => scenario
            .ForModel("llm-tck-chat")
            .WhenUserContains("largest animal")
            .Responds("blue whale", "blue ", "whale"));
});

var app = builder.Build();
app.MapLlmTck();
app.Run();

A fuller config can advertise your application's real model names while still serving deterministic fixtures:

using ManagedCode.LlmTck.Hosting;
using ManagedCode.LlmTck.Models;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddLlmTck(options =>
{
    options.RequireBearerToken("test-key");

    options.AddModel("gpt-4.1-mini", LlmTckModelKind.Chat);
    options.AddModel("text-embedding-3-small", LlmTckModelKind.Embedding);
    options.AddModel("gpt-image-1", LlmTckModelKind.Image);
    options.AddModel("gpt-4o-mini-tts", LlmTckModelKind.Audio);

    options.WithDefaultEmbeddingVector(0.125f, 0.25f, 0.5f, 1.0f);
    options.WithDefaultAudio(File.ReadAllBytes("fixtures/ok.wav"), "audio/wav");

    options.AddChatScenario(
        "invoice-total",
        scenario => scenario
            .ForModel("gpt-4.1-mini")
            .WhenUserContains("invoice total")
            .Responds(
                "{\"total\":42.50}",
                "{\"total\":",
                "42.50",
                "}"));

    options.AddChatScenario(
        "provider-rate-limit",
        scenario => scenario
            .ForModel("gpt-4.1-mini")
            .WhenUserContains("rate limit")
            .Fails(429, "rate_limit_exceeded", "Scripted rate limit from LLM TCK."));

    options.AddChatScenario(
        "slow-response",
        scenario => scenario
            .ForModel("gpt-4.1-mini")
            .WhenUserContains("slow")
            .Responds("done")
            .DelaysBy(250));
});

var app = builder.Build();
app.MapLlmTck();
app.Run();

Aspire AppHost

Install the Aspire integration package in the AppHost:

dotnet add package ManagedCode.LlmTck.Aspire --version 0.0.9

Then add the package-owned TCK resource directly:

using ManagedCode.LlmTck.Aspire;

var builder = DistributedApplication.CreateBuilder(args);

var llmTck = builder
    .AddLlmTck()
    .WithApiKey("test-key");

builder
    .AddProject<Projects.Consumer>("consumer")
    .WithReference(llmTck)
    .WithEnvironment("LLM_PROVIDER_ENDPOINT", llmTck.GetHttpEndpoint())
    .WaitFor(llmTck);

builder.Build().Run();

AddLlmTck() creates a LlmTckResource backed by the packaged .NET LLM TCK service executable and exposes its http endpoint. It does not require a consumer service project reference, generated Projects.* metadata type, project path, Docker, or a container runtime. Consumer resources should reference the TCK resource, wait for it, and use llmTck.GetHttpEndpoint() when they need the provider-compatible base URL. .WithApiKey("test-key") sets LlmTck:RequiredBearerToken so both provider endpoints and /admin/llm-tck/* control endpoints require the same bearer token.

Use AddLlmTckContainer() only when you explicitly want a container-backed resource, for example for a deployment or container-runtime smoke test. The container mode uses the matching versioned image such as ghcr.io/managedcode/llm-tck:0.0.9; it is not the default local Aspire path.

Control Panel And Token Usage

Open the TCK resource endpoint from the Aspire dashboard and add /admin/llm-tck to inspect the running configuration. If the resource was configured with .WithApiKey("test-key") or RequireBearerToken("test-key"), enter the same token in the control panel before refreshing.

The panel reads /admin/llm-tck/models and /admin/llm-tck/assertions. It shows advertised models, assertion counters, request and response previews, and deterministic token usage. Token usage is counted with the repo-owned tiktoken-compatible counter and reported both as summary totals and per runtime event with inputTokens, outputTokens, and totalTokens. Provider response envelopes also receive the same deterministic usage values: OpenAI-compatible chat and Responses usage, Anthropic sync messages and streaming message_start/message_delta, Gemini usageMetadata including long-running video operation results, Cohere chat usage, Ollama prompt/eval counts, and Bedrock Converse usage.

Chat Scenarios

Contains Match

WhenUserContains(...) adds a user-message contains matcher. All configured match messages must be present somewhere in the actual request.

options.AddChatScenario(
    "support-refund",
    scenario => scenario
        .ForModel("gpt-4.1-mini")
        .WhenUserContains("refund")
        .Responds("I can help with a refund."));

Exact Match

Use exact matching when a test must prove the complete prompt contract:

using ManagedCode.LlmTck.Scenarios;

options.AddChatScenario(
    "exact-system-contract",
    scenario => scenario
        .ForModel("gpt-4.1-mini")
        .WithExactMatch(
            new LlmTckMessage { Role = "system", Content = "Return JSON only." },
            new LlmTckMessage { Role = "user", Content = "Give me the invoice total." })
        .Responds("{\"total\":42.50}"));

Queued Responses

Each matched request consumes the next response. This makes multi-turn flows deterministic:

options.AddChatScenario(
    "two-turn-plan",
    scenario => scenario
        .ForModel("gpt-4.1-mini")
        .WhenUserContains("make a plan")
        .Responds("First draft")
        .Responds("Revised draft"));

After the queue is exhausted, the provider returns 409 with llm_tck_scenario_exhausted.

Streaming Chunks

The first Responds argument is the non-streaming response. The remaining arguments are the streaming chunks:

options.AddChatScenario(
    "streaming-answer",
    scenario => scenario
        .ForModel("gpt-4.1-mini")
        .WhenUserContains("stream this")
        .Responds("blue whale", "blue ", "whale"));

For stream: true, LLM TCK emits SSE data: frames and a final data: [DONE] marker. All chunks in one response share the same response id.

Scripted Errors

Use Fails(...) to test client error handling without waiting for a real provider to fail:

options.AddChatScenario(
    "content-policy-error",
    scenario => scenario
        .ForModel("gpt-4.1-mini")
        .WhenUserContains("blocked fixture")
        .Fails(400, "content_filter", "The scripted request was rejected."));

Delays And Cancellation

Use DelaysBy(...) to test timeout and cancellation behavior:

options.AddChatScenario(
    "timeout-path",
    scenario => scenario
        .ForModel("gpt-4.1-mini")
        .WhenUserContains("slow path")
        .Responds("eventual answer")
        .DelaysBy(1_500));

If the caller cancels during the delay, the queued response is not consumed.

Scenario-Specific Token

A scenario can require its own bearer token:

options.AddChatScenario(
    "tenant-a-only",
    scenario => scenario
        .ForModel("gpt-4.1-mini")
        .RequireBearerToken("tenant-a-key")
        .WhenUserContains("tenant secret")
        .Responds("tenant-a response"));

Use either a global token or per-scenario tokens. If both are configured for a scenario, the same request must satisfy both checks, so different global and scenario tokens intentionally make that scenario unreachable.

Modalities

Embeddings

Configure a fixed embedding vector. Every input value receives the same deterministic vector:

options.AddModel("text-embedding-3-small", LlmTckModelKind.Embedding);
options.WithDefaultEmbeddingVector(0.01f, 0.02f, 0.03f, 0.04f);

using ManagedCode.LlmTck.Client;
using Microsoft.Extensions.AI;

using var httpClient = new HttpClient { BaseAddress = new Uri("http://localhost:5000") };
using IEmbeddingGenerator<string, Embedding<float>> embeddings =
    new LlmTckEmbeddingGenerator(httpClient, "text-embedding-3-small");

var result = await embeddings.GenerateAsync(["alpha", "beta"]);

Images

Image generation returns a deterministic base64 PNG by default:

options.AddModel("gpt-image-1", LlmTckModelKind.Image);

using ManagedCode.LlmTck.Client;
using Microsoft.Extensions.AI;

using var httpClient = new HttpClient { BaseAddress = new Uri("http://localhost:5000") };
using IImageGenerator images = new LlmTckImageGenerator(httpClient, "gpt-image-1");

var image = await images.GenerateAsync(
    new ImageGenerationRequest { Prompt = "compatibility marker" });

Audio

Audio speech returns deterministic bytes with a matching media type. The default fixture is a minimal WAV payload:

options.AddModel("gpt-4o-mini-tts", LlmTckModelKind.Audio);
options.WithDefaultAudio(File.ReadAllBytes("fixtures/speech.wav"), "audio/wav");

var audio = await httpClient.PostAsJsonAsync(
    "/openai/v1/audio/speech",
    new { model = "gpt-4o-mini-tts", input = "hello", voice = "alloy" });

audio.EnsureSuccessStatusCode();
var bytes = await audio.Content.ReadAsByteArrayAsync();

Runtime Reconfiguration

Use runtime reconfiguration when each test needs a different provider script.

Configure Before A Test

Use ManagedCode.LlmTck.Client as the universal test control surface. A test can start or discover the hosted TCK, create one LlmTckClient, load all required models, datasets, scripted responses, errors, embeddings, images, and audio fixtures, and then call the provider through regular clients.

using ManagedCode.LlmTck.Client;
using Microsoft.Extensions.AI;

using var httpClient = new HttpClient { BaseAddress = new Uri("http://localhost:5000") };
var tck = LlmTckClient.Create(httpClient, bearerToken: "test-key");

await tck.ConfigureAsync(
    config => config
        .RequireBearerToken("test-key")
        .UseChatModel("test-chat")
        .UseEmbeddingModel("test-embedding")
        .UseImageModel("test-image")
        .UseAudioModel("test-audio")
        .UseEmbeddingVector(0.9f, 0.8f)
        .UseImageDataUri("data:image/png;base64,Zm9v")
        .UseAudio([1, 2, 3, 4], "audio/test")
        .UseDataset(
            "checkout-flow",
            dataset => dataset
                .AddChatScenario(
                    "invoice-total",
                    scenario => scenario
                        .ForModel("test-chat")
                        .WhenUserContains("invoice total")
                        .Responds("{\"total\":42.50}", "{\"total\":", "42.50", "}"))
                .AddChatScenario(
                    "provider-rate-limit",
                    scenario => scenario
                        .ForModel("test-chat")
                        .WhenUserContains("rate limit")
                        .Fails(429, "rate_limit_exceeded", "Scripted rate limit."))),
    resetFirst: true);

using IChatClient chat = tck.CreateChatClient("test-chat");
using IEmbeddingGenerator<string, Embedding<float>> embeddings =
    tck.CreateEmbeddingGenerator("test-embedding");
using IImageGenerator images = tck.CreateImageGenerator("test-image");

var response = await chat.GetResponseAsync(
[
    new ChatMessage(ChatRole.User, "What is the invoice total?"),
]);

var vector = await embeddings.GenerateAsync(["invoice"]);
var image = await images.GenerateAsync(new ImageGenerationRequest { Prompt = "fixture" });
var audio = await tck.GenerateAudioAsync("test-audio", "speak this");
var assertions = await tck.GetAssertionsAsync();

resetFirst: true clears previous assertion events and scenario positions before the new configuration is applied. Named datasets are active after configuration; they group scenarios so a test can load a whole conversation set without flattening everything by hand.

Configure With `LlmTckClient`

using ManagedCode.LlmTck.Client;
using ManagedCode.LlmTck.Configuration;
using ManagedCode.LlmTck.Models;

using var httpClient = new HttpClient { BaseAddress = new Uri("http://localhost:5000") };
httpClient.DefaultRequestHeaders.Authorization =
    new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", "test-key");

var control = new LlmTckClient(httpClient);

var configuration = new LlmTckConfigurationBuilder()
    .RequireBearerToken("test-key")
    .AddModel("docs-chat", LlmTckModelKind.Chat)
    .AddModel("docs-embedding", LlmTckModelKind.Embedding)
    .AddChatScenario(
        "docs-blue-whale",
        scenario => scenario
            .ForModel("docs-chat")
            .WhenUserContains("largest animal")
            .Responds("blue whale", "blue ", "whale"))
    .WithDefaultEmbeddingVector(0.125f, 0.25f)
    .Build();

await control.ConfigureAsync(configuration);

If the current runtime already has a bearer token, the configure request must use that current token. If the new configuration sets a different token, future provider and control requests must use the new token.

Configure With Raw JSON

POST /admin/llm-tck/configure accepts readable enum values such as "chat" and "contains":

curl -X POST http://localhost:5000/admin/llm-tck/configure \
  -H "content-type: application/json" \
  -H "authorization: Bearer test-key" \
  -d '{
    "requiredBearerToken": "test-key",
    "models": [
      { "id": "docs-chat", "kind": "chat" },
      { "id": "docs-embedding", "kind": "embedding" },
      { "id": "docs-image", "kind": "image" },
      { "id": "docs-audio", "kind": "audio" }
    ],
    "chatScenarios": [
      {
        "id": "docs-blue-whale",
        "modelId": "docs-chat",
        "match": {
          "mode": "contains",
          "messages": [
            { "role": "user", "content": "largest animal" }
          ]
        },
        "responses": [
          {
            "content": "blue whale",
            "streamChunks": [ "blue ", "whale" ]
          }
        ]
      }
    ],
    "defaultEmbeddingVector": [ 0.125, 0.25 ],
    "defaultAudioMediaType": "audio/wav"
  }'

Then call the configured model:

curl -X POST http://localhost:5000/openai/v1/chat/completions \
  -H "content-type: application/json" \
  -H "authorization: Bearer test-key" \
  -d '{
    "model": "docs-chat",
    "messages": [
      { "role": "user", "content": "What is the largest animal?" }
    ]
  }'

Microsoft.Extensions.AI

Use the client package when you want tests to call the fake provider through normal Microsoft.Extensions.AI abstractions:

using ManagedCode.LlmTck.Client;
using Microsoft.Extensions.AI;

using var httpClient = new HttpClient
{
    BaseAddress = new Uri("http://localhost:5000"),
};
httpClient.DefaultRequestHeaders.Authorization =
    new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", "test-key");

using IChatClient chatClient = new LlmTckChatClient(httpClient);

var response = await chatClient.GetResponseAsync(
[
    new ChatMessage(ChatRole.User, "What color is the largest animal?"),
]);

Console.WriteLine(response.Text);

When a bearer token is required, prefer creating the modality clients from the control client so the same token is applied consistently:

var tck = LlmTckClient.Create(httpClient, bearerToken: "test-key");

using IChatClient chatClient = tck.CreateChatClient("docs-chat");
using IEmbeddingGenerator<string, Embedding<float>> embeddings =
    tck.CreateEmbeddingGenerator("docs-embedding");
using IImageGenerator images = tck.CreateImageGenerator("docs-image");

Streaming uses the same client:

await foreach (var update in chatClient.GetStreamingResponseAsync(
    [new ChatMessage(ChatRole.User, "stream this")]))
{
    Console.Write(update.Text);
}

Assertions And Reset

The assertion summary is intentionally simple: it tells the test whether requests matched configured scenarios, missed, failed auth, used unknown models, exhausted a response queue, or returned scripted errors.

var control = new LlmTckClient(httpClient);
var assertions = await control.GetAssertionsAsync();

if (assertions.Unmatched > 0 || assertions.ModelNotFound > 0)
{
    throw new InvalidOperationException("The application called an unexpected LLM path.");
}

Reset clears assertion events and scenario response positions without replacing the current configuration:

await control.ResetAsync();

Endpoints

GET /openai/v1/models
POST /openai/v1/chat/completions
POST /openai/v1/embeddings
POST /openai/v1/images/generations
POST /openai/v1/audio/speech
POST /anthropic/v1/messages
POST /gemini/v1beta/models/{model}:generateContent
POST /mistral/v1/chat/completions
POST /ollama/api/chat
POST /cohere/v2/chat
POST /bedrock/model/{modelId}/converse
POST /azure-openai/openai/deployments/{deployment}/chat/completions
POST /microsoft-foundry/chat/completions
GET /admin/llm-tck/models
GET /admin/llm-tck/assertions
POST /admin/llm-tck/configure
POST /admin/llm-tck/reset

Audio speech returns a deterministic WAV fixture by default.

When requiredBearerToken is configured, all endpoints above require Authorization: Bearer <token>.

Common Test Shapes

Replace Provider Calls In Integration Tests

Start the LLM TCK service in Aspire or an ASP.NET Core test host.
Configure the expected scenarios for the test.
Point the application under test at the LLM TCK base URL.
Run the application flow.
Assert GetAssertionsAsync() has no unmatched, unknown-model, auth-failed, or exhausted events.

Test Retry Logic

options.AddChatScenario(
    "retry-then-success",
    scenario => scenario
        .ForModel("gpt-4.1-mini")
        .WhenUserContains("retry")
        .Fails(429, "rate_limit_exceeded", "Try again.")
        .Responds("success after retry"));

Test Prompt Contract Drift

options.AddChatScenario(
    "exact-contract",
    scenario => scenario
        .ForModel("gpt-4.1-mini")
        .WithExactMatch(
            new LlmTckMessage { Role = "system", Content = "Return JSON only." },
            new LlmTckMessage { Role = "user", Content = "Summarize invoice INV-42." })
        .Responds("{\"summary\":\"paid\"}"));

If the application changes the prompt, the request becomes unmatched and the test can fail on the assertion summary.

Test Wrong Model Wiring

Configure only the models your application is allowed to call:

options.AddModel("approved-chat-model", LlmTckModelKind.Chat);

If the app calls old-chat-model or calls approved-chat-model through the embeddings endpoint, LLM TCK returns llm_tck_unknown_model.

Build And Test

dotnet restore ManagedCode.LlmTck.slnx
dotnet build ManagedCode.LlmTck.slnx --configuration Release --no-restore
dotnet test tests/ManagedCode.LlmTck.Tests/ManagedCode.LlmTck.Tests.csproj --configuration Release --no-build --verbosity normal
for project in src/*/*.csproj; do dotnet pack "$project" --configuration Release --no-build --output artifacts/packages; done

global.json opts dotnet test into Microsoft.Testing.Platform for .NET 10.

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
.codex/skills		.codex/skills
.github/workflows		.github/workflows
docs		docs
samples		samples
site		site
src		src
tests		tests
.editorconfig		.editorconfig
.gitignore		.gitignore
AGENTS.md		AGENTS.md
Directory.Build.props		Directory.Build.props
Directory.Packages.props		Directory.Packages.props
LICENSE		LICENSE
ManagedCode.LlmTck.slnx		ManagedCode.LlmTck.slnx
README.md		README.md
dotnet-tools.json		dotnet-tools.json
global.json		global.json
nuget.config		nuget.config

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

LLM TCK

Packages

Provider Packages

Azure OpenAI And Foundry SDKs

Configuration Model

ASP.NET Core Startup

Aspire AppHost

Control Panel And Token Usage

Chat Scenarios

Contains Match

Exact Match

Queued Responses

Streaming Chunks

Scripted Errors

Delays And Cancellation

Scenario-Specific Token

Modalities

Embeddings

Images

Audio

Runtime Reconfiguration

Configure Before A Test

Configure With LlmTckClient

Configure With Raw JSON

Microsoft.Extensions.AI

Assertions And Reset

Endpoints

Common Test Shapes

Replace Provider Calls In Integration Tests

Test Retry Logic

Test Prompt Contract Drift

Test Wrong Model Wiring

Build And Test

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 6

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Configure With `LlmTckClient`

Packages