API-CONTEXTS.md

API Contexts: Maintaining Consistency Across Mock API Calls

New in v2.2.0: Dynamic context memory with automatic expiration and intelligent shared data extraction!

When building and testing applications, one of the biggest challenges with traditional mock APIs is their stateless nature. Each request returns completely random data with no relationship to previous calls. If you fetch a user with ID 123, then fetch their orders, there's no guarantee the order will reference that same user ID.

API Contexts solve this problem by giving your mock API a memory that's now even smarter and more dynamic.

The Problem: Stateless Chaos

Traditional mock APIs generate data independently for each request:

sequenceDiagram
    participant Client
    participant MockAPI
    participant LLM

    Client->>MockAPI: GET /users/1
    MockAPI->>LLM: Generate user data
    LLM-->>MockAPI: {"id": 42, "name": "Alice"}
    MockAPI-->>Client: User data

    Client->>MockAPI: GET /orders?userId=42
    MockAPI->>LLM: Generate order data
    LLM-->>MockAPI: {"userId": 99, ...}
    MockAPI-->>Client: Order data (userId mismatch!)

Loading

Notice the problem? The user had ID 42, but the order came back with userId 99. There's no consistency between related calls.

The Solution: Contextual Memory

With API Contexts, the mock API maintains a shared context across related requests:

sequenceDiagram
    participant Client
    participant MockAPI
    participant Context as Context Manager
    participant LLM

    Client->>MockAPI: GET /users/1?context=session-1
    MockAPI->>Context: Get history for "session-1"
    Context-->>MockAPI: (empty - first call)
    MockAPI->>LLM: Generate user (no context)
    LLM-->>MockAPI: {"id": 42, "name": "Alice"}
    MockAPI->>Context: Store: GET /users/1 → {"id": 42, ...}
    MockAPI-->>Client: User data

    Client->>MockAPI: GET /orders?context=session-1
    MockAPI->>Context: Get history for "session-1"
    Context-->>MockAPI: Previous call: user with id=42, name=Alice
    MockAPI->>LLM: Generate order (with context history)
    LLM-->>MockAPI: {"userId": 42, "customerName": "Alice", ...}
    MockAPI->>Context: Store: GET /orders → {"userId": 42, ...}
    MockAPI-->>Client: Order data (consistent!)

Loading

Now the LLM sees the previous user call and generates orders that reference the same user ID and name. The data forms a coherent story.

How It Works

Architecture

The context system consists of three main components:

graph TD
    A[HTTP Request] --> B[ContextExtractor]
    B --> C{Context Name?}
    C -->|Yes| D[OpenApiContextManager]
    C -->|No| E[Generate without context]
    D --> F[Retrieve Context History]
    F --> G[PromptBuilder]
    E --> G
    G --> H[LLM]
    H --> I[Response]
    I --> J{Context Name?}
    J -->|Yes| K[Store in Context]
    J -->|No| L[Return Response]
    K --> L

Loading

1. ContextExtractor - Extracts the context name from the request 2. OpenApiContextManager - Manages context storage and retrieval 3. PromptBuilder - Includes context history in LLM prompts

Context Storage

New in v2.2.0: Contexts are stored using ASP.NET Core's IMemoryCache with automatic expiration instead of a basic ConcurrentDictionary. This provides intelligent lifecycle management with zero configuration needed.

Dynamic Memory Management

The new MemoryCacheContextStore provides automatic cleanup:

public class MemoryCacheContextStore : IContextStore
{
    private readonly IMemoryCache _cache;
    private readonly TimeSpan _slidingExpiration; // Default: 15 minutes

    public ApiContext GetOrAdd(string contextName, Func<string, ApiContext> factory)
    {
        var cacheKey = GetCacheKey(contextName);

        // Try to get existing context
        if (_cache.TryGetValue<ApiContext>(cacheKey, out var existingContext))
        {
            // Touch the cache to refresh sliding expiration
            SetContextInCache(cacheKey, contextName, existingContext);
            return existingContext;
        }

        // Create new context with automatic expiration
        var newContext = factory(contextName);
        SetContextInCache(cacheKey, contextName, newContext);

        return newContext;
    }

    private void SetContextInCache(string cacheKey, string contextName, ApiContext context)
    {
        var options = new MemoryCacheEntryOptions
        {
            SlidingExpiration = _slidingExpiration, // Resets on every access!
            Priority = CacheItemPriority.Normal
        };

        // Auto-cleanup callback when context expires
        options.RegisterPostEvictionCallback((key, value, reason, state) =>
        {
            if (reason == EvictionReason.Expired)
            {
                _logger.LogInformation(
                    "Context '{ContextName}' expired after {Minutes} minutes of inactivity",
                    contextName, _slidingExpiration.TotalMinutes);
            }
        });

        _cache.Set(cacheKey, context, options);
    }
}

public class OpenApiContextManager
{
    private readonly IContextStore _contextStore; // Now uses IMemoryCache under the hood
    private const int MaxRecentCalls = 15;

    public void AddToContext(
        string contextName,
        string method,
        string path,
        string? requestBody,
        string responseBody)
    {
        var context = _contextStore.GetOrAdd(contextName, _ => new ApiContext
        {
            Name = contextName,
            CreatedAt = DateTimeOffset.UtcNow,
            RecentCalls = new List<RequestSummary>(),
            SharedData = new Dictionary<string, string>(),
            TotalCalls = 0
        });

        context.RecentCalls.Add(new RequestSummary
        {
            Timestamp = DateTimeOffset.UtcNow,
            Method = method,
            Path = path,
            RequestBody = requestBody,
            ResponseBody = responseBody
        });

        // NEW: Dynamic extraction of ALL fields (v2.2.0)
        ExtractAllFields(context, responseBody);

        if (context.RecentCalls.Count > MaxRecentCalls)
        {
            SummarizeOldCalls(context);
        }
    }
}

Why IMemoryCache?

Automatic Lifecycle Management

• Contexts expire after 15 minutes of inactivity (configurable)
• Each API call using a context refreshes the timer (sliding expiration)
• No manual cleanup needed - abandoned contexts disappear automatically
• Zero risk of memory leaks from forgotten test sessions

Perfect for Testing Workflows

• Start testing immediately - contexts created on-demand
• Continue testing as long as you need - timer keeps resetting
• Walk away from your desk - contexts clean themselves up
• CI/CD friendly - no state accumulation between runs

Configuration

{
  "mostlylucid.mockllmapi": {
    "ContextExpirationMinutes": 15  // Default: 15 (range: 5-1440)
  }
}

Recommendations

• 5 minutes: Quick unit tests, CI/CD pipelines
• 15 minutes: Default - general development testing (recommended)
• 30 minutes: Complex multi-step workflows
• 60+ minutes: Long exploratory testing sessions

Automatic Summarization

To prevent context from growing indefinitely and exceeding LLM token limits, the system automatically summarizes old calls when the count exceeds 15:

graph LR
    A[20+ Calls] --> B[Keep 15 Most Recent]
    A --> C[Summarize Older Calls]
    B --> D[Full Request/Response]
    C --> E[Summary: 'GET /users - called 5 times']
    D --> F[Included in LLM Prompt]
    E --> F

Loading

private void SummarizeOldCalls(ApiContext context)
{
    var toSummarize = context.RecentCalls
        .Take(context.RecentCalls.Count - MaxRecentCalls)
        .ToList();

    var summary = new StringBuilder();
    summary.AppendLine($"Earlier calls ({toSummarize.Count}):");

    var groupedByPath = toSummarize
        .GroupBy(c => $"{c.Method} {c.Path.Split('?')[0]}");

    foreach (var group in groupedByPath)
    {
        summary.AppendLine($"  {group.Key} - called {group.Count()} time(s)");
    }

    context.ContextSummary = summary.ToString();
    context.RecentCalls.RemoveRange(0, toSummarize.Count);
}

Shared Data Extraction

New in v2.2.0: The context manager now dynamically extracts ALL fields from responses, not just hardcoded ones!

Intelligent Recursive Extraction

The new ExtractAllFields method recursively walks through your response and captures everything:

private void ExtractAllFields(ApiContext context, string responseBody, int maxDepth = 2)
{
    using var doc = JsonDocument.Parse(responseBody);
    var root = doc.RootElement;

    // Extract based on response type
    if (root.ValueKind == JsonValueKind.Array && root.GetArrayLength() > 0)
    {
        // Store array length
        context.SharedData[$"{arrayName}.length"] = root.GetArrayLength().ToString();

        // Extract all fields from first item
        var firstItem = root[0];
        ExtractFieldsRecursive(context, firstItem, $"{arrayName}[0]", currentDepth: 0);

        // LEGACY: Also extract with old naming for backward compatibility
        ExtractValueIfExists(context, firstItem, "id", "lastId");
    }
    else if (root.ValueKind == JsonValueKind.Object)
    {
        // Extract ALL fields recursively up to maxDepth levels
        ExtractFieldsRecursive(context, root, "", currentDepth: 0);

        // LEGACY: Also extract with old naming for backward compatibility
        ExtractValueIfExists(context, root, "id", "lastId");
        ExtractValueIfExists(context, root, "userId", "lastUserId");
    }
}

private void ExtractFieldsRecursive(
    ApiContext context,
    JsonElement element,
    string prefix,
    int currentDepth)
{
    if (currentDepth >= maxDepth) return;

    foreach (var property in element.EnumerateObject())
    {
        var key = string.IsNullOrEmpty(prefix)
            ? property.Name
            : $"{prefix}.{property.Name}";

        switch (property.Value.ValueKind)
        {
            case JsonValueKind.String:
            case JsonValueKind.Number:
            case JsonValueKind.True:
            case JsonValueKind.False:
                // Store primitive values
                context.SharedData[key] = property.Value.ToString();
                break;

            case JsonValueKind.Object:
                // Recurse into nested objects
                ExtractFieldsRecursive(context, property.Value, key, currentDepth + 1);
                break;

            case JsonValueKind.Array:
                // Store array length and first item
                context.SharedData[$"{key}.length"] = property.Value.GetArrayLength().ToString();
                if (property.Value.GetArrayLength() > 0)
                {
                    ExtractFieldsRecursive(context, property.Value[0], $"{key}[0]", currentDepth + 1);
                }
                break;
        }
    }
}

What This Means For You

Before (v2.1.0): Only id, userId, name, email were extracted

// Response
{"orderId": 123, "sku": "WIDGET-01", "price": 49.99}

// Extracted (only 'id' field)
{"lastId": "123"}

Now (v2.2.0): Everything is extracted automatically

// Response
{
  "orderId": 123,
  "sku": "WIDGET-01",
  "price": 49.99,
  "customer": {
    "customerId": 456,
    "tier": "gold",
    "address": {"city": "Seattle"}
  },
  "items": [
    {"productId": 789, "quantity": 2}
  ]
}

// Extracted (ALL fields up to 2 levels deep)
{
  "orderId": "123",
  "sku": "WIDGET-01",
  "price": "49.99",
  "customer.customerId": "456",
  "customer.tier": "gold",
  "customer.address.city": "Seattle",  // 2 levels deep
  "items.length": "1",
  "items[0].productId": "789",
  "items[0].quantity": "2",
  // LEGACY keys for backward compatibility:
  "lastId": "123"
}

Real-World Impact

E-Commerce Example

### First call - create order
POST /api/mock/orders?context=checkout
{"orderId": 0, "sku": "string", "customer": {"tier": "string"}}

Response: {"orderId": 123, "sku": "WIDGET-01", "customer": {"tier": "gold"}}

### Second call - create shipment
POST /api/mock/shipping?context=checkout
{"shipmentId": 0, "orderId": 0}

Response: {"shipmentId": 456, "orderId": 123}  ← Same orderId!

### Third call - apply discount
POST /api/mock/discounts?context=checkout
{"discountId": 0, "customerTier": "string", "sku": "string"}

Response: {
  "discountId": 789,
  "customerTier": "gold",    ← Consistent tier!
  "sku": "WIDGET-01",        ← Consistent SKU!
  "discount": 0.15
}

The LLM sees all the extracted fields in the prompt and maintains consistency across domain-specific fields like SKUs, tier levels, account numbers, reference codes, etc.

Benefits

• No configuration needed - works out of the box
• Supports ANY domain (healthcare, finance, gaming, etc.)
• Nested relationships automatically tracked
• Array data preserved
• Backward compatible with v2.1.0 code

Using Contexts

Three Ways to Specify Context

You can pass the context name in three different ways, with this precedence order:

1. Query Parameter (highest priority)

GET /api/mock/users?context=my-session
GET /api/mock/users?api-context=my-session

2. HTTP Header

GET /api/mock/users
X-Api-Context: my-session

3. Request Body

POST /api/mock/orders
Content-Type: application/json

{
  "context": "my-session",
  "shape": {"orderId": 0, "userId": 0}
}

Supported Endpoint Types

Contexts work across all endpoint types:

REST APIs

GET /api/mock/users/123?context=session-1

Streaming APIs

GET /api/mock/stream/stock-prices?context=trading-session
Accept: text/event-stream

GraphQL

POST /graphql?context=my-app
Content-Type: application/json

{
  "query": "{ users { id name } }"
}

gRPC

# JSON over HTTP
POST /api/grpc/json/UserService/CreateUser?context=grpc-session
Content-Type: application/json

{
  "name": "Alice",
  "email": "alice@example.com"
}

# Binary Protobuf (context extracted from query param or headers)
POST /api/grpc/proto/UserService/GetUser?context=grpc-session
Content-Type: application/grpc+proto

[binary protobuf data]

OpenAPI

# Dynamically generated endpoints support contexts
GET /petstore/pets/123?context=petstore-session
POST /shop/orders?context=shop-session

SignalR (via configuration)

{
  "mostlylucid.mockllmapi": {
    "HubContexts": [
      {
        "Name": "stock-ticker",
        "Description": "Real-time stock prices",
        "ApiContextName": "stocks-session",
        "Shape": "{\"symbol\":\"string\",\"price\":0}"
      }
    ]
  }
}

Real-World Use Cases

Use Case 1: E-Commerce Flow

Simulate a complete shopping experience with consistent user and order data:

### 1. Create user
POST /api/mock/users?context=checkout-flow
{
  "shape": {
    "userId": 0,
    "name": "string",
    "email": "string",
    "address": {"street": "string", "city": "string"}
  }
}

### Response: {"userId": 42, "name": "Alice", ...}

### 2. Create cart (will reference same user)
POST /api/mock/cart?context=checkout-flow
{
  "shape": {
    "cartId": 0,
    "userId": 0,
    "items": [{"productId": 0, "quantity": 0}]
  }
}

### Response: {"cartId": 123, "userId": 42, ...}

### 3. Create order (consistent user and cart)
POST /api/mock/orders?context=checkout-flow
{
  "shape": {
    "orderId": 0,
    "userId": 0,
    "cartId": 0,
    "total": 0
  }
}

### Response: {"orderId": 789, "userId": 42, "cartId": 123, ...}

Use Case 2: Stock Price Simulation

Generate realistic stock price movements instead of random values:

### First call - establishes baseline
GET /api/mock/stocks?context=market-data
    &shape={"symbol":"string","price":0,"volume":0}

### Response: {"symbol": "ACME", "price": 145.50, "volume": 10000}

### Second call - price changes realistically
GET /api/mock/stocks?context=market-data
    &shape={"symbol":"string","price":0,"volume":0}

### Response: {"symbol": "ACME", "price": 146.20, "volume": 12000}
### Notice: Same symbol, price increased by $0.70 (realistic)

### Third call - continues the trend
GET /api/mock/stocks?context=market-data
    &shape={"symbol":"string","price":0,"volume":0}

### Response: {"symbol": "ACME", "price": 145.80, "volume": 11500}
### Notice: Price fluctuates but stays in realistic range

Without context, each call would return a completely random symbol and price. With context, the LLM maintains the same stock and adjusts prices realistically.

Use Case 3: Game State Progression

Track player progress through game sessions:

### Start game
POST /api/mock/game/start?context=game-session-123
{
  "shape": {
    "playerId": 0,
    "level": 0,
    "health": 0,
    "score": 0,
    "inventory": []
  }
}

### Response: {"playerId": 42, "level": 1, "health": 100, "score": 0}

### Complete quest
POST /api/mock/game/quest?context=game-session-123
{
  "shape": {
    "playerId": 0,
    "level": 0,
    "score": 0,
    "reward": {"item": "string", "value": 0}
  }
}

### Response: {"playerId": 42, "level": 2, "score": 500,
###           "reward": {"item": "Sword", "value": 100}}
### Notice: Same player, level increased, score increased

### Get stats
GET /api/mock/game/player?context=game-session-123
    &shape={"playerId":0,"level":0,"health":0,"score":0}

### Response: {"playerId": 42, "level": 2, "health": 100, "score": 500}
### Notice: Consistent with quest completion

Context Management API

List All Contexts

GET /api/openapi/contexts

### Response:
{
  "contexts": [
    {
      "name": "session-1",
      "totalCalls": 5,
      "recentCallCount": 5,
      "sharedDataCount": 3,
      "createdAt": "2025-01-15T10:00:00Z",
      "lastUsedAt": "2025-01-15T10:05:00Z",
      "hasSummary": false
    }
  ],
  "count": 1
}

Get Context Details

GET /api/openapi/contexts/session-1

### Response shows full context including:
### - All recent calls with timestamps
### - Extracted shared data (IDs, names, emails)
### - Summary of older calls (if any)

Clear a Specific Context

DELETE /api/openapi/contexts/session-1

Clear All Contexts

DELETE /api/openapi/contexts

Implementation Details

Integration in Request Handlers

Each request handler (REST, Streaming, GraphQL, SignalR) follows the same pattern:

public async Task<string> HandleRequestAsync(
    string method,
    string fullPathWithQuery,
    string? body,
    HttpRequest request,
    HttpContext context,
    CancellationToken cancellationToken = default)
{
    // 1. Extract context name from request
    var contextName = _contextExtractor.ExtractContextName(request, body);

    // 2. Get context history if context specified
    var contextHistory = !string.IsNullOrWhiteSpace(contextName)
        ? _contextManager.GetContextForPrompt(contextName)
        : null;

    // 3. Build prompt with context history
    var prompt = _promptBuilder.BuildPrompt(
        method, fullPathWithQuery, body, shapeInfo,
        streaming: false, contextHistory: contextHistory);

    // 4. Get response from LLM
    var response = await _llmClient.GetCompletionAsync(prompt, cancellationToken);

    // 5. Store in context if context name provided
    if (!string.IsNullOrWhiteSpace(contextName))
    {
        _contextManager.AddToContext(
            contextName, method, fullPathWithQuery, body, response);
    }

    return response;
}

Context in LLM Prompts

When a context exists, its history is included in the LLM prompt:

TASK: Generate a varied mock API response.
RULES: Output ONLY valid JSON. No markdown, no comments.

API Context: session-1
Total calls in session: 3

Shared data to maintain consistency:
  lastId: 42
  lastName: Alice
  lastEmail: alice@example.com

Recent API calls:
  [10:00:05] GET /users/42
    Response: {"id": 42, "name": "Alice", "email": "alice@example.com"}
  [10:00:12] GET /orders?userId=42
    Response: {"orderId": 123, "userId": 42, "items": [...]}

Generate a response that maintains consistency with the above context.

Method: POST
Path: /shipping/123
Body: {"orderId": 123}

The LLM sees all previous calls and generates responses that reference the same IDs, names, and other data, maintaining consistency.

Best Practices

1. Use Descriptive Context Names

BAD:  ?context=test1
GOOD: ?context=user-checkout-flow-jan15

2. Clear Contexts When Done

Contexts persist in memory until explicitly cleared or the server restarts:

### After completing your test scenario
DELETE /api/openapi/contexts/user-checkout-flow-jan15

3. Share Contexts Across Related Endpoints

Use the same context name for all related calls:

GET /api/mock/users?context=demo-session
GET /api/mock/orders?context=demo-session
GET /api/mock/shipping?context=demo-session

4. Monitor Context Size

Check context details to see how many calls are stored:

GET /api/openapi/contexts/demo-session

If you have many calls (>100), consider clearing and starting fresh to avoid prompt length issues.

5. Combine with OpenAPI Specs

For maximum realism, use contexts with OpenAPI specs:

### Load spec with context
POST /api/openapi/specs
{
  "name": "petstore",
  "source": "https://petstore3.swagger.io/api/v3/openapi.json",
  "basePath": "/petstore",
  "contextName": "petstore-session"
}

### All petstore endpoints will share the same context

Performance Considerations

Memory Usage (v2.2.0 Update)

Automatic Memory Management 🎉

With v2.2.0's sliding expiration, memory usage is now self-regulating:

Each active context stores:

• Up to 15 recent calls (full request/response)
• Summary of older calls (compressed)
• Extracted shared data (now comprehensive - all fields)
• Typical memory per context: ~50-300 KB depending on response sizes

Automatic Cleanup:

• Contexts expire after 15 minutes of inactivity (configurable)
• No indefinite accumulation
• Memory naturally stabilizes based on your testing activity
• Perfect for CI/CD pipelines - no state buildup between runs

Example Scenario:

Test Session (30 contexts created over 2 hours):
- Active tests (10 contexts): ~2.5 MB (recently used)
- Expired contexts (20 contexts): ~0 MB (automatically removed)
Total Memory: ~2.5 MB (down from potential 7.5 MB)

Thread Safety

OpenApiContextManager + MemoryCacheContextStore are fully thread-safe:

• IMemoryCache handles concurrent access automatically
• ConcurrentDictionary tracks context names
• Multiple requests can safely read/write contexts concurrently
• No locking needed in your code

LLM Token Limits (v2.2.0 Update)

Context history is included in every prompt.

Token Impact of v2.2.0 Changes:

• Shared Data: Increased (~50-200 extra tokens per context)

  • More fields extracted = longer shared data section
  • Nested fields use dot notation (compact)
  • Typical: "customer.tier": "gold" = 10 tokens

• Recent Calls: Unchanged (same 15-call limit)

• Overall Impact: Minor increase (5-10% more tokens)

  • Offset by automatic expiration (fewer contexts exist)
  • Summarization still prevents unbounded growth

Token Budget Example:

Without context: ~500 tokens (prompt only)
With context (v2.1): ~1,500 tokens (prompt + 5 calls)
With context (v2.2): ~1,700 tokens (prompt + 5 calls + comprehensive shared data)

Still well within most LLM limits (4K-128K tokens)

Mitigation Strategies (if needed):

1. Lower ContextExpirationMinutes: 5-10 minutes for rapid turnover
2. Clear contexts periodically: DELETE /api/openapi/contexts
3. Use shorter response shapes: Less data to extract
4. Smaller model context windows: Works fine with 4K token models

Limitations

Current Limitations

1. In-Memory Only - Contexts are lost on server restart

   • v2.2.0 Note: This is now a feature - no cleanup needed after restarts!

2. No Persistence - Not suitable for production data storage

   • Contexts are for testing/development only
   • Use a real database for production state

3. Single Server - Contexts don't sync across multiple server instances

   • Load-balanced environments: each server has independent contexts
   • Workaround: Use sticky sessions or single server for testing

4. LLM Dependent - Consistency quality depends on the LLM's ability to follow instructions

   • Better models (GPT-4, Claude) = better consistency
   • Local models (Llama, Mistral) = good but occasional misses

Non-Limitations (Fixed in v2.2.0) ✅

Memory Leaks from Abandoned Contexts → FIXED

• Contexts now expire automatically after 15 minutes
• No manual cleanup required

Limited Field Extraction → FIXED

• All fields now extracted automatically (nested, arrays, etc.)
• Works with any domain without configuration

Conclusion

API Contexts transform stateless mock APIs into stateful simulations that maintain consistency across related calls. Whether you're testing an e-commerce flow, simulating real-time data feeds, or building game state progression, contexts ensure your mock data tells a coherent story.

With v2.2.0, contexts have become even more powerful and intelligent:

• Zero maintenance: Automatic expiration means you never have to clean up
• Universal compatibility: Dynamic extraction works with any API domain
• Production-ready: Memory-efficient for CI/CD and long-running services
• Backward compatible: Upgrade without changing a single line of code

The combination of automatic memory management, comprehensive shared data extraction, intelligent summarization, and seamless integration across all endpoint types makes contexts a powerful tool for realistic API testing and development.

For more advanced scenarios, combine contexts with OpenAPI specifications to create fully-featured mock APIs that behave like the real thing.

---

Additional Resources

• Context Examples (HTTP file) - Runnable HTTP examples covering all mock types
• Context Memory Guide - Comprehensive guide with detailed examples and troubleshooting
• gRPC Support - Using contexts with gRPC services
• OpenAPI Features - Context support with dynamically generated endpoints

---

Ready to try v2.2.0? See the RELEASE_NOTES.md for upgrade instructions and new features.