Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,11 @@ This repository contains the canonical set of flag configurations and evaluation
```
ffe-system-test-data/
├── ufc-config.json # Master flag configuration (UFC format)
└── evaluation-cases/
└── test-*.json # Evaluation test case files
├── evaluation-cases/
│ └── test-*.json # Evaluation test case files
└── precomputed/
├── README.md # Precomputed assignment fixture format
└── cases/*.json # Client-side precompute response fixtures
```

## Usage
Expand Down Expand Up @@ -106,6 +109,15 @@ The shared fixtures intentionally exclude SDK-specific fields such as `variant`
- **variant**: Derive from the flag configuration in `ufc-config.json` by matching the result value
- **flagMetadata**: Extract from the flag's metadata field in `ufc-config.json`

### Precomputed Assignment Fixtures (`precomputed/cases/*.json`)

Precomputed fixtures cover SDKs that consume Datadog's
`/precompute-assignments` API instead of evaluating UFC locally. Each case
contains the context, mocked precompute response, typed evaluations to run, and
expected exposure/flagevaluation emission counts.

See [precomputed/README.md](precomputed/README.md) for the schema.

## Evaluation Cases

| File | Description |
Expand Down
37 changes: 37 additions & 0 deletions precomputed/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Precomputed Assignment Fixtures

These fixtures cover the Datadog precompute assignment API consumed by mobile
and browser SDKs that do not run the full UFC evaluator locally.

Each case contains:

- `context`: the evaluation context sent to `/precompute-assignments`.
- `response`: a JSON:API-style precompute response.
- `evaluations`: typed client calls to run against the response.
- `expectedEmissions`: expected exposure and flagevaluation HTTP emissions after
the evaluations and an explicit flush.

The precompute response uses the assignment shape returned to client SDKs:

```json
{
"data": {
"attributes": {
"flags": {
"flag-key": {
"allocationKey": "allocation-a",
"variationKey": "variation-a",
"variationType": "boolean|string|integer|float|object",
"variationValue": true,
"reason": "TARGETING_MATCH",
"doLog": true
}
}
}
}
}
```

Downstream SDKs should validate typed values, details metadata, persistence
behavior, exposure emission gates, and flagevaluation aggregation using these
fixtures. Live Datadog credentials are not required.
133 changes: 133 additions & 0 deletions precomputed/cases/all-types-success.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
{
"name": "all-types-success",
"description": "Successful precomputed assignments for all supported Flutter value types.",
"context": {
"targetingKey": "precomputed-user",
"attributes": {
"plan": "pro",
"platform": "flutter"
}
},
"response": {
"data": {
"attributes": {
"flags": {
"flutter.fixture.enabled": {
"allocationKey": "allocation-success",
"variationKey": "enabled",
"variationType": "boolean",
"variationValue": true,
"reason": "TARGETING_MATCH",
"doLog": true
},
"flutter.fixture.title": {
"allocationKey": "allocation-success",
"variationKey": "copy-a",
"variationType": "string",
"variationValue": "Datadog Flags",
"reason": "TARGETING_MATCH",
"doLog": true
},
"flutter.fixture.limit": {
"allocationKey": "allocation-success",
"variationKey": "limit-five",
"variationType": "integer",
"variationValue": 5,
"reason": "TARGETING_MATCH",
"doLog": true
},
"flutter.fixture.ratio": {
"allocationKey": "allocation-success",
"variationKey": "half",
"variationType": "float",
"variationValue": 0.5,
"reason": "TARGETING_MATCH",
"doLog": true
},
"flutter.fixture.config": {
"allocationKey": "allocation-success",
"variationKey": "object-a",
"variationType": "object",
"variationValue": {
"showBanner": true,
"colors": [
"blue",
"green"
]
},
"reason": "TARGETING_MATCH",
"doLog": true
}
}
}
}
},
"evaluations": [
{
"flag": "flutter.fixture.enabled",
"variationType": "boolean",
"defaultValue": false,
"result": {
"value": true,
"variant": "enabled",
"reason": "TARGETING_MATCH",
"error": null
}
},
{
"flag": "flutter.fixture.title",
"variationType": "string",
"defaultValue": "Fallback title",
"result": {
"value": "Datadog Flags",
"variant": "copy-a",
"reason": "TARGETING_MATCH",
"error": null
}
},
{
"flag": "flutter.fixture.limit",
"variationType": "integer",
"defaultValue": 0,
"result": {
"value": 5,
"variant": "limit-five",
"reason": "TARGETING_MATCH",
"error": null
}
},
{
"flag": "flutter.fixture.ratio",
"variationType": "float",
"defaultValue": 0.0,
"result": {
"value": 0.5,
"variant": "half",
"reason": "TARGETING_MATCH",
"error": null
}
},
{
"flag": "flutter.fixture.config",
"variationType": "object",
"defaultValue": {},
"result": {
"value": {
"showBanner": true,
"colors": [
"blue",
"green"
]
},
"variant": "object-a",
"reason": "TARGETING_MATCH",
"error": null
}
}
],
"expectedEmissions": {
"exposures": 5,
"flagevaluationRequests": 1,
"flagevaluationEvents": 5
}
}
94 changes: 94 additions & 0 deletions precomputed/cases/defaults-and-emission-gates.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
{
"name": "defaults-and-emission-gates",
"description": "Precomputed assignments that validate exposure gates, unknown variation isolation, and default/error flagevaluation payloads.",
"context": {
"targetingKey": "precomputed-user",
"attributes": {
"plan": "free",
"platform": "flutter"
}
},
"response": {
"data": {
"attributes": {
"flags": {
"flutter.fixture.silent": {
"allocationKey": "allocation-silent",
"variationKey": "silent-on",
"variationType": "boolean",
"variationValue": true,
"reason": "TARGETING_MATCH",
"doLog": false
},
"flutter.fixture.string": {
"allocationKey": "allocation-string",
"variationKey": "copy-b",
"variationType": "string",
"variationValue": "Actual string",
"reason": "TARGETING_MATCH",
"doLog": true
},
"flutter.fixture.unknown": {
"allocationKey": "allocation-unknown",
"variationKey": "unknown-a",
"variationType": "unsupported",
"variationValue": "Ignored",
"reason": "TARGETING_MATCH",
"doLog": true
}
}
}
}
},
"evaluations": [
{
"flag": "flutter.fixture.silent",
"variationType": "boolean",
"defaultValue": false,
"result": {
"value": true,
"variant": "silent-on",
"reason": "TARGETING_MATCH",
"error": null
}
},
{
"flag": "flutter.fixture.string",
"variationType": "boolean",
"defaultValue": false,
"result": {
"value": false,
"variant": null,
"reason": null,
"error": "typeMismatch"
}
},
{
"flag": "flutter.fixture.unknown",
"variationType": "boolean",
"defaultValue": false,
"result": {
"value": false,
"variant": null,
"reason": null,
"error": "flagNotFound"
}
},
{
"flag": "flutter.fixture.missing",
"variationType": "string",
"defaultValue": "fallback",
"result": {
"value": "fallback",
"variant": null,
"reason": null,
"error": "flagNotFound"
}
}
],
"expectedEmissions": {
"exposures": 0,
"flagevaluationRequests": 1,
"flagevaluationEvents": 4
}
}