DynamoDB Conformance Suite

An independent test suite that validates any DynamoDB-compatible endpoint against real DynamoDB behaviour. It works against DynamoDB, DynamoDB Local, Dynoxide, Dynalite, LocalStack, ExtendDB, Floci, Ministack, or anything else that implements the DynamoDB HTTP API.

Why this exists

There's no official AWS conformance suite for DynamoDB. The closest thing the community has is Dynalite's test suite, but over half of its tests are stale against current DynamoDB behaviour (verified March 2026). DynamoDB Local ships with no test suite at all. Every emulator author ends up guessing at behaviour and testing against their own assumptions.

This suite fixes that by running every test against real DynamoDB first, recording what passes, and using those results as the baseline. An emulator only passes if it gives the same answer DynamoDB does.

Quick start

npm install

# Run against a local target
DYNAMODB_ENDPOINT=http://localhost:8000 npm test

# Quicker run, excludes GSI lifecycle tests (see runtime notes below)
DYNAMODB_ENDPOINT=http://localhost:8000 npm run test:quick

# Run a specific tier
DYNAMODB_ENDPOINT=http://localhost:8000 npm run test:tier1

Results

Scored against real DynamoDB in eu-west-2; behaviour varies by region and over time, so these are point-in-time figures.

Target	Tier 1	Tier 2	Tier 3	Total	Pass	Fail	Skip	Version	Date
DynamoDB	100%	100%	100%	100%	873	0	0	live (AWS)	2026-07-06
Dynoxide	100.0%	100.0%	100.0%	100.0%	859	0	14	0.11.3	2026-07-06
Ministack	91.0%	81.9%	81.6%	86.5%	755	118	0	e99876293803	2026-07-06
ExtendDB	85.0%	89.3%	86.4%	86.0%	714	116	43	v0.1.1	2026-07-06
Floci	91.7%	77.4%	77.2%	84.7%	732	132	9	51c2a38d394f	2026-07-06
LocalStack	93.0%	87.8%	67.3%	84.0%	727	138	8	2026.6.1	2026-07-06
DynamoDB Local	91.7%	84.5%	66.9%	82.7%	711	149	13	d89f8fcc6b1a	2026-07-06
Dynalite	91.3%	12.6%	68.8%	75.2%	605	200	68	4.0.0	2026-07-06

Live results: paritysuite.org - the full table for every target, tracked run over run.

DynamoDB is the ground truth. The percentage is correctness over the operations a target implements - Pass / (Pass + Fail). Skips are not counted against it. A skipped test is deliberate: each test file probes for feature support in beforeAll and skips itself when the target doesn't implement that operation, so a skip records honest scope (an operation the target chose not to implement), not a correctness gap. The Skip column keeps those visible; a Fail is a behaviour that diverges from real DynamoDB.

This table is regenerated by the Update Results Table workflow — automatically when a Conformance Tests run finishes on main, and on demand from the Actions tab. It pulls each target's result artifact, fills the Version (npm version, container image digest, release tag, or live for real AWS) and Run date columns from the run, and commits the refreshed table. Run npm run results:table to preview it locally.

Tiers

Tier 1 - Core. The operations and behaviours that 90% of DynamoDB users rely on. CRUD, queries, scans, batch operations, GSIs, UpdateTable. If an emulator fails Tier 1, it's not usable.

Tier 2 - Complete. Less common but documented features. Transactions, PartiQL, LSIs, TTL, streams, tags. An emulator that passes Tier 1 but fails some Tier 2 is usable with caveats.

Tier 3 - Strict. Validation ordering, error behaviour at a range of strictness (exact where DynamoDB's wording is stable, structural where its rendering is non-deterministic), edge cases around limits, legacy API compatibility (ScanFilter, QueryFilter). An emulator that passes Tier 1 and Tier 2 but fails some Tier 3 is production-quality for local dev.

The tiers give emulator authors something meaningful to report. "100% Tier 1, 95% Tier 2, 80% Tier 3" tells you far more than a single percentage.

Tier 3 structure

Tier 3 splits into four sub-directories by what each test asserts:

validation-ordering/ - which validation fires first when a request has multiple problems. Uses toContain against the message; the wording can drift, the ordering should not.
error-messages/ - the error DynamoDB returns. Uses inline try/catch with expect(err).toBeInstanceOf(...) and expect(err.name).toBe(...); the message is matched exactly where it's stable and structurally (toContain on the field and constraint) where AWS's rendering varies by region or SDK version.
limits/ - hard-coded service limits and the errors that fire when you cross them (item size, batch size, response size, transaction size).
legacy-api/ - the older request shapes (AttributeUpdates, QueryFilter, ScanFilter, Expected, AttributesToGet) for backwards compatibility.

A new test goes in whichever sub-directory matches what it asserts. If you care about the message the service returns, that's error-messages/. If you only care which error fires, that's validation-ordering/.

Filtering by feature

Tiers tell you how strict a target is. Tags tell you which capabilities it implements, and they're an independent axis: a test lives in one tier but carries a tag for the operation it covers, a data-plane or control-plane tag, and any cross-cutting trait that applies. That lets you ask a narrower question than the tier score - "how does this target do on just the features I actually use?"

Filter with vitest's --tags-filter:

# Ignore PartiQL
DYNAMODB_ENDPOINT=http://localhost:8000 npx vitest run --tags-filter='!partiql'

# Ignore the legacy request parameters (AttributeUpdates, QueryFilter, ...)
DYNAMODB_ENDPOINT=http://localhost:8000 npx vitest run --tags-filter='!legacy'

# Only item reads and writes, no table management
DYNAMODB_ENDPOINT=http://localhost:8000 npx vitest run --tags-filter='data-plane'

# Drop the operations no emulator implements
DYNAMODB_ENDPOINT=http://localhost:8000 npx vitest run --tags-filter='!cloud-only'

# Compose them, and with tiers: transactions only, excluding cloud-only async
DYNAMODB_ENDPOINT=http://localhost:8000 npx vitest run tests/tier2 --tags-filter='transactions and !cloud-only'

The grammar takes and / &&, not / !, or, parentheses, and prefix/* wildcards, and it composes with the tier scripts and directory paths.

The vocabulary lives in one place, src/tags.ts, and stays honest two ways: strictTags rejects an undeclared tag the moment the suite runs, and a coverage guard (npm run test:tooling) fails if any test is left untagged. So an exclusion like !partiql can't silently miss a test that someone forgot to tag.

Operation tags - one per test, matching the operation it exercises.

Tag	Plane	Operation
`put-item`	data-plane	PutItem
`get-item`	data-plane	GetItem
`update-item`	data-plane	UpdateItem
`delete-item`	data-plane	DeleteItem
`query`	data-plane	Query
`scan`	data-plane	Scan
`batch`	data-plane	BatchGetItem, BatchWriteItem
`transactions`	data-plane	TransactWriteItems, TransactGetItems
`partiql`	data-plane	ExecuteStatement, BatchExecuteStatement, ExecuteTransaction
`create-table`	control-plane	CreateTable
`update-table`	control-plane	UpdateTable
`delete-table`	control-plane	DeleteTable
`describe-table`	control-plane	DescribeTable
`list-tables`	control-plane	ListTables
`ttl`	control-plane	UpdateTimeToLive, DescribeTimeToLive
`streams`	control-plane	DynamoDB Streams
`resource-tags`	control-plane	TagResource, UntagResource, ListTagsOfResource
`backups`	control-plane	On-demand backups, point-in-time recovery
`export-import`	control-plane	ExportTableToPointInTime, ImportTable
`kinesis`	control-plane	Kinesis streaming destinations
`contributor-insights`	control-plane	UpdateContributorInsights
`resource-policy`	control-plane	PutResourcePolicy, GetResourcePolicy, DeleteResourcePolicy
`account`	control-plane	DescribeLimits, DescribeEndpoints

Cross-cutting tags - applied wherever they fit.

Tag	Meaning
`data-plane`	Reads or writes items
`control-plane`	Manages tables, indexes, or table-level features
`cloud-only`	No emulator implements it; needs real AWS infrastructure, another AWS service, or account/region context
`gsi`	Exercises Global Secondary Indexes
`lsi`	Exercises Local Secondary Indexes
`legacy`	Deprecated request parameters (AttributeUpdates, QueryFilter, ScanFilter, Expected, AttributesToGet)
`slow`	Long-running against real AWS; the set `test:gating` excludes
`negative-path`	Asserts only rejections: every case expects a validation error, conditional-check failure, or transaction cancellation

Running against targets

DynamoDB Local

docker run -d --name ddb-local -p 8000:8000 amazon/dynamodb-local:latest
DYNAMODB_ENDPOINT=http://localhost:8000 npm test
docker stop ddb-local && docker rm ddb-local

Dynoxide

dynoxide --port 8001 &
DYNAMODB_ENDPOINT=http://localhost:8001 npm test
kill %1

Dynalite

npx dynalite --port 8002 &
DYNAMODB_ENDPOINT=http://localhost:8002 npm test
kill %1

LocalStack

LocalStack requires a free account. Sign up at localstack.cloud and set your auth token.

export LOCALSTACK_AUTH_TOKEN=your-token-here
docker run -d --name localstack -p 4566:4566 -e LOCALSTACK_AUTH_TOKEN localstack/localstack
DYNAMODB_ENDPOINT=http://localhost:4566 npm test
docker stop localstack && docker rm localstack

Ministack

docker run -d --name ministack -p 4566:4566 ministackorg/ministack:latest
DYNAMODB_ENDPOINT=http://localhost:4566 npm test
docker stop ministack && docker rm ministack

Floci

docker run -d --name floci -p 4566:4566 floci/floci:latest
DYNAMODB_ENDPOINT=http://localhost:4566 npm test
docker stop floci && docker rm floci

ExtendDB

ExtendDB is heavier than the other local targets: it builds from source (Rust), stores data in PostgreSQL 14+, mandates TLS, and verifies SigV4 against a local IAM store. scripts/run-extenddb.sh automates the whole bring-up (build, init, a dynamodb:* IAM user, access key, serve) against a PostgreSQL instance, and the CI job uses it. To wire it up by hand instead, build ExtendDB, run extenddb init, create an IAM user with a dynamodb:* policy plus an access key (ExtendDB getting-started guide, "Post-init workflow"), then start it and point the suite at it:

./target/release/extenddb serve --config extenddb.toml   # https://127.0.0.1:8000

# The JS SDK ignores AWS_CA_BUNDLE; trust the self-signed cert via NODE_EXTRA_CA_CERTS.
export NODE_EXTRA_CA_CERTS=~/.extenddb/tls/cert.pem
export AWS_ACCESS_KEY_ID=<access-key-id>        # a real key — ExtendDB verifies the signature
export AWS_SECRET_ACCESS_KEY=<secret-access-key>
export AWS_REGION=us-east-1                      # SigV4 signing region for the local endpoint, not the ground-truth region (eu-west-2)
export DYNAMODB_ENDPOINT=https://127.0.0.1:8000
CONFORMANCE_TARGET=extenddb npm test            # writes results/extenddb.json

Use 127.0.0.1 or localhost (both are in the cert's SANs). ExtendDB does not implement PartiQL, so those Tier 2 tests skip.

Real DynamoDB

# Uses the default AWS credential chain (env vars, ~/.aws/credentials, IAM role)
npm test

Expected runtime

Target	`npm test`	`npm run test:quick`
Local emulators	~2-5 seconds	~2-5 seconds
Real DynamoDB	~60-90 minutes	~20-25 minutes

The full suite includes 11 UpdateTable GSI lifecycle tests that add and remove Global Secondary Indexes from existing tables. On real DynamoDB, each GSI creation triggers a backfill that takes 5-15 minutes even on empty tables. These tests are important for conformance but they dominate runtime against real AWS.

test:quick excludes the GSI lifecycle tests for faster local iteration. CI's gating real-DynamoDB job runs test:gating, which drops the GSI lifecycle tests and the S3 and Kinesis integration suites (see "Operations no emulator implements" below), so a slow async import can't redden the build. Those integration suites still run against real AWS in a separate non-gating job via npm run test:integrations. Emulator targets run the full npm test since GSI creation is instant locally. If you're modifying GSI-related code, run the full suite against real DynamoDB manually before merging.

Design principles

Ground truth first. Every test is validated against real DynamoDB, observed in the eu-west-2 region. If DynamoDB's behaviour changes, the suite updates.

Observable behaviour only. Tests verify what comes back over the wire: response bodies, error types, error messages. No testing of internal implementation details.

SDK-driven. Tests use the AWS SDK v3 for JavaScript rather than raw HTTP. This tests what real applications actually experience.

Endpoint-agnostic. A single environment variable (DYNAMODB_ENDPOINT) points the suite at any target. No target-specific code paths, no special cases.

Test organisation

tests/
  tier1/                    # ~300 tests
    createTable/            # basic, gsi, lsi
    putItem/                # basic, conditions, validation, expressions, dataTypes, ...
    getItem/                # basic, validation, projection, consumedCapacity
    deleteItem/             # basic, validation
    updateItem/             # basic, conditions, validation, paths
    query/                  # basic, gsi, lsi, expressions, select, numericKeys, binaryKeys
    scan/                   # basic, validation, gsi, lsi, parallel, select, filterOperators
    batchWriteItem/         # basic, validation
    batchGetItem/           # basic, validation
    deleteTable/            # basic
    describeTable/          # basic
    listTables/             # basic
    updateTable/            # basic
  tier2/                    # ~100 tests
    transactions/           # transactWrite, transactGet
    partiql/                # executeStatement, batchExecuteStatement, executeTransaction
    ttl/                    # basic
    streams/                # basic
    tags/                   # basic
    updateTable/            # gsi
  tier3/                    # ~195 tests
    validation-ordering/    # per-operation validation error ordering
    error-messages/         # exact error message strings
    limits/                 # itemSize, batchLimits, responseSize, transactionLimits,
                            # numberPrecision, emptyValues, reservedWords
    legacy-api/             # expected, attributeUpdates, queryFilter, scanFilter, attributesToGet

Shared infrastructure

src/client.ts - DynamoDB and Streams client, configured from the DYNAMODB_ENDPOINT env var
src/helpers.ts - table lifecycle, assertion helpers (expectDynamoError, cleanupItems, waitForGsiConsistency)
src/setup.ts - global beforeAll/afterAll that creates 5 shared tables
src/types.ts - TestTableDef and KeyDef types

Generating results

# Run against a target and save JSON output
DYNAMODB_ENDPOINT=http://localhost:8000 npx vitest run --reporter=json --outputFile=results/dynamodb-local.json

# Generate the comparison table from all saved results
npm run results:table

SDK blindspots

This suite uses the AWS SDK v3 (not raw HTTP), which means it can't test:

Request signing validation - the SDK always signs correctly
Error wire format - __type field naming, message vs Message casing
Content-type handling - the SDK always sends application/x-amz-json-1.0
Connection-level behaviour - HTTP headers, chunked encoding, CRC32 checks

You'd need a raw HTTP test layer using fetch() with aws4 signing for those. The dynalite test suite is a good reference for that approach.

Contributing

Adding tests

Follow existing patterns in the relevant tier directory
Use expectDynamoError() for error assertions, not try/catch
Use cleanupItems() in afterAll for data cleanup
Use ExpressionAttributeNames for all attribute names in expressions (avoid reserved words)
Use ConsistentRead: true on all read-back assertions
Test against real DynamoDB first - if AWS fails, the test is wrong by definition

Adding a target

Start the target on a port
Run: DYNAMODB_ENDPOINT=http://localhost:<port> npx vitest run --reporter=json --outputFile=results/<target>.json
Generate the table: npm run results:table
Submit a PR with the results JSON

If the target speaks HTTPS only or verifies request signatures (ExtendDB is the first such target), two extra steps apply: trust its certificate with NODE_EXTRA_CA_CERTS=/path/to/cert.pem (the JS SDK does not read AWS_CA_BUNDLE), and pass a real AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY whose policy allows the operations the suite exercises. Before committing the results JSON, grep it for your key to be sure no credential leaked into it.

Test data

All test data must be synthetic. Don't use real names, emails, addresses, or any personally identifiable information in test fixtures.

Operations covered

Operation	Tier 1	Tier 2	Tier 3
PutItem	basic, conditions (incl. parens), validation, expressions, dataTypes, consumedCapacity, itemCollectionMetrics		error messages
GetItem	basic, validation, projection, consumedCapacity		error messages
UpdateItem	basic, conditions (incl. parens, non-existent key branch), validation, paths		error messages
DeleteItem	basic, conditions (incl. parens), validation		error messages
Query	basic, GSI, LSI, expressions (incl. KeyCondition + Filter parens), select, numericKeys, binaryKeys, pagination		error messages, validation ordering
Scan	basic, validation, GSI (incl. pagination), LSI (incl. pagination), parallel, select, filterOperators, filterExpression parens		error messages, validation ordering
BatchWriteItem	basic, validation		error messages
BatchGetItem	basic, validation		error messages
CreateTable	basic, GSI, LSI		error messages, validation ordering
DeleteTable	basic
DescribeTable	basic
ListTables	basic
UpdateTable	basic (throughput, billing mode)	GSI lifecycle
TransactWriteItems		basic, conditions (incl. parens, non-existent key branch), idempotency, cancellation	error messages
TransactGetItems		basic, validation	error messages
ExecuteStatement		INSERT, SELECT, UPDATE, DELETE, parameterised
BatchExecuteStatement		batch, partial failure
ExecuteTransaction		atomic, rollback
UpdateTimeToLive		enable, validation
DescribeTimeToLive		describe
TagResource		add, list, remove, validation
DynamoDB Streams		ListStreams, DescribeStream, GetRecords, view types
Backups		on-demand, continuous (PITR)
ExportTableToPointInTime / ImportTable		S3 export and import
Kinesis streaming destination		enable, describe, disable
UpdateContributorInsights		enable, describe, list
Resource policies		put, get, delete
DescribeLimits / DescribeEndpoints		account reads

Operations every emulator skips

A handful of operations only exist on real AWS or reach into another AWS service, so no emulator implements them and each one skips on every target. The suite still exercises them against real DynamoDB - characterising AWS's own behaviour has value - and they all carry the cloud-only tag, so --tags-filter='!cloud-only' drops the lot:

Import/Export to S3
Kinesis Data Streams integration (streaming destinations)
On-demand backups and Point-in-Time Recovery
Contributor Insights
Resource-based policies
Account reads (DescribeLimits, DescribeEndpoints)

Import/Export and Kinesis lean on slow async control-plane calls that make poor gate material, so they run in a separate non-gating job via npm run test:integrations rather than on the gating run.

Genuinely not covered, with no tests yet:

Global Tables
DynamoDB Accelerator (DAX)

Citing a finding

When the suite surfaces a divergence in a target and you want to reference it from that target's own issue tracker, cite the suite as the independent source it is. The reference carries weight precisely because the suite is not the engine's own test harness: it scores every target against the same live-AWS baseline, so "the conformance suite flags this" says more than a self-written test can.

Fill in the bracketed parts. The block is the same whichever engine the finding concerns:

Found by the Parity Suite (paritysuite.org), an independent DynamoDB conformance suite that scores multiple engines against live AWS DynamoDB.

Operation: [e.g. CreateTable] Expected (real DynamoDB, [region, e.g. eu-west-2]): [the exact response or error message real AWS returns] Observed in [target] [version]: [what the target did instead] Suite test: [public link to the specific test, pinned to a commit or tag]

Two details keep the citation honest:

Link the specific test, and pin it. Use a commit SHA or tag (.../blob/<sha>/...), never .../blob/main/...: a main link rots the moment the file is reformatted or the lines shift, while a pinned link points at the exact assertion for good. Link the test itself, not a bare in-repo path, so it resolves for anyone reading the issue.
Pinned test for a specific finding; site row only for a general claim. The pinned test is durable evidence that this exact case diverged. The row on paritysuite.org is a live score that moves with every run, so it answers "how does this engine do overall", not "what broke here". Don't cite a moving score as evidence for a fixed bug.

Real AWS DynamoDB is the ground truth here as everywhere: the "expected" line is what AWS does, captured against a named region, not what any emulator does.

Community

Contributing and AGENTS.md - how to add tests and targets.
Code of Conduct - the Contributor Covenant.
Security policy - how to report a vulnerability or a leaked credential privately.
Support - where to ask for help.

Licence

Apache License 2.0. See LICENSE and NOTICE.

Name		Name	Last commit message	Last commit date
Latest commit History 193 Commits
.github		.github
captures		captures
results		results
scripts		scripts
src		src
tests		tests
.editorconfig		.editorconfig
.env.example		.env.example
.gitignore		.gitignore
.nvmrc		.nvmrc
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
NOTICE		NOTICE
README.md		README.md
SECURITY.md		SECURITY.md
SUPPORT.md		SUPPORT.md
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json
vitest.config.ts		vitest.config.ts
vitest.tooling.config.ts		vitest.tooling.config.ts

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

DynamoDB Conformance Suite

Why this exists

Quick start

Results

Tiers

Tier 3 structure

Filtering by feature

Running against targets

DynamoDB Local

Dynoxide

Dynalite

LocalStack

Ministack

Floci

ExtendDB

Real DynamoDB

Expected runtime

Design principles

Test organisation

Shared infrastructure

Generating results

SDK blindspots

Contributing

Adding tests

Adding a target

Test data

Operations covered

Operations every emulator skips

Citing a finding

Community

Licence

About

Topics

Resources

License

Code of conduct

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Uh oh!

Contributors

Uh oh!

Languages