Skip to content

Clarify benchmark performance/cost wording in docs #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ethanj merged 1 commit into from
May 22, 2026
+7 −7
Merged

Clarify benchmark performance/cost wording in docs #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,17 +30,17 @@ TypeScript packages, framework adapters, host plugins, and public smoke tests.

## Headline Results

AtomicMemory v66 is cost-Pareto SOTA on BEAM-100K, BEAM-1M, and LoCoMo10 under
AtomicMemory v66 is leading performance/cost on BEAM-100K, BEAM-1M, and LoCoMo10 under
matched methodology against published competitors. On BEAM-10M it matches the
strongest published Mem0-new result while leaving Hindsight-scale temporal
retrieval as the known open frontier.

| Benchmark | AtomicMemory v66 | Position | Cost/Q | Sample |
|---|---:|---|---:|---:|
| **BEAM-100K lenient** | **0.7375** | Parity with Hindsight at 0.75 | $1.26 | n=80 |
| **BEAM-1M lenient** | **0.6625** | Cost-Pareto SOTA; +0.022 vs Mem0 paper | $0.083 | n=80 |
| **BEAM-1M lenient** | **0.6625** | Leading Performance/Cost; +0.022 vs Mem0 paper | $0.083 | n=80 |
| **BEAM-10M lenient** | **0.4875** | Parity with Mem0-new at 0.486 | $0.081 | n=80 |
| **LoCoMo10 GPT-4o-mini binary** | **0.8396** | Cost-Pareto SOTA; +0.171 vs Mem0 paper | $0.066 | n=1540 |
| **LoCoMo10 GPT-4o-mini binary** | **0.8396** | Leading Performance/Cost; +0.171 vs Mem0 paper | $0.066 | n=1540 |

These results put AtomicMemory at or near the published ceiling in each
reported category while preserving the lower-cost operating profile that
Expand Down
6 changes: 3 additions & 3 deletions packages/core/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ Docker-deployable memory backend with durable context, semantic retrieval, and m

## Headline Results

AtomicMemory v66 is cost-Pareto SOTA on BEAM-100K, BEAM-1M, and LoCoMo10 under matched methodology against published competitors. On BEAM-10M it matches the strongest published Mem0-new result while leaving Hindsight-scale temporal retrieval as the known open frontier.
AtomicMemory v66 is leading performance/cost on BEAM-100K, BEAM-1M, and LoCoMo10 under matched methodology against published competitors. On BEAM-10M it matches the strongest published Mem0-new result while leaving Hindsight-scale temporal retrieval as the known open frontier.

| Benchmark | AtomicMemory v66 | Position | Cost/Q | Sample |
|---|---:|---|---:|---:|
| **BEAM-100K lenient** | **0.7375** | Parity with Hindsight at 0.75 | $1.26 | n=80 |
| **BEAM-1M lenient** | **0.6625** | Cost-Pareto SOTA; +0.022 vs Mem0 paper | $0.083 | n=80 |
| **BEAM-1M lenient** | **0.6625** | Leading Performance/Cost; +0.022 vs Mem0 paper | $0.083 | n=80 |
| **BEAM-10M lenient** | **0.4875** | Parity with Mem0-new at 0.486 | $0.081 | n=80 |
| **LoCoMo10 GPT-4o-mini binary** | **0.8396** | Cost-Pareto SOTA; +0.171 vs Mem0 paper | $0.066 | n=1540 |
| **LoCoMo10 GPT-4o-mini binary** | **0.8396** | Leading Performance/Cost; +0.171 vs Mem0 paper | $0.066 | n=1540 |

These results put AtomicMemory at or near the published ceiling in each reported category while preserving the lower-cost operating profile that matters for real applications. Reproducibility artifacts and harness details will be published with the benchmark materials.

Expand Down
2 changes: 1 addition & 1 deletion packages/sdk/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Backend-agnostic memory-layer SDK — pluggable providers, local embeddings, sto

**Docs:** [docs.atomicstrata.ai/sdk](https://docs.atomicstrata.ai/sdk)

AtomicMemory Core currently reaches cost-Pareto SOTA on BEAM-100K, BEAM-1M, and LoCoMo10, with BEAM-10M parity against the strongest published Mem0-new result. The SDK is the typed application surface for building on that memory layer.
AtomicMemory Core currently reaches leading performance/cost on BEAM-100K, BEAM-1M, and LoCoMo10, with BEAM-10M parity against the strongest published Mem0-new result. The SDK is the typed application surface for building on that memory layer.

## What this package provides

Expand Down
Loading