Compare commits

...

1 Commits

Author SHA1 Message Date
kartik-mem0 9897951783 docs: update README benchmarks to current temporal-reasoning results
Refresh the README benchmark table and Research Highlights to match the
current numbers on the Memory Evaluation page (docs/core-concepts/
memory-evaluation.mdx), updated in #6056.

- LoCoMo: 91.6 -> 92.5
- LongMemEval: 94.8 -> 94.4 (94.8 was not sourced by any docs page)
- Assistant memory recall highlight: +53.6 delta -> 98.2 absolute
- State the top_200 retrieval budget and the managed-platform caveat

BEAM (1M / 10M), token counts, and latency are unchanged.
2026-07-09 15:50:16 +05:30
+5 -5
View File
@@ -46,12 +46,12 @@
| Benchmark | Old | New | Tokens | Latency p50 |
| --- | --- | --- | --- | --- |
| **LoCoMo** | 71.4 | **91.6** | 7.0K | 0.88s |
| **LongMemEval** | 67.8 | **94.8** | 6.8K | 1.09s |
| **LoCoMo** | 71.4 | **92.5** | 7.0K | 0.88s |
| **LongMemEval** | 67.8 | **94.4** | 6.8K | 1.09s |
| **BEAM (1M)** | — | **64.1** | 6.7K | 1.00s |
| **BEAM (10M)** | — | **48.6** | 6.9K | 1.05s |
All benchmarks run on the same production-representative model stack. Single-pass retrieval (one call, no agentic loops).
All benchmarks run on the same production-representative model stack. Single-pass retrieval (one call, no agentic loops) at a top_200 retrieval budget. Scores reflect Mem0's managed platform, which includes proprietary optimizations not available in the open-source SDK; open-source users should expect directionally similar gains but not identical numbers.
**What changed:**
- **Single-pass ADD-only extraction** -- one LLM call, no UPDATE/DELETE. Memories accumulate; nothing is overwritten.
@@ -63,8 +63,8 @@ All benchmarks run on the same production-representative model stack. Single-pas
See the [migration guide](https://docs.mem0.ai/migration/oss-v2-to-v3) for upgrade instructions. The [evaluation framework](https://github.com/mem0ai/memory-benchmarks) is open-sourced so anyone can reproduce the numbers.
## Research Highlights
- **91.6 on LoCoMo** -- +20 points over the previous algorithm
- **94.8 on LongMemEval** -- +27 points, with +53.6 on assistant memory recall
- **92.5 on LoCoMo** -- +21 points over the previous algorithm
- **94.4 on LongMemEval** -- +27 points, with 98.2 on assistant memory recall
- **64.1 on BEAM (1M)** -- production-scale memory evaluation at 1M tokens
- [Read the full paper](https://mem0.ai/research)