Compare commits

..
Author SHA1 Message Date
Taranjeet SinghandClaude Opus 4.8 b1418ce93f fix(ts-sdk): format Together LLM provider to pass CI
Reformats the Together provider added in #5798 to match the DeepSeek
reference and satisfy the `prettier --check .` CI gate:

- Collapse manual line-wrapping in together.ts to Prettier style
- Add TOGETHER_API_BASE env override + trailing newline (parity with deepseek.ts)
- Remove stray leading blank line in factory.ts

Closes #5759

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 20:16:49 -07:00
niraja03 ab4c76f197 test(ts-sdk): add Together LLM unit tests 2026-06-24 06:24:06 +05:30
niraja03 14fa2ec49b feat(ts-sdk): add Together LLM provider 2026-06-24 05:09:27 +05:30
68 changed files with 873 additions and 2584 deletions
+1 -1
View File
@@ -12,7 +12,7 @@
"name": "mem0",
"source": "./integrations/mem0-plugin",
"description": "Mem0 memory layer for AI applications. Add persistent memory, personalization, and semantic search to Claude workflows.",
"version": "0.2.11"
"version": "0.2.10"
}
]
}
+1 -1
View File
@@ -12,7 +12,7 @@
"name": "mem0",
"source": "./integrations/mem0-plugin",
"description": "Mem0 memory layer for AI applications. Add persistent memory, personalization, and semantic search.",
"version": "0.2.11"
"version": "0.2.10"
}
]
}
+43 -130
View File
@@ -1,157 +1,72 @@
# Contributing to Mem0
# Contributing to mem0
First off, thank you for taking the time to contribute! 🎉 Mem0 is a
community-driven project and we welcome contributions of all kinds — bug fixes,
new features, documentation, examples, and integrations.
Let us make contribution easy, collaborative and fun.
Mem0 is a polyglot monorepo, and this guide covers contributing to both the
**Python SDK** and the **TypeScript SDK** (and the rest of the repository).
## Submit your Contribution through PR
## Before You Start
To make a contribution, follow these steps:
### 1. Open an Issue First
1. Fork and clone this repository
2. Do the changes on your fork with dedicated feature branch `feature/f1`
3. If you modified the code (new feature or bug-fix), please add tests for it
4. Include proper documentation / docstring and examples to run the feature
5. Ensure that all tests pass
6. Submit a pull request
**Always open an issue before opening a pull request.** This lets us discuss the
change, avoid duplicate effort, and agree on the approach before you invest time
in code.
For more details about pull requests, please read [GitHub's guides](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).
- Search [existing issues](https://github.com/mem0ai/mem0/issues) first to see if
your bug or idea already exists.
- If it doesn't, open a
[bug report](https://github.com/mem0ai/mem0/issues/new?template=bug_report.yml) or
[feature request](https://github.com/mem0ai/mem0/issues/new?template=feature_request.yml).
- For anything beyond a trivial fix, wait for a maintainer to confirm the approach
before starting significant work.
Every pull request must link to an issue using `Closes #<issue-number>`.
### 📦 Development Environment
### 2. Sign the Contributor License Agreement (CLA)
**We cannot accept or merge any pull request until you have signed our Contributor
License Agreement (CLA).**
When you open your first PR, the CLA bot will automatically comment with a link to
sign. Signing takes less than a minute and only needs to be done once. Pull
requests from contributors who have not signed the CLA will be blocked from
merging.
## Repository Layout
The two most common contribution targets are the SDKs:
| Package | Path | Language | Package manager |
| --------------------- | ---------- | ------------ | --------------- |
| Python SDK (`mem0ai`) | `mem0/` | Python 3.9+ | `hatch` |
| TypeScript SDK (`mem0ai`) | `mem0-ts/` | TypeScript | `pnpm` |
Other packages include the CLIs (`cli/python/`, `cli/node/`), integrations
(`integrations/`), the self-hosted `server/`, `openmemory/`, and the docs site
(`docs/`). See [AGENTS.md](./AGENTS.md) for a full map of the repository.
## Development Workflow
1. **Fork** the repository and **clone** your fork.
2. Create a **feature branch** from `main` (e.g. `feature/my-new-feature` or
`fix/issue-1234`).
3. Make your changes — add **tests**, **documentation**, and **examples** as
appropriate.
4. Run **linting and tests** for every package you touched (see below).
5. Commit using [Conventional Commits](https://www.conventionalcommits.org/)
(e.g. `feat:`, `fix:`, `docs:`, `refactor:`, `test:`).
6. Push and open a **pull request** against `main`, linking the issue with
`Closes #<number>` and filling out the
[PR template](./.github/PULL_REQUEST_TEMPLATE.md).
### Contributing to the Python SDK (`mem0/`)
We use [`hatch`](https://hatch.pypa.io/latest/install/) to manage environments.
**Do not use `pip` or `conda` for dependency management.**
We use `hatch` for managing development environments. To set up:
```bash
# Activate a dev environment (3.9 / 3.10 / 3.11 / 3.12)
hatch shell dev_py_3_11
# Activate environment for specific Python version:
hatch shell dev_py_3_9 # Python 3.9
hatch shell dev_py_3_10 # Python 3.10
hatch shell dev_py_3_11 # Python 3.11
hatch shell dev_py_3_12 # Python 3.12
# Install pre-commit hooks (runs ruff + isort on commit)
pre-commit install
# Lint, format, and sort imports
make lint
make format
make sort
# Run the test suite (run `make install_all` first if deps are missing)
# The environment will automatically install all dev dependencies
# Run tests within the activated shell:
make test
```
- **Linter / formatter:** Ruff (line length **120**)
- **Import sorting:** isort (`profile = "black"`)
- **Tests:** pytest (in `tests/`)
### 📌 Pre-commit
See the full [Development guide](https://docs.mem0.ai/contributing/development) for
environment details.
### Contributing to the TypeScript SDK (`mem0-ts/`)
We use [`pnpm`](https://pnpm.io/) (v10+) for all TypeScript packages. **Do not use
`npm` or `yarn`.**
To ensure our standards, make sure to install pre-commit before starting to contribute.
```bash
cd mem0-ts
pnpm install
pnpm run build # tsup (CJS + ESM)
pnpm run test # jest (all tests)
pnpm run test:unit # unit tests with coverage
pre-commit install
```
- **Build:** tsup
- **Formatter:** Prettier
- **Tests:** jest
- Always run type checking after changes: `pnpm run typecheck` (or `tsc --noEmit`).
- Use ES module `import` syntax — never `require()`.
### 🧪 Testing
## Good Contribution Practices
We use `pytest` to test our code across multiple Python versions. You can run tests using:
- **Keep PRs small and focused.** One logical change per PR is easier to review and
merge.
- **Follow existing patterns.** Match the style, structure, and conventions of the
code around you. Don't introduce new frameworks or abstractions without
discussion.
- **Write tests** that would fail without your change — regression tests for bugs,
coverage for new features.
- **Update documentation** in `docs/` for any user-facing change. New `.mdx` pages
must be added to `docs/llms.txt` (run
`python scripts/check-llms-txt-coverage.py --write` to scaffold entries).
- **Add examples** when introducing new user-facing behavior.
- **Run linters and tests locally** before pushing — CI re-runs them on every PR
via the CI Gate.
- **Never commit secrets** — no `.env` files, API keys, or credentials.
- **Don't add core dependencies lightly.** New Python dependencies belong in an
optional group in `pyproject.toml`, not the core `dependencies` list.
- **Be responsive** to review feedback and keep your branch up to date with `main`.
```bash
# Run tests with default Python version
make test
## Pull Request Checklist
# Test specific Python versions:
make test-py-3.9 # Python 3.9 environment
make test-py-3.10 # Python 3.10 environment
make test-py-3.11 # Python 3.11 environment
make test-py-3.12 # Python 3.12 environment
Before requesting review, make sure:
# When using hatch shells, run tests with:
make test # After activating a shell with hatch shell test_XX
```
- [ ] An issue exists and is linked with `Closes #<number>`
- [ ] You have signed the CLA
- [ ] Your code follows the project's style guidelines (lint passes)
- [ ] You performed a self-review of your changes
- [ ] Tests are added/updated and pass locally
- [ ] Documentation is updated if needed
Make sure that all tests pass across all supported Python versions before submitting a pull request.
## Reporting Security Issues
We look forward to your pull requests and can't wait to see your contributions!
**Do not report security vulnerabilities through public issues or pull requests.**
Please follow our [Security Policy](./SECURITY.md) to report them privately.
### 🚀 Releasing
## Releasing
All packages are published automatically via GitHub Actions when a GitHub Release is created with the correct tag prefix.
All packages are published automatically via GitHub Actions when a GitHub Release
is created with the correct tag prefix.
### Tag Prefixes
#### Tag Prefixes
| Package | Registry | Tag Prefix | Example |
|---------|----------|------------|---------|
@@ -162,17 +77,15 @@ is created with the correct tag prefix.
| `@mem0/vercel-ai-provider` | npm | `vercel-ai-v*` | `vercel-ai-v2.0.6` |
| `@mem0/openclaw-mem0` | npm | `openclaw-v*` | `openclaw-v1.0.1` |
### How to Release
#### How to Release
1. Bump the version in `pyproject.toml` (Python) or `package.json` (Node)
2. Create a [GitHub Release](https://github.com/mem0ai/mem0/releases/new) with the matching tag prefix
3. The correct workflow will trigger automatically — verify in the [Actions tab](https://github.com/mem0ai/mem0/actions)
### Publishing Details
#### Publishing Details
- **PyPI packages** use OIDC trusted publishing via `pypa/gh-action-pypi-publish`
- **npm packages** use OIDC trusted publishing via npm CLI (>= 11.5.1) — no tokens or secrets required
- All workflows require `permissions: id-token: write` for OIDC authentication
- First publish of a new npm package must be done manually; OIDC works for subsequent versions
We look forward to your pull requests and can't wait to see your contributions!
-48
View File
@@ -1,48 +0,0 @@
# Security Policy
We take the security of Mem0 and our community seriously. Thank you for helping
keep Mem0 and its users safe by disclosing vulnerabilities responsibly.
## Reporting a Vulnerability
Please **do not** report security vulnerabilities through public GitHub issues,
pull requests, or discussions.
If you believe you have found a security vulnerability in Mem0, please report it
privately through one of the following channels:
1. **GitHub Private Vulnerability Reporting** — open a
[private security advisory](https://github.com/mem0ai/mem0/security/advisories/new)
directly on this repository.
2. **Email** the maintainers at **support@mem0.ai** with the subject line:
`SECURITY: Mem0 vulnerability report`
To help us triage and resolve the issue quickly, please include as much of the
following as you can:
- Affected component or package (e.g. Python SDK, TypeScript SDK, server, OpenMemory)
- Affected version, tag, or commit
- Clear, step-by-step reproduction instructions
- The security impact and a proof of concept, if available
- Any suggested fix or mitigation
## Response Process
- We will acknowledge receipt of your report within **72 hours**.
- We will work with you privately to confirm the issue and assess its impact.
- Once a fix or mitigation is ready, we will coordinate a disclosure timeline
with you and credit you for the discovery, unless you prefer to remain anonymous.
## Public Disclosure
Please avoid sharing technical details of the vulnerability publicly until the
maintainers have reviewed the issue and a fix or mitigation has been released. We
are committed to resolving valid reports promptly and keeping you informed
throughout the process.
## Supported Versions
We release security fixes against the latest published version of each package.
Whenever possible, please reproduce the issue on the most recent release before
reporting.
-84
View File
@@ -7,65 +7,6 @@ mode: "wide"
<Tabs>
<Tab title="Python">
<Update label="2026-06-24" description="v2.0.9">
**Bug Fixes:**
- **Memory (OSS):** Improve entity extraction precision by avoiding sentence-start common noun noise, preserving useful topic phrases, and exact-deduplicating entity links before semantic matching ([#5829](https://github.com/mem0ai/mem0/pull/5829))
</Update>
<Update label="2026-06-24" description="v2.0.8">
**New Features:**
- **Embeddings:** Add native `embed_batch` to five embedders — LM Studio, Together, HuggingFace, Vertex AI, and Google GenAI — for batched embedding requests ([#5609](https://github.com/mem0ai/mem0/pull/5609))
**Bug Fixes:**
- **Core:** Guard against malformed `image_url` entries in `parse_vision_messages` to prevent crashes ([#5631](https://github.com/mem0ai/mem0/pull/5631))
- **Core:** Return `attributed_to` from `get()`, `get_all()`, and `search()` ([#5629](https://github.com/mem0ai/mem0/pull/5629))
- **Core:** Fix `reset()` only dropping the history table and leaving stale messages behind ([#5541](https://github.com/mem0ai/mem0/pull/5541))
- **Core:** Guard against an entity `embed_batch` count mismatch in the v3 add pipeline ([#5604](https://github.com/mem0ai/mem0/pull/5604))
- **Core:** Fix an async `delete_all` race condition that corrupted the entity store's `linked_memory_ids` ([#5553](https://github.com/mem0ai/mem0/pull/5553))
- **LLMs:** Skip the JSON `response_format` for Groq compound models that reject it ([#5513](https://github.com/mem0ai/mem0/pull/5513))
- **LLMs:** Preserve reasoning fields during base-to-provider config conversion ([#5638](https://github.com/mem0ai/mem0/pull/5638))
- **LLMs:** Pass the configured `anthropic_base_url` to the Anthropic client ([#5626](https://github.com/mem0ai/mem0/pull/5626))
- **LLMs:** Stop the Azure provider from mutating and corrupting caller messages during content rewrite ([#5731](https://github.com/mem0ai/mem0/pull/5731))
- **LLMs & Embeddings:** Repair HTTP proxy support for `httpx>=0.28` and preserve `proxies` in `LlmFactory` ([#5447](https://github.com/mem0ai/mem0/pull/5447))
- **Embeddings:** Forward `embedding_dims` to Titan V2 in the AWS Bedrock embedder ([#5671](https://github.com/mem0ai/mem0/pull/5671))
- **Rerankers:** Log reranking failures instead of swallowing them silently ([#5717](https://github.com/mem0ai/mem0/pull/5717))
- **Rerankers:** Clamp out-of-range LLM scores instead of mis-parsing them ([#5635](https://github.com/mem0ai/mem0/pull/5635))
- **Rerankers:** Export all five rerankers from the package root ([#5636](https://github.com/mem0ai/mem0/pull/5636))
- **Vector Stores:** Point the FastEmbed-missing warning at `mem0ai[extras]` ([#5622](https://github.com/mem0ai/mem0/pull/5622))
- **Vector Stores:** Preserve empty Azure AI Search update values ([#5524](https://github.com/mem0ai/mem0/pull/5524))
- **Vector Stores:** Add an `auto_refresh` option for OpenSearch Serverless compatibility ([#3893](https://github.com/mem0ai/mem0/pull/3893))
- **Vector Stores:** Wrap a scalar `vector_id` in a list for Chroma `delete()` ([#5703](https://github.com/mem0ai/mem0/pull/5703))
- **Vector Stores:** Wrap Chroma `update()` ids, embeddings, and metadatas in lists ([#5757](https://github.com/mem0ai/mem0/pull/5757))
- **Vector Stores:** Wrap a scalar `vector_id` in a list for Milvus `delete()` ([#5704](https://github.com/mem0ai/mem0/pull/5704))
- **Vector Stores:** Map all comparison operators in the Pinecone `_create_filter()` ([#5707](https://github.com/mem0ai/mem0/pull/5707))
- **Vector Stores:** Return `None` instead of `{}` from Chroma `_generate_where_clause` for empty filters ([#5713](https://github.com/mem0ai/mem0/pull/5713))
- **Vector Stores:** Return `[[]]` from the OpenSearch `list()` error path to honor the `list()` contract ([#5727](https://github.com/mem0ai/mem0/pull/5727))
- **Vector Stores:** Return `[[]]` from the Pinecone `list()` error path instead of a dict ([#5706](https://github.com/mem0ai/mem0/pull/5706))
- **Vector Stores:** Return `[[]]` for an uninitialized FAISS index to honor the `list()` contract ([#5725](https://github.com/mem0ai/mem0/pull/5725))
- **Vector Stores:** Wrap the MongoDB `list()` return in an outer list to match the interface contract ([#5729](https://github.com/mem0ai/mem0/pull/5729))
- **Vector Stores:** Deep-copy Redis `DEFAULT_FIELDS` so instances keep distinct dims ([#5633](https://github.com/mem0ai/mem0/pull/5633))
- **Vector Stores:** Pass the required `vectors` arg in Vertex AI `list()` and similarity search ([#5627](https://github.com/mem0ai/mem0/pull/5627))
- **Vector Stores:** Return `None` from Redis `get()` for missing IDs ([#5625](https://github.com/mem0ai/mem0/pull/5625))
- **Vector Stores:** Drop a stray `print` in Weaviate `list_cols` ([#5637](https://github.com/mem0ai/mem0/pull/5637))
- **Graph:** Keep distinct entities that share a substring prefix ([#5630](https://github.com/mem0ai/mem0/pull/5630))
- **Client:** Check the HTTP status before parsing the ping response in `_validate_api_key` ([#5639](https://github.com/mem0ai/mem0/pull/5639))
- **Server:** Fetch filtered dashboard memories beyond the default page ([#5753](https://github.com/mem0ai/mem0/pull/5753))
- **Server:** Return 404/400 instead of 502 for not-found and invalid input ([#5634](https://github.com/mem0ai/mem0/pull/5634))
- **Server:** Return 404 instead of 500 for a malformed API key id on revoke ([#5640](https://github.com/mem0ai/mem0/pull/5640))
- **Server:** Use `127.0.0.1` in the dashboard healthcheck to avoid IPv6 localhost resolution ([#5612](https://github.com/mem0ai/mem0/pull/5612))
**Improvements:**
- **Vector Stores:** Batch BM25 sparse encoding in Qdrant insert ([#5592](https://github.com/mem0ai/mem0/pull/5592))
**Security:**
- **Vector Stores:** Sanitize Milvus and Baidu filter values to prevent expression injection ([#5746](https://github.com/mem0ai/mem0/pull/5746))
- **Vector Stores:** Reject dict filter values in MongoDB to prevent NoSQL operator injection ([#5748](https://github.com/mem0ai/mem0/pull/5748))
</Update>
<Update label="2026-06-17" description="v2.0.7">
**New Features:**
@@ -1070,31 +1011,6 @@ See the [OSS v1 to v2 migration guide](https://docs.mem0.ai/migration/oss-v1-to-
<Tab title="TypeScript">
<Update label="2026-06-24" description="v3.0.11">
**Bug Fixes:**
- **Memory (OSS):** Align entity extraction with Python by reducing generic entity noise, preserving useful topic phrases, and exact-deduplicating entity links before semantic matching ([#5829](https://github.com/mem0ai/mem0/pull/5829))
</Update>
<Update label="2026-06-24" description="v3.0.10">
**Bug Fixes:**
- **Memory (OSS):** Guard against malformed `image_url` entries in `parseVisionMessages` to prevent crashes ([#5631](https://github.com/mem0ai/mem0/pull/5631))
- **Memory (OSS):** Return `attributedTo` from `get()`, `search()`, and `getAll()` ([#5675](https://github.com/mem0ai/mem0/pull/5675))
- **Memory (OSS):** Preserve message roles in the extraction input so assistant facts aren't attributed to the user ([#5643](https://github.com/mem0ai/mem0/pull/5643))
- **Memory (OSS):** Reject empty or blank messages in `Memory.add()` to prevent hallucinated memories ([#5545](https://github.com/mem0ai/mem0/pull/5545))
- **Memory (OSS):** Check `message.role` instead of `content` when detecting system messages ([#3921](https://github.com/mem0ai/mem0/pull/3921))
- **LLMs:** Honor the configured `baseURL` in `AnthropicLLM` ([#5740](https://github.com/mem0ai/mem0/pull/5740))
- **Client:** Preserve `customCategories` names through key conversion ([#5741](https://github.com/mem0ai/mem0/pull/5741))
- **Client:** Prevent hallucinated memories on an empty messages payload ([#5613](https://github.com/mem0ai/mem0/pull/5613))
- **Client:** Preserve user metadata keys across the case-conversion round-trip ([#5515](https://github.com/mem0ai/mem0/pull/5515))
**Security:**
- **Dependencies:** Upgrade `form-data` to `>=4.0.6` across pnpm workspaces to remediate CVE-2026-12143 ([#5618](https://github.com/mem0ai/mem0/pull/5618))
</Update>
<Update label="2026-06-17" description="v3.0.9">
**Bug Fixes:**
+1 -28
View File
@@ -7,8 +7,7 @@ To use DeepSeek LLM models, you have to set the `DEEPSEEK_API_KEY` environment v
## Usage
<CodeGroup>
```python Python
```python
import os
from mem0 import Memory
@@ -37,32 +36,6 @@ messages = [
m.add(messages, user_id="alice", metadata={"category": "movies"})
```
```typescript TypeScript
import { Memory } from 'mem0ai/oss';
const config = {
llm: {
provider: 'deepseek',
config: {
apiKey: process.env.DEEPSEEK_API_KEY || '',
model: 'deepseek-chat',
temperature: 0.2,
maxTokens: 2000,
top_p: 1.0,
},
},
};
const memory = new Memory(config);
const messages = [
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
{"role": "user", "content": "I’m not a big fan of thriller movies but I love sci-fi movies."},
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."}
];
await memory.add(messages, { userId: 'alice', metadata: { category: 'movies' } });
```
</CodeGroup>
You can also configure the API base URL in the config:
```python
+37 -35
View File
@@ -21,47 +21,49 @@ from mem0 import Memory
load_dotenv()
config = {
"vector_store": {
"provider": "pgvector",
"config": {
"connection_string": os.environ["DATABASE_URL"],
"collection_name": "memories",
"embedding_model_dims": 1536,
"hnsw": True,
},
},
"vector_store": {
"provider": "pgvector",
"config": {
"connection_string": os.environ["DATABASE_URL"],
"collection_name": "memories",
"embedding_model_dims": 1536,
"hnsw": True,
},
},
}
m = Memory.from_config(config)
messages = [
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."},
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."},
]
m.add(messages, user_id="alice", metadata={"category": "movies"})
results = m.search(
"What movies should I recommend?",
filters={"user_id": "alice"},
"What movies should I recommend?",
filters={"user_id": "alice"},
)
print(results)
````
```
```typescript TypeScript
import "dotenv/config";
import { Memory } from "mem0ai/oss";
const databaseUrl = new URL(process.env.DATABASE_URL!);
const m = new Memory({
vectorStore: {
provider: "pgvector",
config: {
connectionString: process.env.DATABASE_URL!,
ssl: {
rejectUnauthorized: false,
},
user: decodeURIComponent(databaseUrl.username),
password: decodeURIComponent(databaseUrl.password),
host: databaseUrl.hostname,
port: Number(databaseUrl.port || 5432),
dbname: databaseUrl.pathname.slice(1) || "neondb",
collectionName: "memories",
dimension: 1536,
embeddingModelDims: 1536,
@@ -87,8 +89,7 @@ const results = await m.search("What movies should I recommend?", {
});
console.log(results);
````
```
</CodeGroup>
## SQL Migration
@@ -115,19 +116,20 @@ DATABASE_URL=postgresql://user:password@ep-example.us-east-2.aws.neon.tech/neond
| `sslmode` | PostgreSQL SSL mode. Use `require` for Neon. | Driver default |
</Tab>
<Tab title="TypeScript">
Use the Neon `DATABASE_URL` directly with `connectionString`. Set `ssl` if your runtime needs an explicit TLS config object.
| Parameter | Description | Default |
| -------------------- | ---------------------------------------------- | -------------- |
| `connectionString` | Neon Postgres connection string. | Required |
| `ssl` | Optional TLS settings passed directly to `pg`. | Driver default |
| `collectionName` | Name for the vector collection. | `memories` |
| `dimension` | Vector dimension for Mem0 config. | Auto-detected |
| `embeddingModelDims` | Embedding model dimensions for table creation. | Required |
| `hnsw` | Enables HNSW indexing. | `false` |
**TLS note:** `ssl: true` is sufficient for most Neon connections since Neon uses valid certificates. Use `ssl: { rejectUnauthorized: false }` only when connecting through Neon's connection pooler on certain edge runtimes (e.g. Cloudflare Workers) that require it, or when your environment does not trust the Neon CA chain.
The current Mem0 TypeScript `pgvector` adapter takes individual Postgres fields,
so parse `DATABASE_URL` before creating `Memory`.
| Parameter | Description | Default |
| --- | --- | --- |
| `user` | Database user. | Required |
| `password` | Database password. | Required |
| `host` | Database host. | Required |
| `port` | Database port. | `5432` |
| `dbname` | Database name. | `vector_store` |
| `collectionName` | Name for the vector collection. | `memories` |
| `dimension` | Vector dimension for Mem0 config. | Auto-detected |
| `embeddingModelDims` | Embedding model dimensions for table creation. | Required |
| `hnsw` | Enables HNSW indexing. | `false` |
</Tab>
</Tabs>
@@ -56,30 +56,6 @@ config = {
}
```
### Configuration Options
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `collection_name` | string | required | Name of the OpenSearch index |
| `host` | string | required | OpenSearch endpoint URL |
| `port` | int | 9200 | Port number |
| `http_auth` | object | None | Authentication credentials (e.g., AWSV4SignerAuth) |
| `embedding_model_dims` | int | 1536 | Dimension of embedding vectors |
| `use_ssl` | bool | False | Enable SSL/TLS connection |
| `verify_certs` | bool | False | Verify SSL certificates |
| `auto_refresh` | bool | False | Automatically refresh index after insert. OpenSearch refreshes every ~1 second by default, so this is rarely needed. |
<Note>
The defaults above match a local OpenSearch instance. The AWS OpenSearch Serverless
example earlier on this page intentionally overrides them with `port=443`, `use_ssl=True`,
and `verify_certs=True`, which are required when connecting to a Serverless collection.
</Note>
<Note>
For **AWS OpenSearch Serverless**, keep `auto_refresh=False` (the default).
The `indices.refresh()` API is not supported on Serverless collections.
</Note>
### Add Memories
```python
+45 -50
View File
@@ -2,7 +2,6 @@
title: "pgvector"
description: "Use pgvector as a vector store in Mem0 for PostgreSQL-based vector similarity search with open-source simplicity."
---
[pgvector](https://github.com/pgvector/pgvector) is an open-source vector similarity search extension for Postgres. After connecting to Postgres, run `CREATE EXTENSION IF NOT EXISTS vector;` to create the vector extension.
### Usage
@@ -15,38 +14,41 @@ from mem0 import Memory
os.environ["OPENAI_API_KEY"] = "sk-xx"
config = {
"vector_store": {
"provider": "pgvector",
"config": {
"user": "test",
"password": "123",
"host": "127.0.0.1",
"port": "5432",
},
}
"vector_store": {
"provider": "pgvector",
"config": {
"user": "test",
"password": "123",
"host": "127.0.0.1",
"port": "5432",
}
}
}
m = Memory.from_config(config)
messages = [
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."},
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."}
]
m.add(messages, user_id="alice", metadata={"category": "movies"})
````
```
```typescript TypeScript
import { Memory } from "mem0ai/oss";
import { Memory } from 'mem0ai/oss';
const config = {
vectorStore: {
provider: "pgvector",
provider: 'pgvector',
config: {
collectionName: "memories",
collectionName: 'memories',
embeddingModelDims: 1536,
connectionString: "postgresql://test:123@localhost:5432/vector_store",
user: 'test',
password: '123',
host: '127.0.0.1',
port: 5432,
dbname: 'vector_store', // Optional; TypeScript OSS defaults to `vector_store` when omitted
diskann: false, // Optional, requires pgvectorscale extension
hnsw: false, // Optional, for HNSW indexing
},
@@ -55,44 +57,37 @@ const config = {
const memory = new Memory(config);
const messages = [
{ role: "user", content: "I'm planning to watch a movie tonight. Any recommendations?" },
{ role: "assistant", content: "How about thriller movies? They can be quite engaging." },
{ role: "user", content: "I'm not a big fan of thriller movies but I love sci-fi movies." },
{ role: "assistant", content: "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future." },
];
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."}
]
await memory.add(messages, { userId: "alice", metadata: { category: "movies" } });
````
```
</CodeGroup>
### Config
Here are the parameters available for configuring pgvector:
| Parameter | SDK | Description | Default Value |
| -------------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- |
| `connectionString` | TypeScript OSS | PostgreSQL connection string for direct connections. When set, Mem0 connects to the target database directly and skips the bootstrap `postgres` database flow. | `None` |
| `ssl` | TypeScript OSS | SSL option passed directly to `pg`, either `true` or an SSL config object, for both `connectionString` and split-field connections. | `None` |
| `dbname` | TypeScript OSS | Split-field database name. This is only used when `connectionString` is absent. | `vector_store` |
| `collectionName` | TypeScript OSS | Collection name. | `memories` |
| `embeddingModelDims` | TypeScript OSS | Dimensions of the embedding model. | Required |
| `user` | TypeScript OSS + Python | Database user for split-field connections. | `None` |
| `password` | TypeScript OSS + Python | Database password for split-field connections. | `None` |
| `host` | TypeScript OSS + Python | Database host for split-field connections. | `None` |
| `port` | TypeScript OSS + Python | Database port for split-field connections. | `None` |
| `diskann` | TypeScript OSS + Python | Whether to use DiskANN for vector similarity search, requires pgvectorscale. | `False` |
| `hnsw` | TypeScript OSS + Python | Whether to use HNSW for vector similarity search. | TypeScript OSS: `False`, Python: `True` |
| `connection_string` | Python only | PostgreSQL connection string, overrides individual connection parameters. | `None` |
| `sslmode` | Python only | SSL mode for PostgreSQL connections, such as `require`, `prefer`, or `disable`. | `None` |
| `connection_pool` | Python only | psycopg connection pool object, overrides connection string and individual connection parameters. | `None` |
| Parameter | Description | Default Value |
| --- | --- | --- |
| `dbname` | The name of the database | `postgres` |
| `collection_name` | The name of the collection | `mem0` |
| `embedding_model_dims` | Dimensions of the embedding model | `1536` |
| `user` | User name to connect to the database | `None` |
| `password` | Password to connect to the database | `None` |
| `host` | The host where the Postgres server is running | `None` |
| `port` | The port where the Postgres server is running | `None` |
| `diskann` | Whether to use diskann for vector similarity search (requires pgvectorscale) | `True` |
| `hnsw` | Whether to use hnsw for vector similarity search | `False` |
| `sslmode` | SSL mode for PostgreSQL connection (e.g., 'require', 'prefer', 'disable') | `None` |
| `connection_string` | PostgreSQL connection string (overrides individual connection parameters) | `None` |
| `connection_pool` | psycopg2 connection pool object (overrides connection string and individual parameters) | `None` |
**TypeScript OSS:** Use `connectionString` plus optional `ssl` for managed Postgres setups. If you omit `connectionString`, Mem0 falls back to split fields and uses `dbname`, `user`, `password`, `host`, `port`, and optional `ssl`.
**Python:** The Python SDK uses snake_case keys such as `connection_string`, `sslmode`, `collection_name`, and `embedding_model_dims`.
**Python connection priority**:
**Note (TypeScript OSS):** If you omit `dbname`, the TypeScript client uses the database name `vector_store`. Python defaults to `postgres` for `dbname`, as in the table above.
**Note**: The connection parameters have the following priority:
1. `connection_pool` (highest priority)
2. `connection_string`
3. Individual connection parameters (`user`, `password`, `host`, `port`, `sslmode`)
3. Individual connection parameters (`user`, `password`, `host`, `port`, `sslmode`)
+17 -83
View File
@@ -1,65 +1,32 @@
---
title: Development
description: "Guide to contributing code to Mem0, covering the issue-first workflow, the CLA, environment setup for the Python and TypeScript SDKs, and code quality checks."
description: "Guide to contributing code to Mem0, covering the fork and clone workflow, PR submission, and code quality checks."
icon: "code"
---
# Development Contributions
We strive to make contributions **easy, collaborative, and enjoyable**. Mem0 is a
polyglot monorepo containing the **Python SDK** (`mem0/`), the **TypeScript SDK**
(`mem0-ts/`), CLIs, integrations, the self-hosted server, and the docs site.
Follow the steps below for a smooth contribution process.
We strive to make contributions **easy, collaborative, and enjoyable**. Follow the steps below to ensure a smooth contribution process.
<Note>
For the complete contributor checklist, see
[CONTRIBUTING.md](https://github.com/mem0ai/mem0/blob/main/CONTRIBUTING.md) in
the repository root.
</Note>
## Submitting Your Contribution through PR
## Before You Start
### 1. Open an Issue First
**Always open an issue before opening a pull request.** This lets us discuss the
change, avoid duplicate work, and agree on the approach before you write code.
- Search [existing issues](https://github.com/mem0ai/mem0/issues) first.
- If none match, open a
[bug report](https://github.com/mem0ai/mem0/issues/new?template=bug_report.yml)
or [feature request](https://github.com/mem0ai/mem0/issues/new?template=feature_request.yml).
- For anything beyond a trivial fix, wait for a maintainer to confirm the approach.
Every pull request must link to an issue using `Closes #<issue-number>`.
### 2. Sign the Contributor License Agreement (CLA)
**We cannot merge any pull request until you have signed our Contributor License
Agreement (CLA).** When you open your first PR, the CLA bot will comment with a
link to sign — it takes less than a minute and only needs to be done once.
## Submitting Your Contribution through a PR
To contribute, follow these steps:
1. **Fork & Clone** the repository: [Mem0 on GitHub](https://github.com/mem0ai/mem0)
2. **Create a Feature Branch**: Use a dedicated branch, e.g., `feature/my-new-feature`
3. **Implement Changes**: If adding a feature or fixing a bug, be sure to:
2. **Create a Feature Branch**: Use a dedicated branch for your changes, e.g., `feature/my-new-feature`
3. **Implement Changes**: If adding a feature or fixing a bug, ensure to:
- Write necessary **tests**
- Add **documentation, docstrings, and runnable examples**
4. **Code Quality Checks**:
- Run **linting** to catch style issues
- Ensure **all tests pass**
5. **Commit** using [Conventional Commits](https://www.conventionalcommits.org/)
(`feat:`, `fix:`, `docs:`, `refactor:`, `test:`)
6. **Submit a Pull Request** against `main`, linking the issue and filling out the
PR template.
5. **Submit a Pull Request**
For detailed guidance on pull requests, refer to [GitHub's documentation](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).
---
## Python SDK (`mem0/`)
### Dependency Management
## Dependency Management
We use `hatch` as our package manager. Install it by following the [official instructions](https://hatch.pypa.io/latest/install/).
@@ -77,9 +44,13 @@ hatch -e dev_py_3_11 shell # For dev_py_3_11 (differences are mentioned in pypr
make install_all
```
---
## Development Standards
### Pre-commit Hooks
Ensure `pre-commit` is installed before contributing (hooks run ruff + isort):
Ensure `pre-commit` is installed before contributing:
```bash
pre-commit install
@@ -87,7 +58,7 @@ pre-commit install
### Linting with `ruff`
Run the linter and fix any reported issues before submitting your PR (line length **120**):
Run the linter and fix any reported issues before submitting your PR:
```bash
make lint
@@ -95,11 +66,10 @@ make lint
### Code Formatting
To maintain a consistent code style, format your code and sort imports (isort, `profile = "black"`):
To maintain a consistent code style, format your code:
```bash
make format
make sort
```
### Testing with `pytest`
@@ -114,46 +84,10 @@ make test
---
## TypeScript SDK (`mem0-ts/`)
We use [`pnpm`](https://pnpm.io/) (v10+) for all TypeScript packages. **Do NOT use
`npm` or `yarn`.**
```bash
cd mem0-ts
pnpm install
pnpm run build # tsup (CJS + ESM)
pnpm run test # jest (all tests)
pnpm run test:unit # unit tests with coverage
```
### Standards
- **Build:** tsup
- **Formatter:** Prettier
- **Tests:** jest
- Always run type checking after changes: `pnpm run typecheck` (or `tsc --noEmit`)
- Use ES module `import` syntax — never `require()`
---
## Reporting Security Issues
**Do not report security vulnerabilities through public issues or pull requests.**
Please follow our [Security Policy](https://github.com/mem0ai/mem0/blob/main/SECURITY.md)
to report them privately.
---
## Release Process
Packages are published automatically via GitHub Actions when a GitHub Release is
created with the correct tag prefix (e.g. `v*` for the Python SDK, `ts-v*` for the
TypeScript SDK). See
[CONTRIBUTING.md](https://github.com/mem0ai/mem0/blob/main/CONTRIBUTING.md#releasing)
for the full tag-prefix table and publishing details.
Currently, releases are handled manually. We aim for frequent releases, typically when new features or bug fixes are introduced.
---
Thank you for contributing to Mem0!
Thank you for contributing to Mem0!
+1 -1
View File
@@ -345,7 +345,7 @@ When evaluating memory systems, keep these considerations in mind:
<Card title="Research" icon="flask" href="https://mem0.ai/research">
Published research papers and technical reports
</Card>
<Card title="Blog Post" icon="newspaper" href="https://mem0.ai/blog/the-token-efficient-memory-algorithm-now-has-temporal-reasoning">
<Card title="Blog Post" icon="newspaper" href="https://mem0.ai/blog/new-algorithm">
Detailed writeup of the new algorithm design and results
</Card>
<Card title="Platform Migration" icon="arrow-right" href="/migration/platform-v2-to-v3">
-1
View File
@@ -65,7 +65,6 @@ The plugin uses the same shell scripts as Claude Code, Cursor, and Codex — hoo
| **User prompt** | `UserPromptSubmit` | Searches relevant memories before each message |
| **Pre-tool** | `PreToolUse` | Blocks MEMORY.md writes, enforces `user_id`/`app_id` on mem0 tools |
| **Post-tool** | `PostToolUse` | Tracks stats, scans bash errors for related memories |
| **Stop** | `Stop` | Stores a session summary when the session ends |
## Troubleshooting
+2 -3
View File
@@ -64,7 +64,7 @@ Add the Mem0 MCP server directly with a single command:
npx mcp-add \
--name mem0-mcp \
--type http \
--url "https://mcp.mem0.ai/mcp/" \
--url "https://mcp.mem0.ai/mcp" \
--clients "claude code"
```
@@ -142,8 +142,7 @@ When installed via the plugin marketplace, Mem0 hooks into Claude Code's lifecyc
| **User prompt** | `UserPromptSubmit` | Searches relevant memories before each message; skips short prompts |
| **Pre-tool** | `PreToolUse` | Blocks MEMORY.md writes, enforces `user_id`/`app_id` on mem0 tool calls |
| **Post-tool** | `PostToolUse` | Tracks stats, scans bash errors for related memories |
| **Stop** | `Stop` | Stores a session summary when the session ends |
| **Pre-compact** | `PreCompact` | Stores a summary before the context is compacted |
| **Pre-compact** | `PreCompact` | Stores a session summary before context compaction |
## Example Workflow
+9 -23
View File
@@ -41,17 +41,7 @@ Install the full plugin including MCP server, lifecycle hooks, and SDK skill.
codex plugin marketplace add mem0ai/mem0
```
2. Install the plugin:
```bash
codex plugin add mem0@mem0-plugins
```
Or, in the app: restart Codex, open the Plugin Directory, browse the **Mem0 Plugins** marketplace, and install **Mem0**.
<Note>
Step 1 is required for the app UI. Mem0 isn't in OpenAI's curated directory yet, so **without `codex plugin marketplace add`, Mem0 won't appear in the Codex app's Plugin Directory** — searching for it returns nothing. Adding the marketplace surfaces it (under **Created by you**) and makes it installable.
</Note>
2. Restart Codex, open the Plugin Directory, browse the **Mem0 Plugins** marketplace, and install **Mem0**.
<Info>
Do not combine with Option B. The plugin manifest auto-registers the `mem0` MCP server, so adding both will create a duplicate registration.
@@ -59,30 +49,27 @@ Install the full plugin including MCP server, lifecycle hooks, and SDK skill.
### Option B — Direct MCP
The fastest way to connect Codex to Mem0 — no plugin, no marketplace. Add the MCP server with a single command:
```bash
codex mcp add mem0 --url https://mcp.mem0.ai/mcp/ --bearer-token-env-var MEM0_API_KEY
```
Or add it manually to `~/.codex/config.toml`:
The fastest way to connect Codex to Mem0 — no plugin, no marketplace. Add to `~/.codex/config.toml`:
```toml
[mcp_servers.mem0]
url = "https://mcp.mem0.ai/mcp/"
url = "https://mcp.mem0.ai/mcp"
bearer_token_env_var = "MEM0_API_KEY"
```
Make sure `MEM0_API_KEY` is exported in the shell you launch Codex from, then restart Codex.
<Info>
Codex's `codex mcp add` CLI only supports stdio MCP servers. Because Mem0's MCP is HTTP/streamable, you configure it by editing `config.toml` directly (or via the **Plugins → Connect to a custom MCP → Streamable HTTP** UI in the Codex app).
</Info>
This gives you the MCP tools but not the lifecycle hooks or SDK skill.
### Managing the Plugin
```bash
codex plugin marketplace upgrade # pull latest plugin versions
codex plugin remove mem0@mem0-plugins # uninstall the plugin (keeps the marketplace)
codex plugin marketplace remove mem0-plugins # unregister the marketplace entirely
codex plugin marketplace remove mem0-plugins # unregister the marketplace
```
To update, run `codex plugin marketplace upgrade` to pull the latest from the Mem0 repo.
@@ -125,8 +112,7 @@ When installed via the plugin marketplace, Mem0 hooks into Codex's lifecycle to
| **User prompt** | `UserPromptSubmit` | Searches relevant memories before each message |
| **Pre-tool** | `PreToolUse` | Blocks MEMORY.md writes, enforces `user_id`/`app_id` on mem0 tool calls |
| **Post-tool** | `PostToolUse` | Tracks stats, scans bash errors for related memories |
| **Stop** | `Stop` | Stores a session summary when the session ends |
| **Pre-compact** | `PreCompact` | Stores a summary before the context is compacted |
| **Pre-compact** | `PreCompact` | Stores a session summary before context compaction |
## Example Workflow
+2 -3
View File
@@ -47,7 +47,7 @@ The fastest way to get started. Click the link below to install the Mem0 MCP ser
npx mcp-add \
--name mem0-mcp \
--type http \
--url "https://mcp.mem0.ai/mcp/" \
--url "https://mcp.mem0.ai/mcp" \
--clients "cursor"
```
@@ -110,8 +110,7 @@ When installed via the Cursor Marketplace, Mem0 hooks into Cursor's lifecycle:
| **User prompt** | `beforeSubmitPrompt` | Searches relevant memories before each message; skips short prompts |
| **Pre-tool (2 handlers)** | `preToolUse` | Blocks MEMORY.md writes, enforces `user_id`/`app_id` on mem0 tool calls |
| **Post-tool (2 handlers)** | `postToolUse` | Tracks stats, scans bash errors for related memories |
| **Stop** | `stop` | Stores a session summary when the session ends |
| **Pre-compact** | `preCompact` | Stores a summary before the context is compacted |
| **Pre-compact** | `preCompact` | Stores a session summary before context compaction |
## Example Workflow
+55 -54
View File
@@ -100,12 +100,12 @@ This section:
Initialize both the ElevenLabs and Mem0 clients:
```python
# Initialize ElevenLabs client
client = ElevenLabs(api_key=API_KEY)
# Initialize ElevenLabs client
client = ElevenLabs(api_key=API_KEY)
# Initialize memory client and tools
client_tools = ClientTools()
mem0_client = AsyncMemoryClient()
# Initialize memory client and tools
client_tools = ClientTools()
mem0_client = AsyncMemoryClient()
```
Here we:
@@ -118,36 +118,36 @@ Here we:
Define the two key memory functions that will be registered as tools:
```python
# Define memory-related functions for the agent
async def add_memories(parameters):
"""Add a message to the memory store"""
message = parameters.get("message")
await mem0_client.add(
messages=message,
user_id=USER_ID
)
return "Memory added successfully"
# Define memory-related functions for the agent
async def add_memories(parameters):
"""Add a message to the memory store"""
message = parameters.get("message")
await mem0_client.add(
messages=message,
user_id=USER_ID
)
return "Memory added successfully"
async def retrieve_memories(parameters):
"""Retrieve relevant memories based on the input message"""
message = parameters.get("message")
async def retrieve_memories(parameters):
"""Retrieve relevant memories based on the input message"""
message = parameters.get("message")
# For Platform API, user_id goes in filters
filters = {"user_id": USER_ID}
# For Platform API, user_id goes in filters
filters = {"user_id": USER_ID}
# Search for relevant memories using the message as a query
results = await mem0_client.search(
query=message,
filters=filters
)
# Search for relevant memories using the message as a query
results = await mem0_client.search(
query=message,
filters=filters
)
# Extract and join the memory texts
memories = ' '.join([result["memory"] for result in results.get('results', [])])
print("[ Memories ]", memories)
# Extract and join the memory texts
memories = ' '.join([result["memory"] for result in results.get('results', [])])
print("[ Memories ]", memories)
if memories:
return memories
return "No memories found"
if memories:
return memories
return "No memories found"
```
These functions:
@@ -171,9 +171,9 @@ These functions:
Register the memory functions with the ElevenLabs ClientTools system:
```python
# Register the memory functions as tools for the agent
client_tools.register("addMemories", add_memories, is_async=True)
client_tools.register("retrieveMemories", retrieve_memories, is_async=True)
# Register the memory functions as tools for the agent
client_tools.register("addMemories", add_memories, is_async=True)
client_tools.register("retrieveMemories", retrieve_memories, is_async=True)
```
This allows the ElevenLabs agent to:
@@ -186,19 +186,19 @@ This allows the ElevenLabs agent to:
Configure the conversation with ElevenLabs:
```python
# Initialize the conversation
conversation = Conversation(
client,
AGENT_ID,
# Assume auth is required when API_KEY is set
requires_auth=bool(API_KEY),
audio_interface=DefaultAudioInterface(),
client_tools=client_tools,
callback_agent_response=lambda response: print(f"Agent: {response}"),
callback_agent_response_correction=lambda original, corrected: print(f"Agent: {original} -> {corrected}"),
callback_user_transcript=lambda transcript: print(f"User: {transcript}"),
# callback_latency_measurement=lambda latency: print(f"Latency: {latency}ms"),
)
# Initialize the conversation
conversation = Conversation(
client,
AGENT_ID,
# Assume auth is required when API_KEY is set
requires_auth=bool(API_KEY),
audio_interface=DefaultAudioInterface(),
client_tools=client_tools,
callback_agent_response=lambda response: print(f"Agent: {response}"),
callback_agent_response_correction=lambda original, corrected: print(f"Agent: {original} -> {corrected}"),
callback_user_transcript=lambda transcript: print(f"User: {transcript}"),
# callback_latency_measurement=lambda latency: print(f"Latency: {latency}ms"),
)
```
This sets up the conversation with:
@@ -217,16 +217,16 @@ This sets up the conversation with:
Start and manage the conversation:
```python
# Start the conversation
print(f"Starting conversation with user_id: {USER_ID}")
conversation.start_session()
# Start the conversation
print(f"Starting conversation with user_id: {USER_ID}")
conversation.start_session()
# Handle Ctrl+C to gracefully end the session
signal.signal(signal.SIGINT, lambda sig, frame: conversation.end_session())
# Handle Ctrl+C to gracefully end the session
signal.signal(signal.SIGINT, lambda sig, frame: conversation.end_session())
# Wait for the conversation to end and get the conversation ID
conversation_id = conversation.wait_for_session_end()
print(f"Conversation ID: {conversation_id}")
# Wait for the conversation to end and get the conversation ID
conversation_id = conversation.wait_for_session_end()
print(f"Conversation ID: {conversation_id}")
if __name__ == '__main__':
@@ -445,3 +445,4 @@ By integrating ElevenLabs Conversational AI with Mem0, you can create voice agen
Create voice-first AI applications
</Card>
</CardGroup>
+37 -164
View File
@@ -1,42 +1,35 @@
---
title: Hermes Agent
description: "Add long-term memory to Hermes agents with Mem0, on managed Mem0 Cloud or fully self-hosted (OSS), with automatic background sync and zero-latency prefetch."
description: "Add long-term memory to Hermes agents using Mem0 as a pluggable memory provider with automatic background sync and zero-latency prefetch."
---
Add long-term memory to [Hermes Agent](https://github.com/NousResearch/hermes-agent), a self-improving AI agent CLI by Nous Research. Hermes has a pluggable memory system, and Mem0 is one of the supported providers. Once enabled, Mem0 learns facts from your conversations and surfaces relevant ones before each turn, without slowing down the chat.
Add long-term memory to [Hermes Agent](https://github.com/NousResearch/hermes-agent) — a self-improving AI agent CLI by Nous Research. Hermes has a pluggable memory system, and Mem0 is one of the supported providers. Once enabled, Mem0 automatically learns facts from your conversations and surfaces relevant ones before each turn — all without slowing down the chat.
You can run Mem0 in two ways:
## Overview
- **Platform mode** (default): managed Mem0 Cloud. Add your API key and you are ready.
- **OSS mode**: fully self-hosted with your own LLM, embedder, and vector store. No data leaves your machine.
Hermes runs a built-in memory system (file-based `MEMORY.md` and `USER.md`) alongside one external provider. When Mem0 is active, it works additively with the built-in system at three key moments in every conversation turn:
## How It Works
### 1. Before the Agent Responds (Prefetch)
Hermes runs a built-in memory system (file-based `MEMORY.md` and `USER.md`) alongside one external provider. When Mem0 is active, it works additively with the built-in system at three points in every conversation turn.
When you send a message, Hermes checks if it already has cached Mem0 search results from the previous turn. If so, those memories are injected into the system prompt so the LLM can see them. This is **zero-latency** — no waiting for an API call.
### 1. Before the agent responds (prefetch)
### 2. After the Agent Responds (Sync)
When you send a message, Hermes checks for cached Mem0 search results from the previous turn. If they exist, those memories are injected into the system prompt so the model can see them. This is zero-latency, with no waiting on an API call.
Once the LLM finishes responding, Hermes sends the `(user message, assistant response)` pair to Mem0's API in a **background thread**. Mem0's server-side LLM automatically extracts facts (e.g., "user prefers Python", "user works at Acme Corp") — you don't have to tell it what to remember.
### 2. After the agent responds (sync)
### 3. Background Prefetch for Next Turn
Once the model finishes, Hermes sends the `(user message, assistant response)` pair to Mem0 in a background thread. Mem0 extracts facts automatically (for example, "user prefers Python" or "user works at Acme Corp"), so you never have to tell it what to remember. Each write is tagged with the gateway channel it came from.
### 3. Background prefetch for the next turn
At the same time, Hermes runs a background search to pre-load relevant memories for your next message. By the time you type, the results are already cached.
At the same time as sync, Hermes kicks off a background search on Mem0 to pre-load relevant memories for the next turn. By the time you type your next message, the memories are already cached.
## Agent Tools
When Mem0 is active, the model gets five tools it can call during a conversation:
When Mem0 is active, the LLM gets three extra tools it can call during conversations:
| Tool | Description | Parameters |
|------|-------------|------------|
| `mem0_list` | List all stored memories, for a full overview | `page`, `page_size` (default 100, max 200) |
| `mem0_search` | Semantic search by meaning, ranked by relevance | `query` (required), `top_k` (default 10, max 50), `rerank` (default `true`, Platform mode only) |
| `mem0_add` | Store a fact verbatim, with no LLM extraction | `content` (required) |
| `mem0_update` | Update a memory's text by ID | `memory_id`, `text` (both required) |
| `mem0_delete` | Delete a memory by ID | `memory_id` (required) |
| Tool | Description |
|------|-------------|
| `mem0_profile` | Fetch all stored memories about the user |
| `mem0_search` | Semantic search through memories (supports optional reranking via `rerank` and `top_k` parameters) |
| `mem0_conclude` | Store a specific fact verbatim — uses `infer=False` so no server-side LLM extraction happens |
## Installation
@@ -47,19 +40,17 @@ curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scri
source ~/.bashrc
```
The `mem0ai` package is installed automatically when you enable the Mem0 provider, so there is no manual pip step. OSS providers may need extra packages (for example `qdrant-client`, `psycopg2-binary`, or `ollama`), which the setup flow installs for you when you pick them.
The `mem0ai` Python package is automatically installed when you enable the Mem0 provider — no manual pip install needed.
## Platform Setup
## Setup
Platform mode uses managed Mem0 Cloud and is the fastest way to start.
### Option 1: Interactive wizard (recommended)
### Option 1: Interactive Setup Wizard (Recommended)
```bash
hermes memory setup
```
Select **mem0**, choose **Platform**, and paste your API key when prompted. The wizard writes the non-secret settings to `~/.hermes/mem0.json` and keeps the key in `~/.hermes/.env`.
Select **mem0** as the provider and enter your Mem0 API key when prompted. The wizard writes your config to `~/.hermes/mem0.json`.
<Note>Get your API key from <a href="https://app.mem0.ai?utm_source=oss&utm_medium=integration-hermes" rel="nofollow">app.mem0.ai</a>.</Note>
@@ -77,151 +68,33 @@ memory:
provider: mem0
```
That's it. Mem0 runs automatically from here.
That's it — Mem0 runs automatically from this point.
## OSS (Self-Hosted) Setup
## Configuration Options
OSS mode runs Mem0 entirely on your own infrastructure: your LLM, your embedder, and your vector store. No data is sent to Mem0 Cloud, and no Mem0 API key is required.
Configuration is stored in `~/.hermes/mem0.json`. Values can also be set via environment variables.
### Interactive
```bash
hermes memory setup
# Select "mem0", then "Open Source (self-hosted)"
# Follow the prompts for LLM, embedder, and vector store
```
### With flags
```bash
hermes memory setup mem0 --mode oss \
--oss-llm openai --oss-llm-key sk-... \
--oss-vector qdrant
```
### Supported providers
| Component | Providers |
|-----------|-----------|
| LLM | `openai` (default model `gpt-5-mini`), `ollama` (local, default `llama3.1:8b`) |
| Embedder | `openai` (default `text-embedding-3-small`), `ollama` (local, default `nomic-embed-text`) |
| Vector store | `qdrant` (local path or server), `pgvector` |
### Flag reference
| Flag | Description |
|------|-------------|
| `--mode` | `platform` or `oss` |
| `--oss-llm` | LLM provider (`openai` or `ollama`, default `openai`) |
| `--oss-llm-key` | LLM API key (for `openai`) |
| `--oss-llm-model` | Override the LLM model |
| `--oss-llm-url` | LLM base URL (for `ollama` or a custom endpoint) |
| `--oss-embedder` | Embedder provider (default `openai`) |
| `--oss-embedder-key` | Embedder API key |
| `--oss-vector` | Vector store (`qdrant` or `pgvector`, default `qdrant`) |
| `--oss-vector-path` | Local Qdrant storage path |
| `--oss-vector-host`, `--oss-vector-port` | PGVector or remote Qdrant host and port |
| `--oss-vector-user`, `--oss-vector-password`, `--oss-vector-dbname` | PGVector connection details |
| `--user-id` | Canonical user identifier |
| `--dry-run` | Preview the resolved config without writing it |
## Switching Modes
You can move between Platform and OSS at any time. Run the setup command again, or edit `~/.hermes/mem0.json` directly.
```bash
# Platform to OSS
hermes memory setup mem0 --mode oss --oss-llm-key sk-...
# OSS to Platform
hermes memory setup mem0 --mode platform --api-key sk-...
# Preview without writing anything
hermes memory setup mem0 --mode oss --oss-llm-key sk-... --dry-run
```
A self-hosted `~/.hermes/mem0.json` looks like this:
```json
{
"mode": "oss",
"oss": {
"llm": {"provider": "openai", "config": {"model": "gpt-5-mini"}},
"embedder": {"provider": "openai", "config": {"model": "text-embedding-3-small"}},
"vector_store": {"provider": "qdrant", "config": {"path": "~/.hermes/mem0_qdrant"}}
}
}
```
## Configuration
Behavioral settings live in `~/.hermes/mem0.json` and are written for you by `hermes memory setup`. Only the secret `MEM0_API_KEY` belongs in `~/.hermes/.env`.
| Key | Default | Description |
|-----|---------|-------------|
| `mode` | `platform` | `platform` (Mem0 Cloud) or `oss` (self-hosted) |
| `api_key` | none | Mem0 Platform API key, required in Platform mode. Stored in `.env` as `MEM0_API_KEY` |
| `user_id` | `hermes-user` | Identifier that scopes memories. See cross-channel behavior below |
| `agent_id` | `hermes` | Agent identifier attached to writes |
| `rerank` | `true` | Rerank search results for relevance (Platform mode only) |
### Cross-channel memories
Hermes can run from the CLI and from gateways like Telegram, Slack, and Discord. The `user_id` setting controls how memories are scoped across them:
- **Set a `user_id`** and it applies to every gateway, so one person gets a single merged memory store no matter where they talk to the agent.
- **Leave it unset** (or at the default `hermes-user`) and each gateway uses its own native id, keeping per-platform memories separate.
Either way, every write is tagged with `metadata.channel` (for example `telegram` or `cli`), so per-channel views are still possible at query time.
| Key | Env Variable | Default | Description |
|-----|-------------|---------|-------------|
| `api_key` | `MEM0_API_KEY` | — | **Required.** Mem0 Platform API key |
| `user_id` | `MEM0_USER_ID` | `hermes-user` | User identifier for scoping memories |
| `agent_id` | `MEM0_AGENT_ID` | `hermes` | Agent identifier |
| `rerank` | — | `true` | Enable reranking for memory recall |
## Reliability
- **Circuit breaker**: if Mem0 fails five times in a row, Hermes pauses calls for two minutes, then retries. The agent keeps working without memory during that window. Expected client errors, like a 404 on a missing memory id, do not count toward tripping the breaker.
- **Non-blocking**: every Mem0 call runs in a background daemon thread, so a slow or failed call never blocks your conversation.
- **Thread-safe**: the client uses lazy initialization with locking, and the background sync and prefetch threads are guarded so concurrent gateway messages cannot produce duplicate memories.
## Troubleshooting
### "Mem0 temporarily unavailable"
The circuit breaker tripped after five consecutive failures and resets after two minutes.
- **Platform mode**: check your API key and internet connection.
- **OSS mode**: make sure your vector store (Qdrant or PGVector) is running and reachable.
### OSS: vector store connection refused
```bash
# Local Qdrant: confirm the storage path is writable
ls -la ~/.hermes/mem0_qdrant
# Qdrant server: confirm it is reachable
curl http://localhost:6333/healthz
# PGVector: confirm PostgreSQL is accepting connections
pg_isready -h localhost -p 5432
```
### OSS: Ollama not reachable
```bash
curl http://localhost:11434/api/tags
```
### Memories not appearing
- `mem0_add` stores text verbatim with no extraction. Ordinary conversation turns are extracted automatically by the background sync.
- Search is semantic, so try a broader query.
- Confirm `user_id` is the same across sessions (check `~/.hermes/mem0.json`).
- **Circuit Breaker** — If Mem0's API fails 5 times in a row, Hermes stops calling it for 2 minutes, then retries. The agent keeps working fine without memory during that time.
- **Non-blocking** — All Mem0 API calls happen in background daemon threads. A slow or failed API call never blocks your conversation.
- **Thread-safe** — The Mem0 client uses lazy initialization with locking, safe for concurrent access.
## Key Features
1. **Two ways to run**: managed Platform or fully self-hosted OSS, switchable at any time.
2. **Zero-latency recall**: memories are prefetched in the background and cached before you type.
3. **Automatic extraction**: Mem0 extracts and deduplicates facts from each exchange for you.
4. **Non-blocking and fault tolerant**: background threads plus a circuit breaker keep the agent responsive even when Mem0 is unreachable.
5. **Additive memory**: works alongside Hermes' built-in file memory (`MEMORY.md`, `USER.md`).
1. **Zero-Latency Recall** — Memories are prefetched in the background and cached, ready before you type
2. **Server-side Extraction** — Mem0's API automatically extracts and deduplicates facts from each exchange
3. **Non-blocking** — All API calls run in background daemon threads
4. **Fault Tolerant** — Circuit breaker ensures the agent works even if Mem0 is temporarily unreachable
5. **Additive Memory** — Works alongside Hermes' built-in file-based memory system (MEMORY.md, USER.md)
<CardGroup cols={2}>
<Card title="OpenClaw Integration" icon={<svg width="24" height="24" viewBox="0 0 500 500" fill="none" xmlns="http://www.w3.org/2000/svg"><path fill-rule="evenodd" d="m153.5 173.5q24.62 1.46 46 13.5 12.11 8.1 17.5 21.5 0.74 2.45 0.5 5 0.09 0.81 1 1 1.48-4.9 1-10 5.04 10.48 1.5 22-9.81 27.86-35.5 42.5-26.17 14.97-56 19.5-2.77-0.4-2 1 2.86 1.27 6 1 25.64 1.53 48.5-10 0.34 10.08 2 20 1.08 5.76 5 10 1 1.5 0 3-31.11 20.84-68.5 17.5-23.7-5.7-32.5-28.5-4.39-9.18-3.5-19 15.41 6.23 32 4.5-20.68-6.39-39-18-34.81-27.22-12.5-65.5 11.84-14.83 29-23 4.21 7.66 11.5 12.5 3 1 6 0-26.04-34.62-29-78-0.13-8.46 2-16.5 1 6.5 2 13 3.43 39.53 24.5 73 2.03 2.28 4.5 4 0.5-1.25 1-2.5-1.27-6.54-5-12 0.5-0.75 1-1.5 9.72-3.43 20-4 0.55 10.34 8 17.5 1.94 0.74 4 0.5-17.8-64.6 16.5-122 0.98-1.79 1.5 0-28.21 56.64-13.5 118 1.08 1.43 2.5 0.5 2.21-4.98 2-10.5z" fill="currentColor"/><path fill-rule="evenodd" d="m454.5 97.5q-1.33 11.18-8.5 20-21.81 26.28-55.5 32-1.11-0.2-2 0.5 2.31 2.82 5.5 4.5 1 2 0 4-9.56 11.3-19.5 20 19.71-8.72 31-27 2.68-0.43 5 1-14.24 30.97-48 36.5-9.93 1.71-20 1.5-6.8-0.48-13 1 5.81 6.92 14 11-10.78 16.03-27 26.5 27.16-7.4 38-33.5 4.34 1.35 9 1-9.08 23.84-33 33.5-18.45 6.41-38 7 22.59 8.92 45-1 12.05-5.52 24-11 9.01-1.79 17 2.5 5.28-4.38 11-8 12.8-6.07 27-5 0 0.5 0 1-19.34 2.69-34 15.5 0.5 0.25 1 0.5 17.79-8.09 36-15 2.71-0.79 5-2 2.5-1 5-2 5.53-4.04 11-8 11.7-4.18 24-6.5 7.78-1.36 15 1.5-2.97 18.45-13.5 34-34.92 49.37-94.5 62.5-59.27 12.45-108-23-15.53-12.52-21.5-31.5-2.47-14.26 4-27-3.15 24.41 14 42-4.92-10.28-7-22-1.97-17.63 7-33 47.28-69.5 125.5-100 15.86-3.42 32-5.5 18.63-1.47 37 1.5z" fill="currentColor"/><path fill-rule="evenodd" d="m231.5 238.5q1.31-0.2 2 1-3.13 28.62 15 51-16.25 6.75-27-7.5-1-1-2 0 14.73 29.34 46 18.5 1.79 0.52 0 1.5-37.63 16.82-50.5-22.5-5.1-26.48 16.5-42z" fill="currentColor"/><path fill-rule="evenodd" d="m203.5 266.5q1.31-0.2 2 1-2.48 22.08 12 39-6.99 1.35-14 0.5 4.59 4.08 10 7-8.71 0.28-14.5-6.5-16.98-22.76 4.5-41z" fill="currentColor"/><path fill-rule="evenodd" d="m58.5 284.5q9.6-2.17 14.5 6 5.15 14.18-1 28-11.05-13.14-27.5-17.5 5.15-9.9 14-16.5z" fill="currentColor"/><path fill-rule="evenodd" d="m56.5 313.5q3.43 5.43 8 10-4.88 0.44-8 4-1.11-0.2-2 0.5 28.91 1.65 38 28.5 0.45 3.16-1 6-11.02-7.01-23-12.5-4.75-3.75-9.5-7.5 1.47 7.42 7 13 8.34 27.18 32 43 0.99 2.41-1.5 3.5-40.25 5.58-66.5-25.5-15.67-22.01-8-48 10.46-23.87 34.5-15z" fill="currentColor"/><path fill-rule="evenodd" d="m198.5 319.5q1.44 0.68 2.5 2 2.41 8.23 6 16 1.2 2.64-0.5 5-30.65 21.41-68 18.5-25.16-6.17-32.5-30.5 6.96 4.99 15.5 6.5 8.99 0.75 18 0.5 16.25 2.38 32-2.5 15.9-3.94 27-15.5z" fill="currentColor"/><path fill-rule="evenodd" d="m239.5 342.5q7.02-0.25 14 0.5 4.46 1.06 8 3.5-5.2 2.35-10 5.5-3.88 4.65-9 7.5-9.89-3.09-9.5-13 2.36-3.63 6.5-4z" fill="currentColor"/><path fill-rule="evenodd" d="m214.5 349.5q5.96 7.2 13.5 13 1 1 0 2-28.58 23.34-65.5 20.5-18.15-4.24-27.5-19.5 1.13 0.94 2.5 1.5 14.7 1.42 29-1.5 26.57-0.52 48-16z" fill="currentColor"/><path fill-rule="evenodd" d="m302.5 373.5q0.21 2.44-2 3.5-28.69 7.6-50.5-12.5-0.06-6.71 6.5-9 4.45-0.75 9-1 22.26 2.27 37 19z" fill="currentColor"/><path fill-rule="evenodd" d="m232.5 365.5q17.6 6.19 10.5 23-10.6 10.42-25.5 11.5-25.94 3.21-49-9 36.75-1.65 64-25.5z" fill="currentColor"/><path fill-rule="evenodd" d="m113.5 367.5q7.7-0.01 9.5 7-9.69 7.19-18.5 15.5-7.23 5.76-5.5-3.5 3.12-12.84 14.5-19z" fill="currentColor"/><path fill-rule="evenodd" d="m126.5 380.5q7.88-0.4 12 6.5-8.5 7.25-17 14.5-5.62-12.55 5-21z" fill="currentColor"/><path fill-rule="evenodd" d="m283.5 385.5q3.22 2.95 7 5.5 2.8 4.03 6 7.5 0.42 2.77-2 4-15.5-9.75-31-19.5-1.79-0.98 0-1.5 9.96 2.49 20 4z" fill="currentColor"/></svg>} href="/integrations/openclaw">
+10 -18
View File
@@ -62,8 +62,7 @@ def add(
filters: dict = None,
output_format: str = None, # ❌ REMOVED
version: str = None # ❌ REMOVED
) -> Union[List[dict], dict]:
...
) -> Union[List[dict], dict]
```
#### v1.0.0 Signature
@@ -77,8 +76,7 @@ def add(
metadata: dict = None,
filters: dict = None,
infer: bool = True # ✅ NEW: Control memory inference
) -> dict: # Always returns dict with "results" key
...
) -> dict # Always returns dict with "results" key
```
#### Changes Summary
@@ -149,8 +147,7 @@ def search(
filters: dict = None, # Basic key-value only
output_format: str = None, # ❌ REMOVED
version: str = None # ❌ REMOVED
) -> Union[List[dict], dict]:
...
) -> Union[List[dict], dict]
```
#### v1.0.0 Signature
@@ -164,8 +161,7 @@ def search(
limit: int = 100,
filters: dict = None, # ✅ ENHANCED: Advanced operators
rerank: bool = True # ✅ NEW: Reranking support
) -> dict: # Always returns dict with "results" key
...
) -> dict # Always returns dict with "results" key
```
#### Enhanced Filtering
@@ -220,8 +216,7 @@ def get_all(
filters: dict = None,
output_format: str = None, # ❌ REMOVED
version: str = None # ❌ REMOVED
) -> Union[List[dict], dict]:
...
) -> Union[List[dict], dict]
```
#### v1.0.0 Signature
@@ -232,8 +227,7 @@ def get_all(
agent_id: str = None,
run_id: str = None,
filters: dict = None # ✅ ENHANCED: Advanced operators
) -> dict: # Always returns dict with "results" key
...
) -> dict # Always returns dict with "results" key
```
### update() Method
@@ -245,8 +239,7 @@ def update(
self,
memory_id: str,
data: str
) -> dict:
...
) -> dict
```
### delete() Method
@@ -257,8 +250,7 @@ def update(
def delete(
self,
memory_id: str
) -> dict:
...
) -> dict
```
### delete_all() Method
@@ -352,7 +344,7 @@ config = {
### New Configuration Options
#### Reranker Configuration
```text
```python
# Cohere reranker
"reranker": {
"provider": "cohere",
@@ -571,4 +563,4 @@ results = m.search(
<Info>
Use this reference to systematically update your codebase. Test each change thoroughly before deploying to production.
</Info>
</Info>
@@ -117,7 +117,7 @@ results_without_criteria = client.search(
### Compare Results
### Search Results (with Criteria)
```text
```python
[
{"memory": "User feels refreshed and ready to take on anything on a beautiful sunny day", "score": 0.666, ...},
{"memory": "User finally has time to draw something after a long time", "score": 0.616, ...},
@@ -128,7 +128,7 @@ results_without_criteria = client.search(
```
### Search Results (without Criteria)
```text
```python
[
{"memory": "User is happy today", "score": 0.607, ...},
{"memory": "User feels refreshed and ready to take on anything on a beautiful sunny day", "score": 0.512, ...},
+1 -1
View File
@@ -190,7 +190,7 @@ messages = [
client.add(messages, user_id='alice')
```
```text Memories with categories
```python Memories with categories
# Following categories will be created for the memories added
Sometimes draws and sketches in free time (hobbies)
Is quite athletic (sports)
+5 -5
View File
@@ -108,10 +108,10 @@ print(response)
```javascript JavaScript
// Basic Export request
const basicFilters = {"user_id": "alice"};
const filters = {"user_id": "alice"};
const response = await client.createMemoryExport({
schema: json_schema,
filters: basicFilters
filters: filters
});
// Export with custom instructions and additional filters
@@ -124,16 +124,16 @@ const export_instructions = `
`;
// For create operation, using only user_id filter as requested
const exportFilters = {
const filters = {
"AND": [
{"user_id": "alex"},
{"created_at": {"gte": "2024-01-01"}}
]
};
}
const responseWithInstructions = await client.createMemoryExport({
schema: json_schema,
filters: exportFilters,
filters: filters,
exportInstructions: export_instructions
});
@@ -1,6 +1,6 @@
{
"name": "mem0",
"version": "0.2.11",
"version": "0.2.10",
"description": "Persistent memory for Claude Code. Remembers decisions, patterns, and preferences across sessions.",
"author": {
"name": "Mem0",
@@ -1,6 +1,6 @@
{
"name": "mem0",
"version": "0.2.11",
"version": "0.2.10",
"description": "Persistent memory for Codex. Remembers decisions, patterns, and preferences across sessions.",
"author": {
"name": "Mem0",
@@ -1,6 +1,6 @@
{
"name": "mem0",
"version": "0.2.11",
"version": "0.2.10",
"description": "Mem0 memory layer for AI applications. Add persistent memory, personalization, and semantic search using the Mem0 Platform MCP server.",
"author": {
"name": "Mem0",
-12
View File
@@ -2,18 +2,6 @@
All notable changes to the Mem0 plugin will be documented in this file.
## 0.2.11 — Session-summary metadata fix + rerank auto-injected context by default
> Versions: Claude Code / Cursor / Codex `0.2.11`; Antigravity `0.1.3`. All four editors share `scripts/`, so the fix below applies to every editor.
### Fixed
- **`files_touched` was double-JSON-encoded in session summaries (`scripts/capture_session_summary.py`):** the Stop-hook summary set `metadata["files_touched"] = json.dumps(files[:20])` — a pre-serialized JSON string — and then serialized the whole request body again with `json.dumps(body)`. The stored memory therefore carried an escaped string blob (`"[\"mem0/memory/main.py\", \"src/client/index.ts\"]"`) instead of a real array, so file paths surfaced as backslash- and slash-heavy escaped text when those memories were returned by `search_memories`/`get_memories` and shown in Claude Code, Cursor, Codex, and Antigravity. The fix stores the list directly (`metadata["files_touched"] = files[:20]`) so the body is encoded exactly once. New `tests/test_capture_session_summary.py` asserts the posted body contains a JSON array and no escaped-string artifact.
### Changed
- **Auto-injected memory context is now reranked by default (`scripts/_search.py`, `scripts/file_context.py`, `scripts/on_bash_output.sh`, `scripts/on_user_prompt.sh`):** the REST search endpoint does not rerank when `rerank` is omitted, so hook-injected context (file-context, bash-error lookup, session-resume prefetch) was ordered by raw vector similarity and the single most relevant memory could fall outside the injected `top_k` window. A new `should_rerank()` helper turns reranking on for every auto-injection path; the extra ~150–200 ms stays within the hook's curl budget. Opt out with `MEM0_RERANK=0` (also accepts `false`/`no`/`off`). (#5690)
## 0.2.10 — Accurate per-editor telemetry attribution
### Fixed
+1 -1
View File
@@ -1,7 +1,7 @@
{
"id": "mem0",
"name": "mem0",
"version": "0.1.3",
"version": "0.1.2",
"description": "Persistent semantic memory for Antigravity agents. Cross-session, user-level recall via the Mem0 Platform MCP server. 16 slash commands, lifecycle hooks for auto-capture and metadata enforcement.",
"author": { "name": "Mem0", "email": "support@mem0.ai" },
"publisher": "mem0ai",
@@ -173,7 +173,7 @@ def store_summary(
if branch:
metadata["branch"] = branch
if files:
metadata["files_touched"] = files[:20]
metadata["files_touched"] = json.dumps(files[:20])
body = {
"messages": [{"role": "user", "content": summary_prompt}],
@@ -1,79 +0,0 @@
"""Regression tests for capture_session_summary.py request body construction.
Guards against the double-JSON-encoding bug where ``files_touched`` was stored
as a pre-serialized JSON string and then encoded a second time with the rest of
the request body — surfacing as escaped, slash-heavy blobs in the memories shown
inside Claude Code / Cursor / Codex / Antigravity (all four editors share this
script).
"""
from __future__ import annotations
import json
class _FakeResp:
status = 200
def __enter__(self):
return self
def __exit__(self, *_):
return False
def _capture_request_body(monkeypatch):
"""Patch urlopen so store_summary posts nowhere; capture the request body."""
import capture_session_summary as css
captured: dict = {}
def fake_urlopen(req, timeout=0):
captured["raw"] = req.data.decode("utf-8")
captured["body"] = json.loads(captured["raw"])
return _FakeResp()
monkeypatch.setattr(css.urllib.request, "urlopen", fake_urlopen)
return captured, css
def test_files_touched_is_json_array_not_double_encoded(monkeypatch):
"""files_touched must be a real JSON array, encoded exactly once."""
captured, css = _capture_request_body(monkeypatch)
files = ["mem0/memory/main.py", "src/client/index.ts"]
css.store_summary(
api_key="test-key",
summary_prompt="did some work",
user_id="u1",
session_id="s1",
project_id="p1",
branch="main",
files=files,
)
files_touched = captured["body"]["metadata"]["files_touched"]
assert isinstance(files_touched, list), (
"files_touched must be a JSON array, not a double-encoded string; "
f"got {type(files_touched).__name__}: {files_touched!r}"
)
assert files_touched == files
# The file paths must not appear as an escaped JSON string inside the body.
assert '\\"' not in captured["raw"]
def test_files_touched_omitted_when_no_files(monkeypatch):
"""No files touched -> no files_touched key (unchanged behaviour)."""
captured, css = _capture_request_body(monkeypatch)
css.store_summary(
api_key="test-key",
summary_prompt="did some work",
user_id="u1",
session_id="s1",
project_id="p1",
branch="main",
files=[],
)
assert "files_touched" not in captured["body"]["metadata"]
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "mem0ai",
"version": "3.0.11",
"version": "3.0.9",
"description": "The Memory Layer For Your AI Apps",
"main": "./dist/index.js",
"module": "./dist/index.mjs",
+1 -3
View File
@@ -169,9 +169,7 @@ export interface PaginatedMemories {
export interface ProjectResponse {
customInstructions?: string;
// The API returns category objects (`[{ "<name>": "<description>" }]`),
// not bare strings (see issue #5738).
customCategories?: custom_categories[];
customCategories?: string[];
[key: string]: any;
}
-46
View File
@@ -99,50 +99,4 @@ describe("camelToSnakeKeys / snakeToCamelKeys", () => {
});
});
});
describe("user-controlled customCategories names (issue #5738)", () => {
it("converts the outer key but leaves multi-word category names on write", () => {
expect(
camelToSnakeKeys({
customCategories: [
{ work_life_balance: "desc" },
{ AIResearch: "desc" },
],
}),
).toEqual({
// outer SDK key is snake_cased, user-defined category names are not
custom_categories: [
{ work_life_balance: "desc" },
{ AIResearch: "desc" },
],
});
});
it("converts the outer key but leaves category names verbatim on read", () => {
expect(
snakeToCamelKeys({
custom_categories: [
{ work_life_balance: "desc" },
{ AIResearch: "desc" },
],
}),
).toEqual({
customCategories: [
{ work_life_balance: "desc" },
{ AIResearch: "desc" },
],
});
});
it("round-trips category names losslessly (write then read)", () => {
const customCategories = [
{ work_life_balance: "balance between work and life" },
{ AIResearch: "artificial intelligence research" },
];
const roundTripped = snakeToCamelKeys(
camelToSnakeKeys({ customCategories }),
);
expect(roundTripped.customCategories).toEqual(customCategories);
});
});
});
-5
View File
@@ -29,11 +29,6 @@ const OPAQUE_VALUE_KEYS = new Set([
"metadata",
"structuredDataSchema",
"structured_data_schema",
// Custom-category names are user-controlled keys (`[{ "<name>": "<desc>" }]`).
// Listed in both casings so they round-trip verbatim in both directions
// (see issue #5738; same class as `metadata`/`structuredDataSchema`).
"customCategories",
"custom_categories",
]);
/**
+43
View File
@@ -0,0 +1,43 @@
import { OpenAILLM } from "./openai";
import { LLMConfig, Message } from "../types";
import { LLMResponse } from "./base";
export class TogetherLLM extends OpenAILLM {
constructor(config: LLMConfig) {
const apiKey = config.apiKey || process.env.TOGETHER_API_KEY;
if (!apiKey) {
throw new Error("Together API key is required");
}
super({
...config,
apiKey,
baseURL:
config.baseURL ||
process.env.TOGETHER_API_BASE ||
"https://api.together.xyz/v1",
model: config.model || "mistralai/Mixtral-8x7B-Instruct-v0.1",
});
}
async generateResponse(
messages: Message[],
responseFormat?: { type: string },
tools?: any[],
): Promise<string | LLMResponse> {
try {
return await super.generateResponse(messages, responseFormat, tools);
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
throw new Error(`Together LLM failed: ${message}`);
}
}
async generateChat(messages: Message[]): Promise<LLMResponse> {
try {
return await super.generateChat(messages);
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
throw new Error(`Together LLM failed: ${message}`);
}
}
}
+10 -72
View File
@@ -296,44 +296,6 @@ export class Memory {
return filters;
}
private _normalizeEntityText(value: string): string {
return value.trim().toLowerCase().replace(/\s+/g, " ");
}
private async _existingEntitiesByText(
entityStore: VectorStore,
filters: Record<string, any>,
): Promise<Map<string, { id: string; payload: Record<string, any> }>> {
const rowsByText = new Map<
string,
{ id: string; payload: Record<string, any> }
>();
let rows: Array<{ id: string; payload: Record<string, any> }> = [];
try {
const listed = await entityStore.list(filters, 10000);
rows = (
Array.isArray(listed) && Array.isArray(listed[0])
? listed[0]
: (listed as any)
) as Array<{ id: string; payload: Record<string, any> }>;
} catch (e) {
console.debug(
`Exact entity lookup failed, falling back to semantic dedup: ${e}`,
);
return rowsByText;
}
for (const row of rows) {
const text = row.payload?.data;
if (typeof text !== "string") continue;
const key = this._normalizeEntityText(text);
if (key && !rowsByText.has(key)) {
rowsByText.set(key, row);
}
}
return rowsByText;
}
/**
* Remove `memoryId` from every entity record scoped to `filters`.
* If an entity's `linkedMemoryIds` becomes empty after removal, the
@@ -431,10 +393,6 @@ export class Memory {
if (entities.length === 0) return;
const entityStore = await this.getEntityStore();
const exactMatches = await this._existingEntitiesByText(
entityStore,
filters,
);
for (const entity of entities) {
try {
@@ -451,21 +409,12 @@ export class Memory {
score?: number;
payload: Record<string, any>;
}> = [];
const exactMatch = exactMatches.get(
this._normalizeEntityText(entity.text),
);
if (!exactMatch) {
try {
matches = await entityStore.search(entityVec, 1, filters);
} catch {}
}
try {
matches = await entityStore.search(entityVec, 1, filters);
} catch {}
const semanticMatch =
matches.length > 0 && (matches[0].score ?? 0) >= 0.95
? matches[0]
: undefined;
const match = exactMatch ?? semanticMatch;
if (match) {
if (matches.length > 0 && (matches[0].score ?? 0) >= 0.95) {
const match = matches[0];
const payload = match.payload || {};
const linked = new Set<string>(
Array.isArray(payload.linkedMemoryIds)
@@ -1113,10 +1062,6 @@ export class Memory {
if (valid.length > 0) {
const entityStore = await this.getEntityStore();
const exactMatches = await this._existingEntitiesByText(
entityStore,
filters,
);
// 7c: Search for existing entities one by one (no batch search)
const toInsertVectors: number[][] = [];
@@ -1132,20 +1077,13 @@ export class Memory {
score?: number;
payload: Record<string, any>;
}> = [];
const exactMatch = exactMatches.get(key);
if (!exactMatch) {
try {
matches = await entityStore.search(entityVec, 1, filters);
} catch {}
}
try {
matches = await entityStore.search(entityVec, 1, filters);
} catch {}
const semanticMatch =
matches.length > 0 && (matches[0].score ?? 0) >= 0.95
? matches[0]
: undefined;
const match = exactMatch ?? semanticMatch;
if (match) {
if (matches.length > 0 && (matches[0].score ?? 0) >= 0.95) {
// Update existing entity
const match = matches[0];
const payload = match.payload || {};
const linked = new Set<string>(payload.linkedMemoryIds ?? []);
for (const mid of memoryIds) linked.add(mid);
+125 -230
View File
@@ -4,8 +4,8 @@
* Extracts four types of entities from text:
* - PROPER: Capitalized multi-word sequences (person names, places, brands)
* - QUOTED: Text in single or double quotes (titles, specific terms)
* - TOPIC: Multi-word noun/topic phrases with specific modifiers
* - IDENTIFIER: Dotted technical identifiers such as person.properties.email
* - COMPOUND: Multi-word noun phrases with specific modifiers (e.g., "machine learning")
* - NOUN: Single nouns from circumstantial compound patterns
*
* Uses the `compromise` npm package for NLP-based extraction when available.
* Falls back to regex-only extraction if `compromise` is not installed.
@@ -196,25 +196,6 @@ const NON_SPECIFIC_ADJ: Set<string> = new Set([
"final",
"initial",
"side",
"top",
]);
/** Leading words that frame a topic but are not part of the topic itself. */
const TOPIC_PREFIX_WORDS: Set<string> = new Set([
"a",
"an",
"the",
"my",
"your",
"our",
"their",
"his",
"her",
"its",
"this",
"that",
"these",
"those",
]);
/** Generic tail words to strip from compound entities. */
@@ -286,25 +267,6 @@ const GENERIC_CAPS: Set<string> = new Set([
"disadvantages",
]);
/** Generic role/title words that should not become single-token entities. */
const GENERIC_SINGLE_ENTITY_TERMS: Set<string> = new Set([
"user",
"assistant",
"agent",
"customer",
"client",
"person",
"people",
"human",
"memory",
"message",
"conversation",
"chat",
"session",
"system",
"top",
]);
/** Markdown/formatting markers to skip during extraction. */
const FORMATTING_MARKERS: Set<string> = new Set([
"*",
@@ -325,7 +287,7 @@ const FORMATTING_MARKERS: Set<string> = new Set([
// ---------------------------------------------------------------------------
export interface ExtractedEntity {
type: "PROPER" | "QUOTED" | "TOPIC" | "IDENTIFIER";
type: "PROPER" | "QUOTED" | "COMPOUND" | "NOUN";
text: string;
}
@@ -376,96 +338,32 @@ function stripGenericEnding(words: string[]): string[] {
return words;
}
function stripTopicPrefix(words: string[]): string[] {
let start = 0;
while (
start < words.length &&
TOPIC_PREFIX_WORDS.has(words[start].toLowerCase())
) {
start++;
}
return words.slice(start);
}
function cleanToken(token: string): string {
return token.replace(/^[^\w.]+|[^\w.]+$/g, "");
}
function tokenize(text: string): string[] {
return (
text.match(
/[A-Za-z_][\w-]*(?:\.[A-Za-z_][\w-]*)*|\d[\d,]*(?:\.\d+)?|[,:;.!?&]/g,
) ?? []
);
}
function isCapitalized(token: string): boolean {
return /^[A-Z]/.test(token) && /[A-Za-z]/.test(token);
}
function hasInternalCapOrDigit(token: string): boolean {
return (
/\d/.test(token) ||
/[A-Z]/.test(token.slice(1)) ||
/^[A-Z]{2,}$/.test(token)
);
}
function isBadSingleNameToken(token: string): boolean {
const lower = token.toLowerCase();
return GENERIC_SINGLE_ENTITY_TERMS.has(lower) || GENERIC_CAPS.has(lower);
}
function looksLikeMetricCount(token: string): boolean {
return /^\d[\d,]*(?:\.\d+)?$/.test(token);
}
function isMetricListContext(tokens: string[], idx: number): boolean {
const prev = idx > 0 ? tokens[idx - 1] : "";
const next = idx + 1 < tokens.length ? tokens[idx + 1] : "";
return [":", ",", ";"].includes(prev) || [",", ";"].includes(next);
}
function isSentenceStart(tokens: string[], idx: number): boolean {
if (idx === 0) return true;
return (
[".", "!", "?", ":"].includes(tokens[idx - 1]) ||
FORMATTING_MARKERS.has(tokens[idx - 1])
);
}
function isListItemNameToken(tokens: string[], idx: number): boolean {
const token = cleanToken(tokens[idx]);
if (!isCapitalized(token) || isBadSingleNameToken(token)) return false;
const next = idx + 1 < tokens.length ? cleanToken(tokens[idx + 1]) : "";
if (!looksLikeMetricCount(next)) return false;
return (
isMetricListContext(tokens, idx) || isMetricListContext(tokens, idx + 1)
);
}
function isNameToken(tokens: string[], idx: number): boolean {
const token = cleanToken(tokens[idx]);
if (!token || !isCapitalized(token) || isBadSingleNameToken(token))
return false;
if (hasInternalCapOrDigit(token) || isListItemNameToken(tokens, idx))
/**
* Determine if a token position is at the start of a sentence.
* Simple heuristic: index 0, or preceded by sentence-ending punctuation
* or formatting markers.
*/
function isSentenceStart(
tokens: string[],
idx: number,
rawText: string,
): boolean {
if (idx === 0) {
return true;
return !isSentenceStart(tokens, idx);
}
function cleanEntityText(text: string): string {
return text
.replace(/^\*+\s*|\s*\*+$/g, "")
.replace(/\s*:+$/g, "")
.replace(/^\d+\s*\.\s*/, "")
.replace(/\s+\d[\d,]*(?:\.\d+)?$/g, "")
.replace(/[.,;!?]+$/, "")
.trim()
.replace(/\s+/g, " ");
}
function isCoordinatedNameTopic(text: string): boolean {
return /\b[A-Z][\w-]+\s+and\s+[A-Z][\w-]+\b/.test(text);
}
const prev = tokens[idx - 1];
if (/[.!?:]$/.test(prev)) {
return true;
}
if (FORMATTING_MARKERS.has(prev)) {
return true;
}
// Check for newline before this token in the raw text
const tokenStart = rawText.indexOf(tokens[idx]);
if (tokenStart > 0 && rawText.charAt(tokenStart - 1) === "\n") {
return true;
}
return false;
}
// ---------------------------------------------------------------------------
@@ -499,79 +397,86 @@ function extractQuoted(text: string): ExtractedEntity[] {
}
/**
* Extract dotted technical identifiers such as person.properties.email.
*/
function extractIdentifiers(text: string): ExtractedEntity[] {
const entities: ExtractedEntity[] = [];
const identifierRe = /\b[A-Za-z_][\w-]*(?:\.[A-Za-z_][\w-]*)+\b/g;
let match: RegExpExecArray | null;
while ((match = identifierRe.exec(text)) !== null) {
entities.push({ type: "IDENTIFIER", text: match[0] });
}
return entities;
}
/**
* Extract proper names using capitalization and list-context heuristics.
* Extract proper noun sequences using capitalization heuristics.
* Finds sequences of capitalized words that are not at sentence starts.
*/
function extractProper(text: string): ExtractedEntity[] {
const entities: ExtractedEntity[] = [];
const tokens = tokenize(text);
const innerConnectors = new Set(["of", "the", "in", "for", "at"]);
// Tokenize on whitespace, preserving order
const tokens = text.split(/\s+/).filter(Boolean);
const functionWords = new Set([
"'s",
"of",
"the",
"in",
"and",
"for",
"at",
"is",
]);
let i = 0;
while (i < tokens.length) {
const token = cleanToken(tokens[i]);
const next = i + 1 < tokens.length ? tokens[i + 1] : "";
const afterNext = i + 2 < tokens.length ? cleanToken(tokens[i + 2]) : "";
if (
token &&
next === "&" &&
afterNext &&
isCapitalized(token) &&
isCapitalized(afterNext) &&
!isBadSingleNameToken(token) &&
!isBadSingleNameToken(afterNext)
) {
entities.push({
type: "PROPER",
text: cleanEntityText(`${token} & ${afterNext}`),
});
i += 3;
continue;
}
if (!isNameToken(tokens, i)) {
const tok = tokens[i];
// Skip formatting markers
if (FORMATTING_MARKERS.has(tok)) {
i++;
continue;
}
const span = [cleanToken(tokens[i])];
let j = i + 1;
while (j < tokens.length) {
const current = cleanToken(tokens[j]);
if (isNameToken(tokens, j)) {
span.push(current);
j++;
continue;
}
if (
innerConnectors.has(current.toLowerCase()) &&
j + 1 < tokens.length &&
isNameToken(tokens, j + 1)
) {
span.push(current, cleanToken(tokens[j + 1]));
j += 2;
continue;
}
break;
}
const isLabel = i + 1 < tokens.length && tokens[i + 1] === ":";
const isCap =
tok.length > 0 &&
tok.charAt(0) === tok.charAt(0).toUpperCase() &&
/[A-Z]/.test(tok.charAt(0));
const phrase = cleanEntityText(span.join(" "));
if (phrase.length > 2) {
entities.push({ type: "PROPER", text: phrase });
if (isCap && !isLabel) {
const seq: Array<{ token: string; idx: number }> = [
{ token: tok, idx: i },
];
let j = i + 1;
while (j < tokens.length) {
const t = tokens[j];
const tIsCap =
t.length > 0 &&
t.charAt(0) === t.charAt(0).toUpperCase() &&
/[A-Z]/.test(t.charAt(0));
if (tIsCap || functionWords.has(t.toLowerCase())) {
seq.push({ token: t, idx: j });
j++;
} else {
break;
}
}
// Strip trailing function words
while (
seq.length > 0 &&
functionWords.has(seq[seq.length - 1].token.toLowerCase())
) {
seq.pop();
}
if (seq.length > 0) {
// Check for at least one mid-sentence capitalized word
const hasMidCap = seq.some(({ token, idx: tokenIdx }) => {
const isCapWord =
/[A-Z]/.test(token.charAt(0)) &&
!functionWords.has(token.toLowerCase());
return isCapWord && !isSentenceStart(tokens, tokenIdx, text);
});
if (hasMidCap) {
const phrase = seq.map((s) => s.token).join(" ");
if (phrase.length > 2) {
entities.push({ type: "PROPER", text: phrase });
}
}
}
i = j;
} else {
i++;
}
i = Math.max(j, i + 1);
}
return entities;
@@ -579,7 +484,7 @@ function extractProper(text: string): ExtractedEntity[] {
/**
* Extract compound noun phrases using the `compromise` NLP library.
* Returns TOPIC entities derived from noun chunks.
* Returns COMPOUND and NOUN entities derived from noun chunks.
*/
function extractCompoundsWithNlp(text: string): ExtractedEntity[] {
if (!nlp) {
@@ -619,12 +524,12 @@ function extractCompoundsWithNlp(text: string): ExtractedEntity[] {
const filtered = words.filter(
(w) => !NON_SPECIFIC_ADJ.has(w.toLowerCase()),
);
const cleaned = stripGenericEnding(stripTopicPrefix(filtered));
const cleaned = stripGenericEnding(filtered);
if (cleaned.length >= 2) {
const phrase = cleanEntityText(cleaned.join(" "));
const phrase = cleaned.join(" ");
if (phrase.length > 3) {
entities.push({ type: "TOPIC", text: phrase });
entities.push({ type: "COMPOUND", text: phrase });
}
}
}
@@ -642,7 +547,7 @@ function extractCompoundsRegex(text: string): ExtractedEntity[] {
// Multi-word sequences with at least one non-trivial word
// Match sequences like "machine learning", "New York", "data science"
const compoundRe =
/\b([A-Z][a-z]+(?:\s+(?:of|the|for|in)\s+)?[A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)\b/g;
/\b([A-Z][a-z]+(?:\s+(?:of|and|the|for|in)\s+)?[A-Z][a-z]+(?:\s+[A-Z][a-z]+)*)\b/g;
let match: RegExpExecArray | null;
while ((match = compoundRe.exec(text)) !== null) {
const phrase = match[1].trim();
@@ -653,12 +558,9 @@ function extractCompoundsRegex(text: string): ExtractedEntity[] {
const filtered = words.filter(
(w) => !NON_SPECIFIC_ADJ.has(w.toLowerCase()),
);
const cleaned = stripGenericEnding(stripTopicPrefix(filtered));
const cleaned = stripGenericEnding(filtered);
if (cleaned.length >= 2) {
entities.push({
type: "TOPIC",
text: cleanEntityText(cleaned.join(" ")),
});
entities.push({ type: "COMPOUND", text: cleaned.join(" ") });
}
}
}
@@ -688,12 +590,9 @@ function extractCompoundsRegex(text: string): ExtractedEntity[] {
const filtered = words.filter(
(w) => !NON_SPECIFIC_ADJ.has(w.toLowerCase()),
);
const cleaned = stripGenericEnding(stripTopicPrefix(filtered));
const cleaned = stripGenericEnding(filtered);
if (cleaned.length >= 2) {
entities.push({
type: "TOPIC",
text: cleanEntityText(cleaned.join(" ")),
});
entities.push({ type: "COMPOUND", text: cleaned.join(" ") });
}
}
}
@@ -715,9 +614,9 @@ function extractCompoundsRegex(text: string): ExtractedEntity[] {
*
* Entity types (in priority order for deduplication):
* PROPER - Capitalized multi-word sequences not at sentence start
* IDENTIFIER - Dotted technical identifiers
* COMPOUND - Multi-word noun phrases with specific modifiers
* QUOTED - Text in single or double quotes (min 3 chars)
* TOPIC - Multi-word noun/topic phrases with specific modifiers
* NOUN - Single nouns from circumstantial patterns
*
* @param text - Input text to extract entities from.
* @returns Deduplicated list of extracted entities.
@@ -731,10 +630,7 @@ export function extractEntities(text: string): ExtractedEntity[] {
// 2. PROPER entities (capitalization heuristics)
raw.push(...extractProper(text));
// 3. IDENTIFIER entities
raw.push(...extractIdentifiers(text));
// 4. TOPIC entities (NLP or regex fallback)
// 3. COMPOUND entities (NLP or regex fallback)
if (nlp) {
raw.push(...extractCompoundsWithNlp(text));
} else {
@@ -758,17 +654,19 @@ export function extractEntities(text: string): ExtractedEntity[] {
const cleaned: ExtractedEntity[] = [];
for (const entity of deduped) {
let txt = entity.text.trim();
txt = cleanEntityText(txt);
// Strip leading/trailing asterisks
txt = txt.replace(/^\*+\s*|\s*\*+$/g, "");
// Strip trailing colons
txt = txt.replace(/\s*:+$/, "");
// Strip leading numbered list markers
txt = txt.replace(/^\d+\s*\.\s*/, "");
// Strip trailing sentence punctuation (".", ",", ";", "!", "?") — otherwise
// "Paris." and "Paris" produce different embeddings and break entity dedup.
txt = txt.replace(/[.,;!?]+$/, "").trim();
if (!txt || txt.length <= 2 || hasArtifacts(txt)) {
continue;
}
if (
entity.type === "TOPIC" &&
(/^\d/.test(txt) || isCoordinatedNameTopic(txt))
) {
continue;
}
// Filter generic single-word PROPER nouns
if (
@@ -782,12 +680,12 @@ export function extractEntities(text: string): ExtractedEntity[] {
cleaned.push({ type: entity.type, text: txt });
}
// Keep best type per entity (PROPER > IDENTIFIER > QUOTED > TOPIC)
// Keep best type per entity (PROPER > COMPOUND > QUOTED > NOUN)
const typePriority: Record<string, number> = {
PROPER: 0,
IDENTIFIER: 1,
COMPOUND: 1,
QUOTED: 2,
TOPIC: 3,
NOUN: 3,
};
const best = new Map<string, ExtractedEntity>();
for (const entity of cleaned) {
@@ -802,17 +700,14 @@ export function extractEntities(text: string): ExtractedEntity[] {
}
const bestEntities = Array.from(best.values());
// Remove entities that are token substrings of longer entities.
// Remove entities that are substrings of longer entities
const allLower = bestEntities.map((e) => e.text.toLowerCase());
return bestEntities.filter(
(entity) =>
!bestEntities.some(
!allLower.some(
(other) =>
entity.text.toLowerCase() !== other.text.toLowerCase() &&
(typePriority[entity.type] ?? 99) >=
(typePriority[other.type] ?? 99) &&
new RegExp(
`(^|\\s)${entity.text.toLowerCase().replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}(\\s|$)`,
).test(other.text.toLowerCase()),
entity.text.toLowerCase() !== other &&
other.includes(entity.text.toLowerCase()),
),
);
}
+3
View File
@@ -22,6 +22,7 @@ import { RedisDB } from "../vector_stores/redis";
import { OllamaLLM } from "../llms/ollama";
import { LMStudioLLM } from "../llms/lmstudio";
import { DeepSeekLLM } from "../llms/deepseek";
import { TogetherLLM } from "../llms/together";
import { SupabaseDB } from "../vector_stores/supabase";
import { SQLiteManager } from "../storage/SQLiteManager";
import { MemoryHistoryManager } from "../storage/MemoryHistoryManager";
@@ -85,6 +86,8 @@ export class LLMFactory {
return new LangchainLLM(config);
case "deepseek":
return new DeepSeekLLM(config);
case "together":
return new TogetherLLM(config);
default:
throw new Error(`Unsupported LLM provider: ${provider}`);
}
+3 -5
View File
@@ -31,11 +31,9 @@ const parse_vision_messages = async (messages: Message[]) => {
typeof message.content === "object" &&
message.content.type === "image_url"
) {
const imageUrl = message.content.image_url?.url;
if (!imageUrl) {
throw new Error("image_url content part is missing image_url.url");
}
const description = await get_image_description(imageUrl);
const description = await get_image_description(
message.content.image_url.url,
);
new_message.content =
typeof description === "string"
? description
+33 -72
View File
@@ -1,4 +1,4 @@
import type { Client as ClientType, ClientConfig } from "pg";
import type { Client as ClientType } from "pg";
import pkg from "pg";
const { Client, escapeIdentifier } = pkg;
import { VectorStore } from "./base";
@@ -157,90 +157,41 @@ export function buildFilterConditions(
interface PGVectorConfig extends VectorStoreConfig {
dbname?: string;
user?: string;
password?: string;
host?: string;
port?: number;
connectionString?: string;
ssl?: ClientConfig["ssl"];
user: string;
password: string;
host: string;
port: number;
embeddingModelDims: number;
diskann?: boolean;
hnsw?: boolean;
}
function getConnectionString(config: PGVectorConfig): string | undefined {
return config.connectionString?.trim() || undefined;
}
function validateConnectionConfig(config: PGVectorConfig): void {
if (getConnectionString(config)) {
return;
}
const missingFields = ["user", "password", "host", "port"].filter((field) => {
const v = config[field as keyof PGVectorConfig];
return v === undefined || v === null || v === "";
});
if (missingFields.length > 0) {
throw new Error(
`PGVector requires either connectionString or ${missingFields.join(", ")}`,
);
}
}
function buildClientConfig(
config: PGVectorConfig,
database?: string,
): ClientConfig {
const connectionString = getConnectionString(config);
if (connectionString) {
return {
connectionString,
...(config.ssl !== undefined ? { ssl: config.ssl } : {}),
};
}
return {
database,
user: config.user,
password: config.password,
host: config.host,
port: config.port,
...(config.ssl !== undefined ? { ssl: config.ssl } : {}),
};
}
export class PGVector implements VectorStore {
private client: ClientType;
private collectionName: string;
private useDiskann: boolean;
private useHnsw: boolean;
private readonly dbName: string;
private readonly useDirectConnection: boolean;
private config: PGVectorConfig;
private _initPromise?: Promise<void>;
constructor(config: PGVectorConfig) {
validateConnectionConfig(config);
this.collectionName = validateIdentifier(
config.collectionName || "memories",
"collectionName",
);
this.useDiskann = config.diskann || false;
this.useHnsw = config.hnsw || false;
this.useDirectConnection = !!getConnectionString(config);
this.dbName = this.useDirectConnection
? ""
: validateIdentifier(config.dbname || "vector_store", "dbname");
this.dbName = validateIdentifier(config.dbname || "vector_store", "dbname");
this.config = config;
this.client = new Client(
buildClientConfig(
config,
this.useDirectConnection ? undefined : "postgres",
),
);
this.client = new Client({
database: "postgres", // Initially connect to default postgres database
user: config.user,
password: config.password,
host: config.host,
port: config.port,
});
this.initialize().catch(console.error);
}
@@ -259,20 +210,29 @@ export class PGVector implements VectorStore {
try {
await this.client.connect();
if (!this.useDirectConnection) {
const dbExists = await this.checkDatabaseExists(this.dbName);
if (!dbExists) {
await this.createDatabase(this.dbName);
}
await this.client.end();
this.client = new Client(buildClientConfig(this.config, this.dbName));
await this.client.connect();
// Check if database exists
const dbExists = await this.checkDatabaseExists(this.dbName);
if (!dbExists) {
await this.createDatabase(this.dbName);
}
// Disconnect from postgres database
await this.client.end();
// Connect to the target database
this.client = new Client({
database: this.dbName,
user: this.config.user,
password: this.config.password,
host: this.config.host,
port: this.config.port,
});
await this.client.connect();
// Create vector extension
await this.client.query("CREATE EXTENSION IF NOT EXISTS vector");
// Create memory_migrations table
await this.client.query(`
CREATE TABLE IF NOT EXISTS memory_migrations (
id SERIAL PRIMARY KEY,
@@ -280,6 +240,7 @@ export class PGVector implements VectorStore {
)
`);
// Check if the collection exists
const collections = await this.listCols();
if (!collections.includes(this.collectionName)) {
await this.createCol(this.config.embeddingModelDims);
@@ -1,45 +0,0 @@
import { extractEntities } from "../src/utils/entity_extraction";
describe("extractEntities", () => {
it("handles product lists, coordinated names, and identifiers", () => {
const text =
"User reported top inbound integration pages: OpenClaw 25,443, " +
"Claude Code 8,916, Codex 2,573, Dify 656. " +
"User compared Cartesia and Deepgram. " +
"The email field for Mem0 lives at person.properties.email. " +
"The qwen endpoint uses person.properties.email. " +
"Johnson & Johnson was mentioned. " +
"Glasses around my window. " +
"On 2026-05-27 there were 90 days of stats.";
const entityTexts = new Set(
extractEntities(text).map((entity) => entity.text),
);
const normalized = new Set(
[...entityTexts].map((entityText) => entityText.toLowerCase()),
);
for (const expected of [
"OpenClaw",
"Claude Code",
"Codex",
"Dify",
"Cartesia",
"Deepgram",
"Mem0",
]) {
expect(entityTexts.has(expected)).toBe(true);
}
expect(entityTexts.has("person.properties.email")).toBe(true);
expect(entityTexts.has("qwen endpoint")).toBe(true);
expect(entityTexts.has("Johnson & Johnson")).toBe(true);
expect(entityTexts.has("Johnson")).toBe(false);
expect(normalized.has("top")).toBe(false);
expect(normalized.has("glasses")).toBe(false);
expect(entityTexts.has("Cartesia and Deepgram")).toBe(false);
expect(entityTexts.has("Claude Code 8,916")).toBe(false);
for (const rejected of ["8,916", "2,573", "656", "2026-05-27", "90"]) {
expect(entityTexts.has(rejected)).toBe(false);
}
});
});
@@ -1,3 +1,5 @@
/// <reference types="jest" />
jest.mock("pg", () => {
const Client = jest.fn().mockImplementation(() => ({
connect: jest.fn().mockResolvedValue(undefined),
+4 -122
View File
@@ -1,3 +1,5 @@
/// <reference types="jest" />
const searchRows = [
{
id: "a",
@@ -21,13 +23,9 @@ const searchRows = [
},
];
const mockState = {
databaseExists: true,
};
function mockPgQuery(sql: string) {
if (sql.includes("SELECT 1 FROM pg_database")) {
return { rows: mockState.databaseExists ? [{ "?column?": 1 }] : [] };
return { rows: [{ "?column?": 1 }] };
}
if (sql.includes("FROM information_schema.tables")) {
@@ -71,129 +69,13 @@ jest.mock("pg", () => {
import { PGVector } from "../src/vector_stores/pgvector";
function getClientQueries(client: { query: jest.Mock }) {
return client.query.mock.calls.map(([sql]) => sql as string);
}
describe("PGVector", () => {
describe("PGVector - search()", () => {
beforeEach(() => {
const pg = require("pg");
mockState.databaseExists = true;
pg.__mock.Client.mockClear();
pg.__mock.clients.length = 0;
});
test("uses one direct client for connectionString mode and skips bootstrap database creation", async () => {
mockState.databaseExists = false;
const ssl = { rejectUnauthorized: false };
const store = new PGVector({
collectionName: "memories",
connectionString:
"postgresql://postgres:postgres@db.example.com:5432/neondb",
ssl,
embeddingModelDims: 3,
dimension: 3,
} as any);
await store.initialize();
const pg = require("pg");
expect(pg.__mock.Client).toHaveBeenCalledTimes(1);
expect(pg.__mock.Client).toHaveBeenCalledWith({
connectionString:
"postgresql://postgres:postgres@db.example.com:5432/neondb",
ssl,
});
const directClient = pg.__mock.clients[0];
const queries = getClientQueries(directClient);
expect(queries).not.toEqual(
expect.arrayContaining([
expect.stringContaining("SELECT 1 FROM pg_database"),
]),
);
expect(queries).not.toEqual(
expect.arrayContaining([expect.stringContaining("CREATE DATABASE")]),
);
expect(queries).toEqual(
expect.arrayContaining([
"CREATE EXTENSION IF NOT EXISTS vector",
expect.stringContaining("FROM information_schema.tables"),
]),
);
});
test("keeps the split-field bootstrap flow when connectionString is absent", async () => {
mockState.databaseExists = false;
const ssl = { rejectUnauthorized: false };
const store = new PGVector({
collectionName: "memories",
user: "postgres",
password: "postgres",
host: "localhost",
port: 5432,
dbname: "vector_store",
ssl,
embeddingModelDims: 3,
dimension: 3,
} as any);
await store.initialize();
const pg = require("pg");
expect(pg.__mock.Client).toHaveBeenCalledTimes(2);
expect(pg.__mock.Client).toHaveBeenNthCalledWith(1, {
database: "postgres",
user: "postgres",
password: "postgres",
host: "localhost",
port: 5432,
ssl,
});
expect(pg.__mock.Client).toHaveBeenNthCalledWith(2, {
database: "vector_store",
user: "postgres",
password: "postgres",
host: "localhost",
port: 5432,
ssl,
});
const bootstrapClient = pg.__mock.clients[0];
const activeClient = pg.__mock.clients[1];
const bootstrapQueries = getClientQueries(bootstrapClient);
expect(bootstrapQueries).toEqual(
expect.arrayContaining([
"SELECT 1 FROM pg_database WHERE datname = $1",
'CREATE DATABASE "vector_store"',
]),
);
expect(bootstrapClient.end).toHaveBeenCalledTimes(1);
expect(getClientQueries(activeClient)).toEqual(
expect.arrayContaining([
"CREATE EXTENSION IF NOT EXISTS vector",
expect.stringContaining("FROM information_schema.tables"),
]),
);
});
test("throws when connectionString is absent and split-field params are missing", () => {
expect(
() =>
new PGVector({
collectionName: "memories",
embeddingModelDims: 3,
dimension: 3,
} as any),
).toThrow(
"PGVector requires either connectionString or user, password, host, port",
);
});
test("returns similarity score (1 - distance) clamped to [0, 1]", async () => {
const store = new PGVector({
collectionName: "memories",
+124
View File
@@ -0,0 +1,124 @@
/// <reference types="jest" />
/**
* Together LLM — unit tests (mocked OpenAI).
*/
import { TogetherLLM } from "../src/llms/together";
const mockCreate = jest.fn();
jest.mock("openai", () => {
return jest.fn().mockImplementation(() => ({
chat: { completions: { create: mockCreate } },
}));
});
describe("TogetherLLM (unit)", () => {
beforeEach(() => mockCreate.mockClear());
it("throws when no API key is provided", () => {
const original = process.env.TOGETHER_API_KEY;
delete process.env.TOGETHER_API_KEY;
try {
expect(() => new TogetherLLM({})).toThrow("Together API key is required");
} finally {
if (original !== undefined) process.env.TOGETHER_API_KEY = original;
}
});
it("generateResponse() returns a text response", async () => {
mockCreate.mockResolvedValueOnce({
choices: [
{
message: {
content: "Hello, world!",
role: "assistant",
tool_calls: null,
},
},
],
});
const llm = new TogetherLLM({ apiKey: "test-key" });
const result = await llm.generateResponse([
{ role: "user", content: "Hi" },
]);
expect(mockCreate).toHaveBeenCalledTimes(1);
expect(result).toBe("Hello, world!");
});
it("generateResponse() handles tool calls", async () => {
mockCreate.mockResolvedValueOnce({
choices: [
{
message: {
content: "",
role: "assistant",
tool_calls: [
{
function: {
name: "get_weather",
arguments: '{"city": "London"}',
},
},
],
},
},
],
});
const llm = new TogetherLLM({ apiKey: "test-key" });
const result = await llm.generateResponse(
[{ role: "user", content: "What is the weather?" }],
undefined,
[{ type: "function", function: { name: "get_weather" } }],
);
expect(result).toEqual({
content: "",
role: "assistant",
toolCalls: [{ name: "get_weather", arguments: '{"city": "London"}' }],
});
});
it("generateResponse() wraps API errors with a clear message", async () => {
mockCreate.mockRejectedValueOnce(new Error("Connection refused"));
const llm = new TogetherLLM({ apiKey: "test-key" });
await expect(
llm.generateResponse([{ role: "user", content: "Hi" }]),
).rejects.toThrow("Together LLM failed: Connection refused");
});
it("generateChat() returns LLMResponse shape", async () => {
mockCreate.mockResolvedValueOnce({
choices: [
{
message: { content: "I can help with that.", role: "assistant" },
},
],
});
const llm = new TogetherLLM({ apiKey: "test-key" });
const result = await llm.generateChat([
{ role: "user", content: "Help me" },
]);
expect(result).toEqual({
content: "I can help with that.",
role: "assistant",
});
});
it("generateChat() wraps API errors with a clear message", async () => {
mockCreate.mockRejectedValueOnce(new Error("Timeout"));
const llm = new TogetherLLM({ apiKey: "test-key" });
await expect(
llm.generateChat([{ role: "user", content: "Hi" }]),
).rejects.toThrow("Together LLM failed: Timeout");
});
});
-6
View File
@@ -18,12 +18,6 @@ class OpenSearchConfig(BaseModel):
"RequestsHttpConnection", description="Connection class for OpenSearch"
)
pool_maxsize: int = Field(20, description="Maximum number of connections in the pool")
auto_refresh: bool = Field(
False,
description="Automatically refresh index after insert operations to make documents "
"immediately searchable. Disabled by default for OpenSearch Serverless compatibility. "
"OpenSearch automatically refreshes indices every ~1 second, so most users don't need this.",
)
@model_validator(mode="before")
@classmethod
+2 -33
View File
@@ -1,7 +1,6 @@
import json
import logging
import os
from typing import Dict, List, Optional, Union
from typing import Dict, List, Optional
try:
from groq import Groq
@@ -12,8 +11,6 @@ from mem0.configs.llms.base import BaseLlmConfig
from mem0.llms.base import LLMBase
from mem0.memory.utils import extract_json
logger = logging.getLogger(__name__)
class GroqLLM(LLMBase):
def __init__(self, config: Optional[BaseLlmConfig] = None):
@@ -25,24 +22,6 @@ class GroqLLM(LLMBase):
api_key = self.config.api_key or os.getenv("GROQ_API_KEY")
self.client = Groq(api_key=api_key)
@staticmethod
def _supports_json_mode(model: Optional[Union[str, Dict]]) -> bool:
"""
Groq's compound agentic systems (e.g. ``groq/compound``, ``groq/compound-mini``)
do not support the JSON ``response_format`` and return empty or non-JSON content
when it is requested. See https://console.groq.com/docs/structured-outputs.
Non-string models (the config allows a dict) are assumed to support JSON mode,
preserving prior behavior.
"""
if not isinstance(model, str):
return True
# Strip provider prefixes (e.g. "groq/compound-mini" -> "compound-mini"),
# mirroring the _is_reasoning_model heuristic in LLMBase, so the match
# targets the compound family rather than any name containing the substring.
base_model = model.lower().rsplit("/", 1)[-1]
return not base_model.startswith("compound")
def _parse_response(self, response, tools):
"""
Process the response based on whether tools are used or not.
@@ -100,17 +79,7 @@ class GroqLLM(LLMBase):
"top_p": self.config.top_p,
}
if response_format:
requests_json = isinstance(response_format, dict) and response_format.get("type") in (
"json_object",
"json_schema",
)
if requests_json and not self._supports_json_mode(self.config.model):
logger.debug(
f"Model '{self.config.model}' does not support JSON response_format; "
"sending the request without it."
)
else:
params["response_format"] = response_format
params["response_format"] = response_format
if tools:
params["tools"] = tools
params["tool_choice"] = tool_choice
+27 -97
View File
@@ -81,14 +81,6 @@ warnings.filterwarnings("ignore", category=DeprecationWarning, message=".*swigva
logger = logging.getLogger(__name__)
def _vector_store_list_rows(listed):
if isinstance(listed, (list, tuple)) and listed and isinstance(listed[0], list):
return listed[0]
if isinstance(listed, (list, tuple)):
return listed
return []
# Fields that hold runtime auth/connection objects and must be preserved.
# These are non-serializable objects (e.g. AWSV4SignerAuth, RequestsHttpConnection)
# needed by clients like OpenSearch — not sensitive strings to redact.
@@ -507,49 +499,22 @@ class Memory(MemoryBase):
)
return self._entity_store
@staticmethod
def _normalize_entity_text(value: str) -> str:
return " ".join(value.strip().lower().split())
def _existing_entities_by_text(self, filters):
"""Return existing entity rows keyed by normalized payload data."""
try:
listed = self.entity_store.list(filters=filters, top_k=10000)
except Exception as e:
logger.debug(f"Exact entity lookup failed, falling back to semantic dedup: {e}")
return {}
rows_by_text = {}
for row in _vector_store_list_rows(listed):
payload = getattr(row, "payload", None) or {}
text = payload.get("data")
if not isinstance(text, str):
continue
normalized = self._normalize_entity_text(text)
if normalized and normalized not in rows_by_text:
rows_by_text[normalized] = row
return rows_by_text
def _upsert_entity(self, entity_text, entity_type, memory_id, filters):
"""Upsert an entity into the entity store, linking it to a memory."""
try:
entity_embedding = self.embedding_model.embed(entity_text, "add")
search_filters = {k: v for k, v in filters.items() if k in ("user_id", "agent_id", "run_id") and v}
exact_match = self._existing_entities_by_text(search_filters).get(self._normalize_entity_text(entity_text))
existing = []
if exact_match is None:
existing = self.entity_store.search(
query=entity_text,
vectors=entity_embedding,
top_k=1,
filters=search_filters,
)
existing = self.entity_store.search(
query=entity_text,
vectors=entity_embedding,
top_k=1,
filters=search_filters,
)
semantic_match = existing[0] if existing and existing[0].score >= 0.95 else None
match = exact_match or semantic_match
if match:
if existing and existing[0].score >= 0.95:
# Update existing entity's linked_memory_ids
match = existing[0]
payload = match.payload or {}
linked_ids = payload.get("linked_memory_ids", [])
if memory_id not in linked_ids:
@@ -644,7 +609,7 @@ class Memory(MemoryBase):
return
seen = set()
for entity_type, entity_text in entities:
key = self._normalize_entity_text(entity_text)
key = entity_text.strip().lower()
if not key or key in seen:
continue
seen.add(key)
@@ -1006,7 +971,7 @@ class Memory(MemoryBase):
for idx, (memory_id, text, embedding, payload) in enumerate(records):
entities = all_entities[idx] if idx < len(all_entities) else []
for entity_type, entity_text in entities:
key = self._normalize_entity_text(entity_text)
key = entity_text.strip().lower()
if key in global_entities:
global_entities[key][2].add(memory_id)
else:
@@ -1044,7 +1009,6 @@ class Memory(MemoryBase):
if valid:
valid_indices, valid_keys = zip(*valid)
valid_vectors = [entity_embeddings[i] for i in valid_indices]
exact_matches = self._existing_entities_by_text(search_filters)
# 7c: Batch search for existing entities
valid_texts = [global_entities[k][1] for k in valid_keys]
@@ -1060,12 +1024,10 @@ class Memory(MemoryBase):
for j, key in enumerate(valid_keys):
entity_type, entity_text, memory_ids = global_entities[key]
matches = existing_matches[j] if j < len(existing_matches) else []
exact_match = exact_matches.get(key)
semantic_match = matches[0] if matches and matches[0].score >= 0.95 else None
match = exact_match or semantic_match
if match:
if matches and matches[0].score >= 0.95:
# Update existing entity
match = matches[0]
payload = match.payload or {}
linked = set(payload.get("linked_memory_ids", []))
linked |= memory_ids
@@ -1641,7 +1603,7 @@ class Memory(MemoryBase):
seen = set()
deduped = []
for entity_type, entity_text in query_entities[:8]:
key = self._normalize_entity_text(entity_text)
key = entity_text.strip().lower()
if key and key not in seen:
seen.add(key)
deduped.append((entity_type, entity_text))
@@ -2083,51 +2045,22 @@ class AsyncMemory(MemoryBase):
)
return self._entity_store
@staticmethod
def _normalize_entity_text(value: str) -> str:
return " ".join(value.strip().lower().split())
def _existing_entities_by_text(self, filters):
"""Return existing entity rows keyed by normalized payload data."""
try:
listed = self.entity_store.list(filters=filters, top_k=10000)
except Exception as e:
logger.debug(f"Exact entity lookup failed, falling back to semantic dedup: {e}")
return {}
rows_by_text = {}
for row in _vector_store_list_rows(listed):
payload = getattr(row, "payload", None) or {}
text = payload.get("data")
if not isinstance(text, str):
continue
normalized = self._normalize_entity_text(text)
if normalized and normalized not in rows_by_text:
rows_by_text[normalized] = row
return rows_by_text
async def _upsert_entity_async(self, entity_text, entity_type, memory_id, filters):
"""Async variant of `_upsert_entity` — per-entity search-then-update-or-insert."""
try:
entity_embedding = await asyncio.to_thread(self.embedding_model.embed, entity_text, "add")
search_filters = {k: v for k, v in filters.items() if k in ("user_id", "agent_id", "run_id") and v}
exact_match = (
await asyncio.to_thread(self._existing_entities_by_text, search_filters)
).get(self._normalize_entity_text(entity_text))
existing = []
if exact_match is None:
existing = await asyncio.to_thread(
self.entity_store.search,
query=entity_text,
vectors=entity_embedding,
top_k=1,
filters=search_filters,
)
existing = await asyncio.to_thread(
self.entity_store.search,
query=entity_text,
vectors=entity_embedding,
top_k=1,
filters=search_filters,
)
semantic_match = existing[0] if existing and existing[0].score >= 0.95 else None
match = exact_match or semantic_match
if match:
if existing and existing[0].score >= 0.95:
match = existing[0]
payload = match.payload or {}
linked_ids = payload.get("linked_memory_ids", [])
if memory_id not in linked_ids:
@@ -2230,7 +2163,7 @@ class AsyncMemory(MemoryBase):
return
seen = set()
for entity_type, entity_text in entities:
key = self._normalize_entity_text(entity_text)
key = entity_text.strip().lower()
if not key or key in seen:
continue
seen.add(key)
@@ -2579,7 +2512,7 @@ class AsyncMemory(MemoryBase):
for idx, (memory_id, text, embedding, payload) in enumerate(records):
entities = all_entities[idx] if idx < len(all_entities) else []
for entity_type, entity_text in entities:
key = self._normalize_entity_text(entity_text)
key = entity_text.strip().lower()
if key in global_entities:
global_entities[key][2].add(memory_id)
else:
@@ -2614,7 +2547,6 @@ class AsyncMemory(MemoryBase):
if valid:
valid_indices, valid_keys = zip(*valid)
valid_vectors = [entity_embeddings[i] for i in valid_indices]
exact_matches = await asyncio.to_thread(self._existing_entities_by_text, search_filters)
# 7c: Batch search for existing entities
valid_texts = [global_entities[k][1] for k in valid_keys]
@@ -2631,11 +2563,9 @@ class AsyncMemory(MemoryBase):
for j, key in enumerate(valid_keys):
entity_type, entity_text, memory_ids = global_entities[key]
matches = existing_matches[j] if j < len(existing_matches) else []
exact_match = exact_matches.get(key)
semantic_match = matches[0] if matches and matches[0].score >= 0.95 else None
match = exact_match or semantic_match
if match:
if matches and matches[0].score >= 0.95:
match = matches[0]
payload = match.payload or {}
linked = set(payload.get("linked_memory_ids", []))
linked |= memory_ids
@@ -3207,7 +3137,7 @@ class AsyncMemory(MemoryBase):
seen = set()
deduped = []
for entity_type, entity_text in query_entities[:8]:
key = self._normalize_entity_text(entity_text)
key = entity_text.strip().lower()
if key and key not in seen:
seen.add(key)
deduped.append((entity_type, entity_text))
+1 -4
View File
@@ -206,10 +206,7 @@ def parse_vision_messages(messages, llm=None, vision_details="auto"):
elif isinstance(content, dict) and content.get("type") == "image_url":
if llm is None:
continue
image_url_obj = content.get("image_url")
image_url = image_url_obj.get("url") if isinstance(image_url_obj, dict) else None
if not image_url:
raise ValueError("image_url content part is missing image_url.url")
image_url = content["image_url"]["url"]
try:
description = get_image_description(image_url, llm, vision_details)
returned_messages.append({"role": role, "content": description})
+1 -5
View File
@@ -1,4 +1,3 @@
import logging
import os
from typing import List, Dict, Any
@@ -10,8 +9,6 @@ try:
except ImportError:
COHERE_AVAILABLE = False
logger = logging.getLogger(__name__)
class CohereReranker(BaseReranker):
"""Cohere-based reranker implementation."""
@@ -81,9 +78,8 @@ class CohereReranker(BaseReranker):
return reranked_docs
except Exception as e:
except Exception:
# Fallback to original order if reranking fails
logger.warning("Cohere reranking failed, falling back to original order: %s", e)
for doc in documents:
doc['rerank_score'] = 0.0
final_top_k = top_k or self.config.top_k
+1 -5
View File
@@ -1,4 +1,3 @@
import logging
from typing import List, Dict, Any, Union
import numpy as np
@@ -13,8 +12,6 @@ try:
except ImportError:
TRANSFORMERS_AVAILABLE = False
logger = logging.getLogger(__name__)
class HuggingFaceReranker(BaseReranker):
"""HuggingFace Transformers based reranker implementation."""
@@ -142,9 +139,8 @@ class HuggingFaceReranker(BaseReranker):
return reranked_docs
except Exception as e:
except Exception:
# Fallback to original order if reranking fails
logger.warning("HuggingFace reranking failed, falling back to original order: %s", e)
for doc in documents:
doc['rerank_score'] = 0.0
final_top_k = top_k or self.config.top_k
+1 -5
View File
@@ -1,4 +1,3 @@
import logging
import re
from typing import Any, Dict, List, Union
@@ -7,8 +6,6 @@ from mem0.configs.rerankers.llm import LLMRerankerConfig
from mem0.reranker.base import BaseReranker
from mem0.utils.factory import LlmFactory
logger = logging.getLogger(__name__)
class LLMReranker(BaseReranker):
"""LLM-based reranker implementation."""
@@ -154,9 +151,8 @@ class LLMReranker(BaseReranker):
scored_doc['rerank_score'] = score
scored_docs.append(scored_doc)
except Exception as e:
except Exception:
# Fallback: assign neutral score if scoring fails
logger.warning("LLM reranking failed for a document, assigning neutral score: %s", e)
scored_doc = doc.copy()
scored_doc['rerank_score'] = 0.5
scored_docs.append(scored_doc)
@@ -1,4 +1,3 @@
import logging
from typing import List, Dict, Any, Union
import numpy as np
@@ -12,8 +11,6 @@ try:
except ImportError:
SENTENCE_TRANSFORMERS_AVAILABLE = False
logger = logging.getLogger(__name__)
class SentenceTransformerReranker(BaseReranker):
"""Sentence Transformer based reranker implementation."""
@@ -105,9 +102,8 @@ class SentenceTransformerReranker(BaseReranker):
return reranked_docs
except Exception as e:
except Exception:
# Fallback to original order if reranking fails
logger.warning("SentenceTransformer reranking failed, falling back to original order: %s", e)
for doc in documents:
doc['rerank_score'] = 0.0
final_top_k = top_k or self.config.top_k
+1 -5
View File
@@ -1,4 +1,3 @@
import logging
import os
from typing import List, Dict, Any
@@ -10,8 +9,6 @@ try:
except ImportError:
ZERO_ENTROPY_AVAILABLE = False
logger = logging.getLogger(__name__)
class ZeroEntropyReranker(BaseReranker):
"""Zero Entropy-based reranker implementation."""
@@ -92,9 +89,8 @@ class ZeroEntropyReranker(BaseReranker):
return reranked_docs
except Exception as e:
except Exception:
# Fallback to original order if reranking fails
logger.warning("Zero Entropy reranking failed, falling back to original order: %s", e)
for doc in documents:
doc['rerank_score'] = 0.0
final_top_k = top_k or self.config.top_k
+197 -606
View File
@@ -1,352 +1,82 @@
"""
Entity extraction from text using spaCy NLP.
Extracts three types of entities from a spaCy-processed document:
Extracts four types of entities from text:
- **Proper nouns**: Capitalized multi-word sequences (person names, places, brands)
- **Quoted text**: Text in single or double quotes (titles, specific terms)
- **Noun compounds**: Multi-word noun phrases with specific modifiers (e.g., "machine learning")
- **Noun fallback**: Single nouns from circumstantial compound patterns
Public API:
``extract_entities(text)`` accepts a string and owns spaCy model loading.
``extract_entities_batch(texts)`` uses ``nlp.pipe`` for batched extraction.
extract_entities(text: str) -> List[Tuple[str, str]]
Returns:
List of ``(entity_type, entity_text)`` tuples where entity_type is one of
PROPER, QUOTED, TOPIC, or IDENTIFIER. Returns ``[]`` if spaCy is unavailable.
Internal:
_extract_entities_from_doc(doc) -> List[Tuple[str, str]]
"""
from __future__ import annotations
from dataclasses import dataclass
import logging
import re
from typing import List, Tuple
@dataclass(frozen=True)
class _EntityCandidate:
entity_type: str
text: str
source: str
start: int
end: int
confidence: float
priority: int
logger = logging.getLogger(__name__)
# Words that are too generic to be useful as entity heads
_GENERIC_HEADS = {
"thing",
"stuff",
"way",
"time",
"experience",
"situation",
"case",
"fact",
"matter",
"issue",
"idea",
"thought",
"feeling",
"place",
"area",
"part",
"kind",
"type",
"sort",
"lot",
"bit",
"day",
"year",
"week",
"month",
"moment",
"instance",
"example",
"technique",
"method",
"approach",
"process",
"step",
"tool",
"result",
"outcome",
"goal",
"task",
"item",
"topic",
"scale",
"size",
"level",
"degree",
"amount",
"number",
"style",
"look",
"color",
"colour",
"shape",
"form",
"piece",
"section",
"side",
"end",
"edge",
"surface",
"point",
}
# Entity labels emitted by spaCy that are usually safe to treat as named
# entities. Numeric and temporal labels are intentionally excluded.
_ACCEPTED_NER_LABELS = {
"PERSON",
"ORG",
"GPE",
"LOC",
"FAC",
"PRODUCT",
"WORK_OF_ART",
"EVENT",
"NORP",
"LAW",
"LANGUAGE",
}
_REJECTED_NER_LABELS = {
"DATE",
"TIME",
"CARDINAL",
"ORDINAL",
"QUANTITY",
"MONEY",
"PERCENT",
}
# Generic role words and title-cased English words that should not become
# single-token named entities just because spaCy tagged them as PROPN.
_GENERIC_SINGLE_ENTITY_TERMS = {
"user",
"assistant",
"agent",
"customer",
"client",
"person",
"people",
"human",
"memory",
"message",
"conversation",
"chat",
"session",
"system",
"top",
"thing", "stuff", "way", "time", "experience", "situation", "case",
"fact", "matter", "issue", "idea", "thought", "feeling", "place",
"area", "part", "kind", "type", "sort", "lot", "bit", "day", "year",
"week", "month", "moment", "instance", "example", "technique",
"method", "approach", "process", "step", "tool", "result", "outcome",
"goal", "task", "item", "topic", "scale", "size", "level", "degree",
"amount", "number", "style", "look", "color", "colour", "shape",
"form", "piece", "section", "side", "end", "edge", "surface", "point",
}
# Modifiers that describe circumstance, not content
_CIRCUMSTANTIAL_MODS = {
"solo",
"individual",
"team",
"group",
"joint",
"collaborative",
"first",
"last",
"next",
"previous",
"final",
"initial",
"main",
"side",
"top",
"solo", "individual", "team", "group", "joint", "collaborative",
"first", "last", "next", "previous", "final", "initial", "main", "side",
}
# Adjectives too vague to make a compound entity specific
_NON_SPECIFIC_ADJ = {
"many",
"few",
"several",
"some",
"any",
"all",
"most",
"more",
"less",
"much",
"little",
"enough",
"various",
"numerous",
"multiple",
"countless",
"great",
"good",
"bad",
"nice",
"terrible",
"awful",
"awesome",
"amazing",
"wonderful",
"horrible",
"excellent",
"poor",
"best",
"worst",
"fine",
"okay",
"new",
"old",
"recent",
"past",
"future",
"current",
"previous",
"next",
"last",
"first",
"latest",
"early",
"late",
"former",
"modern",
"ancient",
"big",
"small",
"large",
"tiny",
"huge",
"enormous",
"long",
"short",
"tall",
"high",
"low",
"wide",
"narrow",
"thick",
"thin",
"deep",
"shallow",
"similar",
"different",
"same",
"other",
"another",
"such",
"certain",
"important",
"main",
"major",
"minor",
"key",
"primary",
"real",
"actual",
"true",
"whole",
"entire",
"full",
"complete",
"total",
"basic",
"simple",
"interesting",
"boring",
"exciting",
"special",
"particular",
"general",
"common",
"unique",
"rare",
"typical",
"usual",
"normal",
"regular",
"possible",
"likely",
"potential",
"available",
"necessary",
"only",
"solo",
"individual",
"team",
"group",
"joint",
"collaborative",
"final",
"initial",
"side",
"many", "few", "several", "some", "any", "all", "most", "more",
"less", "much", "little", "enough", "various", "numerous", "multiple",
"countless", "great", "good", "bad", "nice", "terrible", "awful",
"awesome", "amazing", "wonderful", "horrible", "excellent", "poor",
"best", "worst", "fine", "okay", "new", "old", "recent", "past",
"future", "current", "previous", "next", "last", "first", "latest",
"early", "late", "former", "modern", "ancient", "big", "small",
"large", "tiny", "huge", "enormous", "long", "short", "tall", "high",
"low", "wide", "narrow", "thick", "thin", "deep", "shallow",
"similar", "different", "same", "other", "another", "such", "certain",
"important", "main", "major", "minor", "key", "primary", "real",
"actual", "true", "whole", "entire", "full", "complete", "total",
"basic", "simple", "interesting", "boring", "exciting", "special",
"particular", "general", "common", "unique", "rare", "typical",
"usual", "normal", "regular", "possible", "likely", "potential",
"available", "necessary", "only", "solo", "individual", "team",
"group", "joint", "collaborative", "final", "initial", "side",
}
# Generic tail words to strip from compound entities
_GENERIC_ENDINGS = {
"work",
"works",
"job",
"jobs",
"task",
"tasks",
"stuff",
"things",
"thing",
"info",
"information",
"details",
"data",
"content",
"material",
"materials",
"activities",
"activity",
"efforts",
"effort",
"options",
"option",
"choices",
"choice",
"results",
"result",
"output",
"outputs",
"products",
"product",
"items",
"item",
"work", "works", "job", "jobs", "task", "tasks", "stuff", "things",
"thing", "info", "information", "details", "data", "content",
"material", "materials", "activities", "activity", "efforts", "effort",
"options", "option", "choices", "choice", "results", "result",
"output", "outputs", "products", "product", "items", "item",
}
# Capitalized single words that are too generic to be proper nouns
_GENERIC_CAPS = {
"works",
"items",
"things",
"stuff",
"resources",
"options",
"tips",
"ideas",
"steps",
"ways",
"methods",
"tools",
"features",
"benefits",
"examples",
"details",
"notes",
"instructions",
"guidelines",
"recommendations",
"suggestions",
"overview",
"summary",
"conclusion",
"introduction",
"pros",
"cons",
"advantages",
"disadvantages",
"works", "items", "things", "stuff", "resources", "options", "tips",
"ideas", "steps", "ways", "methods", "tools", "features", "benefits",
"examples", "details", "notes", "instructions", "guidelines",
"recommendations", "suggestions", "overview", "summary", "conclusion",
"introduction", "pros", "cons", "advantages", "disadvantages",
}
# Markdown/formatting markers to skip during extraction
@@ -390,208 +120,121 @@ def _has_artifacts(txt: str) -> bool:
)
def _clean_text(txt: str) -> str:
txt = re.sub(r"^\*+\s*|\s*\*+$", "", txt.strip())
txt = re.sub(r"\s*:+$", "", txt)
txt = re.sub(r"^\d+\s*\.\s*", "", txt)
return " ".join(txt.split())
def extract_entities(text: str) -> List[Tuple[str, str]]:
"""Extract named entities, quoted text, and noun compounds from text.
This is the public API that accepts a string. It loads the spaCy model
internally and delegates to _extract_entities_from_doc().
Args:
text: Input text to extract entities from.
Returns:
Deduplicated list of (entity_type, entity_text) tuples.
Entity types: PROPER, QUOTED, COMPOUND, NOUN.
Returns empty list if spaCy is unavailable.
"""
from mem0.utils.spacy_models import get_nlp_full
nlp = get_nlp_full()
if nlp is None:
return []
doc = nlp(text)
return _extract_entities_from_doc(doc)
def _norm_text(txt: str) -> str:
return " ".join(txt.lower().split())
def extract_entities_batch(texts: List[str], batch_size: int = 32) -> List[List[Tuple[str, str]]]:
"""Extract entities from multiple texts using spaCy's nlp.pipe() for batched NER.
Uses spaCy's efficient batch processing pipeline instead of calling
nlp() individually per text. Significantly faster for multiple texts.
Args:
texts: List of input texts to extract entities from.
batch_size: Number of texts to process in each spaCy batch.
Returns:
List of entity lists, one per input text. Each entity list contains
(entity_type, entity_text) tuples. Returns list of empty lists if
spaCy is unavailable.
"""
if not texts:
return []
from mem0.utils.spacy_models import get_nlp_full
nlp = get_nlp_full()
if nlp is None:
return [[] for _ in texts]
results = []
for doc in nlp.pipe(texts, batch_size=batch_size):
results.append(_extract_entities_from_doc(doc))
return results
def _looks_like_technical_identifier(text: str) -> bool:
return bool(re.fullmatch(r"[A-Za-z_][\w-]*(?:\.[A-Za-z_][\w-]*)+", text))
def _extract_entities_from_doc(doc) -> List[Tuple[str, str]]:
"""Extract entities from a spaCy Doc object.
def _has_internal_cap_or_digit(text: str) -> bool:
return any(ch.isdigit() for ch in text) or any(ch.isupper() for ch in text[1:])
def _looks_like_metric_count_token(tok) -> bool:
return tok.pos_ == "NUM" and bool(re.fullmatch(r"\d[\d,]*(?:\.\d+)?", tok.text))
def _is_metric_list_context(tokens: list, idx: int) -> bool:
prev_text = tokens[idx - 1].text if idx > 0 else ""
next_text = tokens[idx + 1].text if idx + 1 < len(tokens) else ""
return prev_text in {":", ",", ";"} or next_text in {",", ";"}
def _strip_trailing_metric_counts(span_tokens: list, all_tokens: list) -> list:
while len(span_tokens) > 1 and _looks_like_metric_count_token(span_tokens[-1]):
tok = span_tokens[-1]
if "," not in tok.text and not _is_metric_list_context(all_tokens, tok.i):
break
span_tokens = span_tokens[:-1]
return span_tokens
def _is_list_item_name_token(tokens: list, idx: int) -> bool:
tok = tokens[idx]
if not tok.text or tok.text in _FORMATTING_MARKERS or not tok.text[0].isupper():
return False
if not any(ch.isalpha() for ch in tok.text) or _is_bad_single_name_token(tok):
return False
next_tok = tokens[idx + 1] if idx + 1 < len(tokens) else None
if not next_tok or not _looks_like_metric_count_token(next_tok):
return False
return _is_metric_list_context(tokens, idx) or _is_metric_list_context(tokens, idx + 1)
def _is_name_like_token(tok, tokens: list | None = None, idx: int | None = None) -> bool:
if not tok.text or tok.text in _FORMATTING_MARKERS:
return False
if not tok.text[0].isupper():
return False
if not any(ch.isalpha() for ch in tok.text):
return False
if _is_bad_single_name_token(tok):
return False
if tok.pos_ == "PROPN" or tok.tag_ in {"NNP", "NNPS"}:
return True
if tokens is not None and idx is not None and _is_list_item_name_token(tokens, idx):
return True
if _has_internal_cap_or_digit(tok.text):
return True
return (
tokens is not None
and idx is not None
and tok.pos_ == "NOUN"
and tok.dep_ not in {"compound", "amod"}
and not _is_sentence_start(tokens, idx)
)
def _is_bad_single_name_token(tok) -> bool:
lower = tok.text.lower()
return lower in _GENERIC_SINGLE_ENTITY_TERMS or lower in _GENERIC_CAPS or tok.is_stop
def _add_candidate(
candidates: list[_EntityCandidate],
entity_type: str,
text: str,
source: str,
start: int,
end: int,
confidence: float,
priority: int,
) -> None:
cleaned = _clean_text(text)
if not cleaned or len(cleaned) <= 2 or _has_artifacts(cleaned):
return
candidates.append(
_EntityCandidate(
entity_type=entity_type,
text=cleaned,
source=source,
start=start,
end=end,
confidence=confidence,
priority=priority,
)
)
def _add_ner_candidates(doc, candidates: list[_EntityCandidate]) -> None:
Ported from platform's shared.core.utils.entity_extraction.extract_entities().
"""
entities: List[Tuple[str, str]] = []
text = doc.text
tokens = list(doc)
for ent in doc.ents:
if ent.label_ in _REJECTED_NER_LABELS or ent.label_ not in _ACCEPTED_NER_LABELS:
continue
ent_tokens = _strip_trailing_metric_counts(list(ent), tokens)
if not ent_tokens:
continue
if any(tok.pos_ == "CCONJ" and tok.text.lower() == "and" for tok in ent_tokens):
continue
if len(ent_tokens) == 1 and _is_bad_single_name_token(ent_tokens[0]):
continue
if (
len(ent_tokens) == 1
and ent_tokens[0].dep_ in {"compound", "amod"}
and ent_tokens[0].head.pos_ in {"NOUN", "PROPN"}
):
continue
_add_candidate(
candidates,
"PROPER",
"".join(tok.text_with_ws for tok in ent_tokens).strip(),
"spacy_ner",
ent_tokens[0].i,
ent_tokens[-1].i + 1,
0.95,
0,
)
def _add_technical_identifier_candidates(tokens: list, candidates: list[_EntityCandidate]) -> None:
for tok in tokens:
if _looks_like_technical_identifier(tok.text):
_add_candidate(
candidates,
"IDENTIFIER",
tok.text,
"technical_identifier",
tok.i,
tok.i + 1,
0.9,
1,
)
def _add_proper_name_candidates(tokens: list, candidates: list[_EntityCandidate]) -> None:
allowed_inner_connectors = {"of", "the", "for", "at", "in"}
# === PROPER NOUN SEQUENCES ===
i = 0
while i < len(tokens):
tok = tokens[i]
if not _is_name_like_token(tok, tokens, i):
if tok.text in _FORMATTING_MARKERS:
i += 1
continue
is_cap = tok.text and tok.text[0].isupper()
is_label = i + 1 < len(tokens) and tokens[i + 1].text == ":"
span_tokens = [tok]
j = i + 1
while j < len(tokens):
current = tokens[j]
if _is_name_like_token(current, tokens, j):
span_tokens.append(current)
j += 1
continue
if (
current.text.lower() in allowed_inner_connectors
and j + 1 < len(tokens)
and _is_name_like_token(tokens[j + 1], tokens, j + 1)
):
span_tokens.extend([current, tokens[j + 1]])
j += 2
continue
break
if is_cap and not is_label and tok.pos_ in {"PROPN", "NOUN", "ADJ"}:
seq = [(tok, i)]
j = i + 1
while j < len(tokens):
t = tokens[j]
if (t.text and t.text[0].isupper()) or t.text.lower() in {
"'s", "of", "the", "in", "and", "for", "at", "is",
}:
seq.append((t, j))
j += 1
else:
break
# Strip trailing function words
while seq and seq[-1][0].text.lower() in {"of", "the", "in", "and", "for", "at", "is", "'s"}:
seq.pop()
if seq:
has_mid_cap = any(
not _is_sentence_start(tokens, idx)
for (t, idx) in seq
if t.text[0].isupper() and t.text.lower() not in {"'s", "of", "the", "in", "and", "for", "at", "is"}
)
if has_mid_cap:
phrase = "".join(t.text_with_ws for (t, idx) in seq).strip()
if len(phrase) > 2:
entities.append(("PROPER", phrase))
i = j
else:
i += 1
name_tokens = [
t
for t in span_tokens
if _is_name_like_token(t, tokens, t.i) or (0 <= t.i < len(tokens) and _is_list_item_name_token(tokens, t.i))
]
if len(name_tokens) > 1 or not _is_bad_single_name_token(name_tokens[0]):
text = "".join(t.text_with_ws for t in span_tokens).strip()
_add_candidate(candidates, "PROPER", text, "proper_name_span", i, j, 0.8, 2)
i = max(j, i + 1)
def _add_quoted_candidates(text: str, candidates: list[_EntityCandidate]) -> None:
# === QUOTED TEXT ===
for m in re.finditer(r'"([^"]+)"', text):
if len(m.group(1).strip()) > 2:
_add_candidate(candidates, "QUOTED", m.group(1).strip(), "quoted", -1, -1, 0.75, 3)
entities.append(("QUOTED", m.group(1).strip()))
for m in re.finditer(r"(?:^|[\s\(\[{,;])'([^']+)'(?=[\s\.,;:!?\)\]]|$)", text):
if len(m.group(1).strip()) > 2:
_add_candidate(candidates, "QUOTED", m.group(1).strip(), "quoted", -1, -1, 0.75, 3)
entities.append(("QUOTED", m.group(1).strip()))
def _add_topic_phrase_candidates(doc, candidates: list[_EntityCandidate]) -> None:
# === NOUN-NOUN COMPOUNDS ===
for chunk in doc.noun_chunks:
chunk_tokens = list(chunk)
split_indices: list[int] = []
poss_splits: list[int] = []
split_indices: list = []
poss_splits: list = []
for idx, tok in enumerate(chunk_tokens):
if tok.dep_ == "case" and tok.text in {"'s", "\u2019s", "'"}:
split_indices.append(idx)
@@ -600,14 +243,14 @@ def _add_topic_phrase_candidates(doc, candidates: list[_EntityCandidate]) -> Non
split_indices.append(idx)
if split_indices:
groups: list[list] = []
groups: list = []
prev = 0
for split_idx in split_indices:
if split_idx > prev:
groups.append(chunk_tokens[prev:split_idx])
if split_idx in poss_splits:
next_split = next((s for s in split_indices if s > split_idx), None)
owned = chunk_tokens[split_idx + 1 : next_split if next_split else len(chunk_tokens)]
owned = chunk_tokens[split_idx + 1: next_split if next_split else len(chunk_tokens)]
if owned:
first_content = next((t for t in owned if t.pos_ not in {"PUNCT", "PART"}), None)
if not (first_content and first_content.text and first_content.text[0].isupper()):
@@ -629,8 +272,7 @@ def _add_topic_phrase_candidates(doc, candidates: list[_EntityCandidate]) -> Non
content = [
t
for t in group
if t.pos_ not in {"DET", "PRON", "PUNCT", "PART", "ADP", "SCONJ", "NUM"}
and (t.pos_ == "ADJ" or not t.is_stop)
if t.pos_ not in {"DET", "PRON", "PUNCT", "PART", "ADP", "SCONJ", "NUM"} and (t.pos_ == "ADJ" or not t.is_stop)
]
if not content:
continue
@@ -644,129 +286,78 @@ def _add_topic_phrase_candidates(doc, candidates: list[_EntityCandidate]) -> Non
if compound_toks:
is_circ = any(t.lemma_.lower() in _CIRCUMSTANTIAL_MODS for t in compound_toks)
if is_circ:
val = head.text
val = head.lemma_ if head.pos_ == "NOUN" else head.text
if len(val) > 2:
_add_candidate(
candidates,
"TOPIC",
val,
"topic_phrase",
head.i,
head.i + 1,
0.45,
4,
)
entities.append(("NOUN", val))
else:
filtered = _strip_generic_ending(
[t for t in content if not (t.pos_ == "ADJ" and t.lemma_.lower() in _NON_SPECIFIC_ADJ)]
)
if filtered:
phrase = " ".join(t.text for t in filtered)
phrase = _lemmatize_compound(filtered)
if len(phrase) > 3 and " " in phrase:
_add_candidate(
candidates,
"TOPIC",
phrase,
"topic_phrase",
filtered[0].i,
filtered[-1].i + 1,
0.45,
4,
)
entities.append(("COMPOUND", phrase))
elif len(content) > 1 and has_spec_adj:
filtered = _strip_generic_ending(
[
t
for t in content
if not ((t.pos_ == "ADJ" or t.dep_ == "amod") and t.lemma_.lower() in _NON_SPECIFIC_ADJ)
]
[t for t in content if not ((t.pos_ == "ADJ" or t.dep_ == "amod") and t.lemma_.lower() in _NON_SPECIFIC_ADJ)]
)
if filtered:
phrase = " ".join(t.text for t in filtered)
phrase = _lemmatize_compound(filtered)
if len(phrase) > 3 and " " in phrase:
_add_candidate(
candidates,
"TOPIC",
phrase,
"topic_phrase",
filtered[0].i,
filtered[-1].i + 1,
0.45,
4,
)
entities.append(("COMPOUND", phrase))
# === FALLBACK: Mis-tagged VERB heads ===
processed = {e[1].lower() for e in entities if e[0] == "COMPOUND"}
generic_verb_heads = _GENERIC_HEADS | {"find", "buy", "purchase", "sale", "deal", "trip", "visit"}
def _spans_overlap(a: _EntityCandidate, b: _EntityCandidate) -> bool:
if a.start < 0 or b.start < 0:
return False
return a.start < b.end and b.start < a.end
def collect_compounds(head):
return [t for t in doc if t.head == head and t.dep_ == "compound"]
for tok in doc:
if tok.pos_ == "VERB" and tok.dep_ in {"pobj", "dobj", "nsubj"}:
comps = sorted(collect_compounds(tok), key=lambda t: t.i)
if comps:
phrase_toks = comps if tok.lemma_.lower() in generic_verb_heads else comps + [tok]
phrase = " ".join(t.text for t in phrase_toks)
if phrase.lower() not in processed and len(phrase) > 3 and " " in phrase:
entities.append(("COMPOUND", phrase))
processed.add(phrase.lower())
def _resolve_candidates(candidates: list[_EntityCandidate]) -> list[tuple[str, str]]:
deduped_by_text: dict[str, _EntityCandidate] = {}
for candidate in candidates:
key = _norm_text(candidate.text)
current = deduped_by_text.get(key)
if current is None or (candidate.priority, -candidate.confidence) < (current.priority, -current.confidence):
deduped_by_text[key] = candidate
# === DEDUPLICATION & CLEANUP ===
seen: set = set()
deduped = []
for t, e in entities:
k = e.lower().strip()
if k not in seen and len(k) > 2:
seen.add(k)
deduped.append((t, e))
ordered = sorted(
deduped_by_text.values(),
key=lambda c: (c.priority, -c.confidence, -(c.end - c.start), c.start),
)
accepted: list[_EntityCandidate] = []
for candidate in ordered:
if any(
_spans_overlap(candidate, existing)
and not (candidate.entity_type == "TOPIC" and " " in candidate.text and existing.entity_type == "PROPER")
for existing in accepted
):
cleaned: List[Tuple[str, str]] = []
for etype, etext in deduped:
txt = re.sub(r"^\*+\s*|\s*\*+$", "", etext.strip())
txt = re.sub(r"\s*:+$", "", txt)
txt = re.sub(r"^\d+\s*\.\s*", "", txt)
if not txt or len(txt) <= 2 or _has_artifacts(txt):
continue
accepted.append(candidate)
if etype == "PROPER" and " " not in txt and txt.lower() in _GENERIC_CAPS:
continue
cleaned.append((etype, txt))
accepted.sort(key=lambda c: (c.start if c.start >= 0 else 10**9, c.end, c.priority))
return [(candidate.entity_type, candidate.text) for candidate in accepted]
# Keep best type per entity (PROPER > COMPOUND > QUOTED > NOUN)
type_pri = {"PROPER": 0, "COMPOUND": 1, "QUOTED": 2, "NOUN": 3, "VERB": 4}
best: dict = {}
for t, e in cleaned:
k = e.lower()
if k not in best or type_pri.get(t, 99) < type_pri.get(best[k][0], 99):
best[k] = (t, e)
deduped = list(best.values())
def _extract_entities_from_doc(doc) -> list[tuple[str, str]]:
"""Extract typed entity candidates from a spaCy Doc.
Args:
doc: A spaCy ``Doc`` object (from ``nlp(text)``).
Returns:
Deduplicated list of ``(entity_type, entity_text)`` tuples.
Entity types include PROPER, QUOTED, TOPIC, and IDENTIFIER.
"""
tokens = list(doc)
candidates: list[_EntityCandidate] = []
_add_ner_candidates(doc, candidates)
_add_technical_identifier_candidates(tokens, candidates)
_add_proper_name_candidates(tokens, candidates)
_add_quoted_candidates(doc.text, candidates)
_add_topic_phrase_candidates(doc, candidates)
return _resolve_candidates(candidates)
def extract_entities(text: str) -> list[tuple[str, str]]:
"""Extract typed entity candidates from text."""
from mem0.utils.spacy_models import get_nlp_full
nlp = get_nlp_full()
if nlp is None:
return []
return _extract_entities_from_doc(nlp(text))
def extract_entities_batch(texts: list[str], batch_size: int = 32) -> list[list[tuple[str, str]]]:
"""Extract typed entity candidates from multiple texts."""
if not texts:
return []
from mem0.utils.spacy_models import get_nlp_full
nlp = get_nlp_full()
if nlp is None:
return [[] for _ in texts]
return [_extract_entities_from_doc(doc) for doc in nlp.pipe(texts, batch_size=batch_size)]
# Remove entities that are whole-word substrings of longer entities.
# Word-boundary anchoring avoids dropping distinct entities that only share a
# leading substring (e.g. "Sam" must survive alongside "Samsung").
all_lower = [e[1].lower() for e in deduped]
return [
(t, e)
for t, e in deduped
if not any(e.lower() != o and re.search(rf"\b{re.escape(e.lower())}\b", o) for o in all_lower)
]
+2 -2
View File
@@ -298,9 +298,9 @@ class AzureAISearch(VectorStoreBase):
payload (Dict, optional): Updated payload.
"""
document = {"id": vector_id}
if vector is not None:
if vector:
document["vector"] = vector
if payload is not None:
if payload:
json_payload = json.dumps(payload)
document["payload"] = json_payload
for field in ["user_id", "run_id", "agent_id"]:
+2 -6
View File
@@ -169,7 +169,7 @@ class ChromaDB(VectorStoreBase):
Args:
vector_id (str): ID of the vector to delete.
"""
self.collection.delete(ids=[vector_id])
self.collection.delete(ids=vector_id)
def update(
self,
@@ -185,11 +185,7 @@ class ChromaDB(VectorStoreBase):
vector (Optional[List[float]], optional): Updated vector. Defaults to None.
payload (Optional[Dict], optional): Updated payload. Defaults to None.
"""
self.collection.update(
ids=[vector_id],
embeddings=[vector] if vector is not None else None,
metadatas=[payload] if payload is not None else None,
)
self.collection.update(ids=vector_id, embeddings=vector, metadatas=payload)
def get(self, vector_id: str) -> Optional[OutputData]:
"""
+2 -10
View File
@@ -39,8 +39,6 @@ class OpenSearchDB(VectorStoreBase):
self.collection_name = config.collection_name
self.embedding_model_dims = config.embedding_model_dims
self.auto_refresh = config.auto_refresh
self.create_col(self.collection_name, self.embedding_model_dims)
def create_index(self) -> None:
@@ -150,6 +148,8 @@ class OpenSearchDB(VectorStoreBase):
}
try:
self.client.index(index=self.collection_name, body=body)
# Force refresh to make documents immediately searchable for tests
self.client.indices.refresh(index=self.collection_name)
results.append(
OutputData(
@@ -162,14 +162,6 @@ class OpenSearchDB(VectorStoreBase):
logger.error(f"Error inserting vector {id_}: {e}", exc_info=True)
raise
# Refresh once after the full batch (not per document) if explicitly enabled.
# Disabled by default for Serverless compatibility: OpenSearch Serverless does not
# support the indices.refresh() API, and refreshing per document would cause a
# cluster-level I/O stall on every insert.
# See: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-genref.html
if self.auto_refresh:
self.client.indices.refresh(index=self.collection_name)
return results
def search(
+1 -4
View File
@@ -98,10 +98,7 @@ class Qdrant(VectorStoreBase):
self._bm25_encoder = SparseTextEmbedding(model_name="Qdrant/bm25")
logger.info("BM25 encoder loaded (fastembed Qdrant/bm25)")
except ImportError:
logger.warning(
"fastembed not installed - BM25 keyword search disabled. "
'Install it with: pip install "mem0ai[extras]"'
)
logger.warning("fastembed not installed — BM25 keyword search disabled. Install with: pip install fastembed")
self._bm25_encoder = False # sentinel: tried and failed
except Exception as e:
logger.warning(f"Failed to load BM25 encoder: {e}")
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "mem0ai"
version = "2.0.9"
version = "2.0.7"
description = "Long-term memory for AI Agents"
authors = [
{ name = "Mem0", email = "support@mem0.ai" }
@@ -27,8 +27,6 @@ import { useApiQuery } from "@/hooks/use-api-query";
import { Memory } from "@/types/api";
const PAGE_SIZE = 20;
// Keep in sync with ALL_MEMORIES_LIMIT in server/main.py.
const MEMORY_FETCH_LIMIT = 1000;
export default function MemoriesPage() {
const [userId, setUserId] = useState("");
@@ -43,9 +41,7 @@ export default function MemoriesPage() {
refetch,
} = useApiQuery<Memory[]>(
async () => {
const params = userId.trim()
? { user_id: userId.trim(), top_k: MEMORY_FETCH_LIMIT }
: { top_k: MEMORY_FETCH_LIMIT };
const params = userId.trim() ? { user_id: userId.trim() } : undefined;
const res = await api.get(MEMORY_ENDPOINTS.BASE, { params });
const raw = res.data?.results ?? res.data ?? [];
return Array.isArray(raw) ? raw : [];
@@ -100,7 +96,7 @@ export default function MemoriesPage() {
<div className="space-y-4">
<h1 className="text-xl font-semibold font-fustat">Memories</h1>
{memories.length >= MEMORY_FETCH_LIMIT && (
{memories.length >= 1000 && (
<UpgradeBanner
id="memories-1k"
message="1,000+ memories stored. Categories can help organize them."
+3 -7
View File
@@ -16,7 +16,7 @@ from errors import (
upstream_error,
upstream_error_handler,
)
from fastapi import Depends, FastAPI, HTTPException, Query, Request
from fastapi import Depends, FastAPI, HTTPException, Request
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse, RedirectResponse
from mem0.exceptions import ValidationError as Mem0ValidationError
@@ -409,7 +409,6 @@ def get_all_memories(
user_id: Optional[str] = None,
run_id: Optional[str] = None,
agent_id: Optional[str] = None,
top_k: Optional[int] = Query(None, ge=0, le=ALL_MEMORIES_LIMIT),
_auth=Depends(verify_auth),
):
"""Retrieve stored memories. Lists all memories when no identifier is provided (admin only)."""
@@ -418,14 +417,11 @@ def get_all_memories(
auth_type = getattr(request.state, "auth_type", "none")
if _auth is not None and _auth.role != "admin" and auth_type not in {"admin_api_key", "disabled"}:
raise HTTPException(status_code=403, detail="Admin role required to list all memories.")
return _list_all_memories(limit=top_k if top_k is not None else ALL_MEMORIES_LIMIT)
return _list_all_memories()
filters = {
k: v for k, v in {"user_id": user_id, "run_id": run_id, "agent_id": agent_id}.items() if v is not None
}
params = {"filters": filters}
if top_k is not None:
params["top_k"] = top_k
return get_memory_instance().get_all(**params)
return get_memory_instance().get_all(filters=filters)
except HTTPException:
raise
except Exception:
-98
View File
@@ -84,101 +84,3 @@ def test_generate_response_with_tools(mock_groq_client):
assert len(response["tool_calls"]) == 1
assert response["tool_calls"][0]["name"] == "add_memory"
assert response["tool_calls"][0]["arguments"] == {"data": "Today is a sunny day."}
@pytest.mark.parametrize("model", ["groq/compound", "groq/compound-mini"])
def test_generate_response_skips_json_mode_for_compound_models(mock_groq_client, model):
config = BaseLlmConfig(model=model, temperature=0.7, max_tokens=100, top_p=1.0)
llm = GroqLLM(config)
messages = [{"role": "user", "content": "Hi, I'm Alice and I love hiking."}]
# Compound models answer JSON-mode requests with empty or non-JSON content;
# the mock mirrors that plain-text reply. These tests pin request
# construction (response_format omitted), not end-to-end extraction.
mock_response = Mock()
mock_response.choices = [Mock(message=Mock(content="Alice introduced herself and mentioned she loves hiking."))]
mock_groq_client.chat.completions.create.return_value = mock_response
llm.generate_response(messages, response_format={"type": "json_object"})
_, kwargs = mock_groq_client.chat.completions.create.call_args
assert "response_format" not in kwargs
def test_generate_response_keeps_json_mode_for_standard_model(mock_groq_client):
config = BaseLlmConfig(model="llama-3.3-70b-versatile", temperature=0.7, max_tokens=100, top_p=1.0)
llm = GroqLLM(config)
messages = [{"role": "user", "content": "Hi, I'm Alice and I love hiking."}]
mock_response = Mock()
mock_response.choices = [Mock(message=Mock(content='{"memory": ["Name is Alice", "Loves hiking"]}'))]
mock_groq_client.chat.completions.create.return_value = mock_response
llm.generate_response(messages, response_format={"type": "json_object"})
_, kwargs = mock_groq_client.chat.completions.create.call_args
assert kwargs["response_format"] == {"type": "json_object"}
def test_generate_response_keeps_non_json_response_format_for_compound_model(mock_groq_client):
config = BaseLlmConfig(model="groq/compound", temperature=0.7, max_tokens=100, top_p=1.0)
llm = GroqLLM(config)
messages = [{"role": "user", "content": "Hi, I'm Alice and I love hiking."}]
mock_response = Mock()
mock_response.choices = [Mock(message=Mock(content="Alice loves hiking."))]
mock_groq_client.chat.completions.create.return_value = mock_response
llm.generate_response(messages, response_format={"type": "text"})
_, kwargs = mock_groq_client.chat.completions.create.call_args
assert kwargs["response_format"] == {"type": "text"}
def test_generate_response_keeps_tools_when_skipping_json_mode(mock_groq_client):
config = BaseLlmConfig(model="groq/compound", temperature=0.7, max_tokens=100, top_p=1.0)
llm = GroqLLM(config)
messages = [{"role": "user", "content": "Add a new memory: Today is a sunny day."}]
tools = [
{
"type": "function",
"function": {
"name": "add_memory",
"description": "Add a memory",
"parameters": {
"type": "object",
"properties": {"data": {"type": "string", "description": "Data to add to memory"}},
"required": ["data"],
},
},
}
]
mock_response = Mock()
mock_message = Mock()
mock_message.content = "Done."
mock_message.tool_calls = None
mock_response.choices = [Mock(message=mock_message)]
mock_groq_client.chat.completions.create.return_value = mock_response
llm.generate_response(messages, response_format={"type": "json_object"}, tools=tools)
_, kwargs = mock_groq_client.chat.completions.create.call_args
assert "response_format" not in kwargs
assert kwargs["tools"] == tools
assert kwargs["tool_choice"] == "auto"
def test_generate_response_handles_non_string_model(mock_groq_client):
config = BaseLlmConfig(model={"name": "custom-model"}, temperature=0.7, max_tokens=100, top_p=1.0)
llm = GroqLLM(config)
messages = [{"role": "user", "content": "Hi, I'm Alice and I love hiking."}]
mock_response = Mock()
mock_response.choices = [Mock(message=Mock(content='{"memory": []}'))]
mock_groq_client.chat.completions.create.return_value = mock_response
llm.generate_response(messages, response_format={"type": "json_object"})
_, kwargs = mock_groq_client.chat.completions.create.call_args
assert kwargs["response_format"] == {"type": "json_object"}
-18
View File
@@ -96,24 +96,6 @@ class TestParseVisionMessages:
result = parse_vision_messages(messages, llm=None)
assert result == messages
def test_malformed_image_dict_raises_value_error(self):
# A malformed image part (missing the nested url) used to raise an
# uncaught KeyError that aborted add(); it should raise a clear ValueError.
mock_llm = Mock()
messages = [{"role": "user", "content": {"type": "image_url", "image_url": {}}}]
with pytest.raises(ValueError, match=r"missing image_url\.url"):
parse_vision_messages(messages, llm=mock_llm)
mock_llm.generate_response.assert_not_called()
def test_none_image_url_raises_value_error(self):
# image_url present but None (or any non-dict) must also raise the clear
# ValueError, not an AttributeError from calling .get() on None.
mock_llm = Mock()
messages = [{"role": "user", "content": {"type": "image_url", "image_url": None}}]
with pytest.raises(ValueError, match=r"missing image_url\.url"):
parse_vision_messages(messages, llm=mock_llm)
mock_llm.generate_response.assert_not_called()
class TestRemoveSpacesFromEntities:
"""
@@ -1,32 +0,0 @@
"""Reranker failures must be logged, not silently swallowed.
Uses the LLMReranker because it is constructible without heavy ML deps (the
``mock_llm`` fixture stubs the LLM factory). The fix under test is shared by all
reranker providers: the ``except`` fallback now emits a ``logger.warning`` before
degrading to the original order / a neutral score.
"""
import logging
from mem0.reranker.llm_reranker import LLMReranker
class TestRerankerFailureLogging:
def test_llm_failure_is_logged_and_falls_back(self, mock_llm, caplog):
_factory, llm_instance = mock_llm
llm_instance.generate_response.side_effect = RuntimeError("upstream 500")
reranker = LLMReranker({"provider": "openai"})
docs = [{"memory": "alpha"}, {"memory": "beta"}]
with caplog.at_level(logging.WARNING, logger="mem0.reranker.llm_reranker"):
result = reranker.rerank("q", docs)
# Graceful degradation preserved: every doc still comes back, scored neutral.
assert len(result) == 2
assert all(d["rerank_score"] == 0.5 for d in result)
# The failure is no longer silent.
warnings = [r for r in caplog.records if r.levelno == logging.WARNING]
assert warnings, "expected a warning to be logged on reranking failure"
assert "upstream 500" in caplog.text
-25
View File
@@ -618,31 +618,6 @@ class TestGetMemories:
# 3. Verify the core logic: the param was mapped to the filters dict!
_, kwargs = mock_memory.get_all.call_args
assert kwargs["filters"] == {"user_id": "test_routing_user"}
assert "top_k" not in kwargs
def test_get_memories_entity_filters_forward_top_k(self, client, mock_memory):
response = client.get("/memories?user_id=test_routing_user&top_k=1000")
assert response.status_code == 200
_, kwargs = mock_memory.get_all.call_args
assert kwargs["filters"] == {"user_id": "test_routing_user"}
assert kwargs["top_k"] == 1000
def test_get_memories_admin_top_k_zero_not_defaulted(self, client, mock_memory):
mock_memory.vector_store.list.return_value = []
response = client.get("/memories?top_k=0")
assert response.status_code == 200
_, kwargs = mock_memory.vector_store.list.call_args
assert kwargs["top_k"] == 0
def test_get_memories_rejects_top_k_above_limit(self, client, mock_memory):
response = client.get("/memories?user_id=test_routing_user&top_k=1001")
assert response.status_code == 422
mock_memory.get_all.assert_not_called()
# ===========================================================================
+3 -33
View File
@@ -6,7 +6,6 @@ def _ensure_spacy():
"""Skip tests if spaCy model is not available."""
try:
import spacy
spacy.load("en_core_web_sm")
except Exception:
pytest.skip("spaCy en_core_web_sm model not available")
@@ -34,9 +33,8 @@ class TestExtractEntities:
entities = extract_entities("The machine learning engineer built a neural network")
entity_texts = [e[1].lower() for e in entities]
has_compound = any("machine" in t and "learning" in t for t in entity_texts) or any(
"neural" in t and "network" in t for t in entity_texts
)
has_compound = any("machine" in t and "learning" in t for t in entity_texts) or \
any("neural" in t and "network" in t for t in entity_texts)
assert has_compound, f"Expected compound nouns, got {entities}"
def test_empty_string(self):
@@ -78,37 +76,9 @@ class TestExtractEntities:
for entity in entities:
assert isinstance(entity, tuple)
assert len(entity) == 2
assert entity[0] in ("PROPER", "QUOTED", "TOPIC", "IDENTIFIER")
assert entity[0] in ("PROPER", "QUOTED", "COMPOUND", "NOUN")
assert isinstance(entity[1], str)
def test_handles_names_lists_and_identifiers(self):
from mem0.utils.entity_extraction import extract_entities
text = (
"User reported top inbound integration pages: OpenClaw 25,443, "
"Claude Code 8,916, Codex 2,573, Dify 656. "
"User compared Cartesia and Deepgram. "
"The email field for Mem0 lives at person.properties.email. "
"The qwen endpoint uses person.properties.email. "
"Johnson & Johnson was mentioned. "
"Glasses around my window. "
"On 2026-05-27 there were 90 days of stats."
)
entities = extract_entities(text)
entity_texts = {entity_text for _, entity_text in entities}
normalized = {entity_text.lower() for entity_text in entity_texts}
assert {"OpenClaw", "Claude Code", "Codex", "Dify", "Cartesia", "Deepgram", "Mem0"}.issubset(entity_texts)
assert "person.properties.email" in entity_texts
assert "qwen endpoint" in entity_texts
assert "Johnson & Johnson" in entity_texts
assert "top" not in normalized
assert "glasses" not in normalized
assert "Cartesia and Deepgram" not in entity_texts
assert "Claude Code 8,916" not in entity_texts
assert not {"8,916", "2,573", "656", "2026-05-27", "90"}.intersection(entity_texts)
class TestExtractEntitiesBatch:
def test_batch_processing(self):
@@ -511,30 +511,6 @@ def test_insert_with_http_error(azure_ai_search_instance):
assert "Azure service error" in str(exc_info.value)
def test_update_allows_empty_payload(azure_ai_search_instance):
"""Test updating with an empty payload explicitly clears stored payload data."""
instance, mock_search_client, _ = azure_ai_search_instance
mock_search_client.merge_or_upload_documents.return_value = [{"status": True, "id": "doc1", "status_code": 200}]
instance.update("doc1", payload={})
mock_search_client.merge_or_upload_documents.assert_called_once_with(
documents=[{"id": "doc1", "payload": "{}", "user_id": None, "run_id": None, "agent_id": None}]
)
def test_update_allows_empty_vector(azure_ai_search_instance):
"""Test updating with an empty vector still sends the vector field to Azure."""
instance, mock_search_client, _ = azure_ai_search_instance
mock_search_client.merge_or_upload_documents.return_value = [{"status": True, "id": "doc1", "status_code": 200}]
instance.update("doc1", vector=[])
mock_search_client.merge_or_upload_documents.assert_called_once_with(documents=[{"id": "doc1", "vector": []}])
# --- Tests for search method ---
+2 -26
View File
@@ -122,7 +122,7 @@ def test_delete_vector(chromadb_instance):
chromadb_instance.delete(vector_id=vector_id)
chromadb_instance.collection.delete.assert_called_once_with(ids=[vector_id])
chromadb_instance.collection.delete.assert_called_once_with(ids=vector_id)
def test_update_vector(chromadb_instance):
@@ -133,31 +133,7 @@ def test_update_vector(chromadb_instance):
chromadb_instance.update(vector_id=vector_id, vector=new_vector, payload=new_payload)
chromadb_instance.collection.update.assert_called_once_with(
ids=[vector_id], embeddings=[new_vector], metadatas=[new_payload]
)
def test_update_vector_metadata_only(chromadb_instance):
# Metadata-only update (vector=None) must not wrap None in a list.
vector_id = "id1"
new_payload = {"name": "updated_vector"}
chromadb_instance.update(vector_id=vector_id, vector=None, payload=new_payload)
chromadb_instance.collection.update.assert_called_once_with(
ids=[vector_id], embeddings=None, metadatas=[new_payload]
)
def test_update_vector_embedding_only(chromadb_instance):
# Vector-only update (payload=None) must not wrap None in a list.
vector_id = "id1"
new_vector = [0.7, 0.8, 0.9]
chromadb_instance.update(vector_id=vector_id, vector=new_vector, payload=None)
chromadb_instance.collection.update.assert_called_once_with(
ids=[vector_id], embeddings=[new_vector], metadatas=None
ids=vector_id, embeddings=new_vector, metadatas=new_payload
)
-49
View File
@@ -151,55 +151,6 @@ class TestOpenSearchDB(unittest.TestCase):
self.os_db.create_index()
self.client_mock.indices.create.assert_not_called()
def test_auto_refresh_disabled_by_default(self):
"""Test that auto_refresh is disabled by default (Issue #3739).
This ensures OpenSearch Serverless compatibility out-of-the-box since
the indices.refresh() API is not supported in serverless mode.
"""
# Default instance should have auto_refresh=False
self.assertFalse(self.os_db.auto_refresh)
self.client_mock.reset_mock()
vectors = [[0.1] * 1536]
payloads = [{"key1": "value1"}]
ids = ["id1"]
self.os_db.insert(vectors=vectors, payloads=payloads, ids=ids)
# Verify index was called but refresh was NOT called (default behavior)
self.assertEqual(self.client_mock.index.call_count, 1)
self.client_mock.indices.refresh.assert_not_called()
def test_auto_refresh_enabled(self):
"""Test that refresh is called once per batch (not per document) when auto_refresh=True."""
with patch("mem0.vector_stores.opensearch.OpenSearch", return_value=self.client_mock):
auto_refresh_db = OpenSearchDB(
host="localhost",
port=9200,
collection_name="test_auto_refresh",
embedding_model_dims=1536,
auto_refresh=True, # Enable auto-refresh
)
self.assertTrue(auto_refresh_db.auto_refresh)
# auto_refresh_db reuses self.client_mock (patched above), so reset to drop
# the index calls made during construction before asserting on insert().
self.client_mock.reset_mock()
# Insert a batch of 3 vectors to verify the refresh is hoisted out of the
# per-document loop: index() is called once per document, but refresh()
# must fire exactly once for the whole batch.
vectors = [[0.1] * 1536, [0.2] * 1536, [0.3] * 1536]
payloads = [{"key1": "value1"}, {"key2": "value2"}, {"key3": "value3"}]
ids = ["id1", "id2", "id3"]
auto_refresh_db.insert(vectors=vectors, payloads=payloads, ids=ids)
# index() once per document, but refresh() only once for the batch
self.assertEqual(self.client_mock.index.call_count, 3)
self.assertEqual(self.client_mock.indices.refresh.call_count, 1)
def test_insert(self):
vectors = [[0.1] * 1536, [0.2] * 1536]
payloads = [{"key1": "value1"}, {"key2": "value2"}]
-16
View File
@@ -1025,19 +1025,3 @@ class TestQdrantDatetimeRangeFilters(unittest.TestCase):
types = {type(c.range) for c in result.must}
self.assertIn(DatetimeRange, types)
self.assertIn(Range, types)
def test_missing_fastembed_warns_with_extras_install_hint(self):
"""When fastembed is missing, the BM25 warning must point at the mem0 optional group
that actually provides it (`mem0ai[extras]`), not the bare `fastembed` package -
fastembed is declared under [extras], so that is the discoverable install path."""
self.qdrant._bm25_encoder = None
# Setting the module to None in sys.modules makes `from fastembed import ...` raise ImportError.
with patch.dict("sys.modules", {"fastembed": None}):
with self.assertLogs("mem0.vector_stores.qdrant", level="WARNING") as cm:
result = self.qdrant._get_bm25_encoder()
self.assertIsNone(result) # encoder unavailable -> None
self.assertIs(self.qdrant._bm25_encoder, False) # sentinel: tried and failed, do not retry
joined = "\n".join(cm.output)
self.assertIn("mem0ai[extras]", joined) # points at the right install
self.assertNotIn("pip install fastembed", joined) # not the bare package
self.assertNotIn("\u2014", joined) # no em-dash