Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
368cc5fb1e |
+43
-130
@@ -1,157 +1,72 @@
|
|||||||
# Contributing to Mem0
|
# Contributing to mem0
|
||||||
|
|
||||||
First off, thank you for taking the time to contribute! 🎉 Mem0 is a
|
Let us make contribution easy, collaborative and fun.
|
||||||
community-driven project and we welcome contributions of all kinds — bug fixes,
|
|
||||||
new features, documentation, examples, and integrations.
|
|
||||||
|
|
||||||
Mem0 is a polyglot monorepo, and this guide covers contributing to both the
|
## Submit your Contribution through PR
|
||||||
**Python SDK** and the **TypeScript SDK** (and the rest of the repository).
|
|
||||||
|
|
||||||
## Before You Start
|
To make a contribution, follow these steps:
|
||||||
|
|
||||||
### 1. Open an Issue First
|
1. Fork and clone this repository
|
||||||
|
2. Do the changes on your fork with dedicated feature branch `feature/f1`
|
||||||
|
3. If you modified the code (new feature or bug-fix), please add tests for it
|
||||||
|
4. Include proper documentation / docstring and examples to run the feature
|
||||||
|
5. Ensure that all tests pass
|
||||||
|
6. Submit a pull request
|
||||||
|
|
||||||
**Always open an issue before opening a pull request.** This lets us discuss the
|
For more details about pull requests, please read [GitHub's guides](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).
|
||||||
change, avoid duplicate effort, and agree on the approach before you invest time
|
|
||||||
in code.
|
|
||||||
|
|
||||||
- Search [existing issues](https://github.com/mem0ai/mem0/issues) first to see if
|
|
||||||
your bug or idea already exists.
|
|
||||||
- If it doesn't, open a
|
|
||||||
[bug report](https://github.com/mem0ai/mem0/issues/new?template=bug_report.yml) or
|
|
||||||
[feature request](https://github.com/mem0ai/mem0/issues/new?template=feature_request.yml).
|
|
||||||
- For anything beyond a trivial fix, wait for a maintainer to confirm the approach
|
|
||||||
before starting significant work.
|
|
||||||
|
|
||||||
Every pull request must link to an issue using `Closes #<issue-number>`.
|
### 📦 Development Environment
|
||||||
|
|
||||||
### 2. Sign the Contributor License Agreement (CLA)
|
We use `hatch` for managing development environments. To set up:
|
||||||
|
|
||||||
**We cannot accept or merge any pull request until you have signed our Contributor
|
|
||||||
License Agreement (CLA).**
|
|
||||||
|
|
||||||
When you open your first PR, the CLA bot will automatically comment with a link to
|
|
||||||
sign. Signing takes less than a minute and only needs to be done once. Pull
|
|
||||||
requests from contributors who have not signed the CLA will be blocked from
|
|
||||||
merging.
|
|
||||||
|
|
||||||
## Repository Layout
|
|
||||||
|
|
||||||
The two most common contribution targets are the SDKs:
|
|
||||||
|
|
||||||
| Package | Path | Language | Package manager |
|
|
||||||
| --------------------- | ---------- | ------------ | --------------- |
|
|
||||||
| Python SDK (`mem0ai`) | `mem0/` | Python 3.9+ | `hatch` |
|
|
||||||
| TypeScript SDK (`mem0ai`) | `mem0-ts/` | TypeScript | `pnpm` |
|
|
||||||
|
|
||||||
Other packages include the CLIs (`cli/python/`, `cli/node/`), integrations
|
|
||||||
(`integrations/`), the self-hosted `server/`, `openmemory/`, and the docs site
|
|
||||||
(`docs/`). See [AGENTS.md](./AGENTS.md) for a full map of the repository.
|
|
||||||
|
|
||||||
## Development Workflow
|
|
||||||
|
|
||||||
1. **Fork** the repository and **clone** your fork.
|
|
||||||
2. Create a **feature branch** from `main` (e.g. `feature/my-new-feature` or
|
|
||||||
`fix/issue-1234`).
|
|
||||||
3. Make your changes — add **tests**, **documentation**, and **examples** as
|
|
||||||
appropriate.
|
|
||||||
4. Run **linting and tests** for every package you touched (see below).
|
|
||||||
5. Commit using [Conventional Commits](https://www.conventionalcommits.org/)
|
|
||||||
(e.g. `feat:`, `fix:`, `docs:`, `refactor:`, `test:`).
|
|
||||||
6. Push and open a **pull request** against `main`, linking the issue with
|
|
||||||
`Closes #<number>` and filling out the
|
|
||||||
[PR template](./.github/PULL_REQUEST_TEMPLATE.md).
|
|
||||||
|
|
||||||
### Contributing to the Python SDK (`mem0/`)
|
|
||||||
|
|
||||||
We use [`hatch`](https://hatch.pypa.io/latest/install/) to manage environments.
|
|
||||||
**Do not use `pip` or `conda` for dependency management.**
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Activate a dev environment (3.9 / 3.10 / 3.11 / 3.12)
|
# Activate environment for specific Python version:
|
||||||
hatch shell dev_py_3_11
|
hatch shell dev_py_3_9 # Python 3.9
|
||||||
|
hatch shell dev_py_3_10 # Python 3.10
|
||||||
|
hatch shell dev_py_3_11 # Python 3.11
|
||||||
|
hatch shell dev_py_3_12 # Python 3.12
|
||||||
|
|
||||||
# Install pre-commit hooks (runs ruff + isort on commit)
|
# The environment will automatically install all dev dependencies
|
||||||
pre-commit install
|
# Run tests within the activated shell:
|
||||||
|
|
||||||
# Lint, format, and sort imports
|
|
||||||
make lint
|
|
||||||
make format
|
|
||||||
make sort
|
|
||||||
|
|
||||||
# Run the test suite (run `make install_all` first if deps are missing)
|
|
||||||
make test
|
make test
|
||||||
```
|
```
|
||||||
|
|
||||||
- **Linter / formatter:** Ruff (line length **120**)
|
### 📌 Pre-commit
|
||||||
- **Import sorting:** isort (`profile = "black"`)
|
|
||||||
- **Tests:** pytest (in `tests/`)
|
|
||||||
|
|
||||||
See the full [Development guide](https://docs.mem0.ai/contributing/development) for
|
To ensure our standards, make sure to install pre-commit before starting to contribute.
|
||||||
environment details.
|
|
||||||
|
|
||||||
### Contributing to the TypeScript SDK (`mem0-ts/`)
|
|
||||||
|
|
||||||
We use [`pnpm`](https://pnpm.io/) (v10+) for all TypeScript packages. **Do not use
|
|
||||||
`npm` or `yarn`.**
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd mem0-ts
|
pre-commit install
|
||||||
pnpm install
|
|
||||||
|
|
||||||
pnpm run build # tsup (CJS + ESM)
|
|
||||||
pnpm run test # jest (all tests)
|
|
||||||
pnpm run test:unit # unit tests with coverage
|
|
||||||
```
|
```
|
||||||
|
|
||||||
- **Build:** tsup
|
### 🧪 Testing
|
||||||
- **Formatter:** Prettier
|
|
||||||
- **Tests:** jest
|
|
||||||
- Always run type checking after changes: `pnpm run typecheck` (or `tsc --noEmit`).
|
|
||||||
- Use ES module `import` syntax — never `require()`.
|
|
||||||
|
|
||||||
## Good Contribution Practices
|
We use `pytest` to test our code across multiple Python versions. You can run tests using:
|
||||||
|
|
||||||
- **Keep PRs small and focused.** One logical change per PR is easier to review and
|
```bash
|
||||||
merge.
|
# Run tests with default Python version
|
||||||
- **Follow existing patterns.** Match the style, structure, and conventions of the
|
make test
|
||||||
code around you. Don't introduce new frameworks or abstractions without
|
|
||||||
discussion.
|
|
||||||
- **Write tests** that would fail without your change — regression tests for bugs,
|
|
||||||
coverage for new features.
|
|
||||||
- **Update documentation** in `docs/` for any user-facing change. New `.mdx` pages
|
|
||||||
must be added to `docs/llms.txt` (run
|
|
||||||
`python scripts/check-llms-txt-coverage.py --write` to scaffold entries).
|
|
||||||
- **Add examples** when introducing new user-facing behavior.
|
|
||||||
- **Run linters and tests locally** before pushing — CI re-runs them on every PR
|
|
||||||
via the CI Gate.
|
|
||||||
- **Never commit secrets** — no `.env` files, API keys, or credentials.
|
|
||||||
- **Don't add core dependencies lightly.** New Python dependencies belong in an
|
|
||||||
optional group in `pyproject.toml`, not the core `dependencies` list.
|
|
||||||
- **Be responsive** to review feedback and keep your branch up to date with `main`.
|
|
||||||
|
|
||||||
## Pull Request Checklist
|
# Test specific Python versions:
|
||||||
|
make test-py-3.9 # Python 3.9 environment
|
||||||
|
make test-py-3.10 # Python 3.10 environment
|
||||||
|
make test-py-3.11 # Python 3.11 environment
|
||||||
|
make test-py-3.12 # Python 3.12 environment
|
||||||
|
|
||||||
Before requesting review, make sure:
|
# When using hatch shells, run tests with:
|
||||||
|
make test # After activating a shell with hatch shell test_XX
|
||||||
|
```
|
||||||
|
|
||||||
- [ ] An issue exists and is linked with `Closes #<number>`
|
Make sure that all tests pass across all supported Python versions before submitting a pull request.
|
||||||
- [ ] You have signed the CLA
|
|
||||||
- [ ] Your code follows the project's style guidelines (lint passes)
|
|
||||||
- [ ] You performed a self-review of your changes
|
|
||||||
- [ ] Tests are added/updated and pass locally
|
|
||||||
- [ ] Documentation is updated if needed
|
|
||||||
|
|
||||||
## Reporting Security Issues
|
We look forward to your pull requests and can't wait to see your contributions!
|
||||||
|
|
||||||
**Do not report security vulnerabilities through public issues or pull requests.**
|
### 🚀 Releasing
|
||||||
Please follow our [Security Policy](./SECURITY.md) to report them privately.
|
|
||||||
|
|
||||||
## Releasing
|
All packages are published automatically via GitHub Actions when a GitHub Release is created with the correct tag prefix.
|
||||||
|
|
||||||
All packages are published automatically via GitHub Actions when a GitHub Release
|
#### Tag Prefixes
|
||||||
is created with the correct tag prefix.
|
|
||||||
|
|
||||||
### Tag Prefixes
|
|
||||||
|
|
||||||
| Package | Registry | Tag Prefix | Example |
|
| Package | Registry | Tag Prefix | Example |
|
||||||
|---------|----------|------------|---------|
|
|---------|----------|------------|---------|
|
||||||
@@ -162,17 +77,15 @@ is created with the correct tag prefix.
|
|||||||
| `@mem0/vercel-ai-provider` | npm | `vercel-ai-v*` | `vercel-ai-v2.0.6` |
|
| `@mem0/vercel-ai-provider` | npm | `vercel-ai-v*` | `vercel-ai-v2.0.6` |
|
||||||
| `@mem0/openclaw-mem0` | npm | `openclaw-v*` | `openclaw-v1.0.1` |
|
| `@mem0/openclaw-mem0` | npm | `openclaw-v*` | `openclaw-v1.0.1` |
|
||||||
|
|
||||||
### How to Release
|
#### How to Release
|
||||||
|
|
||||||
1. Bump the version in `pyproject.toml` (Python) or `package.json` (Node)
|
1. Bump the version in `pyproject.toml` (Python) or `package.json` (Node)
|
||||||
2. Create a [GitHub Release](https://github.com/mem0ai/mem0/releases/new) with the matching tag prefix
|
2. Create a [GitHub Release](https://github.com/mem0ai/mem0/releases/new) with the matching tag prefix
|
||||||
3. The correct workflow will trigger automatically — verify in the [Actions tab](https://github.com/mem0ai/mem0/actions)
|
3. The correct workflow will trigger automatically — verify in the [Actions tab](https://github.com/mem0ai/mem0/actions)
|
||||||
|
|
||||||
### Publishing Details
|
#### Publishing Details
|
||||||
|
|
||||||
- **PyPI packages** use OIDC trusted publishing via `pypa/gh-action-pypi-publish`
|
- **PyPI packages** use OIDC trusted publishing via `pypa/gh-action-pypi-publish`
|
||||||
- **npm packages** use OIDC trusted publishing via npm CLI (>= 11.5.1) — no tokens or secrets required
|
- **npm packages** use OIDC trusted publishing via npm CLI (>= 11.5.1) — no tokens or secrets required
|
||||||
- All workflows require `permissions: id-token: write` for OIDC authentication
|
- All workflows require `permissions: id-token: write` for OIDC authentication
|
||||||
- First publish of a new npm package must be done manually; OIDC works for subsequent versions
|
- First publish of a new npm package must be done manually; OIDC works for subsequent versions
|
||||||
|
|
||||||
We look forward to your pull requests and can't wait to see your contributions!
|
|
||||||
|
|||||||
-48
@@ -1,48 +0,0 @@
|
|||||||
# Security Policy
|
|
||||||
|
|
||||||
We take the security of Mem0 and our community seriously. Thank you for helping
|
|
||||||
keep Mem0 and its users safe by disclosing vulnerabilities responsibly.
|
|
||||||
|
|
||||||
## Reporting a Vulnerability
|
|
||||||
|
|
||||||
Please **do not** report security vulnerabilities through public GitHub issues,
|
|
||||||
pull requests, or discussions.
|
|
||||||
|
|
||||||
If you believe you have found a security vulnerability in Mem0, please report it
|
|
||||||
privately through one of the following channels:
|
|
||||||
|
|
||||||
1. **GitHub Private Vulnerability Reporting** — open a
|
|
||||||
[private security advisory](https://github.com/mem0ai/mem0/security/advisories/new)
|
|
||||||
directly on this repository.
|
|
||||||
2. **Email** the maintainers at **support@mem0.ai** with the subject line:
|
|
||||||
|
|
||||||
`SECURITY: Mem0 vulnerability report`
|
|
||||||
|
|
||||||
To help us triage and resolve the issue quickly, please include as much of the
|
|
||||||
following as you can:
|
|
||||||
|
|
||||||
- Affected component or package (e.g. Python SDK, TypeScript SDK, server, OpenMemory)
|
|
||||||
- Affected version, tag, or commit
|
|
||||||
- Clear, step-by-step reproduction instructions
|
|
||||||
- The security impact and a proof of concept, if available
|
|
||||||
- Any suggested fix or mitigation
|
|
||||||
|
|
||||||
## Response Process
|
|
||||||
|
|
||||||
- We will acknowledge receipt of your report within **72 hours**.
|
|
||||||
- We will work with you privately to confirm the issue and assess its impact.
|
|
||||||
- Once a fix or mitigation is ready, we will coordinate a disclosure timeline
|
|
||||||
with you and credit you for the discovery, unless you prefer to remain anonymous.
|
|
||||||
|
|
||||||
## Public Disclosure
|
|
||||||
|
|
||||||
Please avoid sharing technical details of the vulnerability publicly until the
|
|
||||||
maintainers have reviewed the issue and a fix or mitigation has been released. We
|
|
||||||
are committed to resolving valid reports promptly and keeping you informed
|
|
||||||
throughout the process.
|
|
||||||
|
|
||||||
## Supported Versions
|
|
||||||
|
|
||||||
We release security fixes against the latest published version of each package.
|
|
||||||
Whenever possible, please reproduce the issue on the most recent release before
|
|
||||||
reporting.
|
|
||||||
@@ -113,7 +113,7 @@ Launched a unified Mem0 plugin across three major AI development environments
|
|||||||
|
|
||||||
Major expansion of the provider ecosystem:
|
Major expansion of the provider ecosystem:
|
||||||
|
|
||||||
- **Apache AGE** — New graph store support, bringing the total to 4 graph store backends (Neo4j, Memgraph, Kuzu, Apache AGE)
|
- **Apache AGE** — New graph store support, bringing the total to 4 graph store backends (Neo4j, Memgraph, Kuzu, Apache AGE). **Note:** All external graph store backends (Neo4j, Memgraph, Kuzu, Apache AGE) were subsequently removed in v2.0.0 (2026-04-14). Graph memory is now built-in entity linking with no external graph store required; see the [v2.0.0 entry above](#mem0-sdk-v2-0-0-v3-0-0).
|
||||||
- **Turbopuffer** — New vector database provider for Python SDK
|
- **Turbopuffer** — New vector database provider for Python SDK
|
||||||
- **MiniMax** — New LLM provider with dedicated AWS Bedrock support
|
- **MiniMax** — New LLM provider with dedicated AWS Bedrock support
|
||||||
- **pgvector for Node.js** — PostgreSQL vector support added to the TypeScript OSS SDK
|
- **pgvector for Node.js** — PostgreSQL vector support added to the TypeScript OSS SDK
|
||||||
|
|||||||
@@ -121,7 +121,7 @@ mode: "wide"
|
|||||||
|
|
||||||
**New Features:**
|
**New Features:**
|
||||||
- **Memory:** Warn at init time when hybrid/BM25 search silently degrades to semantic-only because the configured vector store does not implement `keyword_search`. Affected stores: Chroma, FAISS, Cassandra, LangChain, Neptune Analytics, S3 Vectors, Supabase, TurboPuffer, Valkey ([#5444](https://github.com/mem0ai/mem0/pull/5444))
|
- **Memory:** Warn at init time when hybrid/BM25 search silently degrades to semantic-only because the configured vector store does not implement `keyword_search`. Affected stores: Chroma, FAISS, Cassandra, LangChain, Neptune Analytics, S3 Vectors, Supabase, TurboPuffer, Valkey ([#5444](https://github.com/mem0ai/mem0/pull/5444))
|
||||||
- **Memory:** Add opt-in `explain=True` parameter to `Memory.search()` and `AsyncMemory.search()`. When enabled, each result includes a `score_breakdown` dict with `semantic`, `keyword` (normalized BM25), `entity_boost`, and `temporal_boost` signals so callers can understand and tune retrieval ranking ([#5102](https://github.com/mem0ai/mem0/pull/5102))
|
- **Memory:** Add opt-in `explain=True` parameter to `Memory.search()` and `AsyncMemory.search()`. When enabled, each result includes a `score_details` dict with `semantic_score`, `bm25_score`, `entity_boost`, `raw_score`, `max_possible_score`, `final_score`, and `threshold` so callers can understand and tune retrieval ranking ([#5102](https://github.com/mem0ai/mem0/pull/5102))
|
||||||
|
|
||||||
**Bug Fixes:**
|
**Bug Fixes:**
|
||||||
- **Vector Stores:** Normalize similarity scores to `[0, 1]` (higher = better) consistently across all backends. 11 adapters previously returned raw distance metrics (lower = better) — FAISS, Chroma, Milvus, Redis, Cassandra, PGVector, S3 Vectors, Supabase, Valkey, Azure MySQL, and Vertex AI Vector Search — causing incorrect ranking in multi-store setups ([#5391](https://github.com/mem0ai/mem0/pull/5391))
|
- **Vector Stores:** Normalize similarity scores to `[0, 1]` (higher = better) consistently across all backends. 11 adapters previously returned raw distance metrics (lower = better) — FAISS, Chroma, Milvus, Redis, Cassandra, PGVector, S3 Vectors, Supabase, Valkey, Azure MySQL, and Vertex AI Vector Search — causing incorrect ranking in multi-store setups ([#5391](https://github.com/mem0ai/mem0/pull/5391))
|
||||||
@@ -229,7 +229,7 @@ mode: "wide"
|
|||||||
**Improvements:**
|
**Improvements:**
|
||||||
- **Telemetry:** Sample OSS hot-path events at 10% via PostHog `before_send` hook to reduce event volume ([#4771](https://github.com/mem0ai/mem0/pull/4771))
|
- **Telemetry:** Sample OSS hot-path events at 10% via PostHog `before_send` hook to reduce event volume ([#4771](https://github.com/mem0ai/mem0/pull/4771))
|
||||||
|
|
||||||
See the [OSS v1 to v2 migration guide](https://docs.mem0.ai/migration/oss-v1-to-v2) and [Platform migration guide](https://docs.mem0.ai/migration/platform-v2-to-v3) for upgrade instructions.
|
See the [OSS v2 to v3 migration guide](https://docs.mem0.ai/migration/oss-v2-to-v3) and [Platform migration guide](https://docs.mem0.ai/migration/platform-v2-to-v3) for upgrade instructions.
|
||||||
|
|
||||||
</Update>
|
</Update>
|
||||||
|
|
||||||
|
|||||||
@@ -21,47 +21,49 @@ from mem0 import Memory
|
|||||||
load_dotenv()
|
load_dotenv()
|
||||||
|
|
||||||
config = {
|
config = {
|
||||||
"vector_store": {
|
"vector_store": {
|
||||||
"provider": "pgvector",
|
"provider": "pgvector",
|
||||||
"config": {
|
"config": {
|
||||||
"connection_string": os.environ["DATABASE_URL"],
|
"connection_string": os.environ["DATABASE_URL"],
|
||||||
"collection_name": "memories",
|
"collection_name": "memories",
|
||||||
"embedding_model_dims": 1536,
|
"embedding_model_dims": 1536,
|
||||||
"hnsw": True,
|
"hnsw": True,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
m = Memory.from_config(config)
|
m = Memory.from_config(config)
|
||||||
messages = [
|
messages = [
|
||||||
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
|
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
|
||||||
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
|
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
|
||||||
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
|
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
|
||||||
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."},
|
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."},
|
||||||
]
|
]
|
||||||
m.add(messages, user_id="alice", metadata={"category": "movies"})
|
m.add(messages, user_id="alice", metadata={"category": "movies"})
|
||||||
|
|
||||||
results = m.search(
|
results = m.search(
|
||||||
"What movies should I recommend?",
|
"What movies should I recommend?",
|
||||||
filters={"user_id": "alice"},
|
filters={"user_id": "alice"},
|
||||||
)
|
)
|
||||||
|
|
||||||
print(results)
|
print(results)
|
||||||
|
```
|
||||||
````
|
|
||||||
|
|
||||||
```typescript TypeScript
|
```typescript TypeScript
|
||||||
import "dotenv/config";
|
import "dotenv/config";
|
||||||
import { Memory } from "mem0ai/oss";
|
import { Memory } from "mem0ai/oss";
|
||||||
|
|
||||||
|
const databaseUrl = new URL(process.env.DATABASE_URL!);
|
||||||
|
|
||||||
const m = new Memory({
|
const m = new Memory({
|
||||||
vectorStore: {
|
vectorStore: {
|
||||||
provider: "pgvector",
|
provider: "pgvector",
|
||||||
config: {
|
config: {
|
||||||
connectionString: process.env.DATABASE_URL!,
|
user: decodeURIComponent(databaseUrl.username),
|
||||||
ssl: {
|
password: decodeURIComponent(databaseUrl.password),
|
||||||
rejectUnauthorized: false,
|
host: databaseUrl.hostname,
|
||||||
},
|
port: Number(databaseUrl.port || 5432),
|
||||||
|
dbname: databaseUrl.pathname.slice(1) || "neondb",
|
||||||
collectionName: "memories",
|
collectionName: "memories",
|
||||||
dimension: 1536,
|
dimension: 1536,
|
||||||
embeddingModelDims: 1536,
|
embeddingModelDims: 1536,
|
||||||
@@ -87,8 +89,7 @@ const results = await m.search("What movies should I recommend?", {
|
|||||||
});
|
});
|
||||||
|
|
||||||
console.log(results);
|
console.log(results);
|
||||||
````
|
```
|
||||||
|
|
||||||
</CodeGroup>
|
</CodeGroup>
|
||||||
|
|
||||||
## SQL Migration
|
## SQL Migration
|
||||||
@@ -115,19 +116,20 @@ DATABASE_URL=postgresql://user:password@ep-example.us-east-2.aws.neon.tech/neond
|
|||||||
| `sslmode` | PostgreSQL SSL mode. Use `require` for Neon. | Driver default |
|
| `sslmode` | PostgreSQL SSL mode. Use `require` for Neon. | Driver default |
|
||||||
</Tab>
|
</Tab>
|
||||||
<Tab title="TypeScript">
|
<Tab title="TypeScript">
|
||||||
Use the Neon `DATABASE_URL` directly with `connectionString`. Set `ssl` if your runtime needs an explicit TLS config object.
|
The current Mem0 TypeScript `pgvector` adapter takes individual Postgres fields,
|
||||||
|
so parse `DATABASE_URL` before creating `Memory`.
|
||||||
| Parameter | Description | Default |
|
|
||||||
| -------------------- | ---------------------------------------------- | -------------- |
|
|
||||||
| `connectionString` | Neon Postgres connection string. | Required |
|
|
||||||
| `ssl` | Optional TLS settings passed directly to `pg`. | Driver default |
|
|
||||||
| `collectionName` | Name for the vector collection. | `memories` |
|
|
||||||
| `dimension` | Vector dimension for Mem0 config. | Auto-detected |
|
|
||||||
| `embeddingModelDims` | Embedding model dimensions for table creation. | Required |
|
|
||||||
| `hnsw` | Enables HNSW indexing. | `false` |
|
|
||||||
|
|
||||||
**TLS note:** `ssl: true` is sufficient for most Neon connections since Neon uses valid certificates. Use `ssl: { rejectUnauthorized: false }` only when connecting through Neon's connection pooler on certain edge runtimes (e.g. Cloudflare Workers) that require it, or when your environment does not trust the Neon CA chain.
|
|
||||||
|
|
||||||
|
| Parameter | Description | Default |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `user` | Database user. | Required |
|
||||||
|
| `password` | Database password. | Required |
|
||||||
|
| `host` | Database host. | Required |
|
||||||
|
| `port` | Database port. | `5432` |
|
||||||
|
| `dbname` | Database name. | `vector_store` |
|
||||||
|
| `collectionName` | Name for the vector collection. | `memories` |
|
||||||
|
| `dimension` | Vector dimension for Mem0 config. | Auto-detected |
|
||||||
|
| `embeddingModelDims` | Embedding model dimensions for table creation. | Required |
|
||||||
|
| `hnsw` | Enables HNSW indexing. | `false` |
|
||||||
</Tab>
|
</Tab>
|
||||||
</Tabs>
|
</Tabs>
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,6 @@
|
|||||||
title: "pgvector"
|
title: "pgvector"
|
||||||
description: "Use pgvector as a vector store in Mem0 for PostgreSQL-based vector similarity search with open-source simplicity."
|
description: "Use pgvector as a vector store in Mem0 for PostgreSQL-based vector similarity search with open-source simplicity."
|
||||||
---
|
---
|
||||||
|
|
||||||
[pgvector](https://github.com/pgvector/pgvector) is an open-source vector similarity search extension for Postgres. After connecting to Postgres, run `CREATE EXTENSION IF NOT EXISTS vector;` to create the vector extension.
|
[pgvector](https://github.com/pgvector/pgvector) is an open-source vector similarity search extension for Postgres. After connecting to Postgres, run `CREATE EXTENSION IF NOT EXISTS vector;` to create the vector extension.
|
||||||
|
|
||||||
### Usage
|
### Usage
|
||||||
@@ -15,38 +14,41 @@ from mem0 import Memory
|
|||||||
os.environ["OPENAI_API_KEY"] = "sk-xx"
|
os.environ["OPENAI_API_KEY"] = "sk-xx"
|
||||||
|
|
||||||
config = {
|
config = {
|
||||||
"vector_store": {
|
"vector_store": {
|
||||||
"provider": "pgvector",
|
"provider": "pgvector",
|
||||||
"config": {
|
"config": {
|
||||||
"user": "test",
|
"user": "test",
|
||||||
"password": "123",
|
"password": "123",
|
||||||
"host": "127.0.0.1",
|
"host": "127.0.0.1",
|
||||||
"port": "5432",
|
"port": "5432",
|
||||||
},
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
m = Memory.from_config(config)
|
m = Memory.from_config(config)
|
||||||
messages = [
|
messages = [
|
||||||
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
|
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
|
||||||
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
|
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
|
||||||
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
|
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
|
||||||
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."},
|
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."}
|
||||||
]
|
]
|
||||||
m.add(messages, user_id="alice", metadata={"category": "movies"})
|
m.add(messages, user_id="alice", metadata={"category": "movies"})
|
||||||
|
```
|
||||||
````
|
|
||||||
|
|
||||||
```typescript TypeScript
|
```typescript TypeScript
|
||||||
import { Memory } from "mem0ai/oss";
|
import { Memory } from 'mem0ai/oss';
|
||||||
|
|
||||||
const config = {
|
const config = {
|
||||||
vectorStore: {
|
vectorStore: {
|
||||||
provider: "pgvector",
|
provider: 'pgvector',
|
||||||
config: {
|
config: {
|
||||||
collectionName: "memories",
|
collectionName: 'memories',
|
||||||
embeddingModelDims: 1536,
|
embeddingModelDims: 1536,
|
||||||
connectionString: "postgresql://test:123@localhost:5432/vector_store",
|
user: 'test',
|
||||||
|
password: '123',
|
||||||
|
host: '127.0.0.1',
|
||||||
|
port: 5432,
|
||||||
|
dbname: 'vector_store', // Optional; TypeScript OSS defaults to `vector_store` when omitted
|
||||||
diskann: false, // Optional, requires pgvectorscale extension
|
diskann: false, // Optional, requires pgvectorscale extension
|
||||||
hnsw: false, // Optional, for HNSW indexing
|
hnsw: false, // Optional, for HNSW indexing
|
||||||
},
|
},
|
||||||
@@ -55,44 +57,37 @@ const config = {
|
|||||||
|
|
||||||
const memory = new Memory(config);
|
const memory = new Memory(config);
|
||||||
const messages = [
|
const messages = [
|
||||||
{ role: "user", content: "I'm planning to watch a movie tonight. Any recommendations?" },
|
{"role": "user", "content": "I'm planning to watch a movie tonight. Any recommendations?"},
|
||||||
{ role: "assistant", content: "How about thriller movies? They can be quite engaging." },
|
{"role": "assistant", "content": "How about thriller movies? They can be quite engaging."},
|
||||||
{ role: "user", content: "I'm not a big fan of thriller movies but I love sci-fi movies." },
|
{"role": "user", "content": "I'm not a big fan of thriller movies but I love sci-fi movies."},
|
||||||
{ role: "assistant", content: "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future." },
|
{"role": "assistant", "content": "Got it! I'll avoid thriller recommendations and suggest sci-fi movies in the future."}
|
||||||
];
|
]
|
||||||
|
|
||||||
await memory.add(messages, { userId: "alice", metadata: { category: "movies" } });
|
await memory.add(messages, { userId: "alice", metadata: { category: "movies" } });
|
||||||
````
|
```
|
||||||
|
|
||||||
</CodeGroup>
|
</CodeGroup>
|
||||||
|
|
||||||
### Config
|
### Config
|
||||||
|
|
||||||
Here are the parameters available for configuring pgvector:
|
Here are the parameters available for configuring pgvector:
|
||||||
|
|
||||||
| Parameter | SDK | Description | Default Value |
|
| Parameter | Description | Default Value |
|
||||||
| -------------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- |
|
| --- | --- | --- |
|
||||||
| `connectionString` | TypeScript OSS | PostgreSQL connection string for direct connections. When set, Mem0 connects to the target database directly and skips the bootstrap `postgres` database flow. | `None` |
|
| `dbname` | The name of the database | `postgres` |
|
||||||
| `ssl` | TypeScript OSS | SSL option passed directly to `pg`, either `true` or an SSL config object, for both `connectionString` and split-field connections. | `None` |
|
| `collection_name` | The name of the collection | `mem0` |
|
||||||
| `dbname` | TypeScript OSS | Split-field database name. This is only used when `connectionString` is absent. | `vector_store` |
|
| `embedding_model_dims` | Dimensions of the embedding model | `1536` |
|
||||||
| `collectionName` | TypeScript OSS | Collection name. | `memories` |
|
| `user` | User name to connect to the database | `None` |
|
||||||
| `embeddingModelDims` | TypeScript OSS | Dimensions of the embedding model. | Required |
|
| `password` | Password to connect to the database | `None` |
|
||||||
| `user` | TypeScript OSS + Python | Database user for split-field connections. | `None` |
|
| `host` | The host where the Postgres server is running | `None` |
|
||||||
| `password` | TypeScript OSS + Python | Database password for split-field connections. | `None` |
|
| `port` | The port where the Postgres server is running | `None` |
|
||||||
| `host` | TypeScript OSS + Python | Database host for split-field connections. | `None` |
|
| `diskann` | Whether to use diskann for vector similarity search (requires pgvectorscale) | `True` |
|
||||||
| `port` | TypeScript OSS + Python | Database port for split-field connections. | `None` |
|
| `hnsw` | Whether to use hnsw for vector similarity search | `False` |
|
||||||
| `diskann` | TypeScript OSS + Python | Whether to use DiskANN for vector similarity search, requires pgvectorscale. | `False` |
|
| `sslmode` | SSL mode for PostgreSQL connection (e.g., 'require', 'prefer', 'disable') | `None` |
|
||||||
| `hnsw` | TypeScript OSS + Python | Whether to use HNSW for vector similarity search. | TypeScript OSS: `False`, Python: `True` |
|
| `connection_string` | PostgreSQL connection string (overrides individual connection parameters) | `None` |
|
||||||
| `connection_string` | Python only | PostgreSQL connection string, overrides individual connection parameters. | `None` |
|
| `connection_pool` | psycopg2 connection pool object (overrides connection string and individual parameters) | `None` |
|
||||||
| `sslmode` | Python only | SSL mode for PostgreSQL connections, such as `require`, `prefer`, or `disable`. | `None` |
|
|
||||||
| `connection_pool` | Python only | psycopg connection pool object, overrides connection string and individual connection parameters. | `None` |
|
|
||||||
|
|
||||||
**TypeScript OSS:** Use `connectionString` plus optional `ssl` for managed Postgres setups. If you omit `connectionString`, Mem0 falls back to split fields and uses `dbname`, `user`, `password`, `host`, `port`, and optional `ssl`.
|
**Note (TypeScript OSS):** If you omit `dbname`, the TypeScript client uses the database name `vector_store`. Python defaults to `postgres` for `dbname`, as in the table above.
|
||||||
|
|
||||||
**Python:** The Python SDK uses snake_case keys such as `connection_string`, `sslmode`, `collection_name`, and `embedding_model_dims`.
|
|
||||||
|
|
||||||
**Python connection priority**:
|
|
||||||
|
|
||||||
|
**Note**: The connection parameters have the following priority:
|
||||||
1. `connection_pool` (highest priority)
|
1. `connection_pool` (highest priority)
|
||||||
2. `connection_string`
|
2. `connection_string`
|
||||||
3. Individual connection parameters (`user`, `password`, `host`, `port`, `sslmode`)
|
3. Individual connection parameters (`user`, `password`, `host`, `port`, `sslmode`)
|
||||||
@@ -1,65 +1,32 @@
|
|||||||
---
|
---
|
||||||
title: Development
|
title: Development
|
||||||
description: "Guide to contributing code to Mem0, covering the issue-first workflow, the CLA, environment setup for the Python and TypeScript SDKs, and code quality checks."
|
description: "Guide to contributing code to Mem0, covering the fork and clone workflow, PR submission, and code quality checks."
|
||||||
icon: "code"
|
icon: "code"
|
||||||
---
|
---
|
||||||
|
|
||||||
# Development Contributions
|
# Development Contributions
|
||||||
|
|
||||||
We strive to make contributions **easy, collaborative, and enjoyable**. Mem0 is a
|
We strive to make contributions **easy, collaborative, and enjoyable**. Follow the steps below to ensure a smooth contribution process.
|
||||||
polyglot monorepo containing the **Python SDK** (`mem0/`), the **TypeScript SDK**
|
|
||||||
(`mem0-ts/`), CLIs, integrations, the self-hosted server, and the docs site.
|
|
||||||
Follow the steps below for a smooth contribution process.
|
|
||||||
|
|
||||||
<Note>
|
## Submitting Your Contribution through PR
|
||||||
For the complete contributor checklist, see
|
|
||||||
[CONTRIBUTING.md](https://github.com/mem0ai/mem0/blob/main/CONTRIBUTING.md) in
|
|
||||||
the repository root.
|
|
||||||
</Note>
|
|
||||||
|
|
||||||
## Before You Start
|
To contribute, follow these steps:
|
||||||
|
|
||||||
### 1. Open an Issue First
|
|
||||||
|
|
||||||
**Always open an issue before opening a pull request.** This lets us discuss the
|
|
||||||
change, avoid duplicate work, and agree on the approach before you write code.
|
|
||||||
|
|
||||||
- Search [existing issues](https://github.com/mem0ai/mem0/issues) first.
|
|
||||||
- If none match, open a
|
|
||||||
[bug report](https://github.com/mem0ai/mem0/issues/new?template=bug_report.yml)
|
|
||||||
or [feature request](https://github.com/mem0ai/mem0/issues/new?template=feature_request.yml).
|
|
||||||
- For anything beyond a trivial fix, wait for a maintainer to confirm the approach.
|
|
||||||
|
|
||||||
Every pull request must link to an issue using `Closes #<issue-number>`.
|
|
||||||
|
|
||||||
### 2. Sign the Contributor License Agreement (CLA)
|
|
||||||
|
|
||||||
**We cannot merge any pull request until you have signed our Contributor License
|
|
||||||
Agreement (CLA).** When you open your first PR, the CLA bot will comment with a
|
|
||||||
link to sign — it takes less than a minute and only needs to be done once.
|
|
||||||
|
|
||||||
## Submitting Your Contribution through a PR
|
|
||||||
|
|
||||||
1. **Fork & Clone** the repository: [Mem0 on GitHub](https://github.com/mem0ai/mem0)
|
1. **Fork & Clone** the repository: [Mem0 on GitHub](https://github.com/mem0ai/mem0)
|
||||||
2. **Create a Feature Branch**: Use a dedicated branch, e.g., `feature/my-new-feature`
|
2. **Create a Feature Branch**: Use a dedicated branch for your changes, e.g., `feature/my-new-feature`
|
||||||
3. **Implement Changes**: If adding a feature or fixing a bug, be sure to:
|
3. **Implement Changes**: If adding a feature or fixing a bug, ensure to:
|
||||||
- Write necessary **tests**
|
- Write necessary **tests**
|
||||||
- Add **documentation, docstrings, and runnable examples**
|
- Add **documentation, docstrings, and runnable examples**
|
||||||
4. **Code Quality Checks**:
|
4. **Code Quality Checks**:
|
||||||
- Run **linting** to catch style issues
|
- Run **linting** to catch style issues
|
||||||
- Ensure **all tests pass**
|
- Ensure **all tests pass**
|
||||||
5. **Commit** using [Conventional Commits](https://www.conventionalcommits.org/)
|
5. **Submit a Pull Request**
|
||||||
(`feat:`, `fix:`, `docs:`, `refactor:`, `test:`)
|
|
||||||
6. **Submit a Pull Request** against `main`, linking the issue and filling out the
|
|
||||||
PR template.
|
|
||||||
|
|
||||||
For detailed guidance on pull requests, refer to [GitHub's documentation](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).
|
For detailed guidance on pull requests, refer to [GitHub's documentation](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Python SDK (`mem0/`)
|
## Dependency Management
|
||||||
|
|
||||||
### Dependency Management
|
|
||||||
|
|
||||||
We use `hatch` as our package manager. Install it by following the [official instructions](https://hatch.pypa.io/latest/install/).
|
We use `hatch` as our package manager. Install it by following the [official instructions](https://hatch.pypa.io/latest/install/).
|
||||||
|
|
||||||
@@ -77,9 +44,13 @@ hatch -e dev_py_3_11 shell # For dev_py_3_11 (differences are mentioned in pypr
|
|||||||
make install_all
|
make install_all
|
||||||
```
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Development Standards
|
||||||
|
|
||||||
### Pre-commit Hooks
|
### Pre-commit Hooks
|
||||||
|
|
||||||
Ensure `pre-commit` is installed before contributing (hooks run ruff + isort):
|
Ensure `pre-commit` is installed before contributing:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pre-commit install
|
pre-commit install
|
||||||
@@ -87,7 +58,7 @@ pre-commit install
|
|||||||
|
|
||||||
### Linting with `ruff`
|
### Linting with `ruff`
|
||||||
|
|
||||||
Run the linter and fix any reported issues before submitting your PR (line length **120**):
|
Run the linter and fix any reported issues before submitting your PR:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
make lint
|
make lint
|
||||||
@@ -95,11 +66,10 @@ make lint
|
|||||||
|
|
||||||
### Code Formatting
|
### Code Formatting
|
||||||
|
|
||||||
To maintain a consistent code style, format your code and sort imports (isort, `profile = "black"`):
|
To maintain a consistent code style, format your code:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
make format
|
make format
|
||||||
make sort
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Testing with `pytest`
|
### Testing with `pytest`
|
||||||
@@ -114,45 +84,9 @@ make test
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## TypeScript SDK (`mem0-ts/`)
|
|
||||||
|
|
||||||
We use [`pnpm`](https://pnpm.io/) (v10+) for all TypeScript packages. **Do NOT use
|
|
||||||
`npm` or `yarn`.**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd mem0-ts
|
|
||||||
pnpm install
|
|
||||||
|
|
||||||
pnpm run build # tsup (CJS + ESM)
|
|
||||||
pnpm run test # jest (all tests)
|
|
||||||
pnpm run test:unit # unit tests with coverage
|
|
||||||
```
|
|
||||||
|
|
||||||
### Standards
|
|
||||||
|
|
||||||
- **Build:** tsup
|
|
||||||
- **Formatter:** Prettier
|
|
||||||
- **Tests:** jest
|
|
||||||
- Always run type checking after changes: `pnpm run typecheck` (or `tsc --noEmit`)
|
|
||||||
- Use ES module `import` syntax — never `require()`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Reporting Security Issues
|
|
||||||
|
|
||||||
**Do not report security vulnerabilities through public issues or pull requests.**
|
|
||||||
Please follow our [Security Policy](https://github.com/mem0ai/mem0/blob/main/SECURITY.md)
|
|
||||||
to report them privately.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Release Process
|
## Release Process
|
||||||
|
|
||||||
Packages are published automatically via GitHub Actions when a GitHub Release is
|
Currently, releases are handled manually. We aim for frequent releases, typically when new features or bug fixes are introduced.
|
||||||
created with the correct tag prefix (e.g. `v*` for the Python SDK, `ts-v*` for the
|
|
||||||
TypeScript SDK). See
|
|
||||||
[CONTRIBUTING.md](https://github.com/mem0ai/mem0/blob/main/CONTRIBUTING.md#releasing)
|
|
||||||
for the full tag-prefix table and publishing details.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import type { Client as ClientType, ClientConfig } from "pg";
|
import type { Client as ClientType } from "pg";
|
||||||
import pkg from "pg";
|
import pkg from "pg";
|
||||||
const { Client, escapeIdentifier } = pkg;
|
const { Client, escapeIdentifier } = pkg;
|
||||||
import { VectorStore } from "./base";
|
import { VectorStore } from "./base";
|
||||||
@@ -157,90 +157,41 @@ export function buildFilterConditions(
|
|||||||
|
|
||||||
interface PGVectorConfig extends VectorStoreConfig {
|
interface PGVectorConfig extends VectorStoreConfig {
|
||||||
dbname?: string;
|
dbname?: string;
|
||||||
user?: string;
|
user: string;
|
||||||
password?: string;
|
password: string;
|
||||||
host?: string;
|
host: string;
|
||||||
port?: number;
|
port: number;
|
||||||
connectionString?: string;
|
|
||||||
ssl?: ClientConfig["ssl"];
|
|
||||||
embeddingModelDims: number;
|
embeddingModelDims: number;
|
||||||
diskann?: boolean;
|
diskann?: boolean;
|
||||||
hnsw?: boolean;
|
hnsw?: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
function getConnectionString(config: PGVectorConfig): string | undefined {
|
|
||||||
return config.connectionString?.trim() || undefined;
|
|
||||||
}
|
|
||||||
|
|
||||||
function validateConnectionConfig(config: PGVectorConfig): void {
|
|
||||||
if (getConnectionString(config)) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
const missingFields = ["user", "password", "host", "port"].filter((field) => {
|
|
||||||
const v = config[field as keyof PGVectorConfig];
|
|
||||||
return v === undefined || v === null || v === "";
|
|
||||||
});
|
|
||||||
|
|
||||||
if (missingFields.length > 0) {
|
|
||||||
throw new Error(
|
|
||||||
`PGVector requires either connectionString or ${missingFields.join(", ")}`,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function buildClientConfig(
|
|
||||||
config: PGVectorConfig,
|
|
||||||
database?: string,
|
|
||||||
): ClientConfig {
|
|
||||||
const connectionString = getConnectionString(config);
|
|
||||||
if (connectionString) {
|
|
||||||
return {
|
|
||||||
connectionString,
|
|
||||||
...(config.ssl !== undefined ? { ssl: config.ssl } : {}),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
return {
|
|
||||||
database,
|
|
||||||
user: config.user,
|
|
||||||
password: config.password,
|
|
||||||
host: config.host,
|
|
||||||
port: config.port,
|
|
||||||
...(config.ssl !== undefined ? { ssl: config.ssl } : {}),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
export class PGVector implements VectorStore {
|
export class PGVector implements VectorStore {
|
||||||
private client: ClientType;
|
private client: ClientType;
|
||||||
private collectionName: string;
|
private collectionName: string;
|
||||||
private useDiskann: boolean;
|
private useDiskann: boolean;
|
||||||
private useHnsw: boolean;
|
private useHnsw: boolean;
|
||||||
private readonly dbName: string;
|
private readonly dbName: string;
|
||||||
private readonly useDirectConnection: boolean;
|
|
||||||
private config: PGVectorConfig;
|
private config: PGVectorConfig;
|
||||||
private _initPromise?: Promise<void>;
|
private _initPromise?: Promise<void>;
|
||||||
|
|
||||||
constructor(config: PGVectorConfig) {
|
constructor(config: PGVectorConfig) {
|
||||||
validateConnectionConfig(config);
|
|
||||||
this.collectionName = validateIdentifier(
|
this.collectionName = validateIdentifier(
|
||||||
config.collectionName || "memories",
|
config.collectionName || "memories",
|
||||||
"collectionName",
|
"collectionName",
|
||||||
);
|
);
|
||||||
this.useDiskann = config.diskann || false;
|
this.useDiskann = config.diskann || false;
|
||||||
this.useHnsw = config.hnsw || false;
|
this.useHnsw = config.hnsw || false;
|
||||||
this.useDirectConnection = !!getConnectionString(config);
|
this.dbName = validateIdentifier(config.dbname || "vector_store", "dbname");
|
||||||
this.dbName = this.useDirectConnection
|
|
||||||
? ""
|
|
||||||
: validateIdentifier(config.dbname || "vector_store", "dbname");
|
|
||||||
this.config = config;
|
this.config = config;
|
||||||
|
|
||||||
this.client = new Client(
|
this.client = new Client({
|
||||||
buildClientConfig(
|
database: "postgres", // Initially connect to default postgres database
|
||||||
config,
|
user: config.user,
|
||||||
this.useDirectConnection ? undefined : "postgres",
|
password: config.password,
|
||||||
),
|
host: config.host,
|
||||||
);
|
port: config.port,
|
||||||
|
});
|
||||||
this.initialize().catch(console.error);
|
this.initialize().catch(console.error);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -259,20 +210,29 @@ export class PGVector implements VectorStore {
|
|||||||
try {
|
try {
|
||||||
await this.client.connect();
|
await this.client.connect();
|
||||||
|
|
||||||
if (!this.useDirectConnection) {
|
// Check if database exists
|
||||||
const dbExists = await this.checkDatabaseExists(this.dbName);
|
const dbExists = await this.checkDatabaseExists(this.dbName);
|
||||||
if (!dbExists) {
|
if (!dbExists) {
|
||||||
await this.createDatabase(this.dbName);
|
await this.createDatabase(this.dbName);
|
||||||
}
|
|
||||||
|
|
||||||
await this.client.end();
|
|
||||||
|
|
||||||
this.client = new Client(buildClientConfig(this.config, this.dbName));
|
|
||||||
await this.client.connect();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Disconnect from postgres database
|
||||||
|
await this.client.end();
|
||||||
|
|
||||||
|
// Connect to the target database
|
||||||
|
this.client = new Client({
|
||||||
|
database: this.dbName,
|
||||||
|
user: this.config.user,
|
||||||
|
password: this.config.password,
|
||||||
|
host: this.config.host,
|
||||||
|
port: this.config.port,
|
||||||
|
});
|
||||||
|
await this.client.connect();
|
||||||
|
|
||||||
|
// Create vector extension
|
||||||
await this.client.query("CREATE EXTENSION IF NOT EXISTS vector");
|
await this.client.query("CREATE EXTENSION IF NOT EXISTS vector");
|
||||||
|
|
||||||
|
// Create memory_migrations table
|
||||||
await this.client.query(`
|
await this.client.query(`
|
||||||
CREATE TABLE IF NOT EXISTS memory_migrations (
|
CREATE TABLE IF NOT EXISTS memory_migrations (
|
||||||
id SERIAL PRIMARY KEY,
|
id SERIAL PRIMARY KEY,
|
||||||
@@ -280,6 +240,7 @@ export class PGVector implements VectorStore {
|
|||||||
)
|
)
|
||||||
`);
|
`);
|
||||||
|
|
||||||
|
// Check if the collection exists
|
||||||
const collections = await this.listCols();
|
const collections = await this.listCols();
|
||||||
if (!collections.includes(this.collectionName)) {
|
if (!collections.includes(this.collectionName)) {
|
||||||
await this.createCol(this.config.embeddingModelDims);
|
await this.createCol(this.config.embeddingModelDims);
|
||||||
|
|||||||
@@ -1,3 +1,5 @@
|
|||||||
|
/// <reference types="jest" />
|
||||||
|
|
||||||
jest.mock("pg", () => {
|
jest.mock("pg", () => {
|
||||||
const Client = jest.fn().mockImplementation(() => ({
|
const Client = jest.fn().mockImplementation(() => ({
|
||||||
connect: jest.fn().mockResolvedValue(undefined),
|
connect: jest.fn().mockResolvedValue(undefined),
|
||||||
|
|||||||
@@ -1,3 +1,5 @@
|
|||||||
|
/// <reference types="jest" />
|
||||||
|
|
||||||
const searchRows = [
|
const searchRows = [
|
||||||
{
|
{
|
||||||
id: "a",
|
id: "a",
|
||||||
@@ -21,13 +23,9 @@ const searchRows = [
|
|||||||
},
|
},
|
||||||
];
|
];
|
||||||
|
|
||||||
const mockState = {
|
|
||||||
databaseExists: true,
|
|
||||||
};
|
|
||||||
|
|
||||||
function mockPgQuery(sql: string) {
|
function mockPgQuery(sql: string) {
|
||||||
if (sql.includes("SELECT 1 FROM pg_database")) {
|
if (sql.includes("SELECT 1 FROM pg_database")) {
|
||||||
return { rows: mockState.databaseExists ? [{ "?column?": 1 }] : [] };
|
return { rows: [{ "?column?": 1 }] };
|
||||||
}
|
}
|
||||||
|
|
||||||
if (sql.includes("FROM information_schema.tables")) {
|
if (sql.includes("FROM information_schema.tables")) {
|
||||||
@@ -71,129 +69,13 @@ jest.mock("pg", () => {
|
|||||||
|
|
||||||
import { PGVector } from "../src/vector_stores/pgvector";
|
import { PGVector } from "../src/vector_stores/pgvector";
|
||||||
|
|
||||||
function getClientQueries(client: { query: jest.Mock }) {
|
describe("PGVector - search()", () => {
|
||||||
return client.query.mock.calls.map(([sql]) => sql as string);
|
|
||||||
}
|
|
||||||
|
|
||||||
describe("PGVector", () => {
|
|
||||||
beforeEach(() => {
|
beforeEach(() => {
|
||||||
const pg = require("pg");
|
const pg = require("pg");
|
||||||
mockState.databaseExists = true;
|
|
||||||
pg.__mock.Client.mockClear();
|
pg.__mock.Client.mockClear();
|
||||||
pg.__mock.clients.length = 0;
|
pg.__mock.clients.length = 0;
|
||||||
});
|
});
|
||||||
|
|
||||||
test("uses one direct client for connectionString mode and skips bootstrap database creation", async () => {
|
|
||||||
mockState.databaseExists = false;
|
|
||||||
|
|
||||||
const ssl = { rejectUnauthorized: false };
|
|
||||||
const store = new PGVector({
|
|
||||||
collectionName: "memories",
|
|
||||||
connectionString:
|
|
||||||
"postgresql://postgres:postgres@db.example.com:5432/neondb",
|
|
||||||
ssl,
|
|
||||||
embeddingModelDims: 3,
|
|
||||||
dimension: 3,
|
|
||||||
} as any);
|
|
||||||
|
|
||||||
await store.initialize();
|
|
||||||
|
|
||||||
const pg = require("pg");
|
|
||||||
expect(pg.__mock.Client).toHaveBeenCalledTimes(1);
|
|
||||||
expect(pg.__mock.Client).toHaveBeenCalledWith({
|
|
||||||
connectionString:
|
|
||||||
"postgresql://postgres:postgres@db.example.com:5432/neondb",
|
|
||||||
ssl,
|
|
||||||
});
|
|
||||||
|
|
||||||
const directClient = pg.__mock.clients[0];
|
|
||||||
const queries = getClientQueries(directClient);
|
|
||||||
|
|
||||||
expect(queries).not.toEqual(
|
|
||||||
expect.arrayContaining([
|
|
||||||
expect.stringContaining("SELECT 1 FROM pg_database"),
|
|
||||||
]),
|
|
||||||
);
|
|
||||||
expect(queries).not.toEqual(
|
|
||||||
expect.arrayContaining([expect.stringContaining("CREATE DATABASE")]),
|
|
||||||
);
|
|
||||||
expect(queries).toEqual(
|
|
||||||
expect.arrayContaining([
|
|
||||||
"CREATE EXTENSION IF NOT EXISTS vector",
|
|
||||||
expect.stringContaining("FROM information_schema.tables"),
|
|
||||||
]),
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("keeps the split-field bootstrap flow when connectionString is absent", async () => {
|
|
||||||
mockState.databaseExists = false;
|
|
||||||
const ssl = { rejectUnauthorized: false };
|
|
||||||
|
|
||||||
const store = new PGVector({
|
|
||||||
collectionName: "memories",
|
|
||||||
user: "postgres",
|
|
||||||
password: "postgres",
|
|
||||||
host: "localhost",
|
|
||||||
port: 5432,
|
|
||||||
dbname: "vector_store",
|
|
||||||
ssl,
|
|
||||||
embeddingModelDims: 3,
|
|
||||||
dimension: 3,
|
|
||||||
} as any);
|
|
||||||
|
|
||||||
await store.initialize();
|
|
||||||
|
|
||||||
const pg = require("pg");
|
|
||||||
expect(pg.__mock.Client).toHaveBeenCalledTimes(2);
|
|
||||||
expect(pg.__mock.Client).toHaveBeenNthCalledWith(1, {
|
|
||||||
database: "postgres",
|
|
||||||
user: "postgres",
|
|
||||||
password: "postgres",
|
|
||||||
host: "localhost",
|
|
||||||
port: 5432,
|
|
||||||
ssl,
|
|
||||||
});
|
|
||||||
expect(pg.__mock.Client).toHaveBeenNthCalledWith(2, {
|
|
||||||
database: "vector_store",
|
|
||||||
user: "postgres",
|
|
||||||
password: "postgres",
|
|
||||||
host: "localhost",
|
|
||||||
port: 5432,
|
|
||||||
ssl,
|
|
||||||
});
|
|
||||||
|
|
||||||
const bootstrapClient = pg.__mock.clients[0];
|
|
||||||
const activeClient = pg.__mock.clients[1];
|
|
||||||
const bootstrapQueries = getClientQueries(bootstrapClient);
|
|
||||||
|
|
||||||
expect(bootstrapQueries).toEqual(
|
|
||||||
expect.arrayContaining([
|
|
||||||
"SELECT 1 FROM pg_database WHERE datname = $1",
|
|
||||||
'CREATE DATABASE "vector_store"',
|
|
||||||
]),
|
|
||||||
);
|
|
||||||
expect(bootstrapClient.end).toHaveBeenCalledTimes(1);
|
|
||||||
expect(getClientQueries(activeClient)).toEqual(
|
|
||||||
expect.arrayContaining([
|
|
||||||
"CREATE EXTENSION IF NOT EXISTS vector",
|
|
||||||
expect.stringContaining("FROM information_schema.tables"),
|
|
||||||
]),
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("throws when connectionString is absent and split-field params are missing", () => {
|
|
||||||
expect(
|
|
||||||
() =>
|
|
||||||
new PGVector({
|
|
||||||
collectionName: "memories",
|
|
||||||
embeddingModelDims: 3,
|
|
||||||
dimension: 3,
|
|
||||||
} as any),
|
|
||||||
).toThrow(
|
|
||||||
"PGVector requires either connectionString or user, password, host, port",
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("returns similarity score (1 - distance) clamped to [0, 1]", async () => {
|
test("returns similarity score (1 - distance) clamped to [0, 1]", async () => {
|
||||||
const store = new PGVector({
|
const store = new PGVector({
|
||||||
collectionName: "memories",
|
collectionName: "memories",
|
||||||
|
|||||||
Reference in New Issue
Block a user