Compare commits

..

31 Commits

Author SHA1 Message Date
kartik-mem0 271c7c23c5 fix(mem0-plugin): remove noisy session-state dumps, match openclaw pattern
- Remove all background REST API transcript dumps (on_pre_compact.py,
  capture_compact_summary.py, on_pre_commit.py calls) that were creating
  noisy session_state and commit_context memories automatically
- Rewrite pre-compaction prompt to extract individual durable facts
  (decisions, learnings, anti-patterns) instead of session summary blobs
- Agent now decides what to store, matching openclaw's triage pattern:
  "Most sessions produce zero memory operations. That is correct."
- Fix auto_import.py to search git repo root for CLAUDE.md/AGENTS.md
  (was only searching plugin cwd, missing files in parent directory)
- Fix health check session tracker to check stats file directly instead
  of depending on CLAUDE_PLUGIN_ROOT env var
- Fix session-end report to print in SessionEnd hook (Stop hook output
  doesn't render on /exit)
- Update v2 API endpoint to v3 in git commit capture search
- Keep auto_import.py as only direct REST write path (project profiles,
  idempotent, hash-gated)
2026-05-22 16:20:31 +05:30
kartik-mem0 c2290299cf docs: add session stats file check and explanation to mem0-health skill 2026-05-22 15:33:59 +05:30
kartik-mem0 1b3a57e8ec refactor: update API endpoint to v3 in git commit capture script 2026-05-22 15:28:50 +05:30
kartik-mem0 245a70d90f refactor: update mem0-plugin scripts to use tool_output instead of tool_result 2026-05-22 14:53:56 +05:30
kartik-mem0 71a6696c87 refactor(mem0-plugin): improve hook scripts, config parsing, and skill definitions
Update git commit capture with better error handling, enhance user prompt hook,
extend config parser capabilities, and refine skill docs for dream, health,
mcp, pin, and stats skills.
2026-05-22 14:23:06 +05:30
kartik-mem0 6ab6ed58a8 refactor: update session start script and mem0-tour skill to display full memory content and use 2026-05-22 13:54:04 +05:30
kartik-mem0 1853543ceb Merge remote-tracking branch 'origin/main' into feat/mem0-plugin 2026-05-22 13:35:12 +05:30
kartik-mem0 93bb3b69c4 feat(mem0-plugin): add PostToolUseFailure, PostCompact, SessionEnd, SubagentStop hooks
Close 4 lifecycle coverage gaps:
- PostToolUseFailure: catch failed mem0 MCP calls, classify errors, inject retry guidance
- PostCompact: inject parallel search commands for context recovery after compaction
- SessionEnd: last-chance transcript capture with dedup against Stop hook
- SubagentStop: remind parent agent to persist subagent learnings

Also patch on_stop.sh to write capture marker for SessionEnd dedup.
2026-05-22 01:09:22 +05:30
kartik-mem0 97a3a2cf8d feat(mem0-plugin): complete all Tier 1-8 spec items, add scheduled dream and bash error detection
- Add /mem0:dream --schedule support via Claude Code /schedule command
  with local launchd/cron fallback for non-cloud users
- Add /mem0:digest --schedule and file output to ~/.mem0/weekly-digest.md
  with digest-history.log for trend tracking
- Add on_post_commit.sh PostToolUse hook: interactive prompt after git
  commit asking user to save change as memory with category suggestion
- Add on_bash_output.sh PostToolUse hook: detects stack traces and error
  patterns in bash command output, injects mem0 search rubric for prior
  occurrences (complements existing user prompt error detection)
- Fix session-end report: now shows per-category breakdown
  "wrote 3 memories (decision 2, anti_pattern 1)" matching spec
- Wire new hooks into all 3 platforms (Claude Code, Cursor, Codex)
- Add post_commit and bash_error telemetry events
- Add CTO demo testing guide with tier-by-tier manual test steps
2026-05-21 23:55:06 +05:30
kartik-mem0 b49626ab11 fix(mem0-plugin): wire pre-commit capture hook, add Setup hook, fix access counters
- Add PreToolUse(Bash) hook: detects `git commit` commands, fires
  on_pre_commit.py in background to capture staged changes as memory
- Add Setup(init|maintenance) hook: installs mem0ai SDK on first plugin
  install via ensure_deps.sh (120s timeout for slow networks)
- Add on_git_commit_capture.sh wrapper: reads tool input JSON, matches
  git commit/merge/rebase, runs capture script fire-and-forget
- Fix #24: add metadata.last_accessed ISO timestamp to access counter
  instructions in mem0-mcp/SKILL.md alongside existing access_count
2026-05-21 23:02:32 +05:30
kartik-mem0 eb1cdabe61 fix(mem0-plugin): remove userConfig from plugin.json, add auto-deps install
- Remove userConfig from .claude-plugin/plugin.json — OAuth flow handles
  auth via mcp.mem0.ai, userConfig prompt was redundant and confusing
- Add ensure_deps.sh to install mem0ai SDK into persistent plugin venv
- Add requirements.txt (mem0ai) for dependency tracking
- Wire SessionStart hook to auto-install deps on first run
- Update setup_coding_categories.py to find venv site-packages
- Update onboarding SKILL.md with venv-aware install steps
2026-05-21 22:52:27 +05:30
kartik-mem0 141782a230 fix(mem0-plugin): use env var in .mcp.json, fix onboarding MCP check
- .mcp.json: use ${MEM0_API_KEY} instead of ${user_config.MEM0_API_KEY}
  so MCP connects for --plugin-dir users with the key in their shell
- mem0-onboard: check MCP tools availability first, stop early with
  clear instructions if MCP server not connected
2026-05-21 21:29:19 +05:30
kartik-mem0 d5b15cebb7 feat(mem0-plugin): add PostHog telemetry across all hooks
Wire fire-and-forget PostHog telemetry into every lifecycle hook
(session_start, stop, pre_compact, task_completed, user_prompt,
post_tool_use) matching the same project key, endpoint, and payload
structure used by the Python SDK, CLI, and OpenClaw.

- 10% sampling on ALL events to keep PostHog costs predictable
- Anonymous distinct_id: MD5(api_key) or SHA-256(username)
- Opt-out via MEM0_TELEMETRY=false
- Platform detection: Claude Code / Cursor / Codex
- Never sends user content, API keys, or raw identifiers
- 16 new tests (110 total passing)
2026-05-21 21:08:21 +05:30
kartik-mem0 c5bb11b8b1 fix(mem0-plugin): graceful degradation without API key, stdin support
- SessionStart shows "Mem0 Inactive" banner with identity info when no
  API key set, instead of silently exiting with zero output
- on_user_prompt.sh fires error/file-path detection even without API key
  (local pattern matching, no API needed). Search rubric still gated.
- parse_export_file.py supports '-' argument for stdin piping
2026-05-21 19:13:13 +05:30
kartik-mem0 1daa7ece66 fix(mem0-plugin): fix 3 critical + 5 important issues from review
Critical fixes:
- C1: Pin/dream update_memory now reads content first via get_memory
  before updating, preventing metadata-only calls that wipe content
- C2: SessionStart memory count uses GET /v3/memories/?page_size=1 for
  real count from pagination envelope instead of search limit=100
- C3: on_task_completed.sh uses ${MEM0_PROJECT_ID:-unknown} to avoid
  crash when identity resolution fails under set -u

Important fixes:
- I1: session_stats.py tracks recent_ids (memory IDs + timestamps) for
  /mem0:forget undo support. Capped at 50 entries.
- I2: /mem0:health --deep mode adds memory quality analysis: duplicate
  detection, stale entry scan, contradiction flags, orphan tagging
- I3: /mem0:stats ASCII histogram for category distribution with bar
  chart, sorted by count descending
- I4: mem0-mcp skill documents access counter pattern: get_memory then
  update_memory to track which memories are actually used
- I5: on_pre_commit.py reads COMMIT_EDITMSG first (current commit),
  falls back to HEAD only if EDITMSG unavailable

Tests: 91 -> 94 (3 new for recent_ids tracking)
2026-05-21 18:46:08 +05:30
kartik-mem0 70f2f60727 chore(mem0-plugin): remove TESTING.md from tracking
Manual testing guide will be local-only, not committed to repo.
2026-05-21 18:42:02 +05:30
kartik-mem0 35da92ba88 docs(mem0-plugin): comprehensive TESTING.md covering all Tiers 1-8
Rewrote TESTING.md with behavioral tests for every tier item:
- v3 endpoint verification for all 5 write scripts + search
- app_id scoping and branch tagging checks
- Skill content validation (confidence, citations, recency, infer=False)
- Robustness checks (loop guard, set flags, fallback stubs)
- 28-item checklist covering all 34 roadmap items
- Fixed stale comment in on_stop.sh
2026-05-21 18:40:48 +05:30
kartik-mem0 4b94902446 fix(mem0-plugin): review fixes — loop guard, v3 response format, robustness
- on_stop_cursor.sh: add loop_count guard to prevent infinite re-entry
- on_stop.sh: drop set -e so stats failure doesn't kill the reminder
- _identity.py: fix fallback stub signatures to match production (accept cwd arg)
- on_pre_compact.py: add missing infer=False on session_state writes
- on_session_start.sh: handle v3 search response format (dict with results key)
- Align all plugin versions to 0.2.1 (Claude Code, Codex, Cursor)
2026-05-21 18:27:47 +05:30
kartik-mem0 470cef6ba6 fix(mem0-plugin): upgrade REST endpoints to v3, fix dream scheduling docs
- All add_memory REST calls now use /v3/memories/add/ (was /v1/memories/)
- Session start search uses /v3/memories/search/ (was /v2/memories/search/)
- Dream skill: removed reference to nonexistent /schedule skill
  (Claude Code has no native scheduling — recommend external cron instead)
2026-05-21 18:14:19 +05:30
kartik-mem0 930f6ec51d feat(mem0-plugin): power-user skills + expanded mem0.md config (Tier 8)
Power-user tier:
- /mem0:remember: quick-add with auto-classification, infer=False, confidence=1.0
- /mem0:forget: search-confirm-delete flow with numbered list
- /mem0:pin: mark memories as pinned via metadata update
- /mem0:peek: compact one-liner search results
- parse_mem0_config.py: expanded to parse ## Search, ## Categories, ## Identity
- on_session_start.sh: inject mem0.md project config into session context
2026-05-21 18:14:09 +05:30
kartik-mem0 cd66050748 feat(mem0-plugin): smart retrieval, citations, pre-commit capture (Tier 7)
Smart retrieval tier:
- on_user_prompt.sh: stack-trace detection (auto-searches anti_patterns on errors)
- on_user_prompt.sh: file-aware retrieval (extracts file paths, suggests scoped search)
- on_user_prompt.sh: recency boost guidance for state-related queries
- mem0-mcp/SKILL.md: inline citation format [mem0:<8char_id>]
- on_pre_commit.py: captures staged diff as commit_context memory via REST API
2026-05-21 18:13:59 +05:30
kartik-mem0 81211be473 feat(mem0-plugin): add /mem0:stats, /mem0:health, /mem0:digest skills (Tier 6)
Observability tier:
- /mem0:stats: session + lifetime memory dashboard with category breakdown
- /mem0:health: 5-check diagnostic (API key, identity, MCP, write/read, tracker)
- /mem0:digest: weekly memory activity summary with time-scoped searches
- session_stats.py: add `peek` command (JSON without clearing), per-category counters
2026-05-21 18:13:44 +05:30
kartik-mem0 82ab253f2c fix(mem0-plugin): sanitize marker path, add pre-compact REST backup, fix cross-tool gaps
- Sanitize project_id in onboard marker path (tr '/:' '--') to handle
  slashes from env var overrides
- Fire on_pre_compact.py REST backup from PreCompact hook (was only
  called from Stop hooks — crash before Stop = data loss)
- Source _identity.sh in on_stop_codex.sh and on_stop_cursor.sh so
  MEM0_PROJECT_ID + API key fallback are available
- Note dream --schedule is Claude Code-only (Codex/Cursor lack schedule)
2026-05-21 17:43:40 +05:30
kartik-mem0 63641f5573 feat(mem0-plugin): add export/import skills + competing tool importers
Tier 3 implementation:
- #10: Unified project_id resolver already complete (shared scripts/)
- #11: /mem0:export dumps all project memories as YAML-frontmatter
  markdown; /mem0:import restores from same format. Round-trip capable.
  parse_export_file.py handles parsing (stdlib-only).
- #12: import_competing_tools.py imports from .cursorrules, copilot
  instructions, cline memory-bank, continue.dev rules. Splits by
  section headers, POSTs with app_id + infer=False.
2026-05-21 17:43:30 +05:30
kartik-mem0 b79be63e30 feat(mem0-plugin): add /mem0:dream consolidation + retention policies
Tier 4 implementation:
- #13+15: /mem0:dream skill — fetches all memories, finds near-dupes,
  proposes merges, flags contradictions, prunes stale entries per
  retention policy, outputs terminal diff for user approval
- #14: --schedule weekly registers cron via Claude Code schedule skill
  (Claude Code only, noted in docs)
- #16: parse_mem0_config.py parses ## Retention section from mem0.md
  (e.g. session_state: 90d, architecture_decisions: forever)
2026-05-21 17:43:14 +05:30
kartik-mem0 6ed5fcf76e feat(mem0-plugin): expand categories to 17, add recall rubric + confidence + files tagging
Tier 5 implementation:
- #17: Expand coding categories from 7 to 17 (dependency_decisions,
  performance_findings, security_constraints, testing_patterns, data_model,
  api_contracts, deployment_runbook, team_norms, domain_glossary,
  experiment_results)
- #18: Category-recall rubric table in mem0-mcp/SKILL.md mapping user
  intent to 2-3 categories for parallel search fan-out
- #19: metadata.confidence scoring rules (1.0/0.8/0.5/0.3 scale)
- #20: metadata.files array tagging for affected file paths
2026-05-21 17:43:01 +05:30
kartik-mem0 f6c3432e19 fix(mem0-plugin): fix userConfig schema and restore ${user_config.*} in .mcp.json
userConfig was missing required `type` and `title` fields per Claude Code
plugin docs — value was never collected, so ${user_config.MEM0_API_KEY}
could not resolve. Added both fields. Restored .mcp.json to use
${user_config.MEM0_API_KEY} for the MCP server Authorization header.

This completes the fix for #4876: plugin now prompts for API key at
install, stores in keychain, and MCP server reads it via user_config
interpolation — no shell env var needed.
2026-05-21 17:15:09 +05:30
kartik-mem0 15916a78e0 fix(mem0-plugin): revert .mcp.json to ${MEM0_API_KEY} env var
${user_config.*} interpolation not supported in HTTP-type MCP server
headers — only works for subprocess-based servers. Revert to shell
env var reference. The userConfig + resolve_api_key() fallback chain
still fixes hook scripts for issue #4876.
2026-05-21 17:13:11 +05:30
kartik-mem0 ff2f6c281a fix(mem0-plugin): resolve API key from userConfig, fix #4876
Root cause: .mcp.json required ${MEM0_API_KEY} shell env var which
OAuth flow never populates — MCP server never started, only auth
stubs were exposed.

Fix: switch .mcp.json to ${user_config.MEM0_API_KEY} so Claude Code
reads the key from plugin userConfig (system keychain). Add
resolve_api_key() fallback chain (MEM0_API_KEY -> CLAUDE_PLUGIN_OPTION_MEM0_API_KEY)
to all hook scripts. Update onboard skill and setup_coding_categories.py
to handle keychain-only users. Add 3 tests for key resolution.
2026-05-21 17:10:17 +05:30
kartik-mem0 66600565ed fix(mem0-plugin): thread cwd to resolvers, add write-path tests, userConfig for API key
- on_pre_compact.py and capture_compact_summary.py now read cwd from
  hook stdin and pass it to resolve_project_id/resolve_branch (monorepo safe)
- 5 new tests verify all write functions use app_id top-level and never
  put project_id in metadata
- plugin.json declares userConfig.MEM0_API_KEY so Claude Code prompts
  for the key at install time (sensitive, stored in system keychain)
2026-05-21 16:50:11 +05:30
kartik-mem0 cf6db6ef88 fix(mem0-plugin): use app_id entity scoping, fix tour, add branch tagging
Tour was filtering on metadata.type which missed platform-categorized
memories. Rewritten to use get_memories (list all) + group by platform
categories field. All write hooks now use app_id as top-level entity
param instead of metadata.project_id. Branch stamped on every write.
2026-05-21 16:38:48 +05:30
769 changed files with 49187 additions and 15041 deletions
+1 -1
View File
@@ -12,7 +12,7 @@
"name": "mem0",
"source": "./mem0-plugin",
"description": "Mem0 memory layer for AI applications. Add persistent memory, personalization, and semantic search to Claude workflows.",
"version": "0.2.8"
"version": "0.1.3"
}
]
}
-20
View File
@@ -1,20 +0,0 @@
{
"name": "mem0-plugins",
"interface": {
"displayName": "Mem0 Plugins"
},
"plugins": [
{
"name": "mem0",
"source": {
"source": "local",
"path": "./mem0-plugin"
},
"policy": {
"installation": "AVAILABLE",
"authentication": "ON_INSTALL"
},
"category": "Productivity"
}
]
}
+1 -1
View File
@@ -12,7 +12,7 @@
"name": "mem0",
"source": "./mem0-plugin",
"description": "Mem0 memory layer for AI applications. Add persistent memory, personalization, and semantic search.",
"version": "0.2.8"
"version": "0.1.1"
}
]
}
-44
View File
@@ -1,44 +0,0 @@
name: Publish @mem0/opencode-plugin 📦 to npm
on:
release:
types: [published]
jobs:
build-n-publish:
name: Build and publish @mem0/opencode-plugin 📦 to npm
if: startsWith(github.event.release.tag_name, 'opencode-v')
runs-on: ubuntu-latest
permissions:
id-token: write
defaults:
run:
working-directory: mem0-plugin/.opencode-plugin
steps:
- uses: actions/checkout@v4
- name: Install Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
registry-url: 'https://registry.npmjs.org'
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Build
run: bun run build
- name: Publish to npm
run: |
if [ "${{ github.event.release.prerelease }}" = "true" ]; then
PREID=$(node -p "require('./package.json').version.split('-')[1].split('.')[0]")
npx npm@latest publish --provenance --access public --tag "$PREID"
else
npx npm@latest publish --provenance --access public
fi
@@ -1,40 +0,0 @@
name: opencode-plugin checks
on:
workflow_dispatch:
push:
branches: [main]
paths:
- 'mem0-plugin/.opencode-plugin/**'
- '.github/workflows/opencode-plugin-checks.yml'
pull_request:
paths:
- 'mem0-plugin/.opencode-plugin/**'
- '.github/workflows/opencode-plugin-checks.yml'
jobs:
build:
runs-on: ubuntu-latest
defaults:
run:
working-directory: mem0-plugin/.opencode-plugin
steps:
- uses: actions/checkout@v4
- name: Install Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Type check
run: bun run type-check
- name: Build
run: bun run build
- name: Verify dist output exists
run: |
test -f dist/index.js || (echo "Build output missing: dist/index.js" && exit 1)
+1
View File
@@ -170,6 +170,7 @@ cython_debug/
# Database
db
test-db
!embedchain/embedchain/core/db/
.vscode
.idea/
+4 -3
View File
@@ -33,6 +33,7 @@ This is a **polyglot monorepo** containing Python and TypeScript packages, CLIs,
| `evaluation/` | Benchmarking framework — LOCOMO evals, experiment runner, score generation |
| `examples/` | Sample projects — demo apps, Chrome extension, multi-agent patterns |
| `cookbooks/` | Jupyter notebooks — customer support chatbot, AutoGen integration |
| `embedchain/` | Legacy Embedchain RAG framework (maintained separately, Poetry-based) |
| `pr-reviews/` | Pull request review materials |
| `scripts/` | Repo-wide utility scripts (e.g., `check-llms-txt-coverage.py` for docs/llms.txt sync) |
@@ -329,7 +330,7 @@ make run-openai # OpenAI comparison
- Root SDK: line length **120**
- Python CLI: line length **100** with extended rule set (UP, B, SIM, RUF)
- **isort** with `profile = "black"` for import sorting.
- Ruff excludes `openmemory/` from root config.
- Ruff excludes `embedchain/` and `openmemory/` from root config.
### TypeScript Conventions
@@ -413,7 +414,7 @@ To add a new LLM, embedding, vector store, or reranker provider:
| Python CLI | `cli-python-ci.yml` | Push to `cli/python/`, PRs, manual | Ruff lint + pytest + hatch build on Python 3.10, 3.11, 3.12 |
| Node CLI | `cli-node-ci.yml` | Push to `cli/node/`, PRs, manual | Biome lint + tsc + vitest + tsup build on Node 20, 22 |
| OpenClaw | `openclaw-checks.yml` | Push to `openclaw/`, PRs, manual | tsc + vitest (with Codecov) + tsup build on Node 20, 22 |
| OpenCode Plugin | `opencode-plugin-checks.yml` | Push to `mem0-plugin/.opencode-plugin/`, PRs, manual | Bun: tsc type-check + build + dist artifact check |
| Embedchain | `ci.yml` (shared) | PRs on `embedchain/` | Ruff + pytest + coverage on Python 3.9–3.12 |
### CD Workflows (automated publishing)
@@ -425,7 +426,6 @@ To add a new LLM, embedding, vector store, or reranker provider:
| Node CLI | `cli-node-cd.yml` | `cli-node-v*` | npm (`@mem0/cli`) |
| Vercel AI SDK | `vercel-ai-cd.yml` | `vercel-ai-v*` | npm (`@mem0/vercel-ai-provider`) |
| OpenClaw | `openclaw-cd.yml` | `openclaw-v*` | npm (`@mem0/openclaw-mem0`) |
| OpenCode Plugin | `opencode-plugin-cd.yml` | `opencode-v*` | npm (`@mem0/opencode-plugin`) |
- All publishing uses **OIDC trusted publishing** — no tokens or secrets required.
- First publish of a new npm package must be done manually; OIDC works for subsequent versions.
@@ -576,6 +576,7 @@ N/A
- Modify CI/CD workflows without explicit approval.
- Add new Python dependencies to the core `dependencies` list in `pyproject.toml` without discussion — use optional dependency groups instead.
- Commit `.env` files, API keys, or credentials.
- Modify `embedchain/` unless specifically working on that package — it has its own build system (Poetry).
- Skip pre-commit hooks.
- Use npm or yarn in TypeScript packages — this repo uses pnpm exclusively.
- Use `require()` for imports in TypeScript — use ES module `import` syntax.
+221
View File
@@ -0,0 +1,221 @@
# Migration Guide: Upgrading to mem0 1.0.0
## TL;DR
**What changed?** We simplified the API by removing confusing version parameters. Now everything returns a consistent format: `{"results": [...]}`.
**What you need to do:**
1. Upgrade: `pip install mem0ai==1.0.0`
2. Remove `version` and `output_format` parameters from your code
3. Update response handling to use `result["results"]` instead of treating responses as lists
**Time needed:** ~5-10 minutes for most projects
---
## Quick Migration Guide
### 1. Install the Update
```bash
pip install mem0ai==1.0.0
```
### 2. Update Your Code
**If you're using the Memory API:**
```python
# Before
memory = Memory(config=MemoryConfig(version="v1.1"))
result = memory.add("I like pizza")
# After
memory = Memory() # That's it - version is automatic now
result = memory.add("I like pizza")
```
**If you're using the Client API:**
```python
# Before
client.add(messages, output_format="v1.1")
client.search(query, version="v2", output_format="v1.1")
# After
client.add(messages) # Just remove those extra parameters
client.search(query)
```
### 3. Update How You Handle Responses
All responses now use the same format: a dictionary with `"results"` key.
```python
# Before - you might have done this
result = memory.add("I like pizza")
for item in result: # Treating it as a list
print(item)
# After - do this instead
result = memory.add("I like pizza")
for item in result["results"]: # Access the results key
print(item)
# Graph relations (if you use them)
if "relations" in result:
for relation in result["relations"]:
print(relation)
```
---
## Enhanced Message Handling
The platform client (MemoryClient) now supports the same flexible message formats as the OSS version:
```python
from mem0 import MemoryClient
client = MemoryClient(api_key="your-key")
# All three formats now work:
# 1. Single string (automatically converted to user message)
client.add("I like pizza", user_id="alice")
# 2. Single message dictionary
client.add({"role": "user", "content": "I like pizza"}, user_id="alice")
# 3. List of messages (conversation)
client.add([
{"role": "user", "content": "I like pizza"},
{"role": "assistant", "content": "I'll remember that!"}
], user_id="alice")
```
### Async Mode Configuration
The `async_mode` parameter now defaults to `True` but can be configured:
```python
# Default behavior (async_mode=True)
client.add(messages, user_id="alice")
# Explicitly set async mode
client.add(messages, user_id="alice", async_mode=True)
# Disable async mode if needed
client.add(messages, user_id="alice", async_mode=False)
```
**Note:** `async_mode=True` provides better performance for most use cases. Only set it to `False` if you have specific synchronous processing requirements.
---
## That's It!
For most users, that's all you need to know. The changes are:
- ✅ No more `version` or `output_format` parameters
- ✅ Consistent `{"results": [...]}` response format
- ✅ Cleaner, simpler API
---
## Common Issues
**Getting `KeyError: 'results'`?**
Your code is still treating the response as a list. Update it:
```python
# Change this:
for memory in response:
# To this:
for memory in response["results"]:
```
**Getting `TypeError: unexpected keyword argument`?**
You're still passing old parameters. Remove them:
```python
# Change this:
client.add(messages, output_format="v1.1")
# To this:
client.add(messages)
```
**Seeing deprecation warnings?**
Remove any explicit `version="v1.0"` from your config:
```python
# Change this:
memory = Memory(config=MemoryConfig(version="v1.0"))
# To this:
memory = Memory()
```
---
## What's New in 1.0.0
- **Better vector stores:** Fixed OpenSearch and improved reliability across all stores
- **Cleaner API:** One way to do things, no more confusing options
- **Enhanced GCP support:** Better Vertex AI configuration options
- **Flexible message input:** Platform client now accepts strings, dicts, and lists (aligned with OSS)
- **Configurable async_mode:** Now defaults to `True` but users can override if needed
---
## Need Help?
- Check [GitHub Issues](https://github.com/mem0ai/mem0/issues)
- Read the [documentation](https://docs.mem0.ai/)
- Open a new issue if you're stuck
---
## Advanced: Configuration Changes
**If you configured vector stores with version:**
```python
# Before
config = MemoryConfig(
version="v1.1",
vector_store=VectorStoreConfig(...)
)
# After
config = MemoryConfig(
vector_store=VectorStoreConfig(...)
)
```
---
## Testing Your Migration
Quick sanity check:
```python
from mem0 import Memory
memory = Memory()
# Add should return a dict with "results"
result = memory.add("I like pizza", user_id="test")
assert "results" in result
# Search should return a dict with "results"
search = memory.search("food", user_id="test")
assert "results" in search
# Get all should return a dict with "results"
all_memories = memory.get_all(user_id="test")
assert "results" in all_memories
print("✅ Migration successful!")
```
-15
View File
@@ -5,21 +5,6 @@ All notable changes to `@mem0/cli` are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.2.8] — 2026-06-01
### Security
- Pinned transitive dependencies via pnpm overrides to remediate high-severity CVEs:
- `jws` → 4.0.1 (CVE-2025-65945)
- `langsmith` → ^0.6.0 (CVE-2026-45134)
- `tar-fs` → ^2.1.4 (CVE-2025-48387, CVE-2025-59343)
- `picomatch` → ^2.3.2 (CVE-2026-33671)
- `minimatch` → ^3.1.3 / ^5.1.8 / ^9.0.7 (CVE-2026-27903, CVE-2026-27904, CVE-2026-26996)
- `path-to-regexp` → ^8.4.0 (CVE-2026-4926)
- `rollup` → ^4.59.0 (CVE-2026-27606)
- `glob` → ^10.5.0 (CVE-2025-64756)
- `@modelcontextprotocol/sdk` → ^1.25.4 (CVE-2025-66414, CVE-2026-0621)
## [0.2.7] — 2026-05-20
### Added
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@mem0/cli",
"version": "0.2.8",
"version": "0.2.7",
"description": "The official CLI for mem0 — the memory layer for AI agents",
"type": "module",
"bin": {
-23
View File
@@ -4,12 +4,6 @@ settings:
autoInstallPeers: true
excludeLinksFromLockfile: false
overrides:
jws@4.0.0: 4.0.1
langsmith@<0.6.0: ^0.6.0
tar-fs@>=2.0.0 <2.1.4: ^2.1.4
picomatch@<2.3.2: ^2.3.2
importers:
.:
@@ -73,28 +67,24 @@ packages:
engines: {node: '>=14.21.3'}
cpu: [arm64]
os: [linux]
libc: [musl]
'@biomejs/cli-linux-arm64@1.9.4':
resolution: {integrity: sha512-fJIW0+LYujdjUgJJuwesP4EjIBl/N/TcOX3IvIHJQNsAqvV2CHIogsmA94BPG6jZATS4Hi+xv4SkBBQSt1N4/g==}
engines: {node: '>=14.21.3'}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@biomejs/cli-linux-x64-musl@1.9.4':
resolution: {integrity: sha512-gEhi/jSBhZ2m6wjV530Yy8+fNqG8PAinM3oV7CyO+6c3CEh16Eizm21uHVsyVBEB6RIM8JHIl6AGYCv6Q6Q9Tg==}
engines: {node: '>=14.21.3'}
cpu: [x64]
os: [linux]
libc: [musl]
'@biomejs/cli-linux-x64@1.9.4':
resolution: {integrity: sha512-lRCJv/Vi3Vlwmbd6K+oQ0KhLHMAysN8lXoCI7XeHlxaajk06u7G+UsFSO01NAs5iYuWKmVZjmiOzJ0OJmGsMwg==}
engines: {node: '>=14.21.3'}
cpu: [x64]
os: [linux]
libc: [glibc]
'@biomejs/cli-win32-arm64@1.9.4':
resolution: {integrity: sha512-tlbhLk+WXZmgwoIKwHIHEBZUwxml7bRJgk0X2sPyNR3S93cdRq6XulAZRQJ17FYGGzWne0fgrXBKpl7l4M87Hg==}
@@ -457,79 +447,66 @@ packages:
resolution: {integrity: sha512-RzeBwv0B3qtVBWtcuABtSuCzToo2IEAIQrcyB/b2zMvBWVbjo8bZDjACUpnaafaxhTw2W+imQbP2BD1usasK4g==}
cpu: [arm]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-arm-musleabihf@4.60.0':
resolution: {integrity: sha512-Sf7zusNI2CIU1HLzuu9Tc5YGAHEZs5Lu7N1ssJG4Tkw6e0MEsN7NdjUDDfGNHy2IU+ENyWT+L2obgWiguWibWQ==}
cpu: [arm]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-arm64-gnu@4.60.0':
resolution: {integrity: sha512-DX2x7CMcrJzsE91q7/O02IJQ5/aLkVtYFryqCjduJhUfGKG6yJV8hxaw8pZa93lLEpPTP/ohdN4wFz7yp/ry9A==}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-arm64-musl@4.60.0':
resolution: {integrity: sha512-09EL+yFVbJZlhcQfShpswwRZ0Rg+z/CsSELFCnPt3iK+iqwGsI4zht3secj5vLEs957QvFFXnzAT0FFPIxSrkQ==}
cpu: [arm64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-loong64-gnu@4.60.0':
resolution: {integrity: sha512-i9IcCMPr3EXm8EQg5jnja0Zyc1iFxJjZWlb4wr7U2Wx/GrddOuEafxRdMPRYVaXjgbhvqalp6np07hN1w9kAKw==}
cpu: [loong64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-loong64-musl@4.60.0':
resolution: {integrity: sha512-DGzdJK9kyJ+B78MCkWeGnpXJ91tK/iKA6HwHxF4TAlPIY7GXEvMe8hBFRgdrR9Ly4qebR/7gfUs9y2IoaVEyog==}
cpu: [loong64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-ppc64-gnu@4.60.0':
resolution: {integrity: sha512-RwpnLsqC8qbS8z1H1AxBA1H6qknR4YpPR9w2XX0vo2Sz10miu57PkNcnHVaZkbqyw/kUWfKMI73jhmfi9BRMUQ==}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-ppc64-musl@4.60.0':
resolution: {integrity: sha512-Z8pPf54Ly3aqtdWC3G4rFigZgNvd+qJlOE52fmko3KST9SoGfAdSRCwyoyG05q1HrrAblLbk1/PSIV+80/pxLg==}
cpu: [ppc64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-riscv64-gnu@4.60.0':
resolution: {integrity: sha512-3a3qQustp3COCGvnP4SvrMHnPQ9d1vzCakQVRTliaz8cIp/wULGjiGpbcqrkv0WrHTEp8bQD/B3HBjzujVWLOA==}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-riscv64-musl@4.60.0':
resolution: {integrity: sha512-pjZDsVH/1VsghMJ2/kAaxt6dL0psT6ZexQVrijczOf+PeP2BUqTHYejk3l6TlPRydggINOeNRhvpLa0AYpCWSQ==}
cpu: [riscv64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-s390x-gnu@4.60.0':
resolution: {integrity: sha512-3ObQs0BhvPgiUVZrN7gqCSvmFuMWvWvsjG5ayJ3Lraqv+2KhOsp+pUbigqbeWqueGIsnn+09HBw27rJ+gYK4VQ==}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-x64-gnu@4.60.0':
resolution: {integrity: sha512-EtylprDtQPdS5rXvAayrNDYoJhIz1/vzN2fEubo3yLE7tfAw+948dO0g4M0vkTVFhKojnF+n6C8bDNe+gDRdTg==}
cpu: [x64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-x64-musl@4.60.0':
resolution: {integrity: sha512-k09oiRCi/bHU9UVFqD17r3eJR9bn03TyKraCrlz5ULFJGdJGi7VOmm9jl44vOJvRJ6P7WuBi/s2A97LxxHGIdw==}
cpu: [x64]
os: [linux]
libc: [musl]
'@rollup/rollup-openbsd-x64@4.60.0':
resolution: {integrity: sha512-1o/0/pIhozoSaDJoDcec+IVLbnRtQmHwPV730+AOD29lHEEo4F5BEUB24H0OBdhbBBDwIOSuf7vgg0Ywxdfiiw==}
-12
View File
@@ -1,12 +0,0 @@
packages:
- '.'
onlyBuiltDependencies:
- "@biomejs/biome"
- esbuild
overrides:
jws@4.0.0: 4.0.1
langsmith@<0.6.0: ^0.6.0
tar-fs@>=2.0.0 <2.1.4: ^2.1.4
picomatch@<2.3.2: ^2.3.2
+2 -2
View File
@@ -46,7 +46,7 @@ export async function cmdAdd(
file?: string;
metadata?: string;
immutable: boolean;
infer?: boolean;
noInfer: boolean;
expires?: string;
categories?: string;
output: string;
@@ -136,7 +136,7 @@ export async function cmdAdd(
runId: opts.runId,
metadata: meta,
immutable: opts.immutable,
infer: opts.infer !== false,
infer: !opts.noInfer,
expires: opts.expires,
categories: cats,
});
+16 -48
View File
@@ -3,7 +3,6 @@
*/
import { describe, it, expect, vi, beforeEach } from "vitest";
import { Command } from "commander";
import { createMockBackend } from "./setup.js";
import type { Backend } from "../src/backend/base.js";
import { setAgentMode } from "../src/state.js";
@@ -42,6 +41,8 @@ describe("cmdAdd", () => {
await cmdAdd(mockBackend, "I prefer dark mode", {
userId: "alice",
immutable: false,
noInfer: false,
output: "text",
});
expect(mockBackend.add).toHaveBeenCalledOnce();
@@ -53,6 +54,8 @@ describe("cmdAdd", () => {
userId: "alice",
messages: JSON.stringify([{ role: "user", content: "I love Python" }]),
immutable: false,
noInfer: false,
output: "text",
});
expect(mockBackend.add).toHaveBeenCalledOnce();
@@ -63,6 +66,8 @@ describe("cmdAdd", () => {
await cmdAdd(mockBackend, "test", {
userId: "alice",
immutable: false,
noInfer: false,
output: "json",
});
expect(output).toContain("results");
@@ -73,59 +78,14 @@ describe("cmdAdd", () => {
await cmdAdd(mockBackend, "test", {
userId: "alice",
immutable: false,
noInfer: false,
output: "quiet",
});
expect(output).not.toContain("dark mode");
});
});
describe("cmdAdd forwards --no-infer (regression for #5261)", () => {
it("forwards infer: false when --no-infer is set", async () => {
const { cmdAdd } = await import("../src/commands/memory.js");
// `infer: false` is the shape Commander produces for `--no-infer`.
await cmdAdd(mockBackend, "store me verbatim", {
userId: "alice",
immutable: false,
infer: false,
output: "text",
});
expect(mockBackend.add).toHaveBeenCalledWith(
"store me verbatim",
undefined,
expect.objectContaining({ infer: false }),
);
});
it("forwards infer: true by default (flag absent)", async () => {
const { cmdAdd } = await import("../src/commands/memory.js");
await cmdAdd(mockBackend, "infer me", {
userId: "alice",
immutable: false,
output: "text",
});
expect(mockBackend.add).toHaveBeenCalledWith(
"infer me",
undefined,
expect.objectContaining({ infer: true }),
);
});
it("Commander stores --no-infer as opts.infer, not opts.noInfer", () => {
// Pins the assumption the fix relies on: Commander's `--no-X` option
// populates the positive camelCase key (`infer`), never `noInfer`.
const withFlag = new Command();
withFlag.option("--no-infer", "Skip inference, store raw.").action(() => {});
withFlag.parse(["--no-infer"], { from: "user" });
expect(withFlag.opts().infer).toBe(false);
expect(withFlag.opts().noInfer).toBeUndefined();
const withoutFlag = new Command();
withoutFlag.option("--no-infer", "Skip inference, store raw.").action(() => {});
withoutFlag.parse([], { from: "user" });
expect(withoutFlag.opts().infer).toBe(true);
});
});
describe("cmdAdd deduplicates PENDING", () => {
const DUPLICATE_PENDING = {
results: [
@@ -140,6 +100,8 @@ describe("cmdAdd deduplicates PENDING", () => {
await cmdAdd(mockBackend, "test", {
userId: "alice",
immutable: false,
noInfer: false,
output: "text",
});
expect(output.match(/Queued/g)?.length).toBe(1);
@@ -151,6 +113,8 @@ describe("cmdAdd deduplicates PENDING", () => {
await cmdAdd(mockBackend, "test", {
userId: "alice",
immutable: false,
noInfer: false,
output: "json",
});
const data = JSON.parse(output);
@@ -165,6 +129,8 @@ describe("cmdAdd deduplicates PENDING", () => {
await cmdAdd(mockBackend, "test", {
userId: "alice",
immutable: false,
noInfer: false,
output: "agent",
});
const data = JSON.parse(output);
@@ -349,6 +315,8 @@ describe("agent mode", () => {
await cmdAdd(mockBackend, "test preference", {
userId: "alice",
immutable: false,
noInfer: false,
output: "agent",
});
const parsed = JSON.parse(output.trim());
@@ -79,7 +79,7 @@ new_project = client.project.create(
### Update Project Settings
Modify project configuration including custom instructions, categories, and language preferences:
Modify project configuration including custom instructions, categories, graph settings, and language preferences:
```python
# Update project with custom categories
-57
View File
@@ -7,21 +7,6 @@ mode: "wide"
<Tabs>
<Tab title="Python">
<Update label="2026-05-27" description="v2.0.4">
**New Features:**
- **Client:** `delete()` and async `delete()` accept `delete_linked` (default `False`). When `True`, deleting a memory also removes the older memories it superseded (the v3 `linked_memory_ids` chain), transitively — the delete-side counterpart of `latest_only`, so a superseded memory does not resurface after the current one is deleted ([#5270](https://github.com/mem0ai/mem0/pull/5270))
</Update>
<Update label="2026-05-26" description="v2.0.3">
**Bug Fixes:**
- **Vector Stores:** PGVector adapter now supports rich filter operators (`eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`, `contains`, `icontains`, wildcard `*`, `$or`, `$not`) in `search()`, `keyword_search()`, and `list()`. Previously only exact-equality filters worked — operator dicts were silently stringified and returned zero results ([#5263](https://github.com/mem0ai/mem0/pull/5263))
- **Server:** Fixed `/search` endpoint returning 502 when `user_id`, `agent_id`, or `run_id` are sent as top-level request fields. The server now maps these into the `filters` dict before calling `Memory.search()`, matching the v3 API contract. Top-level entity ID fields are marked as deprecated in the OpenAPI schema and emit a warning log — clients should migrate to `filters={"user_id": "..."}` ([#5263](https://github.com/mem0ai/mem0/pull/5263))
</Update>
<Update label="2026-05-08" description="v2.0.2">
**Bug Fixes:**
@@ -939,27 +924,6 @@ See the [OSS v1 to v2 migration guide](https://docs.mem0.ai/migration/oss-v1-to-
</Tab>
<Tab title="TypeScript">
<Update label="2026-06-01" description="v3.0.6">
**Security:**
- **Dependencies:** Bumped `axios` to `^1.16.0` to remediate high-severity prototype-pollution CVEs (credential theft, MITM, DoS). Pinned transitive dependencies via pnpm overrides: `jws` → 4.0.1 (CVE-2025-65945), `langsmith` → ^0.6.0 (CVE-2026-45134), `tar-fs` → ^2.1.4 (CVE-2025-48387, CVE-2025-59343), `picomatch` → ^2.3.2 (CVE-2026-33671), `minimatch` → ^3.1.3 / ^5.1.8 / ^9.0.7 (CVE-2026-27903, CVE-2026-27904, CVE-2026-26996), `path-to-regexp` → ^8.4.0 (CVE-2026-4926), `rollup` → ^4.59.0 (CVE-2026-27606), `glob` → ^10.5.0 (CVE-2025-64756), `@modelcontextprotocol/sdk` → ^1.25.4 (CVE-2025-66414, CVE-2026-0621)
</Update>
<Update label="2026-05-27" description="v3.0.5">
**New Features:**
- **Client:** `delete()` accepts an options object with `deleteLinked` (serialized as `delete_linked`, default `false`). When `true`, deleting a memory also removes the older memories it superseded (the v3 linked chain), transitively — the delete-side counterpart of `latestOnly`, so a superseded memory does not resurface after the current one is deleted ([#5270](https://github.com/mem0ai/mem0/pull/5270))
</Update>
<Update label="2026-05-26" description="v3.0.4">
**Bug Fixes:**
- **Vector Stores:** PGVector adapter now supports rich filter operators (`eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`, `contains`, `icontains`, wildcard `*`, `$or`, `$not`) in `search()`, `keywordSearch()`, and `list()`. Previously only exact-equality filters worked — operator objects were passed as raw values and returned incorrect results ([#5263](https://github.com/mem0ai/mem0/pull/5263))
</Update>
<Update label="2026-05-08" description="v3.0.3">
**Bug Fixes:**
@@ -1359,13 +1323,6 @@ See the [TypeScript SDK migration guide](https://docs.mem0.ai/migration/ts-v2-to
<Tab title="CLI">
<Update label="2026-06-01" description="Node v0.2.8">
**Security:**
- **Dependencies:** Pinned transitive dependencies via pnpm overrides to remediate high-severity CVEs: `jws` → 4.0.1 (CVE-2025-65945), `langsmith` → ^0.6.0 (CVE-2026-45134), `tar-fs` → ^2.1.4 (CVE-2025-48387, CVE-2025-59343), `picomatch` → ^2.3.2 (CVE-2026-33671), `minimatch` → ^3.1.3 / ^5.1.8 / ^9.0.7 (CVE-2026-27903, CVE-2026-27904, CVE-2026-26996), `path-to-regexp` → ^8.4.0 (CVE-2026-4926), `rollup` → ^4.59.0 (CVE-2026-27606), `glob` → ^10.5.0 (CVE-2025-64756), `@modelcontextprotocol/sdk` → ^1.25.4 (CVE-2025-66414, CVE-2026-0621)
</Update>
<Update label="2026-05-16" description="Python v0.2.6 / Node v0.2.6">
**Bug Fixes:**
@@ -1482,20 +1439,6 @@ A full-featured command-line interface for Mem0, available in both Python and No
<Tab title="Plugins">
<Update label="2026-06-01" description="openclaw-mem0 v1.0.12">
**Security:**
- **Dependencies:** Pinned transitive dependencies via pnpm overrides to remediate high-severity CVEs: `protobufjs` → ^7.5.5, `vite` → ^8.0.5, `langsmith` → ^0.6.0 (CVE-2026-45134), `picomatch` → ^2.3.2 (CVE-2026-33671), `@qdrant/js-client-rest` → ^1.18.0
</Update>
<Update label="2026-06-01" description="Vercel AI SDK v2.0.6">
**Security:**
- **Dependencies:** Pinned transitive dependencies via pnpm overrides to remediate high-severity CVEs: `glob` → ^10.5.0 (CVE-2025-64756), `minimatch` → ^3.1.3 / ^5.1.8 / ^9.0.7 (CVE-2026-27903, CVE-2026-27904, CVE-2026-26996), `picomatch` → ^2.3.2 (CVE-2026-33671), `rollup` → ^4.59.0 (CVE-2026-27606)
</Update>
<Update label="2026-04-02" description="mem0-plugin v1.0.0">
**Mem0 Plugin for Claude Code, Cursor, and Codex**
@@ -323,6 +323,10 @@ Metadata: {'verified': True, 'updated_date': '2025-04-02'}
That “no duplicates” promise comes from the inference pipeline. Keep `infer=True` when you rely on automatic updates. Raw imports (`infer=False`) skip conflict checks, so mixing the two modes for the same fact will create duplicates.
</Warning>
**Maintains relationships:**
- If using graph memory, connections to other entities persist
### Pick the right inference mode
| Mode | What it does | Best for | Watch out for |
@@ -216,6 +216,7 @@ This information was retrieved from your memory history where you previously men
- **Smart Memory Management** - Organizes memories into searchable information *without setting up vector databases*
- **Fast Retrieval** - Instant lookups with *sub-millisecond ping*, handles large datasets
- **Graph Capabilities** - Builds knowledge *automatically* as you push information
- **Simple Integration** - Uses Mem0 API in the backend, works with *any MCP client* with just a few lines of code
### Gemini 3 + Mem0 Benefits
+8 -1
View File
@@ -156,7 +156,14 @@ Here are some examples of how Mem0 can be integrated into various applications:
icon="aws"
href="/cookbooks/integrations/aws-bedrock"
>
Mem0 with AWS Bedrock.
Mem0 with AWS Bedrock and Neptune.
</Card>
<Card
title="Graph Memory on Neptune"
icon="network-wired"
href="/cookbooks/integrations/neptune-analytics"
>
Graph memory with Neptune Analytics.
</Card>
</CardGroup>
+1 -1
View File
@@ -47,7 +47,7 @@ When a query arrives, the retrieval pipeline scores candidates across three sign
1. **Semantic Search** — Vector similarity scoring against memory embeddings
2. **Keyword Search** — Normalized term matching via BM25 with verb-form lemmatization
3. **Entity Search** — Entity matching boosts memories linked to query entities
3. **Entity Search** — Entity graph matching boosts memories linked to query entities
Results are fused via rank scoring into a final top-K set. Different query types lean on different signals:
+7 -3
View File
@@ -27,11 +27,15 @@ Adding memory is how Mem0 captures useful details from a conversation so your ag
Mem0 offers two flows:
- **Mem0 Platform** – Fully managed API with dashboard and scaling.
- **Mem0 Platform** – Fully managed API with dashboard, scaling, and graph features.
- **Mem0 Open Source** – Local SDK that you run in your own environment.
Both flows take the same payload and pass it through the same pipeline.
<Frame caption="Architecture diagram illustrating the process of adding memories.">
<img src="../../images/add_architecture.png" />
</Frame>
<Steps>
<Step title="Information extraction">
Mem0 sends the messages through an LLM that pulls out key facts, decisions, or preferences to remember.
@@ -40,7 +44,7 @@ Mem0 sends the messages through an LLM that pulls out key facts, decisions, or p
Existing memories are checked for duplicates or contradictions so the latest truth wins.
</Step>
<Step title="Storage">
The resulting memories land in managed vector storage so future searches return them quickly.
The resulting memories land in managed vector storage (and optional graph storage) so future searches return them quickly.
</Step>
</Steps>
@@ -173,7 +177,7 @@ For full list of supported fields, required formats, and advanced options, see t
## Put it into practice
- Review the <Link href="/platform/advanced-memory-operations">Advanced Memory Operations</Link> guide to layer metadata and rerankers.
- Review the <Link href="/platform/advanced-memory-operations">Advanced Memory Operations</Link> guide to layer metadata, rerankers, and graph toggles.
- Explore the <Link href="/api-reference/memory/add-memories">Add Memories API reference</Link> for every request/response field.
## See it live
@@ -25,6 +25,10 @@ Mem0's search operation lets agents ask natural-language questions and get back
## Architecture
<Frame caption="Architecture diagram illustrating the memory search process.">
<img src="../../images/search_architecture.png" />
</Frame>
<Steps>
<Step title="Query processing">
Mem0 cleans and enriches your natural-language query so the downstream embedding search is accurate.
+1 -1
View File
@@ -104,7 +104,7 @@ results = memory.search(
## Put it into practice
- Use the <Link href="/core-concepts/memory-operations/add">Add Memory</Link> guide to persist user preferences.
- Follow <Link href="/platform/advanced-memory-operations">Advanced Memory Operations</Link> to tune metadata and retrieval.
- Follow <Link href="/platform/advanced-memory-operations">Advanced Memory Operations</Link> to tune metadata and graph writes.
## See it live
+1 -3
View File
@@ -452,9 +452,7 @@
"pages": [
"integrations/claude-code",
"integrations/cursor",
"integrations/codex",
"integrations/opencode",
"integrations/antigravity"
"integrations/codex"
]
},
{
Binary file not shown.

After

Width:  |  Height:  |  Size: 276 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 227 KiB

-82
View File
@@ -1,82 +0,0 @@
---
title: Antigravity
description: "Add persistent memory to Google Antigravity with the Mem0 plugin — MCP server, lifecycle hooks, and slash commands."
---
Add persistent memory to [**Google Antigravity**](https://antigravity.google) (`agy` CLI and Desktop IDE) with the Mem0 plugin. Your agent forgets everything between sessions — Mem0 fixes that by storing decisions, preferences, and learnings so they carry over automatically.
## Prerequisites
1. A Mem0 API key (starts with `m0-`):
- <a href="https://app.mem0.ai/dashboard/api-keys?utm_source=oss&utm_medium=integration-antigravity" rel="nofollow">Get your API key</a> (free sign-up at <a href="https://app.mem0.ai?utm_source=oss&utm_medium=integration-antigravity" rel="nofollow">app.mem0.ai</a>)
2. Add it to your shell profile so it persists across sessions:
<CodeGroup>
```bash zsh
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.zshrc && source ~/.zshrc
```
```bash bash
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.bashrc && source ~/.bashrc
```
</CodeGroup>
## Installation
**Option A — degit** (recommended):
```bash
# Install the plugin (MCP server, hooks, scripts)
npx degit mem0ai/mem0/mem0-plugin ~/.gemini/config/plugins/mem0
```
This installs the MCP server, lifecycle hooks, and shared scripts.
## What's Included
| Component | Included |
|-----------|:--------:|
| MCP Server (9 memory tools) | Yes |
| Lifecycle Hooks | Yes |
| 16 Slash Commands | Yes |
## Available MCP Tools
| Tool | Description |
|------|-------------|
| `add_memory` | Save text or conversation history for a user/agent |
| `search_memories` | Semantic search across memories with filters |
| `get_memories` | List memories with filters and pagination |
| `get_memory` | Retrieve a specific memory by ID |
| `update_memory` | Overwrite a memory's text by ID |
| `delete_memory` | Delete a single memory by ID |
| `delete_all_memories` | Bulk delete all memories in scope |
| `delete_entities` | Delete a user/agent/app/run entity and its memories |
| `list_entities` | List users/agents/apps/runs stored in Mem0 |
## Lifecycle Hooks
The plugin uses the same shell scripts as Claude Code, Cursor, and Codex — hooks bridge environment variables using `${extensionPath}` (Antigravity's plugin-root token).
| Hook | Event | What it does |
|------|-------|-------------|
| **Session start** | `SessionStart` | Loads prior memories and displays status banner |
| **User prompt** | `UserPromptSubmit` | Searches relevant memories before each message |
| **Pre-tool** | `PreToolUse` | Blocks MEMORY.md writes, enforces `user_id`/`app_id` on mem0 tools |
| **Post-tool** | `PostToolUse` | Tracks stats, scans bash errors for related memories |
## Troubleshooting
- **No tools appearing** — Restart your Antigravity session after installation
- **"Connection failed"** — Verify your key is set: `echo $MEM0_API_KEY`
- **MCP 401 Unauthorized** — If `${MEM0_API_KEY}` interpolation doesn't work in your `agy` version, replace with your literal key in `mcp_config.json`
<CardGroup cols={2}>
<Card title="Mem0 MCP Setup" icon="puzzle-piece" href="/platform/mem0-mcp">
Detailed MCP configuration for all clients
</Card>
<Card title="OpenCode Integration" icon="code" href="/integrations/opencode">
Add Mem0 memory to OpenCode workflows
</Card>
</CardGroup>
+25 -43
View File
@@ -5,6 +5,13 @@ description: "Add persistent memory to Claude Code and Claude Cowork with the Me
Add persistent memory to [**Claude Code**](https://docs.anthropic.com/en/docs/claude-code) (CLI) and **Claude Cowork** (desktop app) with the Mem0 plugin. Your agent forgets everything between sessions — this plugin fixes that by connecting to Mem0's cloud memory layer via MCP, automatically capturing learnings at key lifecycle points, and retrieving relevant context before every response.
## Overview
1. **MCP Server** — Connect to Mem0's remote MCP server for memory tools (add, search, update, delete)
2. **Lifecycle Hooks** — Automatic memory capture at session start, context compaction, task completion, and session end
3. **SDK Skill** — Teaches the agent how to integrate the Mem0 SDK into your applications
4. **Zero local dependencies** — Cloud-hosted MCP server, no local setup required
## Prerequisites
Before setting up Mem0 with Claude Code, ensure you have:
@@ -15,25 +22,10 @@ Before setting up Mem0 with Claude Code, ensure you have:
2. Claude Code CLI or Claude Cowork desktop app installed
3. Your API key added to your shell profile (persists across sessions):
<CodeGroup>
```bash zsh
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.zshrc
source ~/.zshrc
```
```bash bash
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.bashrc
source ~/.bashrc
```
</CodeGroup>
Confirm it's set:
3. Your API key exported in your shell:
```bash
echo $MEM0_API_KEY
# Should print: m0-your-api-key
export MEM0_API_KEY="m0-your-api-key"
```
## Installation
@@ -92,22 +84,6 @@ Add to your Claude Code MCP config (`.mcp.json`):
Start a new session and ask: *"List my mem0 entities"* or *"Search my memories for hello"*. If the `mem0` tools appear and respond, you're all set.
</Info>
## Post-Installation: Run `/mem0:onboard`
After installing the plugin, start a new Claude Code session and run:
```
/mem0:onboard
```
This runs the setup wizard which:
1. Verifies your API key and MCP connection
2. Detects and imports project files (`CLAUDE.md`, `AGENTS.md`, `.cursorrules`)
3. Installs coding-optimized memory categories
4. Shows your identity (user ID, project scope, branch)
The onboarding is idempotent — safe to re-run anytime. It auto-triggers on first session in a new project, but you can always invoke it manually.
## What's Included
| Component | Plugin Install | MCP Only |
@@ -136,13 +112,20 @@ Once installed, the following tools are available in every Claude Code session:
When installed via the plugin marketplace, Mem0 hooks into Claude Code's lifecycle to automatically manage memory:
| Hook | Event | What it does |
|------|-------|-------------|
| **Session start** | `SessionStart` | Loads prior memories and displays status banner |
| **User prompt** | `UserPromptSubmit` | Searches relevant memories before each message; skips short prompts |
| **Pre-tool** | `PreToolUse` | Blocks MEMORY.md writes, enforces `user_id`/`app_id` on mem0 tool calls |
| **Post-tool** | `PostToolUse` | Tracks stats, scans bash errors for related memories |
| **Pre-compact** | `PreCompact` | Stores a session summary before context compaction |
### Session Start
On every new session, the plugin prompts Claude to call `search_memories` to load relevant context from prior sessions. On resumed or post-compaction sessions, it adjusts the prompt accordingly.
### User Prompt
Before processing each user message, the plugin searches Mem0 for memories relevant to the current prompt and injects them into context. Short prompts (< 20 characters) are skipped to minimize latency.
### Pre-Compaction
Before context compaction, the plugin prompts Claude to store a comprehensive session summary — including goals, accomplishments, decisions, modified files, and current state — so nothing is lost.
### Task Completed
After each task completion, the plugin prompts Claude to extract and store key learnings: successful strategies, failed approaches, architectural decisions, and new conventions.
### Session End
When Claude finishes responding, the plugin prompts for any unstored learnings and captures transcript state via the Mem0 REST API as a background safety net.
## Example Workflow
@@ -166,10 +149,9 @@ You: Add refresh token rotation to the auth system.
## Troubleshooting
- **"Connection failed"** — Verify `MEM0_API_KEY` is set in your shell: `echo $MEM0_API_KEY`. If empty, add it to your shell profile (see Prerequisites)
- **"Connection failed"** — Verify `MEM0_API_KEY` is set in your shell: `echo $MEM0_API_KEY`
- **No tools appearing** — Restart your Claude Code session after installation
- **Memories not being captured** — Ensure you installed via the plugin marketplace (Option A) for lifecycle hooks. MCP-only installs require manual memory operations
- **"Mem0 Inactive" banner every session** — Your API key isn't persisting. Add `export MEM0_API_KEY="m0-..."` to your `~/.zshrc` (or `~/.bashrc`) and run `source ~/.zshrc`
- **Memories not being captured** — Ensure you installed via the plugin marketplace (Option A) for lifecycle hooks. MCP-only installs require manual memory operations.
<CardGroup cols={2}>
<Card title="Mem0 MCP Setup" icon="puzzle-piece" href="/platform/mem0-mcp">
+124 -49
View File
@@ -1,9 +1,16 @@
---
title: Codex
description: "Add persistent memory to OpenAI Codex with the Mem0 plugin — MCP server, lifecycle hooks, and SDK skill."
description: "Add persistent memory to OpenAI Codex with the Mem0 plugin — MCP server, memory protocol skill, and plugin marketplace support."
---
Add persistent memory to [**OpenAI Codex**](https://openai.com/index/codex/) with the Mem0 plugin. Codex forgets everything between tasks — this plugin fixes that by connecting to Mem0's cloud memory layer via MCP, automatically capturing learnings at key lifecycle points, and retrieving relevant context before every response.
Add persistent memory to [**OpenAI Codex**](https://openai.com/index/codex/) with the Mem0 plugin. Codex forgets everything between tasks — this plugin fixes that by connecting to Mem0's cloud memory layer via MCP and using a skill-based memory protocol to automatically retrieve context and store learnings.
## Overview
1. **MCP Server** — Connect to Mem0's remote MCP server for memory tools (add, search, update, delete)
2. **Memory Protocol Skill** — Instructs the agent to retrieve memories at task start, store learnings on completion, and capture session state before context loss
3. **Plugin Marketplace** — Install via Codex's repo-level or personal plugin marketplace
4. **Zero local dependencies** — Cloud-hosted MCP server, no local setup required
## Prerequisites
@@ -15,41 +22,17 @@ Before setting up Mem0 with Codex, ensure you have:
2. OpenAI Codex access
3. Your API key added to your shell profile (persists across sessions):
3. Your API key exported in your shell:
<CodeGroup>
```bash zsh
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.zshrc
source ~/.zshrc
```bash
export MEM0_API_KEY="m0-your-api-key"
```
```bash bash
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.bashrc
source ~/.bashrc
```
</CodeGroup>
## Installation
### Option A — Plugin Marketplace (Recommended)
### Option A — Direct MCP (Recommended)
Install the full plugin including MCP server, lifecycle hooks, and SDK skill.
1. Add the Mem0 marketplace:
```bash
codex plugin marketplace add mem0ai/mem0
```
2. Restart Codex, open the Plugin Directory, browse the **Mem0 Plugins** marketplace, and install **Mem0**.
<Info>
Do not combine with Option B. The plugin manifest auto-registers the `mem0` MCP server, so adding both will create a duplicate registration.
</Info>
### Option B — Direct MCP
The fastest way to connect Codex to Mem0 — no plugin, no marketplace. Add to `~/.codex/config.toml`:
The fastest way to connect Codex to Mem0 — no downloads, no marketplace. Codex reads MCP servers from `~/.codex/config.toml` as TOML. Add:
```toml
[mcp_servers.mem0]
@@ -63,16 +46,71 @@ Make sure `MEM0_API_KEY` is exported in the shell you launch Codex from, then re
Codex's `codex mcp add` CLI only supports stdio MCP servers. Because Mem0's MCP is HTTP/streamable, you configure it by editing `config.toml` directly (or via the **Plugins → Connect to a custom MCP → Streamable HTTP** UI in the Codex app).
</Info>
This gives you the MCP tools but not the lifecycle hooks or SDK skill.
### Option B — Sideload the Plugin (Advanced)
For the full plugin experience — MCP server **plus** the Mem0 SDK skill, memory protocol skill, and opt-in lifecycle hooks — sideload the plugin from a local clone. The Mem0 repo already ships a marketplace manifest at [`.agents/plugins/marketplace.json`](https://github.com/mem0ai/mem0/blob/main/.agents/plugins/marketplace.json), so there's no JSON to author by hand. This follows the Codex [build-plugins](https://developers.openai.com/codex/plugins/build) local-testing workflow.
<Info>
Don't combine Option B with Option A. The plugin manifest declares its MCP server via [`.codex-mcp.json`](https://github.com/mem0ai/mem0/blob/main/mem0-plugin/.codex-mcp.json), so Codex auto-registers the `mem0` MCP server when the plugin loads. Adding the same `[mcp_servers.mem0]` block to `~/.codex/config.toml` will create a duplicate registration.
</Info>
**Step 1.** Clone the Mem0 repository anywhere on disk:
```bash
git clone https://github.com/mem0ai/mem0.git ~/codex-plugins/mem0-source
```
**Step 2.** Register the bundled marketplace with Codex's CLI:
```bash
codex plugin marketplace add ~/codex-plugins/mem0-source
```
This points Codex at the repo's `.agents/plugins/marketplace.json`. The bundled file uses `path: "./mem0-plugin"`, which Codex resolves relative to the clone root.
<Info>
**Why we recommend this over hand-authoring `~/.agents/plugins/marketplace.json`:** Codex requires `source.path` in any marketplace manifest to be **relative** (starting with `./`) and **inside the marketplace root**. The repo's bundled manifest already satisfies this — the marketplace root is the clone directory, and `mem0-plugin/` lives inside it. With a personal `~/.agents/plugins/marketplace.json`, the root is `~/` and the clone has to live under `~/` too. The CLI form sidesteps that constraint.
</Info>
**Step 3.** Restart Codex, run `/plugins`, browse the `Mem0 Plugins` marketplace, and install **Mem0**.
**Step 4 (optional) — enable lifecycle hooks.** Codex doesn't auto-wire hooks from plugin manifests; it only reads them from `~/.codex/hooks.json` (or `<repo>/.codex/hooks.json`). Run the bundled installer once to merge the Mem0 entries into your global hooks file:
```bash
python3 ~/codex-plugins/mem0-source/mem0-plugin/scripts/install_codex_hooks.py
```
Then enable the hooks feature flag in `~/.codex/config.toml`:
```toml
[features]
codex_hooks = true
```
Restart Codex. The installer registers three hooks pointing at scripts inside your clone:
| Event | Behavior |
|-------|----------|
| `SessionStart` | Loads prior memories as bootstrap context |
| `UserPromptSubmit` | Injects relevant memories before each prompt |
| `Stop` | Reminds the agent to persist learnings at turn end |
Re-running the installer is idempotent. To remove the hooks: `python3 ~/codex-plugins/mem0-source/mem0-plugin/scripts/install_codex_hooks.py --uninstall`.
<Warning>
The hooks file stores absolute paths into your clone (e.g. `~/codex-plugins/mem0-source/mem0-plugin/scripts/...`). If you move or delete the clone, the hooks will break silently — re-run the installer from the new location, or run `--uninstall` first.
</Warning>
### Managing the Plugin
Codex provides CLI commands for managing marketplaces after install:
```bash
codex plugin marketplace upgrade # pull latest plugin versions
codex plugin marketplace remove mem0-plugins # unregister the marketplace
```
To update, run `codex plugin marketplace upgrade` to pull the latest from the Mem0 repo.
To pull updates to the plugin source itself, `git pull` inside your clone (`~/codex-plugins/mem0-source`) and then run `codex plugin marketplace upgrade` to refresh Codex's plugin cache. Plugins are cached at `~/.codex/plugins/cache/<marketplace>/<plugin>/<version>/`.
<Info icon="check">
After either option, start a new Codex task and ask: *"List my mem0 entities"* or *"Search my memories for hello"*. If the `mem0` tools appear and respond, you're all set.
@@ -80,11 +118,12 @@ To update, run `codex plugin marketplace upgrade` to pull the latest from the Me
## What's Included
| Component | Plugin Install | MCP Only |
|-----------|:--------------:|:--------:|
| Component | Sideloaded Plugin | Direct MCP |
|-----------|:-----------------:|:----------:|
| MCP Server (9 memory tools) | Yes | Yes |
| Lifecycle Hooks | Yes | No |
| Memory Protocol Skill | Yes | No |
| Mem0 SDK Skill | Yes | No |
| Lifecycle Hooks (opt-in) | Yes | No |
## Available MCP Tools
@@ -102,17 +141,49 @@ Once installed, the following tools are available in every Codex session:
| `delete_entities` | Delete a user/agent/app/run entity and its memories |
| `list_entities` | List users/agents/apps/runs stored in Mem0 |
## Lifecycle Hooks
## Memory Protocol Skill
When installed via the plugin marketplace, Mem0 hooks into Codex's lifecycle to automatically manage memory:
When the plugin is sideloaded, the memory protocol skill instructs the agent to:
| Hook | Event | What it does |
|------|-------|-------------|
| **Session start** | `SessionStart` | Loads prior memories and displays status banner |
| **User prompt** | `UserPromptSubmit` | Searches relevant memories before each message |
| **Pre-tool** | `PreToolUse` | Blocks MEMORY.md writes, enforces `user_id`/`app_id` on mem0 tool calls |
| **Post-tool** | `PostToolUse` | Tracks stats, scans bash errors for related memories |
| **Pre-compact** | `PreCompact` | Stores a session summary before context compaction |
### On Every New Task
1. Call `search_memories` with a query related to the current task to load relevant context
2. Review returned memories to understand what was learned in prior sessions
3. Optionally call `get_memories` to browse all stored memories
### After Completing Significant Work
Store key learnings using `add_memory` with structured metadata:
| What to store | Metadata type |
|--------------|---------------|
| Architectural decisions | `{"type": "decision"}` |
| Strategies that worked | `{"type": "task_learning"}` |
| Failed approaches | `{"type": "anti_pattern"}` |
| User preferences observed | `{"type": "user_preference"}` |
| Environment discoveries | `{"type": "environmental"}` |
| Conventions established | `{"type": "convention"}` |
### Before Losing Context
Store a comprehensive session summary including goals, accomplishments, decisions, files modified, and current state with metadata `{"type": "session_state"}`.
## Plugin Manifest
The Codex plugin manifest (`.codex-plugin/plugin.json`) follows the Codex plugin specification:
```json
{
"name": "mem0",
"version": "0.1.0",
"description": "Mem0 memory layer for AI applications.",
"skills": "./skills/",
"mcpServers": "./.codex-mcp.json",
"interface": {
"displayName": "Mem0",
"shortDescription": "Persistent memory layer for AI coding workflows",
"category": "Productivity",
"capabilities": ["Read", "Write"]
}
}
```
## Example Workflow
@@ -135,10 +206,14 @@ You: Add WebSocket support for real-time notification delivery.
## Troubleshooting
- **"Connection failed"** — Verify `MEM0_API_KEY` is set: `echo $MEM0_API_KEY`
- **No tools appearing** — Restart your Codex session after installation
- **Duplicate `mem0` MCP / "tool collision" errors** — You combined Option A with Option B. Remove the `[mcp_servers.mem0]` block from `~/.codex/config.toml`; the plugin registers it automatically
- **Hooks not firing** — Ensure the plugin is installed via the marketplace (Option A). MCP-only installs do not include hooks
- **"Connection failed"** — Verify `MEM0_API_KEY` is set in your shell: `echo $MEM0_API_KEY`
- **No tools appearing** — Restart your Codex session after plugin installation
- **Duplicate `mem0` MCP server / "tool collision" errors** — You combined Option A (Direct MCP) with Option B (sideload). The sideloaded plugin auto-registers `mem0` from `.codex-mcp.json`, so remove the `[mcp_servers.mem0]` block from `~/.codex/config.toml`.
- **`plugin/read failed in TUI`** — Codex can't find the plugin directory the marketplace points at. If you used `codex plugin marketplace add <path>`, confirm the path is your clone root and that `<clone>/.agents/plugins/marketplace.json` exists. If you hand-authored `~/.agents/plugins/marketplace.json`, `source.path` must be relative (start with `./`), inside the marketplace root (`~/` for personal installs), and end in `mem0-plugin` — e.g. `"./codex-plugins/mem0-source/mem0-plugin"`.
- **Plugin not found in `/plugins`** — Run `codex plugin marketplace add ~/path/to/clone` again, or confirm the marketplace was registered with `codex plugin marketplace remove mem0-plugins` then re-add.
- **Skills not loading** — Verify the `skills` field in `plugin.json` points to a valid directory containing `SKILL.md` files.
- **Hooks not firing** — Confirm `codex_hooks = true` is in `~/.codex/config.toml` under `[features]`, and that `~/.codex/hooks.json` contains the Mem0 entries (re-run the installer if not). Restart Codex after enabling the flag.
- **Hooks broke after moving the clone** — The installer bakes absolute paths into `~/.codex/hooks.json` pointing at scripts inside your clone. If you moved or renamed the clone directory, run `python3 <new-clone>/mem0-plugin/scripts/install_codex_hooks.py` from the new location — the installer is idempotent and replaces the old entries.
<CardGroup cols={2}>
<Card title="Mem0 MCP Setup" icon="puzzle-piece" href="/platform/mem0-mcp">
+18 -18
View File
@@ -5,6 +5,13 @@ description: "Add persistent memory to Cursor with the Mem0 plugin — MCP serve
Add persistent memory to [**Cursor**](https://cursor.com) with the Mem0 plugin. Your AI assistant forgets everything between sessions — this plugin fixes that by connecting to Mem0's cloud memory layer via MCP, automatically capturing learnings at key lifecycle points, and retrieving relevant context before every response.
## Overview
1. **MCP Server** — Connect to Mem0's remote MCP server for memory tools (add, search, update, delete)
2. **Lifecycle Hooks** — Automatic memory capture at session start, compaction, and user prompts (Marketplace install)
3. **SDK Skill** — Teaches the agent how to integrate the Mem0 SDK into your applications
4. **Zero local dependencies** — Cloud-hosted MCP server, no local setup required
## Prerequisites
Before setting up Mem0 with Cursor, ensure you have:
@@ -15,20 +22,12 @@ Before setting up Mem0 with Cursor, ensure you have:
2. Cursor installed ([cursor.com](https://cursor.com))
3. Your API key added to your shell profile (persists across sessions):
3. Your API key exported in your shell:
<CodeGroup>
```bash zsh
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.zshrc
source ~/.zshrc
```bash
export MEM0_API_KEY="m0-your-api-key"
```
```bash bash
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.bashrc
source ~/.bashrc
```
</CodeGroup>
<Warning>
Already have `mem0` configured as an MCP server in Cursor? Remove the existing entry from your Cursor MCP settings before installing to avoid duplicate tools.
</Warning>
@@ -104,13 +103,14 @@ Once installed, the following tools are available in every Cursor session:
When installed via the Cursor Marketplace, Mem0 hooks into Cursor's lifecycle:
| Hook | Event | What it does |
|------|-------|-------------|
| **Session start** | `sessionStart` | Loads prior memories and displays status banner |
| **User prompt** | `beforeSubmitPrompt` | Searches relevant memories before each message; skips short prompts |
| **Pre-tool (2 handlers)** | `preToolUse` | Blocks MEMORY.md writes, enforces `user_id`/`app_id` on mem0 tool calls |
| **Post-tool (2 handlers)** | `postToolUse` | Tracks stats, scans bash errors for related memories |
| **Pre-compact** | `preCompact` | Stores a session summary before context compaction |
### Session Start
On every new session, the plugin prompts the agent to call `search_memories` to load relevant context from prior sessions.
### User Prompt
Before processing each user message, the plugin searches Mem0 for relevant memories and injects them into context. Short prompts are skipped to minimize latency.
### Pre-Compaction
Before context compaction, the plugin captures a comprehensive session summary so nothing is lost when the context window resets.
## Example Workflow
-119
View File
@@ -1,119 +0,0 @@
---
title: OpenCode
description: "Add persistent memory to OpenCode with the Mem0 plugin — MCP server, lifecycle hooks, and slash commands."
---
Add persistent memory to [**OpenCode**](https://opencode.ai) with the Mem0 plugin. Your agent forgets everything between sessions — Mem0 fixes that by storing decisions, preferences, and learnings so they carry over automatically.
## Prerequisites
1. A Mem0 API key (starts with `m0-`):
- <a href="https://app.mem0.ai/dashboard/api-keys?utm_source=oss&utm_medium=integration-opencode" rel="nofollow">Get your API key</a> (free sign-up at <a href="https://app.mem0.ai?utm_source=oss&utm_medium=integration-opencode" rel="nofollow">app.mem0.ai</a>)
2. Add it to your shell profile so it persists across sessions:
<CodeGroup>
```bash zsh
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.zshrc && source ~/.zshrc
```
```bash bash
echo 'export MEM0_API_KEY="m0-your-api-key"' >> ~/.bashrc && source ~/.bashrc
```
</CodeGroup>
## Installation
### Option A — Plugin Install (Recommended)
```bash
opencode plugin @mem0/opencode-plugin
```
Or using this command which does the same thing:
```bash
bunx @mem0/opencode-plugin@latest install
```
**Or let your agent do it** — paste this into OpenCode:
```
Install @mem0/opencode-plugin by following https://raw.githubusercontent.com/mem0ai/mem0/main/mem0-plugin/.opencode-plugin/README.md
```
All commands auto-add the plugin and MCP server to your `~/.config/opencode/opencode.json`. Restart OpenCode — you get the MCP server, lifecycle hooks, and all `/mem0:` slash commands.
### Option B — MCP Only
If you only need the memory tools without hooks or skills, add this to your `opencode.json` (project-level or global at `~/.config/opencode/opencode.json`):
```json
{
"mcp": {
"mem0": {
"type": "remote",
"url": "https://mcp.mem0.ai/mcp/",
"headers": {
"Authorization": "Token {env:MEM0_API_KEY}"
},
"oauth": false
}
}
}
```
## What's Included
| Component | Plugin (A) | MCP Only (B) |
|-----------|:----------:|:------------:|
| MCP Server (9 memory tools) | Yes | Yes |
| Lifecycle Hooks | Yes | No |
| 16 Slash Commands | Yes | No |
## Available MCP Tools
| Tool | Description |
|------|-------------|
| `add_memory` | Save text or conversation history for a user/agent |
| `search_memories` | Semantic search across memories with filters |
| `get_memories` | List memories with filters and pagination |
| `get_memory` | Retrieve a specific memory by ID |
| `update_memory` | Overwrite a memory's text by ID |
| `delete_memory` | Delete a single memory by ID |
| `delete_all_memories` | Bulk delete all memories in scope |
| `delete_entities` | Delete a user/agent/app/run entity and its memories |
| `list_entities` | List users/agents/apps/runs stored in Mem0 |
## Lifecycle Hooks
The plugin uses the [mem0ai](https://www.npmjs.com/package/mem0ai) TypeScript SDK directly — pure TypeScript, no Python, no shell scripts.
| OpenCode Event | Hook | What happens |
|----------------|------|-------------|
| `chat.message` | **Chat message** | Searches prior memories on session start, searches relevant memories before each prompt, auto-captures learnings periodically |
| `tool.execute.before` | **Pre-tool** | Blocks MEMORY.md writes, injects `user_id`/`app_id` on mem0 tool calls |
| `tool.execute.after` | **Post-tool** | Tracks stats, scans Bash errors and pre-fetches related error memories |
| `experimental.chat.system.transform` | **System transform** | Injects memory context (session memories, search results, error lookups) into the system prompt |
| `experimental.session.compacting` | **Compaction** | Stores session state memory, then injects prior memories into compaction context so nothing is lost |
| `shell.env` | **Shell env** | Exports `MEM0_USER_ID`, `MEM0_APP_ID`, `MEM0_SESSION_ID`, and `MEM0_BRANCH` to all shell executions |
## Troubleshooting
- **No tools appearing** — Restart OpenCode after installing
- **"Connection failed"** — Verify your key is set: `echo $MEM0_API_KEY`
- **Plugin not loading** — Run `opencode plugin @mem0/opencode-plugin` again, then restart
- **Hooks not firing** — Hooks require the plugin install (Option A). MCP-only installs don't include hooks.
<CardGroup cols={2}>
<Card title="Mem0 MCP Setup" icon="puzzle-piece" href="/platform/mem0-mcp">
Detailed MCP configuration for all clients
</Card>
<Card title="Antigravity Integration" icon="google" href="/integrations/antigravity">
Add Mem0 memory to Google Antigravity
</Card>
</CardGroup>
+1 -1
View File
@@ -12,7 +12,7 @@ mode: "custom"
</h1>
<p className="max-w-2xl mx-auto text-base text-gray-600 dark:text-zinc-400 leading-relaxed">
Universal, self-improving memory layer for LLM applications.
Universal, Self-improving memory layer for LLM applications.
</p>
<a
+1 -5
View File
@@ -268,8 +268,6 @@ If the user is on a pre-current major (Python < 2, TS < 3, or Platform `output_f
- [Claude Code](https://docs.mem0.ai/integrations/claude-code) [Both]: Use when wiring memory into Claude Code.
- [Cursor](https://docs.mem0.ai/integrations/cursor) [Both]: Use when wiring memory into Cursor.
- [Codex](https://docs.mem0.ai/integrations/codex) [Both]: Use when wiring memory into Codex / other editor assistants.
- [OpenCode](https://docs.mem0.ai/integrations/opencode) [Both]: Use when wiring memory into OpenCode.
- [Antigravity](https://docs.mem0.ai/integrations/antigravity) [Both]: Use when wiring memory into Google Antigravity.
### Voice & Real-time
- [LiveKit](https://docs.mem0.ai/integrations/livekit) [Both]: Use when building real-time voice/video with memory.
@@ -398,15 +396,13 @@ Each subdirectory is a Claude Code Skill (`SKILL.md` + supporting assets). Load
Source: https://github.com/mem0ai/mem0/tree/main/mem0-plugin
The `mem0-plugin/` directory provides MCP server connection, lifecycle hooks, and skill bundling for Claude Code, Cursor, Codex, OpenCode, and Antigravity. It exposes 9 MCP tools: `add_memory`, `search_memories`, `get_memories`, `get_memory`, `update_memory`, `delete_memory`, `delete_all_memories`, `delete_entities`, `list_entities`.
The `mem0-plugin/` directory provides MCP server connection, lifecycle hooks, and skill bundling for Claude Code, Cursor, and Codex. It exposes 9 MCP tools: `add_memory`, `search_memories`, `get_memories`, `get_memory`, `update_memory`, `delete_memory`, `delete_all_memories`, `delete_entities`, `list_entities`.
Editor-specific setup docs (already listed above under `## Integrations > AI Coding Tools`):
- `integrations/claude-code` [Both]
- `integrations/cursor` [Both]
- `integrations/codex` [Both]
- `integrations/opencode` [Both]
- `integrations/antigravity` [Both]
- `integrations/openclaw` [Both]
### MCP Endpoints
@@ -4,7 +4,22 @@ description: Fine-grained metadata queries for precise OSS memory retrieval.
icon: "filter"
---
Enhanced metadata filtering in Mem0 lets you run complex queries across memory metadata. Combine comparisons, logical operators, and wildcard matches to zero in on the exact memories your agent needs.
Enhanced metadata filtering in Mem0 1.0.0 lets you run complex queries across memory metadata. Combine comparisons, logical operators, and wildcard matches to zero in on the exact memories your agent needs.
<Info>
**You’ll use this when…**
- Retrieval must respect multiple metadata conditions before returning context.
- You need to mix numeric, boolean, and string filters in a single query.
- Agents rely on deterministic filtering instead of broad semantic search alone.
</Info>
<Warning>
Enhanced filtering requires Mem0 1.0.0 or later and a vector store that supports the operators you enable. Unsupported operators fall back to simple equality filters.
</Warning>
<Note>
The TypeScript SDK accepts the same filter shape shown here—transpose the dictionaries to objects and reuse the keys unchanged.
</Note>
---
@@ -133,8 +148,8 @@ Combine filters with `AND`, `OR`, and `NOT` to express complex decision trees. N
results = m.search(
"complex query",
filters={
"user_id": "alice",
"AND": [
{"user_id": "alice"},
{"category": "work"},
{"priority": {"gte": 7}},
{"status": {"ne": "completed"}}
@@ -146,11 +161,15 @@ results = m.search(
results = m.search(
"flexible query",
filters={
"user_id": "alice",
"OR": [
{"category": "urgent"},
{"priority": {"gte": 9}},
{"deadline": {"contains": "today"}}
"AND": [
{"user_id": "alice"},
{
"OR": [
{"category": "urgent"},
{"priority": {"gte": 9}},
{"deadline": {"contains": "today"}}
]
}
]
}
)
@@ -159,10 +178,14 @@ results = m.search(
results = m.search(
"exclusion query",
filters={
"user_id": "alice",
"NOT": [
{"category": "archived"},
{"status": "deleted"}
"AND": [
{"user_id": "alice"},
{
"NOT": [
{"category": "archived"},
{"status": "deleted"}
]
}
]
}
)
@@ -171,8 +194,8 @@ results = m.search(
results = m.search(
"advanced query",
filters={
"user_id": "alice",
"AND": [
{"user_id": "alice"},
{
"OR": [
{"category": "work"},
@@ -225,8 +248,8 @@ config = {
```python
# More efficient: Filter on indexed fields first
good_filters = {
"user_id": "alice",
"AND": [
{"user_id": "alice"},
{"category": "work"},
{"content": {"contains": "meeting"}}
]
@@ -234,9 +257,9 @@ good_filters = {
# Less efficient: Complex operations first
avoid_filters = {
"user_id": "alice",
"AND": [
{"description": {"icontains": "complex text search"}}
{"description": {"icontains": "complex text search"}},
{"user_id": "alice"}
]
}
```
@@ -279,8 +302,8 @@ results = m.search(
results = m.search(
"query",
filters={
"user_id": "alice",
"AND": [
{"user_id": "alice"},
{"category": "work"},
{"status": {"ne": "archived"}},
{"priority": {"gte": 5}}
@@ -304,8 +327,8 @@ results = m.search(
results = m.search(
"What tasks need attention?",
filters={
"user_id": "project_manager",
"AND": [
{"user_id": "project_manager"},
{"project": {"in": ["alpha", ""]}},
{"priority": {"gte": 8}},
{"status": {"ne": "completed"}},
@@ -331,8 +354,8 @@ results = m.search(
results = m.search(
"pending support issues",
filters={
"agent_id": "support_bot",
"AND": [
{"agent_id": "support_bot"},
{"ticket_status": {"ne": "resolved"}},
{"priority": {"in": ["high", "critical"]}},
{"created_date": {"gte": "2024-01-01"}},
@@ -357,8 +380,8 @@ results = m.search(
results = m.search(
"recommend content",
filters={
"user_id": "reader123",
"AND": [
{"user_id": "reader123"},
{
"OR": [
{"genre": {"in": ["sci-fi", "fantasy"]}},
+1 -1
View File
@@ -6,7 +6,7 @@ icon: "list"
# Self-Hosting Features Overview
Mem0 Open Source ships with capabilities that adapt memory behavior for production workloads—async operations, multimodal inputs, and fine-tuned retrieval. Configure these features with code or YAML to match your application's needs.
Mem0 Open Source ships with capabilities that adapt memory behavior for production workloads—async operations, graph relationships, multimodal inputs, and fine-tuned retrieval. Configure these features with code or YAML to match your application's needs.
<Info>
Start with the <Link href="/open-source/python-quickstart">Python quickstart</Link> to validate basic memory operations, then enable the features below when you need them.
+2 -2
View File
@@ -119,7 +119,7 @@ docker build -t mem0-api-server .
</Tip>
<Note>
The REST server reads the same configuration you use locally, so you can point it at your preferred LLM, vector store, and reranker without changing code.
The REST server reads the same configuration you use locally, so you can point it at your preferred LLM, vector store, graph backend, and reranker without changing code.
</Note>
---
@@ -319,7 +319,7 @@ The `/auth/*`, `/api-keys`, `/requests`, and `/entities` routes are new to the s
<CardGroup cols={2}>
<Card title="Configure OSS Components" icon="sliders" href="/open-source/configuration">
Fine-tune LLMs, vector stores, and rerankers that power the REST server.
Fine-tune LLMs, vector stores, and graph backends that power the REST server.
</Card>
<Card title="Automate Agent Integrations" icon="plug" href="/cookbooks/integrations/agents-sdk-tool">
See how services call the REST endpoints as part of an automation pipeline.
+3 -3
View File
@@ -64,7 +64,7 @@ const memory = new Memory({ apiKey: process.env.MEM0_API_KEY!, async: true });
</Tab>
</Tabs>
## Add memories with metadata
## Add memories with metadata and graph context
<Tabs>
<Tab title="Python">
@@ -109,7 +109,7 @@ const result = await memory.add(conversation, {
</Tabs>
<Info icon="check">
Successful calls return memories tagged with the metadata you passed. In the dashboard, verify the `trip=japan-2025` tag exists on the new memory.
Successful calls return memories tagged with the metadata you passed. In the dashboard, confirm a graph edge between “Morgan” and “Tokyo” and verify the `trip=japan-2025` tag exists.
</Info>
## Retrieve and refine
@@ -201,7 +201,7 @@ await memory.deleteAll({ userId: "traveler-42", runId: "planning-call-1" });
/>
<Card
title="Explore Reranker Search"
description="See how rerankers boost accuracy after advanced retrieval."
description="See how rerankers boost accuracy after vector + graph retrieval."
icon="sparkles"
href="/open-source/features/reranker-search"
/>
+3
View File
@@ -121,6 +121,7 @@ echo "Loves hiking on weekends" | mem0 add --user-id alice
| `-f, --file` | Read messages from a JSON file |
| `-m, --metadata` | Custom metadata as JSON |
| `--categories` | Categories (JSON array or comma-separated) |
| `--graph / --no-graph` | Enable or disable graph memory extraction |
| `-o, --output` | Output format: `text`, `json`, `quiet` |
### `mem0 search`
@@ -140,6 +141,7 @@ mem0 search "preferred tools" --user-id alice --output json --top-k 5
| `--rerank` | Enable reranking |
| `--keyword` | Use keyword search instead of semantic |
| `--filter` | Advanced filter expression (JSON) |
| `--graph / --no-graph` | Enable or disable graph in search |
| `-o, --output` | Output format: `text`, `json`, `table` |
### `mem0 list`
@@ -434,6 +436,7 @@ For non-interactive environments (CI, agent runtimes), set credentials via `mem0
| `MEM0_AGENT_ID` | Default agent ID |
| `MEM0_APP_ID` | Default app ID |
| `MEM0_RUN_ID` | Default run ID |
| `MEM0_ENABLE_GRAPH` | Enable graph memory (`true` / `false`) |
Environment variables take precedence over values in the config file, which take precedence over defaults.
+1 -1
View File
@@ -9,7 +9,7 @@ iconType: "solid"
<Accordion title="How does Mem0 work?">
Mem0 utilizes a sophisticated hybrid database system to efficiently manage and retrieve memories for AI agents and assistants. Each memory is linked to a unique identifier, such as a user ID or agent ID, enabling Mem0 to organize and access memories tailored to specific individuals or contexts.
When a message is added to Mem0 via the `add` method, the system extracts pertinent facts and preferences, distributing them in a managed vector store. This strategy ensures that diverse types of information are stored optimally, facilitating swift and effective searches.
When a message is added to Mem0 via the `add` method, the system extracts pertinent facts and preferences, distributing them across various data stores: a vector database and a graph database. This hybrid strategy ensures that diverse types of information are stored optimally, facilitating swift and effective searches.
When an AI agent or LLM needs to access memories, it employs the `search` method. Mem0 conducts a comprehensive search across these data stores, retrieving relevant information from each.
@@ -130,6 +130,7 @@ The Mem0 MCP server enables powerful memory capabilities for your AI application
## Performance tips
- Enable graph memories for relationship-aware recall
- Use specific filters when searching large memory sets
- Batch operations when adding multiple memories
- Monitor memory usage in the Mem0 dashboard
+2 -2
View File
@@ -1,10 +1,10 @@
---
title: Overview
description: "See how Mem0 Platform features evolve from baseline filters to advanced retrieval."
description: "See how Mem0 Platform features evolve from baseline filters to graph-powered retrieval."
icon: "list"
---
Mem0 Platform features help managed deployments scale from basic filtering to advanced retrieval and data governance. Use this page to pick the right feature lane for your team.
Mem0 Platform features help managed deployments scale from basic filtering to graph-powered retrieval and data governance. Use this page to pick the right feature lane for your team.
<Info>
New to the platform? Start with the <Link href="/platform/quickstart">Platform quickstart</Link>,
+3 -3
View File
@@ -11,7 +11,7 @@ Mem0 is the memory engine that keeps conversations contextual so users never rep
## Why it matters
- **Personalized replies**: Memories persist across users and agents, cutting prompt bloat and repeat questions.
- **Hosted stack**: Mem0 runs the vector store and rerankers—no provisioning, tuning, or maintenance.
- **Hosted stack**: Mem0 runs the vector store, graph services, and rerankers—no provisioning, tuning, or maintenance.
- **Enterprise controls**: Audit logs and workspace governance ship by default for production readiness.
<AccordionGroup>
@@ -21,7 +21,7 @@ Mem0 is the memory engine that keeps conversations contextual so users never rep
| --- | --- |
| Fast setup | Add a few lines of code and you’re production-ready—no vector database or LLM configuration required. |
| Production scale | Automatic scaling, high availability, and managed infrastructure so you focus on product work. |
| Advanced features | webhooks, multimodal support, and custom categories are ready to enable. |
| Advanced features | Graph memory, webhooks, multimodal support, and custom categories are ready to enable. |
| Enterprise ready | Audit logs, workspace governance, and dedicated support keep security and governance covered. |
</Accordion>
</AccordionGroup>
@@ -49,7 +49,7 @@ Mem0 is the memory engine that keeps conversations contextual so users never rep
Add, search, update, and delete workflows.
</Card>
<Card title="Explore Platform Features" icon="sparkles" href="/platform/features/platform-overview">
async clients and rerankers.
Graph memory, async clients, and rerankers.
</Card>
<Card title="Configure Advanced Operations" icon="bolt" href="/platform/advanced-memory-operations">
Metadata filters and per-request toggles.
+1
View File
@@ -57,6 +57,7 @@ Mem0 offers two powerful ways to add memory to your AI applications. Choose base
<Accordion title="Advanced Capabilities" icon="sparkles">
| Feature | Platform | Open Source |
|---------|----------|-------------|
| **Graph Memory** | ✅ (Managed) | ✅ (Self-configured) |
| **Multimodal support** | ✅ | ✅ |
| **Custom categories** | ✅ | Limited |
| **Advanced retrieval** | ✅ | ✅ |
+1 -1
View File
@@ -155,7 +155,7 @@ Learn how to search, update, and delete memories with complete CRUD operations
</Card>
<Card title="Platform Features" icon="star" href="/platform/features/platform-overview">
Explore advanced features like metadata filtering and webhooks
Explore advanced features like metadata filtering, graph memory, and webhooks
</Card>
<Card title="API Reference" icon="code" href="/api-reference/memory/add-memories">
+1 -1
View File
@@ -96,7 +96,7 @@ applications that gives agents persistent context across sessions.
Mem0 is a memory layer for AI apps — managed (Mem0 Platform) or self-hosted
(Open Source). It stores, retrieves, and manages user memories so agents
remember preferences, learn from interactions, and personalize over time.
Sub-50ms retrieval. Storage: vector embeddings.
Sub-50ms retrieval. Dual storage: vector embeddings + graph databases.
**Architecture Overview:**
- Memory is scoped by user_id, agent_id, or run_id
+8
View File
@@ -0,0 +1,8 @@
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Singh"
given-names: "Taranjeet"
title: "Embedchain"
date-released: 2023-06-20
url: "https://github.com/embedchain/embedchain"
+76
View File
@@ -0,0 +1,76 @@
# Contributing to embedchain
Let us make contribution easy, collaborative and fun.
## Submit your Contribution through PR
To make a contribution, follow these steps:
1. Fork and clone this repository
2. Do the changes on your fork with dedicated feature branch `feature/f1`
3. If you modified the code (new feature or bug-fix), please add tests for it
4. Include proper documentation / docstring and examples to run the feature
5. Check the linting
6. Ensure that all tests pass
7. Submit a pull request
For more details about pull requests, please read [GitHub's guides](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).
### 📦 Package manager
We use `poetry` as our package manager. You can install poetry by following the instructions [here](https://python-poetry.org/docs/#installation).
Please DO NOT use pip or conda to install the dependencies. Instead, use poetry:
```bash
make install_all
#activate
poetry shell
```
### 📌 Pre-commit
To ensure our standards, make sure to install pre-commit before starting to contribute.
```bash
pre-commit install
```
### 🧹 Linting
We use `ruff` to lint our code. You can run the linter by running the following command:
```bash
make lint
```
Make sure that the linter does not report any errors or warnings before submitting a pull request.
### Code Formatting with `black`
We use `black` to reformat the code by running the following command:
```bash
make format
```
### 🧪 Testing
We use `pytest` to test our code. You can run the tests by running the following command:
```bash
poetry run pytest
```
Several packages have been removed from Poetry to make the package lighter. Therefore, it is recommended to run `make install_all` to install the remaining packages and ensure all tests pass.
Make sure that all tests pass before submitting a pull request.
## 🚀 Release Process
At the moment, the release process is manual. We try to make frequent releases. Usually, we release a new version when we have a new feature or bugfix. A developer with admin rights to the repository will create a new release on GitHub, and then publish the new version to PyPI.
@@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [2026] [Taranjeet Singh]
Copyright [2023] [Taranjeet Singh]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
+56
View File
@@ -0,0 +1,56 @@
# Variables
PYTHON := python3
PIP := $(PYTHON) -m pip
PROJECT_NAME := embedchain
# Targets
.PHONY: install format lint clean test ci_lint ci_test coverage
install:
poetry install
# TODO: use a more efficient way to install these packages
install_all:
poetry install --all-extras
poetry run pip install ruff==0.6.9 pinecone-text pinecone-client langchain-anthropic "unstructured[local-inference, all-docs]" ollama langchain_together==0.1.3 \
langchain_cohere==0.1.5 deepgram-sdk==3.2.7 langchain-huggingface psutil clarifai==10.0.1 flask==2.3.3 twilio==8.5.0 fastapi-poe==0.0.16 discord==2.3.2 \
slack-sdk==3.21.3 huggingface_hub==0.23.0 gitpython==3.1.38 yt_dlp==2023.11.14 PyGithub==1.59.1 feedparser==6.0.10 newspaper3k==0.2.8 listparser==0.19 \
modal==0.56.4329 dropbox==11.36.2 boto3==1.34.20 youtube-transcript-api==0.6.1 pytube==15.0.0 beautifulsoup4==4.12.3
install_es:
poetry install --extras elasticsearch
install_opensearch:
poetry install --extras opensearch
install_milvus:
poetry install --extras milvus
shell:
poetry shell
py_shell:
poetry run python
format:
$(PYTHON) -m black .
$(PYTHON) -m isort .
clean:
rm -rf dist build *.egg-info
lint:
poetry run ruff .
build:
poetry build
publish:
poetry publish
# for example: make test file=tests/test_factory.py
test:
poetry run pytest $(file)
coverage:
poetry run pytest --cov=$(PROJECT_NAME) --cov-report=xml
+125
View File
@@ -0,0 +1,125 @@
<p align="center">
<img src="docs/logo/dark.svg" width="400px" alt="Embedchain Logo">
</p>
<p align="center">
<a href="https://pypi.org/project/embedchain/">
<img src="https://img.shields.io/pypi/v/embedchain" alt="PyPI">
</a>
<a href="https://pepy.tech/project/embedchain">
<img src="https://static.pepy.tech/badge/embedchain" alt="Downloads">
</a>
<a href="https://embedchain.ai/slack">
<img src="https://img.shields.io/badge/slack-embedchain-brightgreen.svg?logo=slack" alt="Slack">
</a>
<a href="https://embedchain.ai/discord">
<img src="https://dcbadge.vercel.app/api/server/6PzXDgEjG5?style=flat" alt="Discord">
</a>
<a href="https://twitter.com/embedchain">
<img src="https://img.shields.io/twitter/follow/embedchain" alt="Twitter">
</a>
<a href="https://colab.research.google.com/drive/138lMWhENGeEu7Q1-6lNbNTHGLZXBBz_B?usp=sharing">
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab">
</a>
<a href="https://codecov.io/gh/embedchain/embedchain">
<img src="https://codecov.io/gh/embedchain/embedchain/graph/badge.svg?token=EMRRHZXW1Q" alt="codecov">
</a>
</p>
<hr />
## What is Embedchain?
Embedchain is an Open Source Framework for personalizing LLM responses. It makes it easy to create and deploy personalized AI apps. At its core, Embedchain follows the design principle of being *"Conventional but Configurable"* to serve both software engineers and machine learning engineers.
Embedchain streamlines the creation of personalized LLM applications, offering a seamless process for managing various types of unstructured data. It efficiently segments data into manageable chunks, generates relevant embeddings, and stores them in a vector database for optimized retrieval. With a suite of diverse APIs, it enables users to extract contextual information, find precise answers, or engage in interactive chat conversations, all tailored to their own data.
## 🔧 Quick install
### Python API
```bash
pip install embedchain
```
## ✨ Live demo
Checkout the [Chat with PDF](https://embedchain.ai/demo/chat-pdf) live demo we created using Embedchain. You can find the source code [here](https://github.com/mem0ai/mem0/tree/main/embedchain/examples/chat-pdf).
## 🔍 Usage
<!-- Demo GIF or Image -->
<p align="center">
<img src="docs/images/cover.gif" width="900px" alt="Embedchain Demo">
</p>
For example, you can create an Elon Musk bot using the following code:
```python
import os
from embedchain import App
# Create a bot instance
os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>"
app = App()
# Embed online resources
app.add("https://en.wikipedia.org/wiki/Elon_Musk")
app.add("https://www.forbes.com/profile/elon-musk")
# Query the app
app.query("How many companies does Elon Musk run and name those?")
# Answer: Elon Musk currently runs several companies. As of my knowledge, he is the CEO and lead designer of SpaceX, the CEO and product architect of Tesla, Inc., the CEO and founder of Neuralink, and the CEO and founder of The Boring Company. However, please note that this information may change over time, so it's always good to verify the latest updates.
```
You can also try it in your browser with Google Colab:
[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/17ON1LPonnXAtLaZEebnOktstB_1cJJmh?usp=sharing)
## 📖 Documentation
Comprehensive guides and API documentation are available to help you get the most out of Embedchain:
- [Introduction](https://docs.embedchain.ai/get-started/introduction#what-is-embedchain)
- [Getting Started](https://docs.embedchain.ai/get-started/quickstart)
- [Examples](https://docs.embedchain.ai/examples)
- [Supported data types](https://docs.embedchain.ai/components/data-sources/overview)
## 🔗 Join the Community
* Connect with fellow developers by joining our [Slack Community](https://embedchain.ai/slack) or [Discord Community](https://embedchain.ai/discord).
* Dive into [GitHub Discussions](https://github.com/embedchain/embedchain/discussions), ask questions, or share your experiences.
## 🤝 Schedule a 1-on-1 Session
Book a [1-on-1 Session](https://cal.com/taranjeetio/ec) with the founders, to discuss any issues, provide feedback, or explore how we can improve Embedchain for you.
## 🌐 Contributing
Contributions are welcome! Please check out the issues on the repository, and feel free to open a pull request.
For more information, please see the [contributing guidelines](CONTRIBUTING.md).
For more reference, please go through [Development Guide](https://docs.embedchain.ai/contribution/dev) and [Documentation Guide](https://docs.embedchain.ai/contribution/docs).
<a href="https://github.com/embedchain/embedchain/graphs/contributors">
<img src="https://contrib.rocks/image?repo=embedchain/embedchain" />
</a>
## Anonymous Telemetry
We collect anonymous usage metrics to enhance our package's quality and user experience. This includes data like feature usage frequency and system info, but never personal details. The data helps us prioritize improvements and ensure compatibility. If you wish to opt-out, set the environment variable `EC_TELEMETRY=false`. We prioritize data security and don't share this data externally.
## Citation
If you utilize this repository, please consider citing it with:
```
@misc{embedchain,
author = {Taranjeet Singh, Deshraj Yadav},
title = {Embedchain: The Open Source RAG Framework},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/embedchain/embedchain}},
}
```
+8
View File
@@ -0,0 +1,8 @@
llm:
provider: anthropic
config:
model: 'claude-instant-1'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
+15
View File
@@ -0,0 +1,15 @@
llm:
provider: aws_bedrock
config:
model: amazon.titan-text-express-v1
deployment_name: your_llm_deployment_name
temperature: 0.5
max_tokens: 8192
top_p: 1
stream: false
embedder::
provider: aws_bedrock
config:
model: amazon.titan-embed-text-v2:0
deployment_name: you_embedding_model_deployment_name
+19
View File
@@ -0,0 +1,19 @@
app:
config:
id: azure-openai-app
llm:
provider: azure_openai
config:
model: gpt-35-turbo
deployment_name: your_llm_deployment_name
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
embedder:
provider: azure_openai
config:
model: text-embedding-ada-002
deployment_name: you_embedding_model_deployment_name
+24
View File
@@ -0,0 +1,24 @@
app:
config:
id: 'my-app'
llm:
provider: openai
config:
model: 'gpt-4o-mini'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
vectordb:
provider: chroma
config:
collection_name: 'my-app'
dir: db
allow_reset: true
embedder:
provider: openai
config:
model: 'text-embedding-ada-002'
+4
View File
@@ -0,0 +1,4 @@
chunker:
chunk_size: 100
chunk_overlap: 20
length_function: 'len'
+12
View File
@@ -0,0 +1,12 @@
llm:
provider: clarifai
config:
model: "https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct"
model_kwargs:
temperature: 0.5
max_tokens: 1000
embedder:
provider: clarifai
config:
model: "https://clarifai.com/clarifai/main/models/BAAI-bge-base-en-v15"
+7
View File
@@ -0,0 +1,7 @@
llm:
provider: cohere
config:
model: large
temperature: 0.5
max_tokens: 1000
top_p: 1
+40
View File
@@ -0,0 +1,40 @@
app:
config:
id: 'full-stack-app'
chunker:
chunk_size: 100
chunk_overlap: 20
length_function: 'len'
llm:
provider: openai
config:
model: 'gpt-4o-mini'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
prompt: |
Use the following pieces of context to answer the query at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
$context
Query: $query
Helpful Answer:
system_prompt: |
Act as William Shakespeare. Answer the following questions in the style of William Shakespeare.
vectordb:
provider: chroma
config:
collection_name: 'my-collection-name'
dir: db
allow_reset: true
embedder:
provider: openai
config:
model: 'text-embedding-ada-002'
+13
View File
@@ -0,0 +1,13 @@
llm:
provider: google
config:
model: gemini-pro
max_tokens: 1000
temperature: 0.9
top_p: 1.0
stream: false
embedder:
provider: google
config:
model: models/embedding-001
+8
View File
@@ -0,0 +1,8 @@
llm:
provider: openai
config:
model: 'gpt-4'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
+11
View File
@@ -0,0 +1,11 @@
llm:
provider: gpt4all
config:
model: 'orca-mini-3b-gguf2-q4_0.gguf'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
embedder:
provider: gpt4all
+8
View File
@@ -0,0 +1,8 @@
llm:
provider: huggingface
config:
model: 'google/flan-t5-xxl'
temperature: 0.5
max_tokens: 1000
top_p: 0.5
stream: false
+7
View File
@@ -0,0 +1,7 @@
llm:
provider: jina
config:
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
+8
View File
@@ -0,0 +1,8 @@
llm:
provider: llama2
config:
model: 'a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5'
temperature: 0.5
max_tokens: 1000
top_p: 0.5
stream: false
+14
View File
@@ -0,0 +1,14 @@
llm:
provider: ollama
config:
model: 'llama2'
temperature: 0.5
top_p: 1
stream: true
base_url: http://localhost:11434
embedder:
provider: ollama
config:
model: 'mxbai-embed-large:latest'
base_url: http://localhost:11434
+33
View File
@@ -0,0 +1,33 @@
app:
config:
id: 'my-app'
log_level: 'WARNING'
collect_metrics: true
collection_name: 'my-app'
llm:
provider: openai
config:
model: 'gpt-4o-mini'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
vectordb:
provider: opensearch
config:
opensearch_url: 'https://localhost:9200'
http_auth:
- admin
- admin
vector_dimension: 1536
collection_name: 'my-app'
use_ssl: false
verify_certs: false
embedder:
provider: openai
config:
model: 'text-embedding-ada-002'
deployment_name: 'my-app'
+25
View File
@@ -0,0 +1,25 @@
app:
config:
id: 'open-source-app'
collect_metrics: false
llm:
provider: gpt4all
config:
model: 'orca-mini-3b-gguf2-q4_0.gguf'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
vectordb:
provider: chroma
config:
collection_name: 'open-source-app'
dir: db
allow_reset: true
embedder:
provider: gpt4all
config:
deployment_name: 'test-deployment'
+6
View File
@@ -0,0 +1,6 @@
vectordb:
provider: pinecone
config:
metric: cosine
vector_dimension: 1536
collection_name: my-pinecone-index
+26
View File
@@ -0,0 +1,26 @@
pipeline:
config:
name: Example pipeline
id: pipeline-1 # Make sure that id is different every time you create a new pipeline
vectordb:
provider: chroma
config:
collection_name: pipeline-1
dir: db
allow_reset: true
llm:
provider: gpt4all
config:
model: 'orca-mini-3b-gguf2-q4_0.gguf'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
embedding_model:
provider: gpt4all
config:
model: 'all-MiniLM-L6-v2'
deployment_name: null
+6
View File
@@ -0,0 +1,6 @@
llm:
provider: together
config:
model: mistralai/Mixtral-8x7B-Instruct-v0.1
temperature: 0.5
max_tokens: 1000
+6
View File
@@ -0,0 +1,6 @@
llm:
provider: vertexai
config:
model: 'chat-bison'
temperature: 0.5
top_p: 0.5
+14
View File
@@ -0,0 +1,14 @@
llm:
provider: vllm
config:
model: 'meta-llama/Llama-2-70b-hf'
temperature: 0.5
top_p: 1
top_k: 10
stream: true
trust_remote_code: true
embedder:
provider: huggingface
config:
model: 'BAAI/bge-small-en-v1.5'
+4
View File
@@ -0,0 +1,4 @@
vectordb:
provider: weaviate
config:
collection_name: my_weaviate_index
+10
View File
@@ -0,0 +1,10 @@
install:
npm i -g mintlify
run_local:
mintlify dev
troubleshoot:
mintlify install
.PHONY: install run_local troubleshoot
+25
View File
@@ -0,0 +1,25 @@
# Contributing to embedchain docs
### 👩‍💻 Development
Install the [Mintlify CLI](https://www.npmjs.com/package/mintlify) to preview the documentation changes locally. To install, use the following command
```
npm i -g mintlify
```
Run the following command at the root of your documentation (where mint.json is)
```
mintlify dev
```
### 😎 Publishing Changes
Changes will be deployed to production automatically after your PR is merged to the main branch.
#### Troubleshooting
- Mintlify dev isn't running - Run `mintlify install` it'll re-install dependencies.
- Page loads as a 404 - Make sure you are running in a folder with `mint.json`
+11
View File
@@ -0,0 +1,11 @@
<CardGroup cols={3}>
<Card title="Talk to founders" icon="calendar" href="https://cal.com/taranjeetio/ec">
Schedule a call
</Card>
<Card title="Slack" icon="slack" href="https://embedchain.ai/slack" color="#4A154B">
Join our slack community
</Card>
<Card title="Discord" icon="discord" href="https://discord.gg/6PzXDgEjG5" color="#7289DA">
Join our discord community
</Card>
</CardGroup>
@@ -0,0 +1,19 @@
<p>If you can't find the specific data source, please feel free to request through one of the following channels and help us prioritize.</p>
<CardGroup cols={2}>
<Card title="Google Form" icon="file" href="https://forms.gle/NDRCKsRpUHsz2Wcm8" color="#7387d0">
Fill out this form
</Card>
<Card title="Slack" icon="slack" href="https://embedchain.ai/slack" color="#4A154B">
Let us know on our slack community
</Card>
<Card title="Discord" icon="discord" href="https://discord.gg/6PzXDgEjG5" color="#7289DA">
Let us know on discord community
</Card>
<Card title="GitHub" icon="github" href="https://github.com/embedchain/embedchain/issues/new?assignees=&labels=&projects=&template=feature_request.yml" color="#181717">
Open an issue on our GitHub
</Card>
<Card title="Schedule a call" icon="calendar" href="https://cal.com/taranjeetio/ec">
Schedule a call with Embedchain founder
</Card>
</CardGroup>
@@ -0,0 +1,16 @@
<p>If you can't find the specific LLM you need, no need to fret. We're continuously expanding our support for additional LLMs, and you can help us prioritize by opening an issue on our GitHub or simply reaching out to us on our Slack or Discord community.</p>
<CardGroup cols={2}>
<Card title="Slack" icon="slack" href="https://embedchain.ai/slack" color="#4A154B">
Let us know on our slack community
</Card>
<Card title="Discord" icon="discord" href="https://discord.gg/6PzXDgEjG5" color="#7289DA">
Let us know on discord community
</Card>
<Card title="GitHub" icon="github" href="https://github.com/embedchain/embedchain/issues/new?assignees=&labels=&projects=&template=feature_request.yml" color="#181717">
Open an issue on our GitHub
</Card>
<Card title="Schedule a call" icon="calendar" href="https://cal.com/taranjeetio/ec">
Schedule a call with Embedchain founder
</Card>
</CardGroup>
@@ -0,0 +1,18 @@
<p>If you can't find specific feature or run into issues, please feel free to reach out through one of the following channels.</p>
<CardGroup cols={2}>
<Card title="Slack" icon="slack" href="https://embedchain.ai/slack" color="#4A154B">
Let us know on our slack community
</Card>
<Card title="Discord" icon="discord" href="https://discord.gg/6PzXDgEjG5" color="#7289DA">
Let us know on discord community
</Card>
<Card title="GitHub" icon="github" href="https://github.com/embedchain/embedchain/issues/new?assignees=&labels=&projects=&template=feature_request.yml" color="#181717">
Open an issue on our GitHub
</Card>
<Card title="Schedule a call" icon="calendar" href="https://cal.com/taranjeetio/ec">
Schedule a call with Embedchain founder
</Card>
</CardGroup>
@@ -0,0 +1,273 @@
---
title: 'Custom configurations'
---
Embedchain offers several configuration options for your LLM, vector database, and embedding model. All of these configuration options are optional and have sane defaults.
You can configure different components of your app (`llm`, `embedding model`, or `vector database`) through a simple yaml configuration that Embedchain offers. Here is a generic full-stack example of the yaml config:
<Tip>
Embedchain applications are configurable using YAML file, JSON file or by directly passing the config dictionary. Checkout the [docs here](/api-reference/app/overview#usage) on how to use other formats.
</Tip>
<CodeGroup>
```yaml config.yaml
app:
config:
name: 'full-stack-app'
llm:
provider: openai
config:
model: 'gpt-4o-mini'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
api_key: sk-xxx
model_kwargs:
response_format:
type: json_object
api_version: 2024-02-01
http_client_proxies: http://testproxy.mem0.net:8000
prompt: |
Use the following pieces of context to answer the query at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
$context
Query: $query
Helpful Answer:
system_prompt: |
Act as William Shakespeare. Answer the following questions in the style of William Shakespeare.
vectordb:
provider: chroma
config:
collection_name: 'full-stack-app'
dir: db
allow_reset: true
embedder:
provider: openai
config:
model: 'text-embedding-ada-002'
api_key: sk-xxx
http_client_proxies: http://testproxy.mem0.net:8000
chunker:
chunk_size: 2000
chunk_overlap: 100
length_function: 'len'
min_chunk_size: 0
cache:
similarity_evaluation:
strategy: distance
max_distance: 1.0
config:
similarity_threshold: 0.8
auto_flush: 50
memory:
top_k: 10
```
```json config.json
{
"app": {
"config": {
"name": "full-stack-app"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini",
"temperature": 0.5,
"max_tokens": 1000,
"top_p": 1,
"stream": false,
"prompt": "Use the following pieces of context to answer the query at the end.\nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\n$context\n\nQuery: $query\n\nHelpful Answer:",
"system_prompt": "Act as William Shakespeare. Answer the following questions in the style of William Shakespeare.",
"api_key": "sk-xxx",
"model_kwargs": {"response_format": {"type": "json_object"}},
"api_version": "2024-02-01",
"http_client_proxies": "http://testproxy.mem0.net:8000"
}
},
"vectordb": {
"provider": "chroma",
"config": {
"collection_name": "full-stack-app",
"dir": "db",
"allow_reset": true
}
},
"embedder": {
"provider": "openai",
"config": {
"model": "text-embedding-ada-002",
"api_key": "sk-xxx",
"http_client_proxies": "http://testproxy.mem0.net:8000"
}
},
"chunker": {
"chunk_size": 2000,
"chunk_overlap": 100,
"length_function": "len",
"min_chunk_size": 0
},
"cache": {
"similarity_evaluation": {
"strategy": "distance",
"max_distance": 1.0
},
"config": {
"similarity_threshold": 0.8,
"auto_flush": 50
}
},
"memory": {
"top_k": 10
}
}
```
```python config.py
config = {
'app': {
'config': {
'name': 'full-stack-app'
}
},
'llm': {
'provider': 'openai',
'config': {
'model': 'gpt-4o-mini',
'temperature': 0.5,
'max_tokens': 1000,
'top_p': 1,
'stream': False,
'prompt': (
"Use the following pieces of context to answer the query at the end.\n"
"If you don't know the answer, just say that you don't know, don't try to make up an answer.\n"
"$context\n\nQuery: $query\n\nHelpful Answer:"
),
'system_prompt': (
"Act as William Shakespeare. Answer the following questions in the style of William Shakespeare."
),
'api_key': 'sk-xxx',
"model_kwargs": {"response_format": {"type": "json_object"}},
"http_client_proxies": "http://testproxy.mem0.net:8000",
}
},
'vectordb': {
'provider': 'chroma',
'config': {
'collection_name': 'full-stack-app',
'dir': 'db',
'allow_reset': True
}
},
'embedder': {
'provider': 'openai',
'config': {
'model': 'text-embedding-ada-002',
'api_key': 'sk-xxx',
"http_client_proxies": "http://testproxy.mem0.net:8000",
}
},
'chunker': {
'chunk_size': 2000,
'chunk_overlap': 100,
'length_function': 'len',
'min_chunk_size': 0
},
'cache': {
'similarity_evaluation': {
'strategy': 'distance',
'max_distance': 1.0,
},
'config': {
'similarity_threshold': 0.8,
'auto_flush': 50,
},
},
'memory': {
'top_k': 10,
},
}
```
</CodeGroup>
Alright, let's dive into what each key means in the yaml config above:
1. `app` Section:
- `config`:
- `name` (String): The name of your full-stack application.
- `id` (String): The id of your full-stack application.
<Note>Only use this to reload already created apps. We recommend users not to create their own ids.</Note>
- `collect_metrics` (Boolean): Indicates whether metrics should be collected for the app, defaults to `True`
- `log_level` (String): The log level for the app, defaults to `WARNING`
2. `llm` Section:
- `provider` (String): The provider for the language model, which is set to 'openai'. You can find the full list of llm providers in [our docs](/components/llms).
- `config`:
- `model` (String): The specific model being used, 'gpt-4o-mini'.
- `temperature` (Float): Controls the randomness of the model's output. A higher value (closer to 1) makes the output more random.
- `max_tokens` (Integer): Controls how many tokens are used in the response.
- `top_p` (Float): Controls the diversity of word selection. A higher value (closer to 1) makes word selection more diverse.
- `stream` (Boolean): Controls if the response is streamed back to the user (set to false).
- `online` (Boolean): Controls whether to use internet to get more context for answering query (set to false).
- `token_usage` (Boolean): Controls whether to use token usage for the querying models (set to false).
- `prompt` (String): A prompt for the model to follow when generating responses, requires `$context` and `$query` variables.
- `system_prompt` (String): A system prompt for the model to follow when generating responses, in this case, it's set to the style of William Shakespeare.
- `number_documents` (Integer): Number of documents to pull from the vectordb as context, defaults to 1
- `api_key` (String): The API key for the language model.
- `model_kwargs` (Dict): Keyword arguments to pass to the language model. Used for `aws_bedrock` provider, since it requires different arguments for each model.
- `http_client_proxies` (Dict | String): The proxy server settings used to create `self.http_client` using `httpx.Client(proxies=http_client_proxies)`
- `http_async_client_proxies` (Dict | String): The proxy server settings for async calls used to create `self.http_async_client` using `httpx.AsyncClient(proxies=http_async_client_proxies)`
3. `vectordb` Section:
- `provider` (String): The provider for the vector database, set to 'chroma'. You can find the full list of vector database providers in [our docs](/components/vector-databases).
- `config`:
- `collection_name` (String): The initial collection name for the vectordb, set to 'full-stack-app'.
- `dir` (String): The directory for the local database, set to 'db'.
- `allow_reset` (Boolean): Indicates whether resetting the vectordb is allowed, set to true.
- `batch_size` (Integer): The batch size for docs insertion in vectordb, defaults to `100`
<Note>We recommend you to checkout vectordb specific config [here](https://docs.embedchain.ai/components/vector-databases)</Note>
4. `embedder` Section:
- `provider` (String): The provider for the embedder, set to 'openai'. You can find the full list of embedding model providers in [our docs](/components/embedding-models).
- `config`:
- `model` (String): The specific model used for text embedding, 'text-embedding-ada-002'.
- `vector_dimension` (Integer): The vector dimension of the embedding model. [Defaults](https://github.com/embedchain/embedchain/blob/main/embedchain/models/vector_dimensions.py)
- `api_key` (String): The API key for the embedding model.
- `endpoint` (String): The endpoint for the HuggingFace embedding model.
- `deployment_name` (String): The deployment name for the embedding model.
- `title` (String): The title for the embedding model for Google Embedder.
- `task_type` (String): The task type for the embedding model for Google Embedder.
- `model_kwargs` (Dict): Used to pass extra arguments to embedders.
- `http_client_proxies` (Dict | String): The proxy server settings used to create `self.http_client` using `httpx.Client(proxies=http_client_proxies)`
- `http_async_client_proxies` (Dict | String): The proxy server settings for async calls used to create `self.http_async_client` using `httpx.AsyncClient(proxies=http_async_client_proxies)`
5. `chunker` Section:
- `chunk_size` (Integer): The size of each chunk of text that is sent to the language model.
- `chunk_overlap` (Integer): The amount of overlap between each chunk of text.
- `length_function` (String): The function used to calculate the length of each chunk of text. In this case, it's set to 'len'. You can also use any function import directly as a string here.
- `min_chunk_size` (Integer): The minimum size of each chunk of text that is sent to the language model. Must be less than `chunk_size`, and greater than `chunk_overlap`.
6. `cache` Section: (Optional)
- `similarity_evaluation` (Optional): The config for similarity evaluation strategy. If not provided, the default `distance` based similarity evaluation strategy is used.
- `strategy` (String): The strategy to use for similarity evaluation. Currently, only `distance` and `exact` based similarity evaluation is supported. Defaults to `distance`.
- `max_distance` (Float): The bound of maximum distance. Defaults to `1.0`.
- `positive` (Boolean): If the larger distance indicates more similar of two entities, set it `True`, otherwise `False`. Defaults to `False`.
- `config` (Optional): The config for initializing the cache. If not provided, sensible default values are used as mentioned below.
- `similarity_threshold` (Float): The threshold for similarity evaluation. Defaults to `0.8`.
- `auto_flush` (Integer): The number of queries after which the cache is flushed. Defaults to `20`.
7. `memory` Section: (Optional)
- `top_k` (Integer): The number of top-k results to return. Defaults to `10`.
<Note>
If you provide a cache section, the app will automatically configure and use a cache to store the results of the language model. This is useful if you want to speed up the response time and save inference cost of your app.
</Note>
If you have questions about the configuration above, please feel free to reach out to us using one of the following methods:
<Snippet file="get-help.mdx" />
+47
View File
@@ -0,0 +1,47 @@
---
title: '📊 add'
---
`add()` method is used to load the data sources from different data sources to a RAG pipeline. You can find the signature below:
### Parameters
<ParamField path="source" type="str">
The data to embed, can be a URL, local file or raw content, depending on the data type.. You can find the full list of supported data sources [here](/components/data-sources/overview).
</ParamField>
<ParamField path="data_type" type="str" optional>
Type of data source. It can be automatically detected but user can force what data type to load as.
</ParamField>
<ParamField path="metadata" type="dict" optional>
Any metadata that you want to store with the data source. Metadata is generally really useful for doing metadata filtering on top of semantic search to yield faster search and better results.
</ParamField>
<ParamField path="all_references" type="bool" optional>
This parameter instructs Embedchain to retrieve all the context and information from the specified link, as well as from any reference links on the page.
</ParamField>
## Usage
### Load data from webpage
```python Code example
from embedchain import App
app = App()
app.add("https://www.forbes.com/profile/elon-musk")
# Inserting batches in chromadb: 100%|███████████████| 1/1 [00:00<00:00, 1.19it/s]
# Successfully saved https://www.forbes.com/profile/elon-musk (DataType.WEB_PAGE). New chunks count: 4
```
### Load data from sitemap
```python Code example
from embedchain import App
app = App()
app.add("https://python.langchain.com/sitemap.xml", data_type="sitemap")
# Loading pages: 100%|█████████████| 1108/1108 [00:47<00:00, 23.17it/s]
# Inserting batches in chromadb: 100%|█████████| 111/111 [04:41<00:00, 2.54s/it]
# Successfully saved https://python.langchain.com/sitemap.xml (DataType.SITEMAP). New chunks count: 11024
```
You can find complete list of supported data sources [here](/components/data-sources/overview).
+175
View File
@@ -0,0 +1,175 @@
---
title: '💬 chat'
---
`chat()` method allows you to chat over your data sources using a user-friendly chat API. You can find the signature below:
### Parameters
<ParamField path="input_query" type="str">
Question to ask
</ParamField>
<ParamField path="config" type="BaseLlmConfig" optional>
Configure different llm settings such as prompt, temprature, number_documents etc.
</ParamField>
<ParamField path="dry_run" type="bool" optional>
The purpose is to test the prompt structure without actually running LLM inference. Defaults to `False`
</ParamField>
<ParamField path="where" type="dict" optional>
A dictionary of key-value pairs to filter the chunks from the vector database. Defaults to `None`
</ParamField>
<ParamField path="session_id" type="str" optional>
Session ID of the chat. This can be used to maintain chat history of different user sessions. Default value: `default`
</ParamField>
<ParamField path="citations" type="bool" optional>
Return citations along with the LLM answer. Defaults to `False`
</ParamField>
### Returns
<ResponseField name="answer" type="str | tuple">
If `citations=False`, return a stringified answer to the question asked. <br />
If `citations=True`, returns a tuple with answer and citations respectively.
</ResponseField>
## Usage
### With citations
If you want to get the answer to question and return both answer and citations, use the following code snippet:
```python With Citations
from embedchain import App
# Initialize app
app = App()
# Add data source
app.add("https://www.forbes.com/profile/elon-musk")
# Get relevant answer for your query
answer, sources = app.chat("What is the net worth of Elon?", citations=True)
print(answer)
# Answer: The net worth of Elon Musk is $221.9 billion.
print(sources)
# [
# (
# 'Elon Musk PROFILEElon MuskCEO, Tesla$247.1B$2.3B (0.96%)Real Time Net Worthas of 12/7/23 ...',
# {
# 'url': 'https://www.forbes.com/profile/elon-musk',
# 'score': 0.89,
# ...
# }
# ),
# (
# '74% of the company, which is now called X.Wealth HistoryHOVER TO REVEAL NET WORTH BY YEARForbes ...',
# {
# 'url': 'https://www.forbes.com/profile/elon-musk',
# 'score': 0.81,
# ...
# }
# ),
# (
# 'founded in 2002, is worth nearly $150 billion after a $750 million tender offer in June 2023 ...',
# {
# 'url': 'https://www.forbes.com/profile/elon-musk',
# 'score': 0.73,
# ...
# }
# )
# ]
```
<Note>
When `citations=True`, note that the returned `sources` are a list of tuples where each tuple has two elements (in the following order):
1. source chunk
2. dictionary with metadata about the source chunk
- `url`: url of the source
- `doc_id`: document id (used for book keeping purposes)
- `score`: score of the source chunk with respect to the question
- other metadata you might have added at the time of adding the source
</Note>
### Without citations
If you just want to return answers and don't want to return citations, you can use the following example:
```python Without Citations
from embedchain import App
# Initialize app
app = App()
# Add data source
app.add("https://www.forbes.com/profile/elon-musk")
# Chat on your data using `.chat()`
answer = app.chat("What is the net worth of Elon?")
print(answer)
# Answer: The net worth of Elon Musk is $221.9 billion.
```
### With session id
If you want to maintain chat sessions for different users, you can simply pass the `session_id` keyword argument. See the example below:
```python With session id
from embedchain import App
app = App()
app.add("https://www.forbes.com/profile/elon-musk")
# Chat on your data using `.chat()`
app.chat("What is the net worth of Elon Musk?", session_id="user1")
# 'The net worth of Elon Musk is $250.8 billion.'
app.chat("What is the net worth of Bill Gates?", session_id="user2")
# "I don't know the current net worth of Bill Gates."
app.chat("What was my last question", session_id="user1")
# 'Your last question was "What is the net worth of Elon Musk?"'
```
### With custom context window
If you want to customize the context window that you want to use during chat (default context window is 3 document chunks), you can do using the following code snippet:
```python with custom chunks size
from embedchain import App
from embedchain.config import BaseLlmConfig
app = App()
app.add("https://www.forbes.com/profile/elon-musk")
query_config = BaseLlmConfig(number_documents=5)
app.chat("What is the net worth of Elon Musk?", config=query_config)
```
### With Mem0 to store chat history
Mem0 is a cutting-edge long-term memory for LLMs to enable personalization for the GenAI stack. It enables LLMs to remember past interactions and provide more personalized responses.
In order to use Mem0 to enable memory for personalization in your apps:
- Install the [`mem0`](https://docs.mem0.ai/) package using `pip install mem0ai`.
- Prepare config for `memory`, refer [Configurations](docs/api-reference/advanced/configuration.mdx).
```python with mem0
from embedchain import App
config = {
"memory": {
"top_k": 5
}
}
app = App.from_config(config=config)
app.add("https://www.forbes.com/profile/elon-musk")
app.chat("What is the net worth of Elon Musk?")
```
## How Mem0 works:
- Mem0 saves context derived from each user question into its memory.
- When a user poses a new question, Mem0 retrieves relevant previous memories.
- The `top_k` parameter in the memory configuration specifies the number of top memories to consider during retrieval.
- Mem0 generates the final response by integrating the user's question, context from the data source, and the relevant memories.
@@ -0,0 +1,48 @@
---
title: 🗑 delete
---
## Delete Document
`delete()` method allows you to delete a document previously added to the app.
### Usage
```python
from embedchain import App
app = App()
forbes_doc_id = app.add("https://www.forbes.com/profile/elon-musk")
wiki_doc_id = app.add("https://en.wikipedia.org/wiki/Elon_Musk")
app.delete(forbes_doc_id) # deletes the forbes document
```
<Note>
If you do not have the document id, you can use `app.db.get()` method to get the document and extract the `hash` key from `metadatas` dictionary object, which serves as the document id.
</Note>
## Delete Chat Session History
`delete_session_chat_history()` method allows you to delete all previous messages in a chat history.
### Usage
```python
from embedchain import App
app = App()
app.add("https://www.forbes.com/profile/elon-musk")
app.chat("What is the net worth of Elon Musk?")
app.delete_session_chat_history()
```
<Note>
`delete_session_chat_history(session_id="session_1")` method also accepts `session_id` optional param for deleting chat history of a specific session.
It assumes the default session if no `session_id` is provided.
</Note>
@@ -0,0 +1,5 @@
---
title: 🚀 deploy
---
The `deploy()` method is currently available on an invitation-only basis. To request access, please submit your information via the provided [Google Form](https://forms.gle/vigN11h7b4Ywat668). We will review your request and respond promptly.
@@ -0,0 +1,41 @@
---
title: '📝 evaluate'
---
`evaluate()` method is used to evaluate the performance of a RAG app. You can find the signature below:
### Parameters
<ParamField path="question" type="Union[str, list[str]]">
A question or a list of questions to evaluate your app on.
</ParamField>
<ParamField path="metrics" type="Optional[list[Union[BaseMetric, str]]]" optional>
The metrics to evaluate your app on. Defaults to all metrics: `["context_relevancy", "answer_relevancy", "groundedness"]`
</ParamField>
<ParamField path="num_workers" type="int" optional>
Specify the number of threads to use for parallel processing.
</ParamField>
### Returns
<ResponseField name="metrics" type="dict">
Returns the metrics you have chosen to evaluate your app on as a dictionary.
</ResponseField>
## Usage
```python
from embedchain import App
app = App()
# add data source
app.add("https://www.forbes.com/profile/elon-musk")
# run evaluation
app.evaluate("what is the net worth of Elon Musk?")
# {'answer_relevancy': 0.958019958036268, 'context_relevancy': 0.12903225806451613}
# or
# app.evaluate(["what is the net worth of Elon Musk?", "which companies does Elon Musk own?"])
```
+33
View File
@@ -0,0 +1,33 @@
---
title: 📄 get
---
## Get data sources
`get_data_sources()` returns a list of all the data sources added in the app.
### Usage
```python
from embedchain import App
app = App()
app.add("https://www.forbes.com/profile/elon-musk")
app.add("https://en.wikipedia.org/wiki/Elon_Musk")
data_sources = app.get_data_sources()
# [
# {
# 'data_type': 'web_page',
# 'data_value': 'https://en.wikipedia.org/wiki/Elon_Musk',
# 'metadata': 'null'
# },
# {
# 'data_type': 'web_page',
# 'data_value': 'https://www.forbes.com/profile/elon-musk',
# 'metadata': 'null'
# }
# ]
```
@@ -0,0 +1,130 @@
---
title: "App"
---
Create a RAG app object on Embedchain. This is the main entrypoint for a developer to interact with Embedchain APIs. An app configures the llm, vector database, embedding model, and retrieval strategy of your choice.
### Attributes
<ParamField path="local_id" type="str">
App ID
</ParamField>
<ParamField path="name" type="str" optional>
Name of the app
</ParamField>
<ParamField path="config" type="BaseConfig">
Configuration of the app
</ParamField>
<ParamField path="llm" type="BaseLlm">
Configured LLM for the RAG app
</ParamField>
<ParamField path="db" type="BaseVectorDB">
Configured vector database for the RAG app
</ParamField>
<ParamField path="embedding_model" type="BaseEmbedder">
Configured embedding model for the RAG app
</ParamField>
<ParamField path="chunker" type="ChunkerConfig">
Chunker configuration
</ParamField>
<ParamField path="client" type="Client" optional>
Client object (used to deploy an app to Embedchain platform)
</ParamField>
<ParamField path="logger" type="logging.Logger">
Logger object
</ParamField>
## Usage
You can create an app instance using the following methods:
### Default setting
```python Code Example
from embedchain import App
app = App()
```
### Python Dict
```python Code Example
from embedchain import App
config_dict = {
'llm': {
'provider': 'gpt4all',
'config': {
'model': 'orca-mini-3b-gguf2-q4_0.gguf',
'temperature': 0.5,
'max_tokens': 1000,
'top_p': 1,
'stream': False
}
},
'embedder': {
'provider': 'gpt4all'
}
}
# load llm configuration from config dict
app = App.from_config(config=config_dict)
```
### YAML Config
<CodeGroup>
```python main.py
from embedchain import App
# load llm configuration from config.yaml file
app = App.from_config(config_path="config.yaml")
```
```yaml config.yaml
llm:
provider: gpt4all
config:
model: 'orca-mini-3b-gguf2-q4_0.gguf'
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
embedder:
provider: gpt4all
```
</CodeGroup>
### JSON Config
<CodeGroup>
```python main.py
from embedchain import App
# load llm configuration from config.json file
app = App.from_config(config_path="config.json")
```
```json config.json
{
"llm": {
"provider": "gpt4all",
"config": {
"model": "orca-mini-3b-gguf2-q4_0.gguf",
"temperature": 0.5,
"max_tokens": 1000,
"top_p": 1,
"stream": false
}
},
"embedder": {
"provider": "gpt4all"
}
}
```
</CodeGroup>
+109
View File
@@ -0,0 +1,109 @@
---
title: '❓ query'
---
`.query()` method empowers developers to ask questions and receive relevant answers through a user-friendly query API. Function signature is given below:
### Parameters
<ParamField path="input_query" type="str">
Question to ask
</ParamField>
<ParamField path="config" type="BaseLlmConfig" optional>
Configure different llm settings such as prompt, temprature, number_documents etc.
</ParamField>
<ParamField path="dry_run" type="bool" optional>
The purpose is to test the prompt structure without actually running LLM inference. Defaults to `False`
</ParamField>
<ParamField path="where" type="dict" optional>
A dictionary of key-value pairs to filter the chunks from the vector database. Defaults to `None`
</ParamField>
<ParamField path="citations" type="bool" optional>
Return citations along with the LLM answer. Defaults to `False`
</ParamField>
### Returns
<ResponseField name="answer" type="str | tuple">
If `citations=False`, return a stringified answer to the question asked. <br />
If `citations=True`, returns a tuple with answer and citations respectively.
</ResponseField>
## Usage
### With citations
If you want to get the answer to question and return both answer and citations, use the following code snippet:
```python With Citations
from embedchain import App
# Initialize app
app = App()
# Add data source
app.add("https://www.forbes.com/profile/elon-musk")
# Get relevant answer for your query
answer, sources = app.query("What is the net worth of Elon?", citations=True)
print(answer)
# Answer: The net worth of Elon Musk is $221.9 billion.
print(sources)
# [
# (
# 'Elon Musk PROFILEElon MuskCEO, Tesla$247.1B$2.3B (0.96%)Real Time Net Worthas of 12/7/23 ...',
# {
# 'url': 'https://www.forbes.com/profile/elon-musk',
# 'score': 0.89,
# ...
# }
# ),
# (
# '74% of the company, which is now called X.Wealth HistoryHOVER TO REVEAL NET WORTH BY YEARForbes ...',
# {
# 'url': 'https://www.forbes.com/profile/elon-musk',
# 'score': 0.81,
# ...
# }
# ),
# (
# 'founded in 2002, is worth nearly $150 billion after a $750 million tender offer in June 2023 ...',
# {
# 'url': 'https://www.forbes.com/profile/elon-musk',
# 'score': 0.73,
# ...
# }
# )
# ]
```
<Note>
When `citations=True`, note that the returned `sources` are a list of tuples where each tuple has two elements (in the following order):
1. source chunk
2. dictionary with metadata about the source chunk
- `url`: url of the source
- `doc_id`: document id (used for book keeping purposes)
- `score`: score of the source chunk with respect to the question
- other metadata you might have added at the time of adding the source
</Note>
### Without citations
If you just want to return answers and don't want to return citations, you can use the following example:
```python Without Citations
from embedchain import App
# Initialize app
app = App()
# Add data source
app.add("https://www.forbes.com/profile/elon-musk")
# Get relevant answer for your query
answer = app.query("What is the net worth of Elon?")
print(answer)
# Answer: The net worth of Elon Musk is $221.9 billion.
```
@@ -0,0 +1,17 @@
---
title: 🔄 reset
---
`reset()` method allows you to wipe the data from your RAG application and start from scratch.
## Usage
```python
from embedchain import App
app = App()
app.add("https://www.forbes.com/profile/elon-musk")
# Reset the app
app.reset()
```
@@ -0,0 +1,111 @@
---
title: '🔍 search'
---
`.search()` enables you to uncover the most pertinent context by performing a semantic search across your data sources based on a given query. Refer to the function signature below:
### Parameters
<ParamField path="query" type="str">
Question
</ParamField>
<ParamField path="num_documents" type="int" optional>
Number of relevant documents to fetch. Defaults to `3`
</ParamField>
<ParamField path="where" type="dict" optional>
Key value pair for metadata filtering.
</ParamField>
<ParamField path="raw_filter" type="dict" optional>
Pass raw filter query based on your vector database.
Currently, `raw_filter` param is only supported for Pinecone vector database.
</ParamField>
### Returns
<ResponseField name="answer" type="dict">
Return list of dictionaries that contain the relevant chunk and their source information.
</ResponseField>
## Usage
### Basic
Refer to the following example on how to use the search api:
```python Code example
from embedchain import App
app = App()
app.add("https://www.forbes.com/profile/elon-musk")
context = app.search("What is the net worth of Elon?", num_documents=2)
print(context)
```
### Advanced
#### Metadata filtering using `where` params
Here is an advanced example of `search()` API with metadata filtering on pinecone database:
```python
import os
from embedchain import App
os.environ["PINECONE_API_KEY"] = "xxx"
config = {
"vectordb": {
"provider": "pinecone",
"config": {
"metric": "dotproduct",
"vector_dimension": 1536,
"index_name": "ec-test",
"serverless_config": {"cloud": "aws", "region": "us-west-2"},
},
}
}
app = App.from_config(config=config)
app.add("https://www.forbes.com/profile/bill-gates", metadata={"type": "forbes", "person": "gates"})
app.add("https://en.wikipedia.org/wiki/Bill_Gates", metadata={"type": "wiki", "person": "gates"})
results = app.search("What is the net worth of Bill Gates?", where={"person": "gates"})
print("Num of search results: ", len(results))
```
#### Metadata filtering using `raw_filter` params
Following is an example of metadata filtering by passing the raw filter query that pinecone vector database follows:
```python
import os
from embedchain import App
os.environ["PINECONE_API_KEY"] = "xxx"
config = {
"vectordb": {
"provider": "pinecone",
"config": {
"metric": "dotproduct",
"vector_dimension": 1536,
"index_name": "ec-test",
"serverless_config": {"cloud": "aws", "region": "us-west-2"},
},
}
}
app = App.from_config(config=config)
app.add("https://www.forbes.com/profile/bill-gates", metadata={"year": 2022, "person": "gates"})
app.add("https://en.wikipedia.org/wiki/Bill_Gates", metadata={"year": 2024, "person": "gates"})
print("Filter with person: gates and year > 2023")
raw_filter = {"$and": [{"person": "gates"}, {"year": {"$gt": 2023}}]}
results = app.search("What is the net worth of Bill Gates?", raw_filter=raw_filter)
print("Num of search results: ", len(results))
```
@@ -0,0 +1,54 @@
---
title: 'AI Assistant'
---
The `AIAssistant` class, an alternative to the OpenAI Assistant API, is designed for those who prefer using large language models (LLMs) other than those provided by OpenAI. It facilitates the creation of AI Assistants with several key benefits:
- **Visibility into Citations**: It offers transparent access to the sources and citations used by the AI, enhancing the understanding and trustworthiness of its responses.
- **Debugging Capabilities**: Users have the ability to delve into and debug the AI's processes, allowing for a deeper understanding and fine-tuning of its performance.
- **Customizable Prompts**: The class provides the flexibility to modify and tailor prompts according to specific needs, enabling more precise and relevant interactions.
- **Chain of Thought Integration**: It supports the incorporation of a 'chain of thought' approach, which helps in breaking down complex queries into simpler, sequential steps, thereby improving the clarity and accuracy of responses.
It is ideal for those who value customization, transparency, and detailed control over their AI Assistant's functionalities.
### Arguments
<ParamField path="name" type="string" optional>
Name for your AI assistant
</ParamField>
<ParamField path="instructions" type="string" optional>
How the Assistant and model should behave or respond
</ParamField>
<ParamField path="assistant_id" type="string" optional>
Load existing AI Assistant. If you pass this, you don't have to pass other arguments.
</ParamField>
<ParamField path="thread_id" type="string" optional>
Existing thread id if exists
</ParamField>
<ParamField path="yaml_path" type="str" Optional>
Embedchain pipeline config yaml path to use. This will define the configuration of the AI Assistant (such as configuring the LLM, vector database, and embedding model)
</ParamField>
<ParamField path="data_sources" type="list" default="[]">
Add data sources to your assistant. You can add in the following format: `[{"source": "https://example.com", "data_type": "web_page"}]`
</ParamField>
<ParamField path="collect_metrics" type="boolean" default="True">
Anonymous telemetry (doesn't collect any user information or user's files). Used to improve the Embedchain package utilization. Default is `True`.
</ParamField>
## Usage
For detailed guidance on creating your own AI Assistant, click the link below. It provides step-by-step instructions to help you through the process:
<Card title="Guide to Creating Your AI Assistant" icon="link" href="/examples/opensource-assistant">
Learn how to build a customized AI Assistant using the `AIAssistant` class.
</Card>
@@ -0,0 +1,45 @@
---
title: 'OpenAI Assistant'
---
### Arguments
<ParamField path="name" type="string">
Name for your AI assistant
</ParamField>
<ParamField path="instructions" type="string">
how the Assistant and model should behave or respond
</ParamField>
<ParamField path="assistant_id" type="string">
Load existing OpenAI Assistant. If you pass this, you don't have to pass other arguments.
</ParamField>
<ParamField path="thread_id" type="string">
Existing OpenAI thread id if exists
</ParamField>
<ParamField path="model" type="str" default="gpt-4-1106-preview">
OpenAI model to use
</ParamField>
<ParamField path="tools" type="list">
OpenAI tools to use. Default set to `[{"type": "retrieval"}]`
</ParamField>
<ParamField path="data_sources" type="list" default="[]">
Add data sources to your assistant. You can add in the following format: `[{"source": "https://example.com", "data_type": "web_page"}]`
</ParamField>
<ParamField path="telemetry" type="boolean" default="True">
Anonymous telemetry (doesn't collect any user information or user's files). Used to improve the Embedchain package utilization. Default is `True`.
</ParamField>
## Usage
For detailed guidance on creating your own OpenAI Assistant, click the link below. It provides step-by-step instructions to help you through the process:
<Card title="Guide to Creating Your OpenAI Assistant" icon="link" href="/examples/openai-assistant">
Learn how to build an OpenAI Assistant using the `OpenAIAssistant` class.
</Card>
@@ -0,0 +1,28 @@
---
title: 🤝 Connect with Us
---
We believe in building a vibrant and supportive community around embedchain. There are various channels through which you can connect with us, stay updated, and contribute to the ongoing discussions:
<CardGroup cols={3}>
<Card title="Twitter" icon="twitter" href="https://twitter.com/embedchain">
Follow us on Twitter
</Card>
<Card title="Slack" icon="slack" href="https://embedchain.ai/slack" color="#4A154B">
Join our slack community
</Card>
<Card title="Discord" icon="discord" href="https://discord.gg/6PzXDgEjG5" color="#7289DA">
Join our discord community
</Card>
<Card title="LinkedIn" icon="linkedin" href="https://www.linkedin.com/company/embedchain/">
Connect with us on LinkedIn
</Card>
<Card title="Schedule a call" icon="calendar" href="https://cal.com/taranjeetio/ec">
Schedule a call with Embedchain founder
</Card>
<Card title="Newsletter" icon="message" href="https://embedchain.substack.com/">
Subscribe to our newsletter
</Card>
</CardGroup>
We look forward to connecting with you and seeing how we can create amazing things together!
@@ -0,0 +1,25 @@
---
title: "🎤 Audio"
---
To use an audio as data source, just add `data_type` as `audio` and pass in the path of the audio (local or hosted).
We use [Deepgram](https://developers.deepgram.com/docs/introduction) to transcribe the audiot to text, and then use the generated text as the data source.
You would require an Deepgram API key which is available [here](https://console.deepgram.com/signup?jump=keys) to use this feature.
### Without customization
```python
import os
from embedchain import App
os.environ["DEEPGRAM_API_KEY"] = "153xxx"
app = App()
app.add("introduction.wav", data_type="audio")
response = app.query("What is my name and how old am I?")
print(response)
# Answer: Your name is Dave and you are 21 years old.
```
@@ -0,0 +1,16 @@
---
title: "🐝 Beehiiv"
---
To add any Beehiiv data sources to your app, just add the base url as the source and set the data_type to `beehiiv`.
```python
from embedchain import App
app = App()
# source: just add the base url and set the data_type to 'beehiiv'
app.add('https://aibreakfast.beehiiv.com', data_type='beehiiv')
app.query("How much is OpenAI paying developers?")
# Answer: OpenAI is aggressively recruiting Google's top AI researchers with offers ranging between $5 to $10 million annually, primarily in stock options.
```
@@ -0,0 +1,28 @@
---
title: '📊 CSV'
---
You can load any csv file from your local file system or through a URL. Headers are included for each line, so if you have an `age` column, `18` will be added as `age: 18`.
## Usage
### Load from a local file
```python
from embedchain import App
app = App()
app.add('/path/to/file.csv', data_type='csv')
```
### Load from URL
```python
from embedchain import App
app = App()
app.add('https://people.sc.fsu.edu/~jburkardt/data/csv/airtravel.csv', data_type="csv")
```
<Note>
There is a size limit allowed for csv file beyond which it can throw error. This limit is set by the LLMs. Please consider chunking large csv files into smaller csv files.
</Note>
@@ -0,0 +1,42 @@
---
title: '⚙️ Custom'
---
When we say "custom", we mean that you can customize the loader and chunker to your needs. This is done by passing a custom loader and chunker to the `add` method.
```python
from embedchain import App
import your_loader
from my_module import CustomLoader
from my_module import CustomChunker
app = App()
loader = CustomLoader()
chunker = CustomChunker()
app.add("source", data_type="custom", loader=loader, chunker=chunker)
```
<Note>
The custom loader and chunker must be a class that inherits from the [`BaseLoader`](https://github.com/embedchain/embedchain/blob/main/embedchain/loaders/base_loader.py) and [`BaseChunker`](https://github.com/embedchain/embedchain/blob/main/embedchain/chunkers/base_chunker.py) classes respectively.
</Note>
<Note>
If the `data_type` is not a valid data type, the `add` method will fallback to the `custom` data type and expect a custom loader and chunker to be passed by the user.
</Note>
Example:
```python
from embedchain import App
from embedchain.loaders.github import GithubLoader
app = App()
loader = GithubLoader(config={"token": "ghp_xxx"})
app.add("repo:embedchain/embedchain type:repo", data_type="github", loader=loader)
app.query("What is Embedchain?")
# Answer: Embedchain is a Data Platform for Large Language Models (LLMs). It allows users to seamlessly load, index, retrieve, and sync unstructured data in order to build dynamic, LLM-powered applications. There is also a JavaScript implementation called embedchain-js available on GitHub.
```
@@ -0,0 +1,85 @@
---
title: 'Data type handling'
---
## Automatic data type detection
The add method automatically tries to detect the data_type, based on your input for the source argument. So `app.add('https://www.youtube.com/watch?v=dQw4w9WgXcQ')` is enough to embed a YouTube video.
This detection is implemented for all formats. It is based on factors such as whether it's a URL, a local file, the source data type, etc.
### Debugging automatic detection
Set `log_level: DEBUG` in the config yaml to debug if the data type detection is done right or not. Otherwise, you will not know when, for instance, an invalid filepath is interpreted as raw text instead.
### Forcing a data type
To omit any issues with the data type detection, you can **force** a data_type by adding it as a `add` method argument.
The examples below show you the keyword to force the respective `data_type`.
Forcing can also be used for edge cases, such as interpreting a sitemap as a web_page, for reading its raw text instead of following links.
## Remote data types
<Tip>
**Use local files in remote data types**
Some data_types are meant for remote content and only work with URLs.
You can pass local files by formatting the path using the `file:` [URI scheme](https://en.wikipedia.org/wiki/File_URI_scheme), e.g. `file:///info.pdf`.
</Tip>
## Reusing a vector database
Default behavior is to create a persistent vector db in the directory **./db**. You can split your application into two Python scripts: one to create a local vector db and the other to reuse this local persistent vector db. This is useful when you want to index hundreds of documents and separately implement a chat interface.
Create a local index:
```python
from embedchain import App
config = {
"app": {
"config": {
"id": "app-1"
}
}
}
naval_chat_bot = App.from_config(config=config)
naval_chat_bot.add("https://www.youtube.com/watch?v=3qHkcs3kG44")
naval_chat_bot.add("https://navalmanack.s3.amazonaws.com/Eric-Jorgenson_The-Almanack-of-Naval-Ravikant_Final.pdf")
```
You can reuse the local index with the same code, but without adding new documents:
```python
from embedchain import App
config = {
"app": {
"config": {
"id": "app-1"
}
}
}
naval_chat_bot = App.from_config(config=config)
print(naval_chat_bot.query("What unique capacity does Naval argue humans possess when it comes to understanding explanations or concepts?"))
```
## Resetting an app and vector database
You can reset the app by simply calling the `reset` method. This will delete the vector database and all other app related files.
```python
from embedchain import App
app = App()config = {
"app": {
"config": {
"id": "app-1"
}
}
}
naval_chat_bot = App.from_config(config=config)
app.add("https://www.youtube.com/watch?v=3qHkcs3kG44")
app.reset()
```
@@ -0,0 +1,41 @@
---
title: '📁 Directory/Folder'
---
To use an entire directory as data source, just add `data_type` as `directory` and pass in the path of the local directory.
### Without customization
```python
import os
from embedchain import App
os.environ["OPENAI_API_KEY"] = "sk-xxx"
app = App()
app.add("./elon-musk", data_type="directory")
response = app.query("list all files")
print(response)
# Answer: Files are elon-musk-1.txt, elon-musk-2.pdf.
```
### Customization
```python
import os
from embedchain import App
from embedchain.loaders.directory_loader import DirectoryLoader
os.environ["OPENAI_API_KEY"] = "sk-xxx"
lconfig = {
"recursive": True,
"extensions": [".txt"]
}
loader = DirectoryLoader(config=lconfig)
app = App()
app.add("./elon-musk", loader=loader)
response = app.query("what are all the files related to?")
print(response)
# Answer: The files are related to Elon Musk.
```

Some files were not shown because too many files have changed in this diff Show More