JDSS/mem0
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Aspen theme for 1.x (#3473)

Browse Source
This commit is contained in:
Parshva Daftari
2025-09-18 21:03:17 +05:30
committed by GitHub
parent 6b5582f474
commit ac72eb5ecc
135 changed files with 19053 additions and 360 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/open-source/features/metadata-filtering.mdx
+389
View File
@@ -0,0 +1,389 @@
---
title: Enhanced Metadata Filtering
description: 'Advanced filtering capabilities for precise memory retrieval in Mem0 1.0.0 Beta'
icon: "filter"
iconType: "solid"
---
<Info>
Enhanced metadata filtering is available in **Mem0 1.0.0 Beta** and later versions. This feature provides powerful filtering capabilities with logical operators and comparison functions.
</Info>
## Overview
Mem0 1.0.0 Beta introduces enhanced metadata filtering that allows you to perform complex queries on your memory metadata. You can now use logical operators, comparison functions, and advanced filtering patterns to retrieve exactly the memories you need.
## Basic Filtering
### Simple Key-Value Filtering
```python
from mem0 import Memory
m = Memory()
# Search with simple metadata filters
results = m.search(
"What are my preferences?",
user_id="alice",
filters={"category": "preferences"}
)
```
### Exact Match Filtering
```python
# Multiple exact match filters
results = m.search(
"movie recommendations",
user_id="alice",
filters={
"category": "entertainment",
"type": "recommendation",
"priority": "high"
}
)
```
## Advanced Filtering with Operators
### Comparison Operators
```python
# Greater than / Less than
results = m.search(
"recent activities",
user_id="alice",
filters={
"score": {"gt": 0.8}, # score > 0.8
"priority": {"gte": 5}, # priority >= 5
"confidence": {"lt": 0.9}, # confidence < 0.9
"rating": {"lte": 3} # rating <= 3
}
)
# Equality operators
results = m.search(
"specific content",
user_id="alice",
filters={
"status": {"eq": "active"}, # status == "active"
"archived": {"ne": True} # archived != True
}
)
```
### List-based Operators
```python
# In / Not in operators
results = m.search(
"multi-category search",
user_id="alice",
filters={
"category": {"in": ["food", "travel", "entertainment"]},
"status": {"nin": ["deleted", "archived"]}
}
)
```
### String Operators
```python
# Text matching operators
results = m.search(
"content search",
user_id="alice",
filters={
"title": {"contains": "meeting"}, # case-sensitive contains
"description": {"icontains": "important"}, # case-insensitive contains
"tags": {"contains": "urgent"}
}
)
```
### Wildcard Matching
```python
# Match any value for a field
results = m.search(
"all with category",
user_id="alice",
filters={
"category": "*" # Any memory that has a category field
}
)
```
## Logical Operators
### AND Operations
```python
# Logical AND - all conditions must be true
results = m.search(
"complex query",
user_id="alice",
filters={
"AND": [
{"category": "work"},
{"priority": {"gte": 7}},
{"status": {"ne": "completed"}}
]
}
)
```
### OR Operations
```python
# Logical OR - any condition can be true
results = m.search(
"flexible query",
user_id="alice",
filters={
"OR": [
{"category": "urgent"},
{"priority": {"gte": 9}},
{"deadline": {"contains": "today"}}
]
}
)
```
### NOT Operations
```python
# Logical NOT - exclude matches
results = m.search(
"exclusion query",
user_id="alice",
filters={
"NOT": [
{"category": "archived"},
{"status": "deleted"}
]
}
)
```
### Complex Nested Logic
```python
# Combine multiple logical operators
results = m.search(
"advanced query",
user_id="alice",
filters={
"AND": [
{
"OR": [
{"category": "work"},
{"category": "personal"}
]
},
{"priority": {"gte": 5}},
{
"NOT": [
{"status": "archived"}
]
}
]
}
)
```
## Real-world Examples
### Project Management Filtering
```python
# Find high-priority active tasks
results = m.search(
"What tasks need attention?",
user_id="project_manager",
filters={
"AND": [
{"project": {"in": ["alpha", "beta"]}},
{"priority": {"gte": 8}},
{"status": {"ne": "completed"}},
{
"OR": [
{"assignee": "alice"},
{"assignee": "bob"}
]
}
]
}
)
```
### Customer Support Filtering
```python
# Find recent unresolved tickets
results = m.search(
"pending support issues",
agent_id="support_bot",
filters={
"AND": [
{"ticket_status": {"ne": "resolved"}},
{"priority": {"in": ["high", "critical"]}},
{"created_date": {"gte": "2024-01-01"}},
{
"NOT": [
{"category": "spam"}
]
}
]
}
)
```
### Content Recommendation Filtering
```python
# Personalized content filtering
results = m.search(
"recommend content",
user_id="reader123",
filters={
"AND": [
{
"OR": [
{"genre": {"in": ["sci-fi", "fantasy"]}},
{"author": {"contains": "favorite"}}
]
},
{"rating": {"gte": 4.0}},
{"read_status": {"ne": "completed"}},
{"language": "english"}
]
}
)
```
## Performance Considerations
### Indexing Strategy
```python
# Ensure your vector store supports indexing on filtered fields
config = {
"vector_store": {
"provider": "qdrant",
"config": {
"host": "localhost",
"port": 6333,
# Enable indexing on frequently filtered fields
"indexed_fields": ["category", "priority", "status", "user_id"]
}
}
}
```
### Filter Optimization
```python
# More efficient: Filter on indexed fields first
good_filters = {
"AND": [
{"user_id": "alice"}, # Indexed field first
{"category": "work"}, # Then other indexed fields
{"content": {"contains": "meeting"}} # Text search last
]
}
# Less efficient: Complex operations first
avoid_filters = {
"AND": [
{"description": {"icontains": "complex text search"}}, # Expensive first
{"user_id": "alice"} # Indexed field last
]
}
```
## Vector Store Compatibility
Different vector stores support different filtering capabilities:
### Qdrant
-  Full support for all operators
-  Efficient nested logical operations
-  Indexed field optimization
### Chroma
-  Basic operators (eq, ne, gt, lt, gte, lte)
-  Simple logical operations
-   Limited nested operations
### Pinecone
-  Good support for comparison operators
-  In/nin operations
-   Limited text operations
### Weaviate
-  Full operator support
-  Advanced text operations
-  Efficient filtering
## Error Handling
```python
try:
results = m.search(
"test query",
user_id="alice",
filters={
"invalid_operator": {"unknown": "value"}
}
)
except ValueError as e:
print(f"Filter error: {e}")
# Fallback to simple filtering
results = m.search(
"test query",
user_id="alice",
filters={"category": "general"}
)
```
## Migration from Simple Filters
### Before (v0.x)
```python
# Simple key-value filtering only
results = m.search(
"query",
user_id="alice",
filters={"category": "work", "status": "active"}
)
```
### After (v1.0.0 Beta)
```python
# Enhanced filtering with operators
results = m.search(
"query",
user_id="alice",
filters={
"AND": [
{"category": "work"},
{"status": {"ne": "archived"}},
{"priority": {"gte": 5}}
]
}
)
```
## Best Practices
1. **Use Indexed Fields**: Filter on indexed fields for better performance
2. **Combine Operators**: Use logical operators to create precise queries
3. **Test Filter Performance**: Benchmark complex filters with your data
4. **Graceful Degradation**: Implement fallbacks for unsupported operations
5. **Validate Filters**: Check filter syntax before executing queries
<Info>
Enhanced metadata filtering provides powerful capabilities for precise memory retrieval. Start with simple filters and gradually adopt more complex patterns as needed.
</Info>
docs/open-source/features/overview.mdx
+3 -3
View File
@@ -44,14 +44,14 @@ Choose your preferred approach:
- **[Python Quickstart](../python-quickstart)**: Get started with Python SDK
- **[Node.js Quickstart](../node-quickstart)**: Use Mem0 with Node.js/TypeScript
- **[Examples](../examples)**: Explore real-world use cases and implementations
- **[Examples](/examples)**: Explore real-world use cases and implementations
## Next Steps
- Explore [specific features](./async-memory) in detail
- Learn about [graph memory](../graph_memory/overview) capabilities
- Set up [vector databases](../components/vectordbs/overview) and [LLM integrations](../components/llms/overview)
- Check out our [examples](../examples) for practical implementations
- Set up [vector databases](/components/vectordbs/overview) and [LLM integrations](/components/llms/overview)
- Check out our [examples](/examples) for practical implementations
- Join our [Discord community](https://mem0.dev/DiD) for support
We're excited to see what you'll build with Mem0 open-source. Let's create smarter, more personalized AI experiences together!
docs/open-source/features/reranker-search.mdx
+418
View File
@@ -0,0 +1,418 @@
---
title: Reranker-Enhanced Search
description: 'Improve search relevance with reranking models in Mem0 1.0.0 Beta'
icon: "sort"
iconType: "solid"
---
<Info>
Reranker-enhanced search is available in **Mem0 1.0.0 Beta** and later versions. This feature significantly improves search relevance by using specialized reranking models to reorder search results.
</Info>
## Overview
Rerankers are specialized models that improve the quality of search results by reordering initially retrieved memories. They work as a second-stage ranking system that analyzes the semantic relationship between your query and retrieved memories to provide more relevant results.
## How Reranking Works
1. **Initial Vector Search**: Retrieves candidate memories using vector similarity
2. **Reranking**: Specialized model analyzes query-memory relationships
3. **Reordering**: Results are reordered based on semantic relevance
4. **Enhanced Results**: Final results with improved relevance scores
## Configuration
### Basic Setup
```python
from mem0 import Memory
config = {
"reranker": {
"provider": "cohere",
"config": {
"model": "rerank-english-v3.0",
"api_key": "your-cohere-api-key"
}
}
}
m = Memory.from_config(config)
```
### Supported Providers
#### Cohere Reranker
```python
config = {
"reranker": {
"provider": "cohere",
"config": {
"model": "rerank-english-v3.0",
"api_key": "your-cohere-api-key",
"top_k": 10, # Number of results to rerank
"return_documents": True
}
}
}
```
#### Sentence Transformer Reranker
```python
config = {
"reranker": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cuda", # Use GPU if available
"max_length": 512
}
}
}
```
#### Hugging Face Reranker
```python
config = {
"reranker": {
"provider": "huggingface",
"config": {
"model": "BAAI/bge-reranker-base",
"device": "cuda",
"batch_size": 32
}
}
}
```
#### LLM-based Reranker
```python
config = {
"reranker": {
"provider": "llm_reranker",
"config": {
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4",
"api_key": "your-openai-api-key"
}
},
"top_k": 5
}
}
}
```
## Usage Examples
### Basic Reranked Search
```python
# Reranking is enabled by default when configured
results = m.search(
"What are my food preferences?",
user_id="alice"
)
# Results are automatically reranked for better relevance
for result in results["results"]:
print(f"Memory: {result['memory']}")
print(f"Score: {result['score']}")
```
### Controlling Reranking
```python
# Enable reranking explicitly
results_with_rerank = m.search(
"What movies do I like?",
user_id="alice",
rerank=True
)
# Disable reranking for this search
results_without_rerank = m.search(
"What movies do I like?",
user_id="alice",
rerank=False
)
# Compare the difference in results
print("With reranking:", len(results_with_rerank["results"]))
print("Without reranking:", len(results_without_rerank["results"]))
```
### Combining with Filters
```python
# Reranking works with metadata filtering
results = m.search(
"important work tasks",
user_id="alice",
filters={
"AND": [
{"category": "work"},
{"priority": {"gte": 7}}
]
},
rerank=True,
limit=20
)
```
## Advanced Configuration
### Complete Configuration Example
```python
config = {
"vector_store": {
"provider": "qdrant",
"config": {
"host": "localhost",
"port": 6333
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4",
"api_key": "your-openai-api-key"
}
},
"embedder": {
"provider": "openai",
"config": {
"model": "text-embedding-3-small",
"api_key": "your-openai-api-key"
}
},
"reranker": {
"provider": "cohere",
"config": {
"model": "rerank-english-v3.0",
"api_key": "your-cohere-api-key",
"top_k": 15,
"return_documents": True
}
}
}
m = Memory.from_config(config)
```
### Async Support
```python
from mem0 import AsyncMemory
# Reranking works with async operations
async_memory = AsyncMemory.from_config(config)
async def search_with_rerank():
results = await async_memory.search(
"What are my preferences?",
user_id="alice",
rerank=True
)
return results
# Use in async context
import asyncio
results = asyncio.run(search_with_rerank())
```
## Performance Considerations
### When to Use Reranking
✅ **Good Use Cases:**
- Complex semantic queries
- Domain-specific searches
- When precision is more important than speed
- Large memory collections
- Ambiguous or nuanced queries
❌ **Avoid When:**
- Simple keyword matching
- Real-time applications with strict latency requirements
- Small memory collections
- High-frequency searches where cost matters
### Performance Optimization
```python
# Optimize reranking performance
config = {
"reranker": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cuda", # Use GPU
"batch_size": 32, # Process in batches
"top_k": 10, # Limit candidates
"max_length": 256 # Reduce if appropriate
}
}
}
```
### Cost Management
```python
# For API-based rerankers like Cohere
config = {
"reranker": {
"provider": "cohere",
"config": {
"model": "rerank-english-v3.0",
"api_key": "your-cohere-api-key",
"top_k": 5, # Reduce to control API costs
}
}
}
# Use reranking selectively
def smart_search(query, user_id, use_rerank=None):
# Automatically decide when to use reranking
if use_rerank is None:
use_rerank = len(query.split()) > 3 # Complex queries only
return m.search(query, user_id=user_id, rerank=use_rerank)
```
## Error Handling
```python
try:
results = m.search(
"test query",
user_id="alice",
rerank=True
)
except Exception as e:
print(f"Reranking failed: {e}")
# Gracefully fall back to vector search
results = m.search(
"test query",
user_id="alice",
rerank=False
)
```
## Reranker Comparison
| Provider | Latency | Quality | Cost | Local Deploy |
|----------|---------|---------|------|--------------|
| Cohere | Medium | High | API Cost | ❌ |
| Sentence Transformer | Low | Good | Free | ✅ |
| Hugging Face | Low-Medium | Variable | Free | ✅ |
| LLM Reranker | High | Very High | API Cost | Depends |
## Real-world Examples
### Customer Support
```python
# Improve support ticket relevance
config = {
"reranker": {
"provider": "cohere",
"config": {
"model": "rerank-english-v3.0",
"api_key": "your-cohere-api-key"
}
}
}
m = Memory.from_config(config)
# Find relevant support cases
results = m.search(
"customer having login issues with mobile app",
agent_id="support_bot",
filters={"category": "technical_support"},
rerank=True
)
```
### Content Recommendation
```python
# Better content matching
results = m.search(
"science fiction books with space exploration themes",
user_id="reader123",
filters={"content_type": "book_recommendation"},
rerank=True,
limit=10
)
for result in results["results"]:
print(f"Recommendation: {result['memory']}")
print(f"Relevance: {result['score']:.3f}")
```
### Personal Assistant
```python
# Enhanced personal queries
results = m.search(
"What restaurants did I enjoy last month that had good vegetarian options?",
user_id="foodie_user",
filters={
"AND": [
{"category": "dining"},
{"rating": {"gte": 4}},
{"date": {"gte": "2024-01-01"}}
]
},
rerank=True
)
```
## Migration Guide
### From v0.x (No Reranking)
```python
# v0.x - basic vector search
results = m.search("query", user_id="alice")
```
### To v1.0.0 Beta (With Reranking)
```python
# Add reranker configuration
config = {
"reranker": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2"
}
}
}
m = Memory.from_config(config)
# Same search API, better results
results = m.search("query", user_id="alice") # Automatically reranked
```
## Best Practices
1. **Start Simple**: Begin with Sentence Transformers for local deployment
2. **Monitor Performance**: Track both relevance improvements and latency
3. **Cost Awareness**: Use API-based rerankers judiciously
4. **Selective Usage**: Apply reranking where it provides the most value
5. **Fallback Strategy**: Always handle reranking failures gracefully
6. **Test Different Models**: Experiment to find the best fit for your domain
<Info>
Reranker-enhanced search significantly improves result relevance. Start with a local model and upgrade to API-based solutions as your needs grow.
</Info>
docs/open-source/graph_memory/overview.mdx
+1 -1
View File
@@ -285,7 +285,7 @@ m = Memory.from_config(config_dict=config)
- For issues related to authentication, refer to the [boto3 client configuration options](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html).
- For more details on how to connect, configure, and use the graph_memory graph store, see the [Neptune Analytics example notebook](examples/graph-db-demo/neptune-analytics-example.ipynb).
- For more details on how to connect, configure, and use the graph_memory graph store, see the Neptune Analytics example in our [AWS example guide](/examples/aws_example#aws-bedrock-and-aoss).
### Initialize Kuzu