JDSS/mem0
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Mem0 1.0.0 (#3545)

Browse Source
This commit is contained in:
Parshva Daftari
2025-10-16 15:50:20 +05:30
committed by GitHub
parent 8f5151c344
commit 394203d1b5
77 changed files with 3445 additions and 990 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/components/rerankers/config.mdx
+76 -114
View File
@@ -1,145 +1,107 @@
---
title: Configuration
title: Config
description: 'Configuration options for rerankers in Mem0'
icon: "gear"
iconType: "solid"
---
## How to define configurations?
## Common Configuration Parameters
The `reranker` configuration is defined as an object with two main keys:
- `provider`: The name of the reranker provider (e.g., "cohere", "sentence_transformer", "huggingface", "llm_reranker")
- `config`: A nested dictionary containing provider-specific settings
All rerankers share these common configuration parameters:
## Basic Configuration
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `provider` | Reranker provider name | `str` | Required |
| `top_k` | Maximum number of results to return after reranking | `int` | `None` |
| `api_key` | API key for the reranker service | `str` | `None` |
Here's how to configure a reranker with Mem0:
## Provider-Specific Configuration
```python
from mem0 import Memory
### Zero Entropy
config = {
"reranker": {
"provider": "cohere",
"config": {
"api_key": "your-api-key",
"top_n": 10,
"model": "rerank-english-v3.0"
}
}
}
memory = Memory.from_config(config)
```
## Configuration Parameters
| Parameter | Description | Required | Default |
|-----------|-------------|----------|---------|
| `provider` | Reranker provider name | Yes | - |
| `config` | Provider-specific configuration | Yes | - |
### Common Config Parameters
| Parameter | Description | Providers |
|-----------|-------------|-----------|
| `api_key` | API key for the service | Cohere, Hugging Face |
| `model` | Model name to use | All |
| `top_n` | Number of results to return | All |
| `device` | Device to run on (cpu/cuda/mps) | Sentence Transformer, Hugging Face |
## Provider-Specific Examples
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | Model to use: `zerank-1` or `zerank-1-small` | `str` | `"zerank-1"` |
| `api_key` | Zero Entropy API key | `str` | `None` |
### Cohere
```python
config = {
"reranker": {
"provider": "cohere",
"config": {
"api_key": "your-cohere-api-key",
"model": "rerank-english-v3.0",
"top_n": 5
}
}
}
```
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | Cohere rerank model | `str` | `"rerank-english-v3.0"` |
| `api_key` | Cohere API key | `str` | `None` |
| `return_documents` | Whether to return document texts in response | `bool` | `False` |
| `max_chunks_per_doc` | Maximum chunks per document | `int` | `None` |
### Sentence Transformer
```python
config = {
"reranker": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cpu",
"top_n": 10
}
}
}
```
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | HuggingFace cross-encoder model name | `str` | `"cross-encoder/ms-marco-MiniLM-L-6-v2"` |
| `device` | Device to run model on (`cpu`, `cuda`, etc.) | `str` | `None` |
| `batch_size` | Batch size for processing | `int` | `32` |
| `show_progress_bar` | Show progress during processing | `bool` | `False` |
### Hugging Face
```python
config = {
"reranker": {
"provider": "huggingface",
"config": {
"api_key": "your-hf-token",
"model": "BAAI/bge-reranker-large",
"top_n": 8
}
}
}
```
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | HuggingFace reranker model name | `str` | `"BAAI/bge-reranker-large"` |
| `api_key` | HuggingFace API token | `str` | `None` |
| `device` | Device to run model on (`cpu`, `cuda`, etc.) | `str` | `None` |
### LLM-based
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | LLM model to use for scoring | `str` | `"gpt-4o-mini"` |
| `provider` | LLM provider (`openai`, `anthropic`, etc.) | `str` | `"openai"` |
| `api_key` | API key for LLM provider | `str` | `None` |
| `temperature` | Temperature for LLM generation | `float` | `0.0` |
| `max_tokens` | Maximum tokens for LLM response | `int` | `100` |
| `scoring_prompt` | Custom prompt template for scoring | `str` | Default scoring prompt |
### LLM Reranker
```python
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `llm.provider` | LLM provider for reranking | `str` | Required |
| `llm.config` | LLM configuration object | `dict` | Required |
| `top_n` | Number of results to return | `int` | `None` |
## Environment Variables
You can set API keys using environment variables:
- `ZERO_ENTROPY_API_KEY` - Zero Entropy API key
- `COHERE_API_KEY` - Cohere API key
- `HUGGINGFACE_API_KEY` - HuggingFace API token
- `OPENAI_API_KEY` - OpenAI API key (for LLM-based reranker)
- `ANTHROPIC_API_KEY` - Anthropic API key (for LLM-based reranker)
## Basic Configuration Example
```python Python
config = {
"reranker": {
"provider": "llm_reranker",
"vector_store": {
"provider": "chroma",
"config": {
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4",
"api_key": "your-openai-key"
}
},
"top_n": 5
"collection_name": "my_memories",
"path": "./chroma_db"
}
}
}
```
## Advanced Configuration
You can combine rerankers with other components:
```python
config = {
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4",
"api_key": "your-openai-key"
"model": "gpt-4o-mini"
}
},
"vector_store": {
"provider": "qdrant",
"rerank": {
"provider": "zero_entropy",
"config": {
"collection_name": "memories",
"host": "localhost",
"port": 6333
}
},
"reranker": {
"provider": "cohere",
"config": {
"api_key": "your-cohere-key",
"model": "rerank-english-v3.0",
"top_n": 10
"model": "zerank-1",
"top_k": 5
}
}
}
```
For provider-specific configuration details, visit the individual reranker pages.
docs/components/rerankers/models/cohere.mdx
+108 -77
View File
@@ -1,116 +1,147 @@
---
title: Cohere
description: 'Reranking with Cohere'
icon: "building"
iconType: "solid"
---
Cohere provides state-of-the-art reranking models that can significantly improve the relevance of search results. Cohere's rerankers are optimized for various languages and use cases.
Cohere provides enterprise-grade reranking models with excellent multilingual support and production-ready performance.
## Usage
## Models
To use Cohere's reranker with Mem0:
Cohere offers several reranking models:
```python
import os
- **`rerank-english-v3.0`**: Latest English reranker with best performance
- **`rerank-multilingual-v3.0`**: Multilingual support for global applications
- **`rerank-english-v2.0`**: Previous generation English reranker
## Installation
```bash
pip install cohere
```
## Configuration
```python Python
from mem0 import Memory
os.environ["COHERE_API_KEY"] = "your-cohere-api-key"
config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "my_memories",
"path": "./chroma_db"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini"
}
},
"reranker": {
"provider": "cohere",
"config": {
"api_key": "your-cohere-api-key", # Can also use environment variable
"model": "rerank-english-v3.0",
"top_n": 10
"api_key": "your-cohere-api-key", # or set COHERE_API_KEY
"top_k": 5,
"return_documents": False,
"max_chunks_per_doc": None
}
}
}
memory = Memory.from_config(config)
```
## Environment Variables
Set your API key as an environment variable:
```bash
export COHERE_API_KEY="your-api-key"
```
## Usage Example
```python Python
import os
from mem0 import Memory
# Set API key
os.environ["COHERE_API_KEY"] = "your-api-key"
# Initialize memory with Cohere reranker
config = {
"vector_store": {"provider": "chroma"},
"llm": {"provider": "openai", "config": {"model": "gpt-4o-mini"}},
"rerank": {
"provider": "cohere",
"config": {
"model": "rerank-english-v3.0",
"top_k": 3
}
}
}
memory = Memory.from_config(config)
# Use memory as usual
memory.add("I love playing basketball", user_id="alice")
memory.add("I enjoy watching movies", user_id="alice")
# Add memories
messages = [
{"role": "user", "content": "I work as a data scientist at Microsoft"},
{"role": "user", "content": "I specialize in machine learning and NLP"},
{"role": "user", "content": "I enjoy playing tennis on weekends"}
]
# Search will now use Cohere reranking
results = memory.search("What sports does Alice like?", user_id="alice")
memory.add(messages, user_id="bob")
# Search with reranking
results = memory.search("What is the user's profession?", user_id="bob")
for result in results['results']:
print(f"Memory: {result['memory']}")
print(f"Vector Score: {result['score']:.3f}")
print(f"Rerank Score: {result['rerank_score']:.3f}")
print()
```
## Configuration
## Multilingual Support
| Parameter | Description | Default |
|-----------|-------------|---------|
| `api_key` | Cohere API key | Required |
| `model` | Cohere rerank model | `rerank-english-v3.0` |
| `top_n` | Number of results to return | `10` |
For multilingual applications, use the multilingual model:
## Available Models
- `rerank-english-v3.0`: Latest English reranking model
- `rerank-multilingual-v3.0`: Multilingual reranking model
- `rerank-english-v2.0`: Previous English model
- `rerank-multilingual-v2.0`: Previous multilingual model
## Example with Different Models
### English Reranker
```python
```python Python
config = {
"reranker": {
"rerank": {
"provider": "cohere",
"config": {
"api_key": "your-cohere-api-key",
"model": "rerank-english-v3.0",
"top_n": 5
}
}
}
```
### Multilingual Reranker
```python
config = {
"reranker": {
"provider": "cohere",
"config": {
"api_key": "your-cohere-api-key",
"model": "rerank-multilingual-v3.0",
"top_n": 8
"top_k": 5
}
}
}
```
## Environment Variables
## Configuration Parameters
You can set your Cohere API key as an environment variable:
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | Cohere rerank model to use | `str` | `"rerank-english-v3.0"` |
| `api_key` | Cohere API key | `str` | `None` |
| `top_k` | Maximum documents to return | `int` | `None` |
| `return_documents` | Whether to return document texts | `bool` | `False` |
| `max_chunks_per_doc` | Maximum chunks per document | `int` | `None` |
```bash
export COHERE_API_KEY="your-cohere-api-key"
```
## Features
Then use the config without specifying the API key:
- **High Quality**: Enterprise-grade relevance scoring
- **Multilingual**: Support for 100+ languages
- **Scalable**: Production-ready with high throughput
- **Reliable**: SLA-backed service with 99.9% uptime
```python
config = {
"reranker": {
"provider": "cohere",
"config": {
"model": "rerank-english-v3.0",
"top_n": 10
}
}
}
```
## Best Practices
## Getting Your API Key
1. Sign up at [Cohere](https://cohere.ai/)
2. Navigate to the API keys section in your dashboard
3. Generate a new API key
4. Use this key in your configuration
## Performance Considerations
- Cohere rerankers work best with 10-100 candidate documents
- Higher `top_n` values provide more comprehensive reranking but may increase latency
- The v3.0 models generally provide better performance than v2.0 models
1. **Model Selection**: Use `rerank-english-v3.0` for English, `rerank-multilingual-v3.0` for other languages
2. **Batch Processing**: Process multiple queries efficiently
3. **Error Handling**: Implement retry logic for production systems
4. **Monitoring**: Track reranking performance and costs
docs/components/rerankers/models/llm.mdx
+224
View File
@@ -0,0 +1,224 @@
---
title: LLM as Reranker
description: 'Flexible reranking using LLMs'
icon: "robot"
iconType: "solid"
---
LLM-based reranker provides maximum flexibility by using any Large Language Model to score document relevance. This approach allows for custom prompts and domain-specific scoring logic.
## Supported LLM Providers
Any LLM provider supported by Mem0 can be used for reranking:
- **OpenAI**: GPT-4, GPT-3.5-turbo, etc.
- **Anthropic**: Claude models
- **Together**: Open-source models
- **Groq**: Fast inference
- **Ollama**: Local models
- And more...
## Configuration
```python Python
from mem0 import Memory
config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "my_memories",
"path": "./chroma_db"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini"
}
},
"rerank": {
"provider": "llm",
"config": {
"model": "gpt-4o-mini",
"provider": "openai",
"api_key": "your-openai-api-key", # or set OPENAI_API_KEY
"top_k": 5,
"temperature": 0.0
}
}
}
memory = Memory.from_config(config)
```
## Custom Scoring Prompt
You can provide a custom prompt for relevance scoring:
```python Python
custom_prompt = """You are a relevance scoring assistant. Rate how well this document answers the query.
Query: "{query}"
Document: "{document}"
Score from 0.0 to 1.0 where:
- 1.0: Perfect match, directly answers the query
- 0.8-0.9: Highly relevant, good match
- 0.6-0.7: Moderately relevant, partial match
- 0.4-0.5: Slightly relevant, limited useful information
- 0.0-0.3: Not relevant or no useful information
Provide only a single numerical score between 0.0 and 1.0."""
config["rerank"]["config"]["scoring_prompt"] = custom_prompt
```
## Usage Example
```python Python
import os
from mem0 import Memory
# Set API key
os.environ["OPENAI_API_KEY"] = "your-api-key"
# Initialize memory with LLM reranker
config = {
"vector_store": {"provider": "chroma"},
"llm": {"provider": "openai", "config": {"model": "gpt-4o-mini"}},
"rerank": {
"provider": "llm",
"config": {
"model": "gpt-4o-mini",
"provider": "openai",
"temperature": 0.0
}
}
}
memory = Memory.from_config(config)
# Add memories
messages = [
{"role": "user", "content": "I'm learning Python programming"},
{"role": "user", "content": "I find object-oriented programming challenging"},
{"role": "user", "content": "I love hiking in national parks"}
]
memory.add(messages, user_id="david")
# Search with LLM reranking
results = memory.search("What programming topics is the user studying?", user_id="david")
for result in results['results']:
print(f"Memory: {result['memory']}")
print(f"Vector Score: {result['score']:.3f}")
print(f"Rerank Score: {result['rerank_score']:.3f}")
print()
```
```text Output
Memory: I'm learning Python programming
Vector Score: 0.856
Rerank Score: 0.920
Memory: I find object-oriented programming challenging
Vector Score: 0.782
Rerank Score: 0.850
```
## Domain-Specific Scoring
Create specialized scoring for your domain:
```python Python
medical_prompt = """You are a medical relevance expert. Score how relevant this medical record is to the clinical query.
Clinical Query: "{query}"
Medical Record: "{document}"
Consider:
- Clinical relevance and accuracy
- Patient safety implications
- Diagnostic value
- Treatment relevance
Score from 0.0 to 1.0. Provide only the numerical score."""
config = {
"rerank": {
"provider": "llm",
"config": {
"model": "gpt-4o-mini",
"provider": "openai",
"scoring_prompt": medical_prompt,
"temperature": 0.0
}
}
}
```
## Multiple LLM Providers
Use different LLM providers for reranking:
```python Python
# Using Anthropic Claude
anthropic_config = {
"rerank": {
"provider": "llm",
"config": {
"model": "claude-3-haiku-20240307",
"provider": "anthropic",
"temperature": 0.0
}
}
}
# Using local Ollama model
ollama_config = {
"rerank": {
"provider": "llm",
"config": {
"model": "llama2:7b",
"provider": "ollama",
"temperature": 0.0
}
}
}
```
## Configuration Parameters
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | LLM model to use for scoring | `str` | `"gpt-4o-mini"` |
| `provider` | LLM provider name | `str` | `"openai"` |
| `api_key` | API key for the LLM provider | `str` | `None` |
| `top_k` | Maximum documents to return | `int` | `None` |
| `temperature` | Temperature for LLM generation | `float` | `0.0` |
| `max_tokens` | Maximum tokens for LLM response | `int` | `100` |
| `scoring_prompt` | Custom prompt template | `str` | Default prompt |
## Advantages
- **Maximum Flexibility**: Custom prompts for any use case
- **Domain Expertise**: Leverage LLM knowledge for specialized domains
- **Interpretability**: Understand scoring through prompt engineering
- **Multi-criteria**: Score based on multiple relevance factors
## Considerations
- **Latency**: Higher latency than specialized rerankers
- **Cost**: LLM API costs per reranking operation
- **Consistency**: May have slight variations in scoring
- **Prompt Engineering**: Requires careful prompt design
## Best Practices
1. **Temperature**: Use 0.0 for consistent scoring
2. **Prompt Design**: Be specific about scoring criteria
3. **Token Efficiency**: Keep prompts concise to reduce costs
4. **Caching**: Cache results for repeated queries when possible
5. **Fallback**: Handle API errors gracefully
docs/components/rerankers/models/sentence_transformer.mdx
+125 -126
View File
@@ -1,162 +1,161 @@
---
title: Sentence Transformer
description: 'Local reranking with HuggingFace cross-encoder models'
icon: "server"
iconType: "solid"
---
Sentence Transformer rerankers use cross-encoder models that are specifically designed for ranking tasks. These models can run locally and provide good reranking performance without external API calls.
Sentence Transformer reranker provides local reranking using HuggingFace cross-encoder models, perfect for privacy-focused deployments where you want to keep data on-premises.
## Usage
## Models
To use Sentence Transformer reranker with Mem0:
Any HuggingFace cross-encoder model can be used. Popular choices include:
```python
- **`cross-encoder/ms-marco-MiniLM-L-6-v2`**: Default, good balance of speed and accuracy
- **`cross-encoder/ms-marco-TinyBERT-L-2-v2`**: Fastest, smaller model size
- **`cross-encoder/ms-marco-electra-base`**: Higher accuracy, larger model
- **`cross-encoder/stsb-distilroberta-base`**: Good for semantic similarity tasks
## Installation
```bash
pip install sentence-transformers
```
## Configuration
```python Python
from mem0 import Memory
config = {
"reranker": {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "my_memories",
"path": "./chroma_db"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini"
}
},
"rerank": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cpu",
"top_n": 10
"device": "cpu", # or "cuda" for GPU
"batch_size": 32,
"show_progress_bar": False,
"top_k": 5
}
}
}
memory = Memory.from_config(config)
```
## GPU Acceleration
For better performance, use GPU acceleration:
```python Python
config = {
"rerank": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cuda", # Use GPU
"batch_size": 64 # high batch size for high memory GPUs
}
}
}
```
## Usage Example
```python Python
from mem0 import Memory
# Initialize memory with local reranker
config = {
"vector_store": {"provider": "chroma"},
"llm": {"provider": "openai", "config": {"model": "gpt-4o-mini"}},
"rerank": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cpu"
}
}
}
memory = Memory.from_config(config)
# Use memory as usual
memory.add("I love playing basketball", user_id="alice")
memory.add("I enjoy watching movies", user_id="alice")
# Add memories
messages = [
{"role": "user", "content": "I love reading science fiction novels"},
{"role": "user", "content": "My favorite author is Isaac Asimov"},
{"role": "user", "content": "I also enjoy watching sci-fi movies"}
]
# Search will now use Sentence Transformer reranking
results = memory.search("What sports does Alice like?", user_id="alice")
```
memory.add(messages, user_id="charlie")
## Configuration
# Search with local reranking
results = memory.search("What books does the user like?", user_id="charlie")
| Parameter | Description | Default |
|-----------|-------------|---------|
| `model` | Sentence Transformer cross-encoder model | `cross-encoder/ms-marco-MiniLM-L-6-v2` |
| `device` | Device to run on (`cpu`, `cuda`, `mps`) | `cpu` |
| `top_n` | Number of results to return | `10` |
## Popular Models
### Lightweight Models
- `cross-encoder/ms-marco-MiniLM-L-6-v2`: Fast and efficient
- `cross-encoder/ms-marco-MiniLM-L-4-v2`: Even faster, slightly lower accuracy
- `cross-encoder/ms-marco-MiniLM-L-2-v2`: Fastest, good for real-time applications
### High-Performance Models
- `cross-encoder/ms-marco-electra-base`: Better accuracy, larger model
- `ms-marco-MiniLM-L-12-v2`: Balanced performance and speed
- `cross-encoder/qnli-electra-base`: Good for question-answering tasks
## Device Configuration
### CPU Usage
```python
config = {
"reranker": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cpu",
"top_n": 10
}
}
}
```
### GPU Usage (CUDA)
```python
config = {
"reranker": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-electra-base",
"device": "cuda",
"top_n": 15
}
}
}
```
### Apple Silicon (MPS)
```python
config = {
"reranker": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "mps",
"top_n": 10
}
}
}
```
## Installation
The sentence-transformers library is required:
```bash
pip install sentence-transformers
```
For GPU support with CUDA:
```bash
pip install sentence-transformers torch
```
## Performance Optimization
### Model Selection
- Use MiniLM models for faster inference
- Use larger models (electra-base) for better accuracy
- Consider the trade-off between speed and quality
### Device Optimization
- Use GPU (`cuda` or `mps`) for larger models
- CPU is sufficient for MiniLM models
- Batch processing improves GPU utilization
### Memory Considerations
```python
# For memory-constrained environments
config = {
"reranker": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-2-v2", # Smallest model
"device": "cpu",
"top_n": 5 # Fewer results to process
}
}
}
for result in results['results']:
print(f"Memory: {result['memory']}")
print(f"Vector Score: {result['score']:.3f}")
print(f"Rerank Score: {result['rerank_score']:.3f}")
print()
```
## Custom Models
You can use any Sentence Transformer cross-encoder model:
You can use any HuggingFace cross-encoder model:
```python
```python Python
# Using a different model
config = {
"reranker": {
"provider": "sentence_transformer",
"rerank": {
"provider": "sentence_transformer",
"config": {
"model": "your-custom-model-name",
"device": "cpu",
"top_n": 10
"model": "cross-encoder/stsb-distilroberta-base",
"device": "cpu"
}
}
}
```
## Configuration Parameters
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | HuggingFace cross-encoder model name | `str` | `"cross-encoder/ms-marco-MiniLM-L-6-v2"` |
| `device` | Device to run model on (`cpu`, `cuda`, etc.) | `str` | `None` |
| `batch_size` | Batch size for processing documents | `int` | `32` |
| `show_progress_bar` | Show progress bar during processing | `bool` | `False` |
| `top_k` | Maximum documents to return | `int` | `None` |
## Advantages
- **Local Processing**: No external API calls required
- **Privacy**: Data stays on your infrastructure
- **Cost Effective**: No per-request charges
- **Fast**: Especially with GPU acceleration
- **Customizable**: Can fine-tune on your specific data
- **Privacy**: Complete local processing, no external API calls
- **Cost**: No per-token charges after initial model download
- **Customization**: Use any HuggingFace cross-encoder model
- **Offline**: Works without internet connection after model download
## Performance Considerations
- **First Run**: Model download may take time initially
- **Memory Usage**: Models require GPU/CPU memory
- **Batch Size**: Optimize batch size based on available memory
- **Device**: GPU acceleration significantly improves speed
## Best Practices
1. **Model Selection**: Choose model based on accuracy vs speed requirements
2. **Device Management**: Use GPU when available for better performance
3. **Batch Processing**: Process multiple documents together for efficiency
4. **Memory Monitoring**: Monitor system memory usage with larger models
docs/components/rerankers/models/zero_entropy.mdx
+119
View File
@@ -0,0 +1,119 @@
---
title: Zero Entropy
description: 'Neural reranking with Zero Entropy'
icon: "sparkles"
iconType: "solid"
---
[Zero Entropy](https://www.zeroentropy.dev) provides neural reranking models that significantly improve search relevance with fast performance.
## Models
Zero Entropy offers two reranking models:
- **`zerank-1`**: Flagship state-of-the-art reranker (non-commercial license)
- **`zerank-1-small`**: Open-source model (Apache 2.0 license)
## Installation
```bash
pip install zeroentropy
```
## Configuration
```python Python
from mem0 import Memory
config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "my_memories",
"path": "./chroma_db"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini"
}
},
"rerank": {
"provider": "zero_entropy",
"config": {
"model": "zerank-1", # or "zerank-1-small"
"api_key": "your-zero-entropy-api-key", # or set ZERO_ENTROPY_API_KEY
"top_k": 5
}
}
}
memory = Memory.from_config(config)
```
## Environment Variables
Set your API key as an environment variable:
```bash
export ZERO_ENTROPY_API_KEY="your-api-key"
```
## Usage Example
```python Python
import os
from mem0 import Memory
# Set API key
os.environ["ZERO_ENTROPY_API_KEY"] = "your-api-key"
# Initialize memory with Zero Entropy reranker
config = {
"vector_store": {"provider": "chroma"},
"llm": {"provider": "openai", "config": {"model": "gpt-4o-mini"}},
"rerank": {"provider": "zero_entropy", "config": {"model": "zerank-1"}}
}
memory = Memory.from_config(config)
# Add memories
messages = [
{"role": "user", "content": "I love Italian pasta, especially carbonara"},
{"role": "user", "content": "Japanese sushi is also amazing"},
{"role": "user", "content": "I enjoy cooking Mediterranean dishes"}
]
memory.add(messages, user_id="alice")
# Search with reranking
results = memory.search("What Italian food does the user like?", user_id="alice")
for result in results['results']:
print(f"Memory: {result['memory']}")
print(f"Vector Score: {result['score']:.3f}")
print(f"Rerank Score: {result['rerank_score']:.3f}")
print()
```
## Configuration Parameters
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | Model to use: `"zerank-1"` or `"zerank-1-small"` | `str` | `"zerank-1"` |
| `api_key` | Zero Entropy API key | `str` | `None` |
| `top_k` | Maximum documents to return after reranking | `int` | `None` |
## Performance
- **Fast**: Optimized neural architecture for low latency
- **Accurate**: State-of-the-art relevance scoring
- **Cost-effective**: ~$0.025/1M tokens processed
## Best Practices
1. **Model Selection**: Use `zerank-1` for best quality, `zerank-1-small` for faster processing
2. **Batch Size**: Process multiple queries together when possible
3. **Top-k Limiting**: Set reasonable `top_k` values (5-20) for best performance
4. **API Key Management**: Use environment variables for secure key storage
docs/components/rerankers/overview.mdx
+56 -38
View File
@@ -1,52 +1,70 @@
---
title: Overview
icon: "info"
icon: "arrow-up-arrow-down"
iconType: "solid"
---
Rerankers enhance the quality of search results by re-ordering the initial retrieval results using more sophisticated scoring mechanisms. They act as a secondary ranking layer that can significantly improve the relevance of retrieved memories.
## How Rerankers Work
1. **Initial Retrieval**: Vector search returns candidate memories based on semantic similarity
2. **Reranking**: The reranker evaluates and re-scores these candidates using more complex criteria
3. **Final Results**: Returns the top-k memories with improved relevance ordering
## Benefits
- **Improved Precision**: Better ranking of relevant memories
- **Context Awareness**: More sophisticated understanding of query-memory relationships
- **Performance**: Can improve results without changing the underlying vector store
## Supported Rerankers
Mem0 supports several reranker models:
<CardGroup cols={2}>
<Card title="Cohere" href="/components/rerankers/models/cohere" />
<Card title="Sentence Transformer" href="/components/rerankers/models/sentence_transformer" />
<Card title="Hugging Face" href="/components/rerankers/models/huggingface" />
<Card title="LLM Reranker" href="/components/rerankers/models/llm_reranker" />
</CardGroup>
Mem0 includes built-in support for various reranking providers to improve the relevance of memory search results. Rerankers post-process initial vector search results by re-scoring and re-ordering them using more sophisticated relevance models.
## Usage
Rerankers are configured as part of the memory configuration:
To use a reranker:
1. **Configure**: Add a `rerank` configuration section in your memory config
2. **Search**: Reranking is automatically enabled for all searches (default: `rerank=True`)
If no reranker is configured, search results will rely on vector similarity scoring alone.
For comprehensive configuration parameters for each reranker, please refer to [Config](./config).
### Controlling Reranking Per Search
Once configured, reranking is enabled by default. You can control it per-search:
```python
from mem0 import Memory
# Reranking enabled (default)
results = memory.search("query", user_id="user1")
config = {
"reranker": {
"provider": "cohere",
"config": {
"api_key": "your-api-key",
"top_n": 10
}
}
}
# Explicitly enable reranking
results = memory.search("query", user_id="user1", rerank=True)
memory = Memory.from_config(config)
# Disable reranking for this specific search
results = memory.search("query", user_id="user1", rerank=False)
```
For detailed configuration options, see the [Config](./config) page.
## How Reranking Works
1. **Initial Search**: Vector similarity search retrieves candidate memories
2. **Reranking** (if enabled): Selected reranker re-scores candidates using advanced models
3. **Final Results**: Re-ordered results with both vector and rerank scores
<Note>
Reranking operates as a post-processing step and can significantly improve search relevance at the cost of additional latency and API calls.
</Note>
## Supported Rerankers
See the list of supported rerankers below.
<CardGroup cols={2}>
<Card title="Zero Entropy" href="/components/rerankers/models/zero_entropy" />
<Card title="Cohere" href="/components/rerankers/models/cohere" />
<Card title="Sentence Transformer" href="/components/rerankers/models/sentence_transformer" />
<Card title="Hugging Face" href="/components/rerankers/models/huggingface" />
<Card title="LLM-based" href="/components/rerankers/models/llm" />
<Card title="LLM Reranker" href="/components/rerankers/models/llm_reranker" />
</CardGroup>
## When to Use Reranking
- **Improved Relevance**: When vector search alone doesn't provide sufficiently relevant results
- **Domain-Specific Queries**: For specialized terminology or context that benefits from advanced models
- **Quality vs Speed Trade-off**: When you can accept higher latency for better search quality
- **Production Systems**: Where search quality directly impacts user experience
Choose the reranker that best fits your use case:
- **Zero Entropy**: Best balance of speed and quality for general use
- **Cohere**: Enterprise-grade with excellent multilingual support
- **Sentence Transformer**: Local deployment for privacy-sensitive applications
- **Hugging Face**: Wide variety of pre-trained models for specialized use cases
- **LLM-based**: Maximum customization with custom prompts and logic