JDSS/mem0
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Enhanced reranker and updated corresponding docs

Browse Source
This commit is contained in:
parshvadaftari
2025-09-11 02:33:55 +05:30
parent 5182c8f311
commit f0d98ec4cb
15 changed files with 1358 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/components/rerankers/config.mdx
+90
View File
@@ -0,0 +1,90 @@
---
title: Config
description: 'Configuration options for rerankers in Mem0'
icon: "gear"
iconType: "solid"
---
## Common Configuration Parameters
All rerankers share these common configuration parameters:
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `provider` | Reranker provider name | `str` | Required |
| `top_k` | Maximum number of results to return after reranking | `int` | `None` |
| `api_key` | API key for the reranker service | `str` | `None` |
## Provider-Specific Configuration
### Zero Entropy
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | Model to use: `zerank-1` or `zerank-1-small` | `str` | `"zerank-1"` |
| `api_key` | Zero Entropy API key | `str` | `None` |
### Cohere
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | Cohere rerank model | `str` | `"rerank-english-v3.0"` |
| `api_key` | Cohere API key | `str` | `None` |
| `return_documents` | Whether to return document texts in response | `bool` | `False` |
| `max_chunks_per_doc` | Maximum chunks per document | `int` | `None` |
### Sentence Transformer
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | HuggingFace cross-encoder model name | `str` | `"cross-encoder/ms-marco-MiniLM-L-6-v2"` |
| `device` | Device to run model on (`cpu`, `cuda`, etc.) | `str` | `None` |
| `batch_size` | Batch size for processing | `int` | `32` |
| `show_progress_bar` | Show progress during processing | `bool` | `False` |
### LLM-based
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | LLM model to use for scoring | `str` | `"gpt-4o-mini"` |
| `provider` | LLM provider (`openai`, `anthropic`, etc.) | `str` | `"openai"` |
| `api_key` | API key for LLM provider | `str` | `None` |
| `temperature` | Temperature for LLM generation | `float` | `0.0` |
| `max_tokens` | Maximum tokens for LLM response | `int` | `100` |
| `scoring_prompt` | Custom prompt template for scoring | `str` | Default scoring prompt |
## Environment Variables
You can set API keys using environment variables:
- `ZERO_ENTROPY_API_KEY` - Zero Entropy API key
- `COHERE_API_KEY` - Cohere API key
- `OPENAI_API_KEY` - OpenAI API key (for LLM-based reranker)
- `ANTHROPIC_API_KEY` - Anthropic API key (for LLM-based reranker)
## Basic Configuration Example
```python Python
config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "my_memories",
"path": "./chroma_db"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini"
}
},
"rerank": {
"provider": "zero_entropy",
"config": {
"model": "zerank-1",
"top_k": 5
}
}
}
```
docs/components/rerankers/models/cohere.mdx
+147
View File
@@ -0,0 +1,147 @@
---
title: Cohere
description: 'Enterprise-grade reranking with Cohere'
icon: "building"
iconType: "solid"
---
Cohere provides enterprise-grade reranking models with excellent multilingual support and production-ready performance.
## Models
Cohere offers several reranking models:
- **`rerank-english-v3.0`**: Latest English reranker with best performance
- **`rerank-multilingual-v3.0`**: Multilingual support for global applications
- **`rerank-english-v2.0`**: Previous generation English reranker
## Installation
```bash
pip install cohere
```
## Configuration
```python Python
from mem0 import Memory
config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "my_memories",
"path": "./chroma_db"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini"
}
},
"rerank": {
"provider": "cohere",
"config": {
"model": "rerank-english-v3.0",
"api_key": "your-cohere-api-key", # or set COHERE_API_KEY
"top_k": 5,
"return_documents": False,
"max_chunks_per_doc": None
}
}
}
memory = Memory.from_config(config)
```
## Environment Variables
Set your API key as an environment variable:
```bash
export COHERE_API_KEY="your-api-key"
```
## Usage Example
```python Python
import os
from mem0 import Memory
# Set API key
os.environ["COHERE_API_KEY"] = "your-api-key"
# Initialize memory with Cohere reranker
config = {
"vector_store": {"provider": "chroma"},
"llm": {"provider": "openai", "config": {"model": "gpt-4o-mini"}},
"rerank": {
"provider": "cohere",
"config": {
"model": "rerank-english-v3.0",
"top_k": 3
}
}
}
memory = Memory.from_config(config)
# Add memories
messages = [
{"role": "user", "content": "I work as a data scientist at Microsoft"},
{"role": "user", "content": "I specialize in machine learning and NLP"},
{"role": "user", "content": "I enjoy playing tennis on weekends"}
]
memory.add(messages, user_id="bob")
# Search with reranking
results = memory.search("What is the user's profession?", user_id="bob")
for result in results['results']:
print(f"Memory: {result['memory']}")
print(f"Vector Score: {result['score']:.3f}")
print(f"Rerank Score: {result['rerank_score']:.3f}")
print()
```
## Multilingual Support
For multilingual applications, use the multilingual model:
```python Python
config = {
"rerank": {
"provider": "cohere",
"config": {
"model": "rerank-multilingual-v3.0",
"top_k": 5
}
}
}
```
## Configuration Parameters
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | Cohere rerank model to use | `str` | `"rerank-english-v3.0"` |
| `api_key` | Cohere API key | `str` | `None` |
| `top_k` | Maximum documents to return | `int` | `None` |
| `return_documents` | Whether to return document texts | `bool` | `False` |
| `max_chunks_per_doc` | Maximum chunks per document | `int` | `None` |
## Features
- **High Quality**: Enterprise-grade relevance scoring
- **Multilingual**: Support for 100+ languages
- **Scalable**: Production-ready with high throughput
- **Reliable**: SLA-backed service with 99.9% uptime
## Best Practices
1. **Model Selection**: Use `rerank-english-v3.0` for English, `rerank-multilingual-v3.0` for other languages
2. **Batch Processing**: Process multiple queries efficiently
3. **Error Handling**: Implement retry logic for production systems
4. **Monitoring**: Track reranking performance and costs
docs/components/rerankers/models/llm.mdx
+214
View File
@@ -0,0 +1,214 @@
---
title: LLM-based
description: 'Flexible reranking using any Large Language Model'
icon: "robot"
iconType: "solid"
---
LLM-based reranker provides maximum flexibility by using any Large Language Model to score document relevance. This approach allows for custom prompts and domain-specific scoring logic.
## Supported LLM Providers
Any LLM provider supported by Mem0 can be used for reranking:
- **OpenAI**: GPT-4, GPT-3.5-turbo, etc.
- **Anthropic**: Claude models
- **Together**: Open-source models
- **Groq**: Fast inference
- **Ollama**: Local models
- And more...
## Configuration
```python Python
from mem0 import Memory
config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "my_memories",
"path": "./chroma_db"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini"
}
},
"rerank": {
"provider": "llm",
"config": {
"model": "gpt-4o-mini",
"provider": "openai",
"api_key": "your-openai-api-key", # or set OPENAI_API_KEY
"top_k": 5,
"temperature": 0.0
}
}
}
memory = Memory.from_config(config)
```
## Custom Scoring Prompt
You can provide a custom prompt for relevance scoring:
```python Python
custom_prompt = """You are a relevance scoring assistant. Rate how well this document answers the query.
Query: "{query}"
Document: "{document}"
Score from 0.0 to 1.0 where:
- 1.0: Perfect match, directly answers the query
- 0.8-0.9: Highly relevant, good match
- 0.6-0.7: Moderately relevant, partial match
- 0.4-0.5: Slightly relevant, limited useful information
- 0.0-0.3: Not relevant or no useful information
Provide only a single numerical score between 0.0 and 1.0."""
config["rerank"]["config"]["scoring_prompt"] = custom_prompt
```
## Usage Example
```python Python
import os
from mem0 import Memory
# Set API key
os.environ["OPENAI_API_KEY"] = "your-api-key"
# Initialize memory with LLM reranker
config = {
"vector_store": {"provider": "chroma"},
"llm": {"provider": "openai", "config": {"model": "gpt-4o-mini"}},
"rerank": {
"provider": "llm",
"config": {
"model": "gpt-4o-mini",
"provider": "openai",
"temperature": 0.0
}
}
}
memory = Memory.from_config(config)
# Add memories
messages = [
{"role": "user", "content": "I'm learning Python programming"},
{"role": "user", "content": "I find object-oriented programming challenging"},
{"role": "user", "content": "I love hiking in national parks"}
]
memory.add(messages, user_id="david")
# Search with LLM reranking
results = memory.search("What programming topics is the user studying?", user_id="david")
for result in results['results']:
print(f"Memory: {result['memory']}")
print(f"Vector Score: {result['score']:.3f}")
print(f"Rerank Score: {result['rerank_score']:.3f}")
print()
```
## Domain-Specific Scoring
Create specialized scoring for your domain:
```python Python
medical_prompt = """You are a medical relevance expert. Score how relevant this medical record is to the clinical query.
Clinical Query: "{query}"
Medical Record: "{document}"
Consider:
- Clinical relevance and accuracy
- Patient safety implications
- Diagnostic value
- Treatment relevance
Score from 0.0 to 1.0. Provide only the numerical score."""
config = {
"rerank": {
"provider": "llm",
"config": {
"model": "gpt-4o-mini",
"provider": "openai",
"scoring_prompt": medical_prompt,
"temperature": 0.0
}
}
}
```
## Multiple LLM Providers
Use different LLM providers for reranking:
```python Python
# Using Anthropic Claude
anthropic_config = {
"rerank": {
"provider": "llm",
"config": {
"model": "claude-3-haiku-20240307",
"provider": "anthropic",
"temperature": 0.0
}
}
}
# Using local Ollama model
ollama_config = {
"rerank": {
"provider": "llm",
"config": {
"model": "llama2:7b",
"provider": "ollama",
"temperature": 0.0
}
}
}
```
## Configuration Parameters
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | LLM model to use for scoring | `str` | `"gpt-4o-mini"` |
| `provider` | LLM provider name | `str` | `"openai"` |
| `api_key` | API key for the LLM provider | `str` | `None` |
| `top_k` | Maximum documents to return | `int` | `None` |
| `temperature` | Temperature for LLM generation | `float` | `0.0` |
| `max_tokens` | Maximum tokens for LLM response | `int` | `100` |
| `scoring_prompt` | Custom prompt template | `str` | Default prompt |
## Advantages
- **Maximum Flexibility**: Custom prompts for any use case
- **Domain Expertise**: Leverage LLM knowledge for specialized domains
- **Interpretability**: Understand scoring through prompt engineering
- **Multi-criteria**: Score based on multiple relevance factors
## Considerations
- **Latency**: Higher latency than specialized rerankers
- **Cost**: LLM API costs per reranking operation
- **Consistency**: May have slight variations in scoring
- **Prompt Engineering**: Requires careful prompt design
## Best Practices
1. **Temperature**: Use 0.0 for consistent scoring
2. **Prompt Design**: Be specific about scoring criteria
3. **Token Efficiency**: Keep prompts concise to reduce costs
4. **Caching**: Cache results for repeated queries when possible
5. **Fallback**: Handle API errors gracefully
docs/components/rerankers/models/sentence_transformer.mdx
+161
View File
@@ -0,0 +1,161 @@
---
title: Sentence Transformer
description: 'Local reranking with HuggingFace cross-encoder models'
icon: "server"
iconType: "solid"
---
Sentence Transformer reranker provides local reranking using HuggingFace cross-encoder models, perfect for privacy-focused deployments where you want to keep data on-premises.
## Models
Any HuggingFace cross-encoder model can be used. Popular choices include:
- **`cross-encoder/ms-marco-MiniLM-L-6-v2`**: Default, good balance of speed and accuracy
- **`cross-encoder/ms-marco-TinyBERT-L-2-v2`**: Fastest, smaller model size
- **`cross-encoder/ms-marco-electra-base`**: Higher accuracy, larger model
- **`cross-encoder/stsb-distilroberta-base`**: Good for semantic similarity tasks
## Installation
```bash
pip install sentence-transformers
```
## Configuration
```python Python
from mem0 import Memory
config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "my_memories",
"path": "./chroma_db"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini"
}
},
"rerank": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cpu", # or "cuda" for GPU
"batch_size": 32,
"show_progress_bar": False,
"top_k": 5
}
}
}
memory = Memory.from_config(config)
```
## GPU Acceleration
For better performance, use GPU acceleration:
```python Python
config = {
"rerank": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cuda", # Use GPU
"batch_size": 64 # Larger batch size for GPU
}
}
}
```
## Usage Example
```python Python
from mem0 import Memory
# Initialize memory with local reranker
config = {
"vector_store": {"provider": "chroma"},
"llm": {"provider": "openai", "config": {"model": "gpt-4o-mini"}},
"rerank": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/ms-marco-MiniLM-L-6-v2",
"device": "cpu"
}
}
}
memory = Memory.from_config(config)
# Add memories
messages = [
{"role": "user", "content": "I love reading science fiction novels"},
{"role": "user", "content": "My favorite author is Isaac Asimov"},
{"role": "user", "content": "I also enjoy watching sci-fi movies"}
]
memory.add(messages, user_id="charlie")
# Search with local reranking
results = memory.search("What books does the user like?", user_id="charlie")
for result in results['results']:
print(f"Memory: {result['memory']}")
print(f"Vector Score: {result['score']:.3f}")
print(f"Rerank Score: {result['rerank_score']:.3f}")
print()
```
## Custom Models
You can use any HuggingFace cross-encoder model:
```python Python
# Using a different model
config = {
"rerank": {
"provider": "sentence_transformer",
"config": {
"model": "cross-encoder/stsb-distilroberta-base",
"device": "cpu"
}
}
}
```
## Configuration Parameters
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | HuggingFace cross-encoder model name | `str` | `"cross-encoder/ms-marco-MiniLM-L-6-v2"` |
| `device` | Device to run model on (`cpu`, `cuda`, etc.) | `str` | `None` |
| `batch_size` | Batch size for processing documents | `int` | `32` |
| `show_progress_bar` | Show progress bar during processing | `bool` | `False` |
| `top_k` | Maximum documents to return | `int` | `None` |
## Advantages
- **Privacy**: Complete local processing, no external API calls
- **Cost**: No per-token charges after initial model download
- **Customization**: Use any HuggingFace cross-encoder model
- **Offline**: Works without internet connection after model download
## Performance Considerations
- **First Run**: Model download may take time initially
- **Memory Usage**: Models require GPU/CPU memory
- **Batch Size**: Optimize batch size based on available memory
- **Device**: GPU acceleration significantly improves speed
## Best Practices
1. **Model Selection**: Choose model based on accuracy vs speed requirements
2. **Device Management**: Use GPU when available for better performance
3. **Batch Processing**: Process multiple documents together for efficiency
4. **Memory Monitoring**: Monitor system memory usage with larger models
docs/components/rerankers/models/zero_entropy.mdx
+119
View File
@@ -0,0 +1,119 @@
---
title: Zero Entropy
description: 'State-of-the-art neural reranking with Zero Entropy'
icon: "sparkles"
iconType: "solid"
---
[Zero Entropy](https://www.zeroentropy.dev) provides state-of-the-art neural reranking models that significantly improve search relevance with fast performance.
## Models
Zero Entropy offers two reranking models:
- **`zerank-1`**: Flagship state-of-the-art reranker (non-commercial license)
- **`zerank-1-small`**: Open-source model (Apache 2.0 license)
## Installation
```bash
pip install zeroentropy
```
## Configuration
```python Python
from mem0 import Memory
config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "my_memories",
"path": "./chroma_db"
}
},
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini"
}
},
"rerank": {
"provider": "zero_entropy",
"config": {
"model": "zerank-1", # or "zerank-1-small"
"api_key": "your-zero-entropy-api-key", # or set ZERO_ENTROPY_API_KEY
"top_k": 5
}
}
}
memory = Memory.from_config(config)
```
## Environment Variables
Set your API key as an environment variable:
```bash
export ZERO_ENTROPY_API_KEY="your-api-key"
```
## Usage Example
```python Python
import os
from mem0 import Memory
# Set API key
os.environ["ZERO_ENTROPY_API_KEY"] = "your-api-key"
# Initialize memory with Zero Entropy reranker
config = {
"vector_store": {"provider": "chroma"},
"llm": {"provider": "openai", "config": {"model": "gpt-4o-mini"}},
"rerank": {"provider": "zero_entropy", "config": {"model": "zerank-1"}}
}
memory = Memory.from_config(config)
# Add memories
messages = [
{"role": "user", "content": "I love Italian pasta, especially carbonara"},
{"role": "user", "content": "Japanese sushi is also amazing"},
{"role": "user", "content": "I enjoy cooking Mediterranean dishes"}
]
memory.add(messages, user_id="alice")
# Search with reranking
results = memory.search("What Italian food does the user like?", user_id="alice")
for result in results['results']:
print(f"Memory: {result['memory']}")
print(f"Vector Score: {result['score']:.3f}")
print(f"Rerank Score: {result['rerank_score']:.3f}")
print()
```
## Configuration Parameters
| Parameter | Description | Type | Default |
|-----------|-------------|------|---------|
| `model` | Model to use: `"zerank-1"` or `"zerank-1-small"` | `str` | `"zerank-1"` |
| `api_key` | Zero Entropy API key | `str` | `None` |
| `top_k` | Maximum documents to return after reranking | `int` | `None` |
## Performance
- **Fast**: Optimized neural architecture for low latency
- **Accurate**: State-of-the-art relevance scoring
- **Cost-effective**: ~$0.025/1M tokens processed
## Best Practices
1. **Model Selection**: Use `zerank-1` for best quality, `zerank-1-small` for faster processing
2. **Batch Size**: Process multiple queries together when possible
3. **Top-k Limiting**: Set reasonable `top_k` values (5-20) for best performance
4. **API Key Management**: Use environment variables for secure key storage
docs/components/rerankers/overview.mdx
+47
View File
@@ -0,0 +1,47 @@
---
title: Overview
icon: "arrow-up-arrow-down"
iconType: "solid"
---
Mem0 includes built-in support for various reranking providers to improve the relevance of memory search results. Rerankers post-process initial vector search results by re-scoring and re-ordering them using more sophisticated relevance models.
## Usage
To use a reranker, you must provide a `rerank` configuration section in your memory config. If no reranker is configured, search results will rely on vector similarity scoring alone.
For comprehensive configuration parameters for each reranker, please refer to [Config](./config).
## How Reranking Works
1. **Initial Search**: Vector similarity search retrieves candidate memories
2. **Reranking**: Selected reranker re-scores candidates using advanced models
3. **Final Results**: Re-ordered results with both vector and rerank scores
<Note>
Reranking operates as a post-processing step and can significantly improve search relevance at the cost of additional latency and API calls.
</Note>
## Supported Rerankers
See the list of supported rerankers below.
<CardGroup cols={2}>
<Card title="Zero Entropy" href="/components/rerankers/models/zero_entropy" />
<Card title="Cohere" href="/components/rerankers/models/cohere" />
<Card title="Sentence Transformer" href="/components/rerankers/models/sentence_transformer" />
<Card title="LLM-based" href="/components/rerankers/models/llm" />
</CardGroup>
## When to Use Reranking
- **Improved Relevance**: When vector search alone doesn't provide sufficiently relevant results
- **Domain-Specific Queries**: For specialized terminology or context that benefits from advanced models
- **Quality vs Speed Trade-off**: When you can accept higher latency for better search quality
- **Production Systems**: Where search quality directly impacts user experience
Choose the reranker that best fits your use case:
- **Zero Entropy**: Best balance of speed and quality for general use
- **Cohere**: Enterprise-grade with excellent multilingual support
- **Sentence Transformer**: Local deployment for privacy-sensitive applications
- **LLM-based**: Maximum customization with custom prompts and logic