JDSS/mem0
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: fix LLM reranker config examples and field names (#4539)

Browse Source 
Co-authored-by: utkarsh240799 <utkarsh240799@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Utkarsh
2026-03-26 19:58:05 +05:30
committed by GitHub
co-authored by utkarsh240799 Claude Opus 4.6
parent 1aecfadf45
commit 7fff26f374
3 changed files with 171 additions and 226 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/components/rerankers/custom-prompts.mdx
+67 -87
View File
@@ -7,58 +7,51 @@ When using LLM rerankers, you can customize the prompts used for ranking to bett
## Default Prompt ## Default Prompt
The default LLM reranker prompt is designed to be general-purpose: The default LLM reranker prompt scores each memory individually on a 0.0-1.0 scale:
``` ```
Given a query and a list of memory entries, rank the memory entries based on their relevance to the query. You are a relevance scoring assistant. Given a query and a document, you need to score how relevant the document is to the query.
Rate each memory on a scale of 1-10 where 10 is most relevant.
Query: {query} Score the relevance on a scale from 0.0 to 1.0, where:
- 1.0 = Perfectly relevant and directly answers the query
- 0.8-0.9 = Highly relevant with good information
- 0.6-0.7 = Moderately relevant with some useful information
- 0.4-0.5 = Slightly relevant with limited useful information
- 0.0-0.3 = Not relevant or no useful information
Memory entries: Query: "{query}"
{memories} Document: "{document}"
Provide your ranking as a JSON array with scores for each memory. Provide only a single numerical score between 0.0 and 1.0. Do not include any explanation or additional text.
``` ```
## Custom Prompt Configuration ## Custom Prompt Configuration
You can provide a custom prompt template when configuring the LLM reranker: You can provide a custom prompt template using the `scoring_prompt` parameter:
```python ```python
from mem0 import Memory from mem0 import Memory
custom_prompt = """ custom_prompt = """
You are an expert at ranking memories for a personal AI assistant. You are an expert at evaluating memories for a personal AI assistant.
Given a user query and a list of memory entries, rank each memory based on: Given a user query and a memory entry, score how relevant the memory is.
1. Direct relevance to the query Consider direct relevance, temporal relevance, and actionability.
2. Temporal relevance (recent memories may be more important)
3. Emotional significance
4. Actionability
Query: {query} Query: "{query}"
User Context: {user_context} Memory: "{document}"
Memory entries: Provide only a single numerical score between 0.0 and 1.0.
{memories}
Rate each memory from 1-10 and provide reasoning.
Return as JSON: {{"rankings": [{{"index": 0, "score": 8, "reason": "..."}}]}}
""" """
config = { config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"llm": { "provider": "openai",
"provider": "openai", "model": "gpt-4o-mini",
"config": { "api_key": "your-openai-key",
"model": "gpt-4.1-nano-2025-04-14", "scoring_prompt": custom_prompt,
"api_key": "your-openai-key" "top_k": 5
}
},
"custom_prompt": custom_prompt,
"top_n": 5
} }
} }
} }
@@ -70,12 +63,14 @@ memory = Memory.from_config(config)
Your custom prompt can use the following variables: Your custom prompt can use the following variables:
| Variable | Description | | Variable | Description |
| ---------------- | ------------------------------------- | | ------------ | ----------------------------- |
| `{query}` | The search query | | `{query}` | The search query |
| `{memories}` | The list of memory entries to rank | | `{document}` | The memory entry being scored |
| `{user_id}` | The user ID (if available) |
| `{user_context}` | Additional user context (if provided) | <Note>
Both `{query}` and `{document}` are required in your custom prompt. The LLM reranker scores each memory individually against the query, so the prompt is called once per candidate memory.
</Note>
## Domain-Specific Examples ## Domain-Specific Examples
@@ -89,13 +84,10 @@ Prioritize memories that:
- Show previous resolution patterns - Show previous resolution patterns
- Indicate customer preferences or constraints - Indicate customer preferences or constraints
Query: {query} Query: "{query}"
Customer Context: Previous interactions with this customer Memory: "{document}"
Memories: Score relevance from 0.0 to 1.0.
{memories}
Rank each memory 1-10 based on support relevance.
""" """
``` ```
@@ -103,19 +95,16 @@ Rank each memory 1-10 based on support relevance.
```python ```python
educational_prompt = """ educational_prompt = """
Rank these learning memories for a student query. Score this learning memory for relevance to a student query.
Consider: Consider:
- Prerequisite knowledge requirements - Prerequisite knowledge requirements
- Learning progression and difficulty - Learning progression and difficulty
- Relevance to current learning objectives - Relevance to current learning objectives
Student Query: {query} Student Query: "{query}"
Learning Context: {user_context} Memory: "{document}"
Available memories: Score educational relevance from 0.0 to 1.0.
{memories}
Score each memory for educational value (1-10).
""" """
``` ```
@@ -123,73 +112,64 @@ Score each memory for educational value (1-10).
```python ```python
personal_assistant_prompt = """ personal_assistant_prompt = """
Rank personal memories for relevance to the user's query. Score this personal memory for relevance to the user's query.
Consider: Consider:
- Recent vs. historical importance - Recent vs. historical importance
- Personal preferences and habits - Personal preferences and habits
- Contextual relationships between memories - Contextual relationships
Query: {query} Query: "{query}"
Personal context: {user_context} Memory: "{document}"
Memories to rank: Provide relevance score from 0.0 to 1.0.
{memories}
Provide relevance scores (1-10) with brief explanations.
""" """
``` ```
## Advanced Prompt Techniques ## Advanced Prompt Techniques
### Multi-Criteria Ranking ### Multi-Criteria Scoring
```python ```python
multi_criteria_prompt = """ multi_criteria_prompt = """
Evaluate memories using multiple criteria: Evaluate this memory using multiple criteria:
1. RELEVANCE (40%): How directly related to the query 1. RELEVANCE (40%): How directly related to the query
2. RECENCY (20%): How recent the memory is 2. RECENCY (20%): How recent the memory appears to be
3. IMPORTANCE (25%): Personal or business significance 3. IMPORTANCE (25%): Personal or business significance
4. ACTIONABILITY (15%): How useful for next steps 4. ACTIONABILITY (15%): How useful for next steps
Query: {query} Query: "{query}"
Context: {user_context} Memory: "{document}"
Memories: Compute a weighted score from 0.0 to 1.0 based on these criteria.
{memories} Provide only the final numerical score.
For each memory, provide:
- Overall score (1-10)
- Breakdown by criteria
- Final ranking recommendation
Format: JSON with detailed scoring
""" """
``` ```
### Contextual Ranking ### Chain-of-Thought Scoring
```python ```python
contextual_prompt = """ reasoning_prompt = """
Consider the following context when ranking memories: Evaluate this memory's relevance step by step:
- Current user situation: {user_context}
- Time of day: {current_time}
- Recent activities: {recent_activities}
Query: {query} 1. What is the main intent of the query?
2. What key information does the memory contain?
3. How directly does the memory address the query?
Rank these memories considering both direct relevance and contextual appropriateness: Based on this analysis, provide a single relevance score from 0.0 to 1.0.
{memories}
Provide contextually-aware relevance scores (1-10). Query: "{query}"
Memory: "{document}"
Score:
""" """
``` ```
## Best Practices ## Best Practices
1. **Be Specific**: Clearly define what makes a memory relevant for your use case 1. **Be Specific**: Clearly define what makes a memory relevant for your use case
2. **Use Examples**: Include examples in your prompt for better model understanding 2. **Use 0.0-1.0 Scale**: The score extractor expects values between 0.0 and 1.0
3. **Structure Output**: Specify the exact JSON format you want returned 3. **Request Only the Score**: Ask for just the numerical score to improve extraction reliability
4. **Test Iteratively**: Refine your prompt based on actual ranking performance 4. **Test Iteratively**: Refine your prompt based on actual ranking performance
5. **Consider Token Limits**: Keep prompts concise while being comprehensive 5. **Consider Token Limits**: Keep prompts concise while being comprehensive
@@ -206,7 +186,7 @@ prompts = [
] ]
for i, prompt in enumerate(prompts): for i, prompt in enumerate(prompts):
config["reranker"]["config"]["custom_prompt"] = prompt config["reranker"]["config"]["scoring_prompt"] = prompt
memory = Memory.from_config(config) memory = Memory.from_config(config)
results = memory.search("test query", user_id="test_user") results = memory.search("test query", user_id="test_user")
@@ -216,6 +196,6 @@ for i, prompt in enumerate(prompts):
## Common Issues ## Common Issues
- **Too Long**: Keep prompts under token limits for your chosen LLM - **Too Long**: Keep prompts under token limits for your chosen LLM
- **Too Vague**: Be specific about ranking criteria - **Too Vague**: Be specific about scoring criteria
- **Inconsistent Format**: Ensure JSON output format is clearly specified - **Wrong Scale**: Use 0.0-1.0 scale to match the default score extractor
- **Missing Context**: Include relevant variables for your use case - **Extra Output**: Ask for only the numeric score — extra text can confuse score extraction
docs/components/rerankers/models/llm_reranker.mdx
+101 -132
View File
@@ -18,13 +18,9 @@ config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"llm": { "provider": "openai",
"provider": "openai", "model": "gpt-4o-mini",
"config": { "api_key": "your-openai-api-key"
"model": "gpt-4",
"api_key": "your-openai-api-key"
}
}
} }
} }
} }
@@ -36,11 +32,14 @@ m = Memory.from_config(config)
| Parameter | Type | Default | Description | | Parameter | Type | Default | Description |
|-----------|------|---------|-------------| |-----------|------|---------|-------------|
| `llm` | dict | Required | LLM configuration object | | `provider` | str | `"openai"` | LLM provider (openai, anthropic, etc.) |
| `top_k` | int | 10 | Number of results to rerank | | `model` | str | `"gpt-4o-mini"` | LLM model to use for reranking |
| `api_key` | str | None | API key for the LLM provider |
| `top_k` | int | None | Number of top documents to return after reranking |
| `temperature` | float | 0.0 | LLM temperature for consistency | | `temperature` | float | 0.0 | LLM temperature for consistency |
| `custom_prompt` | str | None | Custom reranking prompt | | `max_tokens` | int | 100 | Maximum tokens for LLM response |
| `score_range` | tuple | (0, 10) | Score range for relevance | | `scoring_prompt` | str | None | Custom prompt template for scoring documents |
| `llm` | dict | None | Optional nested LLM config for provider-specific fields (e.g., `ollama_base_url`, `azure_endpoint`). Overrides top-level `provider`/`model`/`api_key` when provided. |
### Advanced Configuration ### Advanced Configuration
@@ -49,20 +48,19 @@ config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"llm": { "provider": "anthropic",
"provider": "anthropic", "model": "claude-sonnet-4-20250514",
"config": { "api_key": "your-anthropic-api-key",
"model": "claude-3-sonnet-20240229",
"api_key": "your-anthropic-api-key"
}
},
"top_k": 15, "top_k": 15,
"temperature": 0.0, "temperature": 0.0,
"score_range": (1, 5), "scoring_prompt": """
"custom_prompt": """ Rate the relevance of each memory to the query on a scale of 0.0-1.0.
Rate the relevance of each memory to the query on a scale of 1-5.
Consider semantic similarity, context, and practical utility. Consider semantic similarity, context, and practical utility.
Only provide the numeric score. Only provide the numeric score.
Query: "{query}"
Document: "{document}"
Score:
""" """
} }
} }
@@ -78,14 +76,10 @@ config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"llm": { "provider": "openai",
"provider": "openai", "model": "gpt-4o-mini",
"config": { "api_key": "your-openai-api-key",
"model": "gpt-4", "temperature": 0.0
"api_key": "your-openai-api-key",
"temperature": 0.0
}
}
} }
} }
} }
@@ -98,13 +92,9 @@ config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"llm": { "provider": "anthropic",
"provider": "anthropic", "model": "claude-sonnet-4-20250514",
"config": { "api_key": "your-anthropic-api-key"
"model": "claude-3-sonnet-20240229",
"api_key": "your-anthropic-api-key"
}
}
} }
} }
} }
@@ -117,10 +107,12 @@ config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"provider": "ollama",
"model": "llama3.2",
"llm": { "llm": {
"provider": "ollama", "provider": "ollama",
"config": { "config": {
"model": "llama2", "model": "llama3.2",
"ollama_base_url": "http://localhost:11434" "ollama_base_url": "http://localhost:11434"
} }
} }
@@ -129,6 +121,10 @@ config = {
} }
``` ```
<Note>
For providers like Ollama that need extra fields (e.g., `ollama_base_url`), use the optional nested `llm` key to pass provider-specific configuration. The nested `llm` config overrides top-level `provider`/`model`/`api_key` when provided.
</Note>
### Azure OpenAI ### Azure OpenAI
```python ```python
@@ -136,13 +132,16 @@ config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"provider": "azure_openai",
"model": "gpt-4o-mini",
"api_key": "your-azure-api-key",
"llm": { "llm": {
"provider": "azure_openai", "provider": "azure_openai",
"config": { "config": {
"model": "gpt-4", "model": "gpt-4o-mini",
"api_key": "your-azure-api-key", "api_key": "your-azure-api-key",
"azure_endpoint": "https://your-resource.openai.azure.com/", "azure_endpoint": "https://your-resource.openai.azure.com/",
"azure_deployment": "gpt-4-deployment" "azure_deployment": "gpt-4o-mini-deployment"
} }
} }
} }
@@ -154,16 +153,22 @@ config = {
### Default Prompt Behavior ### Default Prompt Behavior
The default prompt asks the LLM to score relevance on a 0-10 scale: The default prompt asks the LLM to score relevance on a 0.0-1.0 scale:
``` ```
Given a query and a memory, rate how relevant the memory is to answering the query. You are a relevance scoring assistant. Given a query and a document, you need to score how relevant the document is to the query.
Score from 0 (completely irrelevant) to 10 (perfectly relevant).
Only provide the numeric score.
Query: {query} Score the relevance on a scale from 0.0 to 1.0, where:
Memory: {memory} - 1.0 = Perfectly relevant and directly answers the query
Score: - 0.8-0.9 = Highly relevant with good information
- 0.6-0.7 = Moderately relevant with some useful information
- 0.4-0.5 = Slightly relevant with limited useful information
- 0.0-0.3 = Not relevant or no useful information
Query: "{query}"
Document: "{document}"
Provide only a single numerical score between 0.0 and 1.0. Do not include any explanation or additional text.
``` ```
### Custom Prompt Examples ### Custom Prompt Examples
@@ -174,14 +179,14 @@ Score:
custom_prompt = """ custom_prompt = """
You are a medical information specialist. Rate how relevant each memory is for answering the medical query. You are a medical information specialist. Rate how relevant each memory is for answering the medical query.
Consider clinical accuracy, specificity, and practical applicability. Consider clinical accuracy, specificity, and practical applicability.
Rate from 1-10 where: Rate from 0.0 to 1.0 where:
- 1-3: Irrelevant or potentially harmful - 0.0-0.3: Irrelevant or potentially harmful
- 4-6: Somewhat relevant but incomplete - 0.4-0.6: Somewhat relevant but incomplete
- 7-8: Relevant and helpful - 0.7-0.8: Relevant and helpful
- 9-10: Highly relevant and clinically useful - 0.9-1.0: Highly relevant and clinically useful
Query: {query} Query: "{query}"
Memory: {memory} Document: "{document}"
Score: Score:
""" """
@@ -189,14 +194,10 @@ config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"llm": { "provider": "openai",
"provider": "openai", "model": "gpt-4o-mini",
"config": { "api_key": "your-api-key",
"model": "gpt-4", "scoring_prompt": custom_prompt
"api_key": "your-api-key"
}
},
"custom_prompt": custom_prompt
} }
} }
} }
@@ -213,15 +214,15 @@ Consider:
- Recency and accuracy - Recency and accuracy
- Practical usefulness - Practical usefulness
Rate 1-5: Rate 0.0-1.0:
1 = Not relevant 0.0 = Not relevant
2 = Slightly relevant 0.25 = Slightly relevant
3 = Moderately relevant 0.5 = Moderately relevant
4 = Very relevant 0.75 = Very relevant
5 = Perfectly answers the question 1.0 = Perfectly answers the question
Query: {query} Query: "{query}"
Memory: {memory} Document: "{document}"
Score: Score:
""" """
``` ```
@@ -239,13 +240,17 @@ Consider:
- Factual accuracy - Factual accuracy
- Conversation flow - Conversation flow
Rate 0-10: Rate 0.0-1.0:
Query: {query} Query: "{query}"
Memory: {memory} Document: "{document}"
Score: Score:
""" """
``` ```
<Note>
Custom prompts must include `{query}` and `{document}` placeholders. The LLM response should contain a numerical score which is automatically extracted.
</Note>
## Usage Examples ## Usage Examples
### Basic Usage ### Basic Usage
@@ -295,9 +300,9 @@ results = safe_llm_rerank_search("What are my preferences?", "alice")
| Model Type | Speed | Quality | Cost | Best For | | Model Type | Speed | Quality | Cost | Best For |
|------------|-------|---------|------|----------| |------------|-------|---------|------|----------|
| GPT-3.5 Turbo | Fast | Good | Low | High-volume applications | | GPT-4o mini | Fast | Good | Low | High-volume applications |
| GPT-4 | Medium | Excellent | Medium | Quality-critical applications | | GPT-4o | Medium | Excellent | Medium | Quality-critical applications |
| Claude 3 Sonnet | Medium | Excellent | Medium | Balanced performance | | Claude Sonnet | Medium | Excellent | Medium | Balanced performance |
| Ollama Local | Variable | Good | Free | Privacy-sensitive applications | | Ollama Local | Variable | Good | Free | Privacy-sensitive applications |
### Optimization Strategies ### Optimization Strategies
@@ -308,14 +313,10 @@ fast_config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"llm": { "provider": "openai",
"provider": "openai", "model": "gpt-4o-mini",
"config": { "api_key": "your-api-key",
"model": "gpt-3.5-turbo", "top_k": 5,
"api_key": "your-api-key"
}
},
"top_k": 5, # Limit candidates
"temperature": 0.0 "temperature": 0.0
} }
} }
@@ -326,13 +327,9 @@ quality_config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"llm": { "provider": "openai",
"provider": "openai", "model": "gpt-4o",
"config": { "api_key": "your-api-key",
"model": "gpt-4",
"api_key": "your-api-key"
}
},
"top_k": 15, "top_k": 15,
"temperature": 0.0 "temperature": 0.0
} }
@@ -353,10 +350,10 @@ Evaluate this memory's relevance using multi-step reasoning:
3. How directly does the memory address the query? 3. How directly does the memory address the query?
4. What additional context might be needed? 4. What additional context might be needed?
Based on this analysis, rate relevance 1-10: Based on this analysis, rate relevance 0.0-1.0:
Query: {query} Query: "{query}"
Memory: {memory} Document: "{document}"
Analysis: Analysis:
Step 1 (Intent): Step 1 (Intent):
@@ -367,42 +364,6 @@ Final Score:
""" """
``` ```
### Comparative Ranking
```python
comparative_prompt = """
You will see a query and multiple memories. Rank them in order of relevance.
Consider which memories best answer the question and would be most helpful.
Query: {query}
Memories to rank:
{memories}
Provide scores 1-10 for each memory, considering their relative usefulness.
"""
```
### Emotional Intelligence
```python
emotional_prompt = """
Consider both factual relevance and emotional appropriateness.
Rate how suitable this memory is for responding to the user's query.
Factors to consider:
- Factual accuracy and relevance
- Emotional tone and sensitivity
- User's likely emotional state
- Appropriateness of response
Query: {query}
Memory: {memory}
Emotional Context: {context}
Score (1-10):
"""
```
## Error Handling and Fallbacks ## Error Handling and Fallbacks
```python ```python
@@ -433,14 +394,22 @@ class RobustLLMReranker:
primary_config = { primary_config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": {"llm": {"provider": "openai", "config": {"model": "gpt-4"}}} "config": {
"provider": "openai",
"model": "gpt-4o",
"api_key": "your-api-key"
}
} }
} }
fallback_config = { fallback_config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": {"llm": {"provider": "openai", "config": {"model": "gpt-3.5-turbo"}}} "config": {
"provider": "openai",
"model": "gpt-4o-mini",
"api_key": "your-api-key"
}
} }
} }
@@ -486,4 +455,4 @@ results = reranker.search("What are my preferences?", "alice")
<Card title="Performance Optimization" icon="bolt" href="/components/rerankers/optimization"> <Card title="Performance Optimization" icon="bolt" href="/components/rerankers/optimization">
Optimize LLM reranker performance Optimize LLM reranker performance
</Card> </Card>
</CardGroup> </CardGroup>
docs/open-source/features/reranker-search.mdx
+3 -7
View File
@@ -123,13 +123,9 @@ config = {
"reranker": { "reranker": {
"provider": "llm_reranker", "provider": "llm_reranker",
"config": { "config": {
"llm": { "provider": "openai",
"provider": "openai", "model": "gpt-4o-mini",
"config": { "api_key": "your-openai-api-key",
"model": "gpt-4",
"api_key": "your-openai-api-key"
}
},
"top_k": 5 "top_k": 5
} }
} }