JDSS/mem0
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: fix LLM reranker config examples and field names (#4539)

Browse Source 
Co-authored-by: utkarsh240799 <utkarsh240799@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Utkarsh
2026-03-26 19:58:05 +05:30
committed by GitHub
co-authored by utkarsh240799 Claude Opus 4.6
parent 1aecfadf45
commit 7fff26f374
3 changed files with 171 additions and 226 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/components/rerankers/custom-prompts.mdx
+67 -87
View File
@@ -7,58 +7,51 @@ When using LLM rerankers, you can customize the prompts used for ranking to bett
## Default Prompt
The default LLM reranker prompt is designed to be general-purpose:
The default LLM reranker prompt scores each memory individually on a 0.0-1.0 scale:
```
Given a query and a list of memory entries, rank the memory entries based on their relevance to the query.
Rate each memory on a scale of 1-10 where 10 is most relevant.
You are a relevance scoring assistant. Given a query and a document, you need to score how relevant the document is to the query.
Query: {query}
Score the relevance on a scale from 0.0 to 1.0, where:
- 1.0 = Perfectly relevant and directly answers the query
- 0.8-0.9 = Highly relevant with good information
- 0.6-0.7 = Moderately relevant with some useful information
- 0.4-0.5 = Slightly relevant with limited useful information
- 0.0-0.3 = Not relevant or no useful information
Memory entries:
{memories}
Query: "{query}"
Document: "{document}"
Provide your ranking as a JSON array with scores for each memory.
Provide only a single numerical score between 0.0 and 1.0. Do not include any explanation or additional text.
```
## Custom Prompt Configuration
You can provide a custom prompt template when configuring the LLM reranker:
You can provide a custom prompt template using the `scoring_prompt` parameter:
```python
from mem0 import Memory
custom_prompt = """
You are an expert at ranking memories for a personal AI assistant.
Given a user query and a list of memory entries, rank each memory based on:
1. Direct relevance to the query
2. Temporal relevance (recent memories may be more important)
3. Emotional significance
4. Actionability
You are an expert at evaluating memories for a personal AI assistant.
Given a user query and a memory entry, score how relevant the memory is.
Consider direct relevance, temporal relevance, and actionability.
Query: {query}
User Context: {user_context}
Query: "{query}"
Memory: "{document}"
Memory entries:
{memories}
Rate each memory from 1-10 and provide reasoning.
Return as JSON: {{"rankings": [{{"index": 0, "score": 8, "reason": "..."}}]}}
Provide only a single numerical score between 0.0 and 1.0.
"""
config = {
"reranker": {
"provider": "llm_reranker",
"config": {
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4.1-nano-2025-04-14",
"api_key": "your-openai-key"
}
},
"custom_prompt": custom_prompt,
"top_n": 5
"provider": "openai",
"model": "gpt-4o-mini",
"api_key": "your-openai-key",
"scoring_prompt": custom_prompt,
"top_k": 5
}
}
}
@@ -70,12 +63,14 @@ memory = Memory.from_config(config)
Your custom prompt can use the following variables:
| Variable | Description |
| ---------------- | ------------------------------------- |
| `{query}` | The search query |
| `{memories}` | The list of memory entries to rank |
| `{user_id}` | The user ID (if available) |
| `{user_context}` | Additional user context (if provided) |
| Variable | Description |
| ------------ | ----------------------------- |
| `{query}` | The search query |
| `{document}` | The memory entry being scored |
<Note>
Both `{query}` and `{document}` are required in your custom prompt. The LLM reranker scores each memory individually against the query, so the prompt is called once per candidate memory.
</Note>
## Domain-Specific Examples
@@ -89,13 +84,10 @@ Prioritize memories that:
- Show previous resolution patterns
- Indicate customer preferences or constraints
Query: {query}
Customer Context: Previous interactions with this customer
Query: "{query}"
Memory: "{document}"
Memories:
{memories}
Rank each memory 1-10 based on support relevance.
Score relevance from 0.0 to 1.0.
"""
```
@@ -103,19 +95,16 @@ Rank each memory 1-10 based on support relevance.
```python
educational_prompt = """
Rank these learning memories for a student query.
Score this learning memory for relevance to a student query.
Consider:
- Prerequisite knowledge requirements
- Learning progression and difficulty
- Relevance to current learning objectives
Student Query: {query}
Learning Context: {user_context}
Student Query: "{query}"
Memory: "{document}"
Available memories:
{memories}
Score each memory for educational value (1-10).
Score educational relevance from 0.0 to 1.0.
"""
```
@@ -123,73 +112,64 @@ Score each memory for educational value (1-10).
```python
personal_assistant_prompt = """
Rank personal memories for relevance to the user's query.
Score this personal memory for relevance to the user's query.
Consider:
- Recent vs. historical importance
- Personal preferences and habits
- Contextual relationships between memories
- Contextual relationships
Query: {query}
Personal context: {user_context}
Query: "{query}"
Memory: "{document}"
Memories to rank:
{memories}
Provide relevance scores (1-10) with brief explanations.
Provide relevance score from 0.0 to 1.0.
"""
```
## Advanced Prompt Techniques
### Multi-Criteria Ranking
### Multi-Criteria Scoring
```python
multi_criteria_prompt = """
Evaluate memories using multiple criteria:
Evaluate this memory using multiple criteria:
1. RELEVANCE (40%): How directly related to the query
2. RECENCY (20%): How recent the memory is
2. RECENCY (20%): How recent the memory appears to be
3. IMPORTANCE (25%): Personal or business significance
4. ACTIONABILITY (15%): How useful for next steps
Query: {query}
Context: {user_context}
Query: "{query}"
Memory: "{document}"
Memories:
{memories}
For each memory, provide:
- Overall score (1-10)
- Breakdown by criteria
- Final ranking recommendation
Format: JSON with detailed scoring
Compute a weighted score from 0.0 to 1.0 based on these criteria.
Provide only the final numerical score.
"""
```
### Contextual Ranking
### Chain-of-Thought Scoring
```python
contextual_prompt = """
Consider the following context when ranking memories:
- Current user situation: {user_context}
- Time of day: {current_time}
- Recent activities: {recent_activities}
reasoning_prompt = """
Evaluate this memory's relevance step by step:
Query: {query}
1. What is the main intent of the query?
2. What key information does the memory contain?
3. How directly does the memory address the query?
Rank these memories considering both direct relevance and contextual appropriateness:
{memories}
Based on this analysis, provide a single relevance score from 0.0 to 1.0.
Provide contextually-aware relevance scores (1-10).
Query: "{query}"
Memory: "{document}"
Score:
"""
```
## Best Practices
1. **Be Specific**: Clearly define what makes a memory relevant for your use case
2. **Use Examples**: Include examples in your prompt for better model understanding
3. **Structure Output**: Specify the exact JSON format you want returned
2. **Use 0.0-1.0 Scale**: The score extractor expects values between 0.0 and 1.0
3. **Request Only the Score**: Ask for just the numerical score to improve extraction reliability
4. **Test Iteratively**: Refine your prompt based on actual ranking performance
5. **Consider Token Limits**: Keep prompts concise while being comprehensive
@@ -206,7 +186,7 @@ prompts = [
]
for i, prompt in enumerate(prompts):
config["reranker"]["config"]["custom_prompt"] = prompt
config["reranker"]["config"]["scoring_prompt"] = prompt
memory = Memory.from_config(config)
results = memory.search("test query", user_id="test_user")
@@ -216,6 +196,6 @@ for i, prompt in enumerate(prompts):
## Common Issues
- **Too Long**: Keep prompts under token limits for your chosen LLM
- **Too Vague**: Be specific about ranking criteria
- **Inconsistent Format**: Ensure JSON output format is clearly specified
- **Missing Context**: Include relevant variables for your use case
- **Too Vague**: Be specific about scoring criteria
- **Wrong Scale**: Use 0.0-1.0 scale to match the default score extractor
- **Extra Output**: Ask for only the numeric score — extra text can confuse score extraction