diff --git a/docs/components/rerankers/custom-prompts.mdx b/docs/components/rerankers/custom-prompts.mdx index 5ca736064..1d72137c0 100644 --- a/docs/components/rerankers/custom-prompts.mdx +++ b/docs/components/rerankers/custom-prompts.mdx @@ -7,58 +7,51 @@ When using LLM rerankers, you can customize the prompts used for ranking to bett ## Default Prompt -The default LLM reranker prompt is designed to be general-purpose: +The default LLM reranker prompt scores each memory individually on a 0.0-1.0 scale: ``` -Given a query and a list of memory entries, rank the memory entries based on their relevance to the query. -Rate each memory on a scale of 1-10 where 10 is most relevant. +You are a relevance scoring assistant. Given a query and a document, you need to score how relevant the document is to the query. -Query: {query} +Score the relevance on a scale from 0.0 to 1.0, where: +- 1.0 = Perfectly relevant and directly answers the query +- 0.8-0.9 = Highly relevant with good information +- 0.6-0.7 = Moderately relevant with some useful information +- 0.4-0.5 = Slightly relevant with limited useful information +- 0.0-0.3 = Not relevant or no useful information -Memory entries: -{memories} +Query: "{query}" +Document: "{document}" -Provide your ranking as a JSON array with scores for each memory. +Provide only a single numerical score between 0.0 and 1.0. Do not include any explanation or additional text. ``` ## Custom Prompt Configuration -You can provide a custom prompt template when configuring the LLM reranker: +You can provide a custom prompt template using the `scoring_prompt` parameter: ```python from mem0 import Memory custom_prompt = """ -You are an expert at ranking memories for a personal AI assistant. -Given a user query and a list of memory entries, rank each memory based on: -1. Direct relevance to the query -2. Temporal relevance (recent memories may be more important) -3. Emotional significance -4. Actionability +You are an expert at evaluating memories for a personal AI assistant. +Given a user query and a memory entry, score how relevant the memory is. +Consider direct relevance, temporal relevance, and actionability. -Query: {query} -User Context: {user_context} +Query: "{query}" +Memory: "{document}" -Memory entries: -{memories} - -Rate each memory from 1-10 and provide reasoning. -Return as JSON: {{"rankings": [{{"index": 0, "score": 8, "reason": "..."}}]}} +Provide only a single numerical score between 0.0 and 1.0. """ config = { "reranker": { "provider": "llm_reranker", "config": { - "llm": { - "provider": "openai", - "config": { - "model": "gpt-4.1-nano-2025-04-14", - "api_key": "your-openai-key" - } - }, - "custom_prompt": custom_prompt, - "top_n": 5 + "provider": "openai", + "model": "gpt-4o-mini", + "api_key": "your-openai-key", + "scoring_prompt": custom_prompt, + "top_k": 5 } } } @@ -70,12 +63,14 @@ memory = Memory.from_config(config) Your custom prompt can use the following variables: -| Variable | Description | -| ---------------- | ------------------------------------- | -| `{query}` | The search query | -| `{memories}` | The list of memory entries to rank | -| `{user_id}` | The user ID (if available) | -| `{user_context}` | Additional user context (if provided) | +| Variable | Description | +| ------------ | ----------------------------- | +| `{query}` | The search query | +| `{document}` | The memory entry being scored | + + + Both `{query}` and `{document}` are required in your custom prompt. The LLM reranker scores each memory individually against the query, so the prompt is called once per candidate memory. + ## Domain-Specific Examples @@ -89,13 +84,10 @@ Prioritize memories that: - Show previous resolution patterns - Indicate customer preferences or constraints -Query: {query} -Customer Context: Previous interactions with this customer +Query: "{query}" +Memory: "{document}" -Memories: -{memories} - -Rank each memory 1-10 based on support relevance. +Score relevance from 0.0 to 1.0. """ ``` @@ -103,19 +95,16 @@ Rank each memory 1-10 based on support relevance. ```python educational_prompt = """ -Rank these learning memories for a student query. +Score this learning memory for relevance to a student query. Consider: - Prerequisite knowledge requirements - Learning progression and difficulty - Relevance to current learning objectives -Student Query: {query} -Learning Context: {user_context} +Student Query: "{query}" +Memory: "{document}" -Available memories: -{memories} - -Score each memory for educational value (1-10). +Score educational relevance from 0.0 to 1.0. """ ``` @@ -123,73 +112,64 @@ Score each memory for educational value (1-10). ```python personal_assistant_prompt = """ -Rank personal memories for relevance to the user's query. +Score this personal memory for relevance to the user's query. Consider: - Recent vs. historical importance - Personal preferences and habits -- Contextual relationships between memories +- Contextual relationships -Query: {query} -Personal context: {user_context} +Query: "{query}" +Memory: "{document}" -Memories to rank: -{memories} - -Provide relevance scores (1-10) with brief explanations. +Provide relevance score from 0.0 to 1.0. """ ``` ## Advanced Prompt Techniques -### Multi-Criteria Ranking +### Multi-Criteria Scoring ```python multi_criteria_prompt = """ -Evaluate memories using multiple criteria: +Evaluate this memory using multiple criteria: 1. RELEVANCE (40%): How directly related to the query -2. RECENCY (20%): How recent the memory is +2. RECENCY (20%): How recent the memory appears to be 3. IMPORTANCE (25%): Personal or business significance 4. ACTIONABILITY (15%): How useful for next steps -Query: {query} -Context: {user_context} +Query: "{query}" +Memory: "{document}" -Memories: -{memories} - -For each memory, provide: -- Overall score (1-10) -- Breakdown by criteria -- Final ranking recommendation - -Format: JSON with detailed scoring +Compute a weighted score from 0.0 to 1.0 based on these criteria. +Provide only the final numerical score. """ ``` -### Contextual Ranking +### Chain-of-Thought Scoring ```python -contextual_prompt = """ -Consider the following context when ranking memories: -- Current user situation: {user_context} -- Time of day: {current_time} -- Recent activities: {recent_activities} +reasoning_prompt = """ +Evaluate this memory's relevance step by step: -Query: {query} +1. What is the main intent of the query? +2. What key information does the memory contain? +3. How directly does the memory address the query? -Rank these memories considering both direct relevance and contextual appropriateness: -{memories} +Based on this analysis, provide a single relevance score from 0.0 to 1.0. -Provide contextually-aware relevance scores (1-10). +Query: "{query}" +Memory: "{document}" + +Score: """ ``` ## Best Practices 1. **Be Specific**: Clearly define what makes a memory relevant for your use case -2. **Use Examples**: Include examples in your prompt for better model understanding -3. **Structure Output**: Specify the exact JSON format you want returned +2. **Use 0.0-1.0 Scale**: The score extractor expects values between 0.0 and 1.0 +3. **Request Only the Score**: Ask for just the numerical score to improve extraction reliability 4. **Test Iteratively**: Refine your prompt based on actual ranking performance 5. **Consider Token Limits**: Keep prompts concise while being comprehensive @@ -206,7 +186,7 @@ prompts = [ ] for i, prompt in enumerate(prompts): - config["reranker"]["config"]["custom_prompt"] = prompt + config["reranker"]["config"]["scoring_prompt"] = prompt memory = Memory.from_config(config) results = memory.search("test query", user_id="test_user") @@ -216,6 +196,6 @@ for i, prompt in enumerate(prompts): ## Common Issues - **Too Long**: Keep prompts under token limits for your chosen LLM -- **Too Vague**: Be specific about ranking criteria -- **Inconsistent Format**: Ensure JSON output format is clearly specified -- **Missing Context**: Include relevant variables for your use case +- **Too Vague**: Be specific about scoring criteria +- **Wrong Scale**: Use 0.0-1.0 scale to match the default score extractor +- **Extra Output**: Ask for only the numeric score — extra text can confuse score extraction diff --git a/docs/components/rerankers/models/llm_reranker.mdx b/docs/components/rerankers/models/llm_reranker.mdx index 5fa1582ce..3abb827dc 100644 --- a/docs/components/rerankers/models/llm_reranker.mdx +++ b/docs/components/rerankers/models/llm_reranker.mdx @@ -18,13 +18,9 @@ config = { "reranker": { "provider": "llm_reranker", "config": { - "llm": { - "provider": "openai", - "config": { - "model": "gpt-4", - "api_key": "your-openai-api-key" - } - } + "provider": "openai", + "model": "gpt-4o-mini", + "api_key": "your-openai-api-key" } } } @@ -36,11 +32,14 @@ m = Memory.from_config(config) | Parameter | Type | Default | Description | |-----------|------|---------|-------------| -| `llm` | dict | Required | LLM configuration object | -| `top_k` | int | 10 | Number of results to rerank | +| `provider` | str | `"openai"` | LLM provider (openai, anthropic, etc.) | +| `model` | str | `"gpt-4o-mini"` | LLM model to use for reranking | +| `api_key` | str | None | API key for the LLM provider | +| `top_k` | int | None | Number of top documents to return after reranking | | `temperature` | float | 0.0 | LLM temperature for consistency | -| `custom_prompt` | str | None | Custom reranking prompt | -| `score_range` | tuple | (0, 10) | Score range for relevance | +| `max_tokens` | int | 100 | Maximum tokens for LLM response | +| `scoring_prompt` | str | None | Custom prompt template for scoring documents | +| `llm` | dict | None | Optional nested LLM config for provider-specific fields (e.g., `ollama_base_url`, `azure_endpoint`). Overrides top-level `provider`/`model`/`api_key` when provided. | ### Advanced Configuration @@ -49,20 +48,19 @@ config = { "reranker": { "provider": "llm_reranker", "config": { - "llm": { - "provider": "anthropic", - "config": { - "model": "claude-3-sonnet-20240229", - "api_key": "your-anthropic-api-key" - } - }, + "provider": "anthropic", + "model": "claude-sonnet-4-20250514", + "api_key": "your-anthropic-api-key", "top_k": 15, "temperature": 0.0, - "score_range": (1, 5), - "custom_prompt": """ - Rate the relevance of each memory to the query on a scale of 1-5. + "scoring_prompt": """ + Rate the relevance of each memory to the query on a scale of 0.0-1.0. Consider semantic similarity, context, and practical utility. Only provide the numeric score. + + Query: "{query}" + Document: "{document}" + Score: """ } } @@ -78,14 +76,10 @@ config = { "reranker": { "provider": "llm_reranker", "config": { - "llm": { - "provider": "openai", - "config": { - "model": "gpt-4", - "api_key": "your-openai-api-key", - "temperature": 0.0 - } - } + "provider": "openai", + "model": "gpt-4o-mini", + "api_key": "your-openai-api-key", + "temperature": 0.0 } } } @@ -98,13 +92,9 @@ config = { "reranker": { "provider": "llm_reranker", "config": { - "llm": { - "provider": "anthropic", - "config": { - "model": "claude-3-sonnet-20240229", - "api_key": "your-anthropic-api-key" - } - } + "provider": "anthropic", + "model": "claude-sonnet-4-20250514", + "api_key": "your-anthropic-api-key" } } } @@ -117,10 +107,12 @@ config = { "reranker": { "provider": "llm_reranker", "config": { + "provider": "ollama", + "model": "llama3.2", "llm": { "provider": "ollama", "config": { - "model": "llama2", + "model": "llama3.2", "ollama_base_url": "http://localhost:11434" } } @@ -129,6 +121,10 @@ config = { } ``` + + For providers like Ollama that need extra fields (e.g., `ollama_base_url`), use the optional nested `llm` key to pass provider-specific configuration. The nested `llm` config overrides top-level `provider`/`model`/`api_key` when provided. + + ### Azure OpenAI ```python @@ -136,13 +132,16 @@ config = { "reranker": { "provider": "llm_reranker", "config": { + "provider": "azure_openai", + "model": "gpt-4o-mini", + "api_key": "your-azure-api-key", "llm": { "provider": "azure_openai", "config": { - "model": "gpt-4", + "model": "gpt-4o-mini", "api_key": "your-azure-api-key", "azure_endpoint": "https://your-resource.openai.azure.com/", - "azure_deployment": "gpt-4-deployment" + "azure_deployment": "gpt-4o-mini-deployment" } } } @@ -154,16 +153,22 @@ config = { ### Default Prompt Behavior -The default prompt asks the LLM to score relevance on a 0-10 scale: +The default prompt asks the LLM to score relevance on a 0.0-1.0 scale: ``` -Given a query and a memory, rate how relevant the memory is to answering the query. -Score from 0 (completely irrelevant) to 10 (perfectly relevant). -Only provide the numeric score. +You are a relevance scoring assistant. Given a query and a document, you need to score how relevant the document is to the query. -Query: {query} -Memory: {memory} -Score: +Score the relevance on a scale from 0.0 to 1.0, where: +- 1.0 = Perfectly relevant and directly answers the query +- 0.8-0.9 = Highly relevant with good information +- 0.6-0.7 = Moderately relevant with some useful information +- 0.4-0.5 = Slightly relevant with limited useful information +- 0.0-0.3 = Not relevant or no useful information + +Query: "{query}" +Document: "{document}" + +Provide only a single numerical score between 0.0 and 1.0. Do not include any explanation or additional text. ``` ### Custom Prompt Examples @@ -174,14 +179,14 @@ Score: custom_prompt = """ You are a medical information specialist. Rate how relevant each memory is for answering the medical query. Consider clinical accuracy, specificity, and practical applicability. -Rate from 1-10 where: -- 1-3: Irrelevant or potentially harmful -- 4-6: Somewhat relevant but incomplete -- 7-8: Relevant and helpful -- 9-10: Highly relevant and clinically useful +Rate from 0.0 to 1.0 where: +- 0.0-0.3: Irrelevant or potentially harmful +- 0.4-0.6: Somewhat relevant but incomplete +- 0.7-0.8: Relevant and helpful +- 0.9-1.0: Highly relevant and clinically useful -Query: {query} -Memory: {memory} +Query: "{query}" +Document: "{document}" Score: """ @@ -189,14 +194,10 @@ config = { "reranker": { "provider": "llm_reranker", "config": { - "llm": { - "provider": "openai", - "config": { - "model": "gpt-4", - "api_key": "your-api-key" - } - }, - "custom_prompt": custom_prompt + "provider": "openai", + "model": "gpt-4o-mini", + "api_key": "your-api-key", + "scoring_prompt": custom_prompt } } } @@ -213,15 +214,15 @@ Consider: - Recency and accuracy - Practical usefulness -Rate 1-5: -1 = Not relevant -2 = Slightly relevant -3 = Moderately relevant -4 = Very relevant -5 = Perfectly answers the question +Rate 0.0-1.0: +0.0 = Not relevant +0.25 = Slightly relevant +0.5 = Moderately relevant +0.75 = Very relevant +1.0 = Perfectly answers the question -Query: {query} -Memory: {memory} +Query: "{query}" +Document: "{document}" Score: """ ``` @@ -239,13 +240,17 @@ Consider: - Factual accuracy - Conversation flow -Rate 0-10: -Query: {query} -Memory: {memory} +Rate 0.0-1.0: +Query: "{query}" +Document: "{document}" Score: """ ``` + + Custom prompts must include `{query}` and `{document}` placeholders. The LLM response should contain a numerical score which is automatically extracted. + + ## Usage Examples ### Basic Usage @@ -295,9 +300,9 @@ results = safe_llm_rerank_search("What are my preferences?", "alice") | Model Type | Speed | Quality | Cost | Best For | |------------|-------|---------|------|----------| -| GPT-3.5 Turbo | Fast | Good | Low | High-volume applications | -| GPT-4 | Medium | Excellent | Medium | Quality-critical applications | -| Claude 3 Sonnet | Medium | Excellent | Medium | Balanced performance | +| GPT-4o mini | Fast | Good | Low | High-volume applications | +| GPT-4o | Medium | Excellent | Medium | Quality-critical applications | +| Claude Sonnet | Medium | Excellent | Medium | Balanced performance | | Ollama Local | Variable | Good | Free | Privacy-sensitive applications | ### Optimization Strategies @@ -308,14 +313,10 @@ fast_config = { "reranker": { "provider": "llm_reranker", "config": { - "llm": { - "provider": "openai", - "config": { - "model": "gpt-3.5-turbo", - "api_key": "your-api-key" - } - }, - "top_k": 5, # Limit candidates + "provider": "openai", + "model": "gpt-4o-mini", + "api_key": "your-api-key", + "top_k": 5, "temperature": 0.0 } } @@ -326,13 +327,9 @@ quality_config = { "reranker": { "provider": "llm_reranker", "config": { - "llm": { - "provider": "openai", - "config": { - "model": "gpt-4", - "api_key": "your-api-key" - } - }, + "provider": "openai", + "model": "gpt-4o", + "api_key": "your-api-key", "top_k": 15, "temperature": 0.0 } @@ -353,10 +350,10 @@ Evaluate this memory's relevance using multi-step reasoning: 3. How directly does the memory address the query? 4. What additional context might be needed? -Based on this analysis, rate relevance 1-10: +Based on this analysis, rate relevance 0.0-1.0: -Query: {query} -Memory: {memory} +Query: "{query}" +Document: "{document}" Analysis: Step 1 (Intent): @@ -367,42 +364,6 @@ Final Score: """ ``` -### Comparative Ranking - -```python -comparative_prompt = """ -You will see a query and multiple memories. Rank them in order of relevance. -Consider which memories best answer the question and would be most helpful. - -Query: {query} - -Memories to rank: -{memories} - -Provide scores 1-10 for each memory, considering their relative usefulness. -""" -``` - -### Emotional Intelligence - -```python -emotional_prompt = """ -Consider both factual relevance and emotional appropriateness. -Rate how suitable this memory is for responding to the user's query. - -Factors to consider: -- Factual accuracy and relevance -- Emotional tone and sensitivity -- User's likely emotional state -- Appropriateness of response - -Query: {query} -Memory: {memory} -Emotional Context: {context} -Score (1-10): -""" -``` - ## Error Handling and Fallbacks ```python @@ -433,14 +394,22 @@ class RobustLLMReranker: primary_config = { "reranker": { "provider": "llm_reranker", - "config": {"llm": {"provider": "openai", "config": {"model": "gpt-4"}}} + "config": { + "provider": "openai", + "model": "gpt-4o", + "api_key": "your-api-key" + } } } fallback_config = { "reranker": { "provider": "llm_reranker", - "config": {"llm": {"provider": "openai", "config": {"model": "gpt-3.5-turbo"}}} + "config": { + "provider": "openai", + "model": "gpt-4o-mini", + "api_key": "your-api-key" + } } } @@ -486,4 +455,4 @@ results = reranker.search("What are my preferences?", "alice") Optimize LLM reranker performance - \ No newline at end of file + diff --git a/docs/open-source/features/reranker-search.mdx b/docs/open-source/features/reranker-search.mdx index fbae84182..893d9a7af 100644 --- a/docs/open-source/features/reranker-search.mdx +++ b/docs/open-source/features/reranker-search.mdx @@ -123,13 +123,9 @@ config = { "reranker": { "provider": "llm_reranker", "config": { - "llm": { - "provider": "openai", - "config": { - "model": "gpt-4", - "api_key": "your-openai-api-key" - } - }, + "provider": "openai", + "model": "gpt-4o-mini", + "api_key": "your-openai-api-key", "top_k": 5 } }