Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6883caac58 | ||
|
|
d724ceb521 |
+9
-59
@@ -8,56 +8,20 @@ mode: "wide"
|
||||
<Tabs>
|
||||
<Tab title="Python">
|
||||
|
||||
<Update label="2026-03-26" description="v1.0.8">
|
||||
|
||||
**New Features & Updates:**
|
||||
- **Vector Stores:** Integrated Turbopuffer as a vector database provider ([#4428](https://github.com/mem0ai/mem0/pull/4428))
|
||||
- **LLMs:** Added MiniMax LLM provider ([#4431](https://github.com/mem0ai/mem0/pull/4431))
|
||||
|
||||
**Bug Fixes:**
|
||||
- **Core:** Fixed merging of multiple filter operators for the same key ([#4559](https://github.com/mem0ai/mem0/pull/4559))
|
||||
- **Core:** Prevented in-place mutation of metadata in `_create_memory` ([#4529](https://github.com/mem0ai/mem0/pull/4529))
|
||||
- **Core:** Preserved custom metadata when updating memory ([#4495](https://github.com/mem0ai/mem0/pull/4495))
|
||||
- **Core:** Handled chatty LLM responses in JSON parsing ([#4525](https://github.com/mem0ai/mem0/pull/4525))
|
||||
- **Core:** Prevented double embedding in `mem0.add` ([#3996](https://github.com/mem0ai/mem0/pull/3996))
|
||||
- **Core:** Raised `ValueError` when deleting nonexistent memory ([#4455](https://github.com/mem0ai/mem0/pull/4455))
|
||||
- **Core:** Cleaned up graph store data on `Memory.delete()` ([#4505](https://github.com/mem0ai/mem0/pull/4505))
|
||||
- **Vector Stores:** Prevented SQL injection in Databricks vector store ([#4558](https://github.com/mem0ai/mem0/pull/4558))
|
||||
- **Vector Stores:** Upgraded MongoDB vector store from deprecated `knnVector` to GA `vectorSearch` ([#3995](https://github.com/mem0ai/mem0/pull/3995))
|
||||
- **Vector Stores:** Prevented embedding corruption in Valkey and Redis when vector is `None` ([#4362](https://github.com/mem0ai/mem0/pull/4362))
|
||||
- **Vector Stores:** Accepted default `/tmp/chroma` path in `ChromaDbConfig` validator ([#4179](https://github.com/mem0ai/mem0/pull/4179))
|
||||
- **Vector Stores:** Wrapped vector and payload in lists for `Langchain.update` ([#4446](https://github.com/mem0ai/mem0/pull/4446))
|
||||
- **Graph:** Soft-delete graph relationships instead of hard `DELETE` ([#4188](https://github.com/mem0ai/mem0/pull/4188))
|
||||
- **Graph:** Sanitized hyphens in Neo4j Cypher relationship names ([#4154](https://github.com/mem0ai/mem0/pull/4154))
|
||||
- **Graph:** Used root LLM config as fallback for graph store instead of hardcoded OpenAI default ([#4466](https://github.com/mem0ai/mem0/pull/4466))
|
||||
- **Qdrant:** Fixed `do not remove local path on init` ([#4475](https://github.com/mem0ai/mem0/pull/4475))
|
||||
- **Qdrant:** Implemented enhanced metadata filtering operators ([#4127](https://github.com/mem0ai/mem0/pull/4127))
|
||||
- **Embeddings:** Fixed OpenAI embedding dimensions ([#4481](https://github.com/mem0ai/mem0/pull/4481))
|
||||
- **LLMs:** Omitted `topP` for Anthropic Converse in Bedrock; used `AWSBedrockConfig` in `LlmFactory` ([#4469](https://github.com/mem0ai/mem0/pull/4469))
|
||||
- **LLMs:** Avoided sending both `temperature` and `top_p` to Anthropic API ([#4471](https://github.com/mem0ai/mem0/pull/4471))
|
||||
- **LLMs:** Handled `None` content and empty candidates in `GeminiLLM` parsing ([#4462](https://github.com/mem0ai/mem0/pull/4462))
|
||||
- **LLMs:** Added missing `_parse_response` to `AzureOpenAIStructuredLLM` ([#4434](https://github.com/mem0ai/mem0/pull/4434))
|
||||
- **History:** Added timestamps for `DELETE` operations in history ([#4492](https://github.com/mem0ai/mem0/pull/4492))
|
||||
|
||||
**Improvements:**
|
||||
- **Vector Stores:** Added vector validation to OpenSearchDB to ensure non-null, non-empty, and correct-dimension vectors ([#4472](https://github.com/mem0ai/mem0/pull/4472))
|
||||
|
||||
</Update>
|
||||
|
||||
<Update label="2026-03-19" description="v1.0.7">
|
||||
|
||||
**Bug Fixes:**
|
||||
- **Core:** Fixed control characters in LLM JSON responses causing parse failures ([#4420](https://github.com/mem0ai/mem0/pull/4420))
|
||||
- **Core:** Replaced hardcoded US/Pacific timezone references with `timezone.utc` ([#4404](https://github.com/mem0ai/mem0/pull/4404))
|
||||
- **Core:** Preserved `http_auth` in `_safe_deepcopy_config` for OpenSearch ([#4418](https://github.com/mem0ai/mem0/pull/4418))
|
||||
- **Core:** Normalized malformed LLM fact output before embedding ([#4224](https://github.com/mem0ai/mem0/pull/4224))
|
||||
- **Embeddings:** Pass `encoding_format='float'` in OpenAI embeddings for proxy compatibility ([#4058](https://github.com/mem0ai/mem0/pull/4058))
|
||||
- **LLMs:** Fixed Ollama to pass tools to `client.chat` and parse `tool_calls` from response ([#4176](https://github.com/mem0ai/mem0/pull/4176))
|
||||
- **Reranker:** Support nested LLM config in `LLMReranker` for non-OpenAI providers ([#4405](https://github.com/mem0ai/mem0/pull/4405))
|
||||
- **Vector Stores:** Cast `vector_distance` to float in Redis search ([#4377](https://github.com/mem0ai/mem0/pull/4377))
|
||||
- **Core:** Fixed control characters in LLM JSON responses causing parse failures (#4420)
|
||||
- **Core:** Replaced hardcoded US/Pacific timezone references with `timezone.utc` (#4404)
|
||||
- **Core:** Preserved `http_auth` in `_safe_deepcopy_config` for OpenSearch (#4418)
|
||||
- **Core:** Normalized malformed LLM fact output before embedding (#4224)
|
||||
- **Embeddings:** Pass `encoding_format='float'` in OpenAI embeddings for proxy compatibility (#4058)
|
||||
- **LLMs:** Fixed Ollama to pass tools to `client.chat` and parse `tool_calls` from response (#4176)
|
||||
- **Reranker:** Support nested LLM config in `LLMReranker` for non-OpenAI providers (#4405)
|
||||
- **Vector Stores:** Cast `vector_distance` to float in Redis search (#4377)
|
||||
|
||||
**Improvements:**
|
||||
- **Embeddings:** Improved Ollama embedder with model name normalization and error handling ([#4403](https://github.com/mem0ai/mem0/pull/4403))
|
||||
- **Embeddings:** Improved Ollama embedder with model name normalization and error handling (#4403)
|
||||
|
||||
</Update>
|
||||
|
||||
@@ -799,20 +763,6 @@ mode: "wide"
|
||||
|
||||
<Tab title="TypeScript">
|
||||
|
||||
<Update label="2026-03-26" description="v2.4.3">
|
||||
|
||||
**New Features & Updates:**
|
||||
- **OSS:** Added pgvector support to NodeJS OSS `VectorStoreFactory` ([#3997](https://github.com/mem0ai/mem0/pull/3997))
|
||||
|
||||
**Bug Fixes:**
|
||||
- **OSS:** Made pgvector `pg` import compatible with ESM ([#4544](https://github.com/mem0ai/mem0/pull/4544))
|
||||
- **OSS:** Registered pgvector in `VectorStoreFactory` ([#4502](https://github.com/mem0ai/mem0/pull/4502))
|
||||
- **OSS:** Used root LLM config as fallback for graph store instead of hardcoded OpenAI default ([#4466](https://github.com/mem0ai/mem0/pull/4466))
|
||||
- **OSS:** Fixed `toCamelCase` in Redis `get` method for the payload ([#3172](https://github.com/mem0ai/mem0/pull/3172))
|
||||
- **Client:** Fixed Zod Schema incompatibility with OpenAI Structured Outputs API ([#3462](https://github.com/mem0ai/mem0/pull/3462))
|
||||
|
||||
</Update>
|
||||
|
||||
<Update label="2026-03-19" description="v2.4.2">
|
||||
|
||||
**Bug Fixes:**
|
||||
|
||||
@@ -7,51 +7,58 @@ When using LLM rerankers, you can customize the prompts used for ranking to bett
|
||||
|
||||
## Default Prompt
|
||||
|
||||
The default LLM reranker prompt scores each memory individually on a 0.0-1.0 scale:
|
||||
The default LLM reranker prompt is designed to be general-purpose:
|
||||
|
||||
```
|
||||
You are a relevance scoring assistant. Given a query and a document, you need to score how relevant the document is to the query.
|
||||
Given a query and a list of memory entries, rank the memory entries based on their relevance to the query.
|
||||
Rate each memory on a scale of 1-10 where 10 is most relevant.
|
||||
|
||||
Score the relevance on a scale from 0.0 to 1.0, where:
|
||||
- 1.0 = Perfectly relevant and directly answers the query
|
||||
- 0.8-0.9 = Highly relevant with good information
|
||||
- 0.6-0.7 = Moderately relevant with some useful information
|
||||
- 0.4-0.5 = Slightly relevant with limited useful information
|
||||
- 0.0-0.3 = Not relevant or no useful information
|
||||
Query: {query}
|
||||
|
||||
Query: "{query}"
|
||||
Document: "{document}"
|
||||
Memory entries:
|
||||
{memories}
|
||||
|
||||
Provide only a single numerical score between 0.0 and 1.0. Do not include any explanation or additional text.
|
||||
Provide your ranking as a JSON array with scores for each memory.
|
||||
```
|
||||
|
||||
## Custom Prompt Configuration
|
||||
|
||||
You can provide a custom prompt template using the `scoring_prompt` parameter:
|
||||
You can provide a custom prompt template when configuring the LLM reranker:
|
||||
|
||||
```python
|
||||
from mem0 import Memory
|
||||
|
||||
custom_prompt = """
|
||||
You are an expert at evaluating memories for a personal AI assistant.
|
||||
Given a user query and a memory entry, score how relevant the memory is.
|
||||
Consider direct relevance, temporal relevance, and actionability.
|
||||
You are an expert at ranking memories for a personal AI assistant.
|
||||
Given a user query and a list of memory entries, rank each memory based on:
|
||||
1. Direct relevance to the query
|
||||
2. Temporal relevance (recent memories may be more important)
|
||||
3. Emotional significance
|
||||
4. Actionability
|
||||
|
||||
Query: "{query}"
|
||||
Memory: "{document}"
|
||||
Query: {query}
|
||||
User Context: {user_context}
|
||||
|
||||
Provide only a single numerical score between 0.0 and 1.0.
|
||||
Memory entries:
|
||||
{memories}
|
||||
|
||||
Rate each memory from 1-10 and provide reasoning.
|
||||
Return as JSON: {{"rankings": [{{"index": 0, "score": 8, "reason": "..."}}]}}
|
||||
"""
|
||||
|
||||
config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"api_key": "your-openai-key",
|
||||
"scoring_prompt": custom_prompt,
|
||||
"top_k": 5
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-4.1-nano-2025-04-14",
|
||||
"api_key": "your-openai-key"
|
||||
}
|
||||
},
|
||||
"custom_prompt": custom_prompt,
|
||||
"top_n": 5
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -63,14 +70,12 @@ memory = Memory.from_config(config)
|
||||
|
||||
Your custom prompt can use the following variables:
|
||||
|
||||
| Variable | Description |
|
||||
| ------------ | ----------------------------- |
|
||||
| `{query}` | The search query |
|
||||
| `{document}` | The memory entry being scored |
|
||||
|
||||
<Note>
|
||||
Both `{query}` and `{document}` are required in your custom prompt. The LLM reranker scores each memory individually against the query, so the prompt is called once per candidate memory.
|
||||
</Note>
|
||||
| Variable | Description |
|
||||
| ---------------- | ------------------------------------- |
|
||||
| `{query}` | The search query |
|
||||
| `{memories}` | The list of memory entries to rank |
|
||||
| `{user_id}` | The user ID (if available) |
|
||||
| `{user_context}` | Additional user context (if provided) |
|
||||
|
||||
## Domain-Specific Examples
|
||||
|
||||
@@ -84,10 +89,13 @@ Prioritize memories that:
|
||||
- Show previous resolution patterns
|
||||
- Indicate customer preferences or constraints
|
||||
|
||||
Query: "{query}"
|
||||
Memory: "{document}"
|
||||
Query: {query}
|
||||
Customer Context: Previous interactions with this customer
|
||||
|
||||
Score relevance from 0.0 to 1.0.
|
||||
Memories:
|
||||
{memories}
|
||||
|
||||
Rank each memory 1-10 based on support relevance.
|
||||
"""
|
||||
```
|
||||
|
||||
@@ -95,16 +103,19 @@ Score relevance from 0.0 to 1.0.
|
||||
|
||||
```python
|
||||
educational_prompt = """
|
||||
Score this learning memory for relevance to a student query.
|
||||
Rank these learning memories for a student query.
|
||||
Consider:
|
||||
- Prerequisite knowledge requirements
|
||||
- Learning progression and difficulty
|
||||
- Relevance to current learning objectives
|
||||
|
||||
Student Query: "{query}"
|
||||
Memory: "{document}"
|
||||
Student Query: {query}
|
||||
Learning Context: {user_context}
|
||||
|
||||
Score educational relevance from 0.0 to 1.0.
|
||||
Available memories:
|
||||
{memories}
|
||||
|
||||
Score each memory for educational value (1-10).
|
||||
"""
|
||||
```
|
||||
|
||||
@@ -112,64 +123,73 @@ Score educational relevance from 0.0 to 1.0.
|
||||
|
||||
```python
|
||||
personal_assistant_prompt = """
|
||||
Score this personal memory for relevance to the user's query.
|
||||
Rank personal memories for relevance to the user's query.
|
||||
Consider:
|
||||
- Recent vs. historical importance
|
||||
- Personal preferences and habits
|
||||
- Contextual relationships
|
||||
- Contextual relationships between memories
|
||||
|
||||
Query: "{query}"
|
||||
Memory: "{document}"
|
||||
Query: {query}
|
||||
Personal context: {user_context}
|
||||
|
||||
Provide relevance score from 0.0 to 1.0.
|
||||
Memories to rank:
|
||||
{memories}
|
||||
|
||||
Provide relevance scores (1-10) with brief explanations.
|
||||
"""
|
||||
```
|
||||
|
||||
## Advanced Prompt Techniques
|
||||
|
||||
### Multi-Criteria Scoring
|
||||
### Multi-Criteria Ranking
|
||||
|
||||
```python
|
||||
multi_criteria_prompt = """
|
||||
Evaluate this memory using multiple criteria:
|
||||
Evaluate memories using multiple criteria:
|
||||
|
||||
1. RELEVANCE (40%): How directly related to the query
|
||||
2. RECENCY (20%): How recent the memory appears to be
|
||||
2. RECENCY (20%): How recent the memory is
|
||||
3. IMPORTANCE (25%): Personal or business significance
|
||||
4. ACTIONABILITY (15%): How useful for next steps
|
||||
|
||||
Query: "{query}"
|
||||
Memory: "{document}"
|
||||
Query: {query}
|
||||
Context: {user_context}
|
||||
|
||||
Compute a weighted score from 0.0 to 1.0 based on these criteria.
|
||||
Provide only the final numerical score.
|
||||
Memories:
|
||||
{memories}
|
||||
|
||||
For each memory, provide:
|
||||
- Overall score (1-10)
|
||||
- Breakdown by criteria
|
||||
- Final ranking recommendation
|
||||
|
||||
Format: JSON with detailed scoring
|
||||
"""
|
||||
```
|
||||
|
||||
### Chain-of-Thought Scoring
|
||||
### Contextual Ranking
|
||||
|
||||
```python
|
||||
reasoning_prompt = """
|
||||
Evaluate this memory's relevance step by step:
|
||||
contextual_prompt = """
|
||||
Consider the following context when ranking memories:
|
||||
- Current user situation: {user_context}
|
||||
- Time of day: {current_time}
|
||||
- Recent activities: {recent_activities}
|
||||
|
||||
1. What is the main intent of the query?
|
||||
2. What key information does the memory contain?
|
||||
3. How directly does the memory address the query?
|
||||
Query: {query}
|
||||
|
||||
Based on this analysis, provide a single relevance score from 0.0 to 1.0.
|
||||
Rank these memories considering both direct relevance and contextual appropriateness:
|
||||
{memories}
|
||||
|
||||
Query: "{query}"
|
||||
Memory: "{document}"
|
||||
|
||||
Score:
|
||||
Provide contextually-aware relevance scores (1-10).
|
||||
"""
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Be Specific**: Clearly define what makes a memory relevant for your use case
|
||||
2. **Use 0.0-1.0 Scale**: The score extractor expects values between 0.0 and 1.0
|
||||
3. **Request Only the Score**: Ask for just the numerical score to improve extraction reliability
|
||||
2. **Use Examples**: Include examples in your prompt for better model understanding
|
||||
3. **Structure Output**: Specify the exact JSON format you want returned
|
||||
4. **Test Iteratively**: Refine your prompt based on actual ranking performance
|
||||
5. **Consider Token Limits**: Keep prompts concise while being comprehensive
|
||||
|
||||
@@ -186,7 +206,7 @@ prompts = [
|
||||
]
|
||||
|
||||
for i, prompt in enumerate(prompts):
|
||||
config["reranker"]["config"]["scoring_prompt"] = prompt
|
||||
config["reranker"]["config"]["custom_prompt"] = prompt
|
||||
memory = Memory.from_config(config)
|
||||
|
||||
results = memory.search("test query", user_id="test_user")
|
||||
@@ -196,6 +216,6 @@ for i, prompt in enumerate(prompts):
|
||||
## Common Issues
|
||||
|
||||
- **Too Long**: Keep prompts under token limits for your chosen LLM
|
||||
- **Too Vague**: Be specific about scoring criteria
|
||||
- **Wrong Scale**: Use 0.0-1.0 scale to match the default score extractor
|
||||
- **Extra Output**: Ask for only the numeric score — extra text can confuse score extraction
|
||||
- **Too Vague**: Be specific about ranking criteria
|
||||
- **Inconsistent Format**: Ensure JSON output format is clearly specified
|
||||
- **Missing Context**: Include relevant variables for your use case
|
||||
|
||||
@@ -18,9 +18,13 @@ config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"api_key": "your-openai-api-key"
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-4",
|
||||
"api_key": "your-openai-api-key"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -32,14 +36,11 @@ m = Memory.from_config(config)
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `provider` | str | `"openai"` | LLM provider (openai, anthropic, etc.) |
|
||||
| `model` | str | `"gpt-4o-mini"` | LLM model to use for reranking |
|
||||
| `api_key` | str | None | API key for the LLM provider |
|
||||
| `top_k` | int | None | Number of top documents to return after reranking |
|
||||
| `llm` | dict | Required | LLM configuration object |
|
||||
| `top_k` | int | 10 | Number of results to rerank |
|
||||
| `temperature` | float | 0.0 | LLM temperature for consistency |
|
||||
| `max_tokens` | int | 100 | Maximum tokens for LLM response |
|
||||
| `scoring_prompt` | str | None | Custom prompt template for scoring documents |
|
||||
| `llm` | dict | None | Optional nested LLM config for provider-specific fields (e.g., `ollama_base_url`, `azure_endpoint`). Overrides top-level `provider`/`model`/`api_key` when provided. |
|
||||
| `custom_prompt` | str | None | Custom reranking prompt |
|
||||
| `score_range` | tuple | (0, 10) | Score range for relevance |
|
||||
|
||||
### Advanced Configuration
|
||||
|
||||
@@ -48,19 +49,20 @@ config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "anthropic",
|
||||
"model": "claude-sonnet-4-20250514",
|
||||
"api_key": "your-anthropic-api-key",
|
||||
"llm": {
|
||||
"provider": "anthropic",
|
||||
"config": {
|
||||
"model": "claude-3-sonnet-20240229",
|
||||
"api_key": "your-anthropic-api-key"
|
||||
}
|
||||
},
|
||||
"top_k": 15,
|
||||
"temperature": 0.0,
|
||||
"scoring_prompt": """
|
||||
Rate the relevance of each memory to the query on a scale of 0.0-1.0.
|
||||
"score_range": (1, 5),
|
||||
"custom_prompt": """
|
||||
Rate the relevance of each memory to the query on a scale of 1-5.
|
||||
Consider semantic similarity, context, and practical utility.
|
||||
Only provide the numeric score.
|
||||
|
||||
Query: "{query}"
|
||||
Document: "{document}"
|
||||
Score:
|
||||
"""
|
||||
}
|
||||
}
|
||||
@@ -76,10 +78,14 @@ config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"api_key": "your-openai-api-key",
|
||||
"temperature": 0.0
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-4",
|
||||
"api_key": "your-openai-api-key",
|
||||
"temperature": 0.0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -92,9 +98,13 @@ config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "anthropic",
|
||||
"model": "claude-sonnet-4-20250514",
|
||||
"api_key": "your-anthropic-api-key"
|
||||
"llm": {
|
||||
"provider": "anthropic",
|
||||
"config": {
|
||||
"model": "claude-3-sonnet-20240229",
|
||||
"api_key": "your-anthropic-api-key"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -107,12 +117,10 @@ config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "ollama",
|
||||
"model": "llama3.2",
|
||||
"llm": {
|
||||
"provider": "ollama",
|
||||
"config": {
|
||||
"model": "llama3.2",
|
||||
"model": "llama2",
|
||||
"ollama_base_url": "http://localhost:11434"
|
||||
}
|
||||
}
|
||||
@@ -121,10 +129,6 @@ config = {
|
||||
}
|
||||
```
|
||||
|
||||
<Note>
|
||||
For providers like Ollama that need extra fields (e.g., `ollama_base_url`), use the optional nested `llm` key to pass provider-specific configuration. The nested `llm` config overrides top-level `provider`/`model`/`api_key` when provided.
|
||||
</Note>
|
||||
|
||||
### Azure OpenAI
|
||||
|
||||
```python
|
||||
@@ -132,16 +136,13 @@ config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "azure_openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"api_key": "your-azure-api-key",
|
||||
"llm": {
|
||||
"provider": "azure_openai",
|
||||
"config": {
|
||||
"model": "gpt-4o-mini",
|
||||
"model": "gpt-4",
|
||||
"api_key": "your-azure-api-key",
|
||||
"azure_endpoint": "https://your-resource.openai.azure.com/",
|
||||
"azure_deployment": "gpt-4o-mini-deployment"
|
||||
"azure_deployment": "gpt-4-deployment"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -153,22 +154,16 @@ config = {
|
||||
|
||||
### Default Prompt Behavior
|
||||
|
||||
The default prompt asks the LLM to score relevance on a 0.0-1.0 scale:
|
||||
The default prompt asks the LLM to score relevance on a 0-10 scale:
|
||||
|
||||
```
|
||||
You are a relevance scoring assistant. Given a query and a document, you need to score how relevant the document is to the query.
|
||||
Given a query and a memory, rate how relevant the memory is to answering the query.
|
||||
Score from 0 (completely irrelevant) to 10 (perfectly relevant).
|
||||
Only provide the numeric score.
|
||||
|
||||
Score the relevance on a scale from 0.0 to 1.0, where:
|
||||
- 1.0 = Perfectly relevant and directly answers the query
|
||||
- 0.8-0.9 = Highly relevant with good information
|
||||
- 0.6-0.7 = Moderately relevant with some useful information
|
||||
- 0.4-0.5 = Slightly relevant with limited useful information
|
||||
- 0.0-0.3 = Not relevant or no useful information
|
||||
|
||||
Query: "{query}"
|
||||
Document: "{document}"
|
||||
|
||||
Provide only a single numerical score between 0.0 and 1.0. Do not include any explanation or additional text.
|
||||
Query: {query}
|
||||
Memory: {memory}
|
||||
Score:
|
||||
```
|
||||
|
||||
### Custom Prompt Examples
|
||||
@@ -179,14 +174,14 @@ Provide only a single numerical score between 0.0 and 1.0. Do not include any ex
|
||||
custom_prompt = """
|
||||
You are a medical information specialist. Rate how relevant each memory is for answering the medical query.
|
||||
Consider clinical accuracy, specificity, and practical applicability.
|
||||
Rate from 0.0 to 1.0 where:
|
||||
- 0.0-0.3: Irrelevant or potentially harmful
|
||||
- 0.4-0.6: Somewhat relevant but incomplete
|
||||
- 0.7-0.8: Relevant and helpful
|
||||
- 0.9-1.0: Highly relevant and clinically useful
|
||||
Rate from 1-10 where:
|
||||
- 1-3: Irrelevant or potentially harmful
|
||||
- 4-6: Somewhat relevant but incomplete
|
||||
- 7-8: Relevant and helpful
|
||||
- 9-10: Highly relevant and clinically useful
|
||||
|
||||
Query: "{query}"
|
||||
Document: "{document}"
|
||||
Query: {query}
|
||||
Memory: {memory}
|
||||
Score:
|
||||
"""
|
||||
|
||||
@@ -194,10 +189,14 @@ config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"api_key": "your-api-key",
|
||||
"scoring_prompt": custom_prompt
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-4",
|
||||
"api_key": "your-api-key"
|
||||
}
|
||||
},
|
||||
"custom_prompt": custom_prompt
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -214,15 +213,15 @@ Consider:
|
||||
- Recency and accuracy
|
||||
- Practical usefulness
|
||||
|
||||
Rate 0.0-1.0:
|
||||
0.0 = Not relevant
|
||||
0.25 = Slightly relevant
|
||||
0.5 = Moderately relevant
|
||||
0.75 = Very relevant
|
||||
1.0 = Perfectly answers the question
|
||||
Rate 1-5:
|
||||
1 = Not relevant
|
||||
2 = Slightly relevant
|
||||
3 = Moderately relevant
|
||||
4 = Very relevant
|
||||
5 = Perfectly answers the question
|
||||
|
||||
Query: "{query}"
|
||||
Document: "{document}"
|
||||
Query: {query}
|
||||
Memory: {memory}
|
||||
Score:
|
||||
"""
|
||||
```
|
||||
@@ -240,17 +239,13 @@ Consider:
|
||||
- Factual accuracy
|
||||
- Conversation flow
|
||||
|
||||
Rate 0.0-1.0:
|
||||
Query: "{query}"
|
||||
Document: "{document}"
|
||||
Rate 0-10:
|
||||
Query: {query}
|
||||
Memory: {memory}
|
||||
Score:
|
||||
"""
|
||||
```
|
||||
|
||||
<Note>
|
||||
Custom prompts must include `{query}` and `{document}` placeholders. The LLM response should contain a numerical score which is automatically extracted.
|
||||
</Note>
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic Usage
|
||||
@@ -300,9 +295,9 @@ results = safe_llm_rerank_search("What are my preferences?", "alice")
|
||||
|
||||
| Model Type | Speed | Quality | Cost | Best For |
|
||||
|------------|-------|---------|------|----------|
|
||||
| GPT-4o mini | Fast | Good | Low | High-volume applications |
|
||||
| GPT-4o | Medium | Excellent | Medium | Quality-critical applications |
|
||||
| Claude Sonnet | Medium | Excellent | Medium | Balanced performance |
|
||||
| GPT-3.5 Turbo | Fast | Good | Low | High-volume applications |
|
||||
| GPT-4 | Medium | Excellent | Medium | Quality-critical applications |
|
||||
| Claude 3 Sonnet | Medium | Excellent | Medium | Balanced performance |
|
||||
| Ollama Local | Variable | Good | Free | Privacy-sensitive applications |
|
||||
|
||||
### Optimization Strategies
|
||||
@@ -313,10 +308,14 @@ fast_config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"api_key": "your-api-key",
|
||||
"top_k": 5,
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-3.5-turbo",
|
||||
"api_key": "your-api-key"
|
||||
}
|
||||
},
|
||||
"top_k": 5, # Limit candidates
|
||||
"temperature": 0.0
|
||||
}
|
||||
}
|
||||
@@ -327,9 +326,13 @@ quality_config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"api_key": "your-api-key",
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-4",
|
||||
"api_key": "your-api-key"
|
||||
}
|
||||
},
|
||||
"top_k": 15,
|
||||
"temperature": 0.0
|
||||
}
|
||||
@@ -350,10 +353,10 @@ Evaluate this memory's relevance using multi-step reasoning:
|
||||
3. How directly does the memory address the query?
|
||||
4. What additional context might be needed?
|
||||
|
||||
Based on this analysis, rate relevance 0.0-1.0:
|
||||
Based on this analysis, rate relevance 1-10:
|
||||
|
||||
Query: "{query}"
|
||||
Document: "{document}"
|
||||
Query: {query}
|
||||
Memory: {memory}
|
||||
|
||||
Analysis:
|
||||
Step 1 (Intent):
|
||||
@@ -364,6 +367,42 @@ Final Score:
|
||||
"""
|
||||
```
|
||||
|
||||
### Comparative Ranking
|
||||
|
||||
```python
|
||||
comparative_prompt = """
|
||||
You will see a query and multiple memories. Rank them in order of relevance.
|
||||
Consider which memories best answer the question and would be most helpful.
|
||||
|
||||
Query: {query}
|
||||
|
||||
Memories to rank:
|
||||
{memories}
|
||||
|
||||
Provide scores 1-10 for each memory, considering their relative usefulness.
|
||||
"""
|
||||
```
|
||||
|
||||
### Emotional Intelligence
|
||||
|
||||
```python
|
||||
emotional_prompt = """
|
||||
Consider both factual relevance and emotional appropriateness.
|
||||
Rate how suitable this memory is for responding to the user's query.
|
||||
|
||||
Factors to consider:
|
||||
- Factual accuracy and relevance
|
||||
- Emotional tone and sensitivity
|
||||
- User's likely emotional state
|
||||
- Appropriateness of response
|
||||
|
||||
Query: {query}
|
||||
Memory: {memory}
|
||||
Emotional Context: {context}
|
||||
Score (1-10):
|
||||
"""
|
||||
```
|
||||
|
||||
## Error Handling and Fallbacks
|
||||
|
||||
```python
|
||||
@@ -394,22 +433,14 @@ class RobustLLMReranker:
|
||||
primary_config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"api_key": "your-api-key"
|
||||
}
|
||||
"config": {"llm": {"provider": "openai", "config": {"model": "gpt-4"}}}
|
||||
}
|
||||
}
|
||||
|
||||
fallback_config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"api_key": "your-api-key"
|
||||
}
|
||||
"config": {"llm": {"provider": "openai", "config": {"model": "gpt-3.5-turbo"}}}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -455,4 +486,4 @@ results = reranker.search("What are my preferences?", "alice")
|
||||
<Card title="Performance Optimization" icon="bolt" href="/components/rerankers/optimization">
|
||||
Optimize LLM reranker performance
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</CardGroup>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 282 KiB After Width: | Height: | Size: 178 KiB |
@@ -123,9 +123,13 @@ config = {
|
||||
"reranker": {
|
||||
"provider": "llm_reranker",
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"api_key": "your-openai-api-key",
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": "gpt-4",
|
||||
"api_key": "your-openai-api-key"
|
||||
}
|
||||
},
|
||||
"top_k": 5
|
||||
}
|
||||
}
|
||||
|
||||
@@ -13,10 +13,6 @@ The Mem0 REST API server exposes every OSS memory operation over HTTP. Run it al
|
||||
- You plan to explore or debug endpoints through the built-in OpenAPI page at `/docs`.
|
||||
</Info>
|
||||
|
||||
<Warning>
|
||||
**OSS vs Platform API paths:** The self-hosted OSS server does **not** use the `/v1/` prefix. For example, the endpoint is `POST /memories`, not `POST /v1/memories/`. The [API Reference](/api-reference) documents the hosted platform at `api.mem0.ai` which uses `/v1/` paths — those do not apply to the OSS server.
|
||||
</Warning>
|
||||
|
||||
<Warning>
|
||||
Enable API key authentication (see below) and HTTPS before exposing the server to anything beyond your internal network.
|
||||
</Warning>
|
||||
@@ -151,18 +147,13 @@ curl -X POST http://localhost:8000/memories \
|
||||
</Info>
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/search \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"query": "vegetable",
|
||||
"user_id": "alice"
|
||||
}'
|
||||
curl "http://localhost:8000/memories/search?user_id=alice&query=vegetable"
|
||||
```
|
||||
|
||||
### Explore with OpenAPI docs
|
||||
|
||||
1. Navigate to `http://localhost:8000/docs`.
|
||||
2. Pick an endpoint (e.g., `POST /search`).
|
||||
1. Navigate to `http://localhost:8000/docs`.
|
||||
2. Pick an endpoint (e.g., `POST /memories/search`).
|
||||
3. Fill in parameters and click **Execute** to try requests in-browser.
|
||||
|
||||
<Tip>
|
||||
@@ -171,25 +162,6 @@ curl -X POST http://localhost:8000/search \
|
||||
|
||||
---
|
||||
|
||||
## Endpoint reference
|
||||
|
||||
The OSS REST server exposes the following endpoints. None use the `/v1/` prefix.
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `POST` | `/configure` | Set memory configuration |
|
||||
| `POST` | `/memories` | Create memories |
|
||||
| `GET` | `/memories` | Get all memories (filter by `user_id`, `agent_id`, or `run_id`) |
|
||||
| `GET` | `/memories/{memory_id}` | Get a specific memory |
|
||||
| `PUT` | `/memories/{memory_id}` | Update a memory |
|
||||
| `DELETE` | `/memories/{memory_id}` | Delete a specific memory |
|
||||
| `DELETE` | `/memories` | Delete all memories for an identifier |
|
||||
| `GET` | `/memories/{memory_id}/history` | Get memory history |
|
||||
| `POST` | `/search` | Search memories |
|
||||
| `POST` | `/reset` | Reset all memories |
|
||||
|
||||
---
|
||||
|
||||
## Verify the feature is working
|
||||
|
||||
- Hit the root route and `/docs` to confirm the server is reachable.
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Integration Guide Template
|
||||
description: "A reusable template for writing integration guides that pair Mem0 with third-party tools and services."
|
||||
description: "Pattern for pairing Mem0 with third-party tools."
|
||||
icon: "plug"
|
||||
---
|
||||
|
||||
|
||||
@@ -4,9 +4,8 @@
|
||||
"description": "Mem0 memory layer for AI applications. Add persistent memory, personalization, and semantic search to Claude workflows using the Mem0 Platform MCP server.",
|
||||
"author": {
|
||||
"name": "Mem0",
|
||||
"email": "support@mem0.ai"
|
||||
"url": "https://github.com/mem0ai"
|
||||
},
|
||||
"homepage": "https://mem0.ai",
|
||||
"repository": "https://github.com/mem0ai/mem0",
|
||||
"logo": "logo.svg",
|
||||
"license": "Apache-2.0"
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"mem0": {
|
||||
"url": "https://mcp.mem0.ai/mcp/",
|
||||
"headers": {
|
||||
"Authorization": "Token ${env:MEM0_API_KEY}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,14 +4,9 @@
|
||||
"description": "Mem0 memory layer for AI applications. Add persistent memory, personalization, and semantic search using the Mem0 Platform MCP server.",
|
||||
"author": {
|
||||
"name": "Mem0",
|
||||
"email": "support@mem0.ai"
|
||||
"url": "https://github.com/mem0ai"
|
||||
},
|
||||
"homepage": "https://mem0.ai",
|
||||
"repository": "https://github.com/mem0ai/mem0",
|
||||
"logo": "logo.svg",
|
||||
"license": "Apache-2.0",
|
||||
"keywords": ["mem0", "memory", "mcp", "personalization", "semantic-search"],
|
||||
"skills": "./skills/",
|
||||
"hooks": "./hooks/cursor-hooks.json",
|
||||
"mcpServers": ".cursor-mcp.json"
|
||||
"license": "Apache-2.0"
|
||||
}
|
||||
|
||||
+9
-45
@@ -1,10 +1,10 @@
|
||||
# Mem0 Plugin for Claude Code, Claude Cowork & Cursor
|
||||
# Mem0 Plugin for Claude & Cursor
|
||||
|
||||
Add persistent memory to your AI workflows. Store, retrieve, and manage memories across sessions using the Mem0 Platform. Works with **Claude Code** (CLI), **Claude Cowork** (desktop app), and **Cursor**.
|
||||
Add persistent memory to your AI coding workflows. Store, retrieve, and manage memories across sessions using the Mem0 Platform. Works with both **Claude Code** and **Cursor**.
|
||||
|
||||
## Step 1: Set your API key
|
||||
|
||||
> **You must complete this step before installing the plugin.**
|
||||
> **You must complete this step before installing the plugin for either Claude Code or Cursor.**
|
||||
|
||||
1. Sign up at [app.mem0.ai](https://app.mem0.ai) if you haven't already
|
||||
2. Go to [app.mem0.ai/dashboard/api-keys](https://app.mem0.ai/dashboard/api-keys)
|
||||
@@ -30,51 +30,22 @@ Add persistent memory to your AI workflows. Store, retrieve, and manage memories
|
||||
|
||||
## Step 2: Install the plugin
|
||||
|
||||
Choose one of the options below. All require `MEM0_API_KEY` to be set first (see above).
|
||||
Choose one of the options below. Both require `MEM0_API_KEY` to be set first (see above).
|
||||
|
||||
### Claude Code (CLI) / Claude Cowork (Desktop)
|
||||
|
||||
Claude Code and Claude Cowork share the same plugin system.
|
||||
|
||||
**CLI:**
|
||||
### Claude Code
|
||||
|
||||
```
|
||||
/plugin marketplace add mem0ai/mem0
|
||||
/plugin install mem0@mem0-plugins
|
||||
```
|
||||
|
||||
**Cowork desktop app:** Open the Cowork tab, click **Customize** in the sidebar, click **Browse plugins**, and install Mem0.
|
||||
|
||||
This installs the full plugin including the MCP server, lifecycle hooks (automatic memory capture), and the Mem0 SDK skill.
|
||||
|
||||
### Cursor
|
||||
|
||||
> **Already have `mem0` configured as an MCP server?** Remove the existing entry from your Cursor MCP settings before installing to avoid duplicate tools.
|
||||
Click the deeplink below to install the Mem0 MCP server in Cursor:
|
||||
|
||||
**Option A — One-click deeplink** (installs MCP server only):
|
||||
[Install Mem0 MCP in Cursor](cursor://anysphere.cursor-deeplink/mcp/install?name=mem0&config=eyJtY3BTZXJ2ZXJzIjp7Im1lbTAiOnsidHlwZSI6Imh0dHAiLCJ1cmwiOiJodHRwczovL21jcC5tZW0wLmFpL21jcC8iLCJoZWFkZXJzIjp7IkF1dGhvcml6YXRpb24iOiJUb2tlbiAke01FTTBfQVBJX0tFWX0ifX19fQ==)
|
||||
|
||||
[Install Mem0 MCP in Cursor](cursor://anysphere.cursor-deeplink/mcp/install?name=mem0&config=eyJtY3BTZXJ2ZXJzIjp7Im1lbTAiOnsidXJsIjoiaHR0cHM6Ly9tY3AubWVtMC5haS9tY3AvIiwiaGVhZGVycyI6eyJBdXRob3JpemF0aW9uIjoiVG9rZW4gJHtlbnY6TUVNMF9BUElfS0VZfSJ9fX19)
|
||||
|
||||
**Option B — Manual configuration** (MCP server only):
|
||||
|
||||
Add the following to your `.cursor/mcp.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"mem0": {
|
||||
"url": "https://mcp.mem0.ai/mcp/",
|
||||
"headers": {
|
||||
"Authorization": "Token ${env:MEM0_API_KEY}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Option C — Cursor Marketplace** (full plugin with hooks and skills):
|
||||
|
||||
Install from the [Cursor Marketplace](https://cursor.com/marketplace) for the complete experience including lifecycle hooks and the Mem0 SDK skill.
|
||||
> **Already have `mem0` configured as an MCP server?** Remove the existing entry from your `.mcp.json` or settings before installing this plugin to avoid duplicate tools.
|
||||
|
||||
## Verify it works
|
||||
|
||||
@@ -86,15 +57,8 @@ After installing, confirm the MCP server is connected:
|
||||
|
||||
## What's included
|
||||
|
||||
| Component | Claude Code / Cowork | Cursor (Marketplace) | Cursor (Deeplink/Manual) |
|
||||
|-----------|:--------------------:|:--------------------:|:------------------------:|
|
||||
| MCP Server | Yes | Yes | Yes |
|
||||
| Lifecycle Hooks | Yes | Yes | No |
|
||||
| Mem0 SDK Skill | Yes | Yes | No |
|
||||
|
||||
- **MCP Server** — Connects to the Mem0 remote MCP server (`mcp.mem0.ai`), providing tools to add, search, update, and delete memories. No local dependencies required.
|
||||
- **Lifecycle Hooks** — Automatic memory capture at key points: session start, context compaction, task completion, and session end.
|
||||
- **Mem0 SDK Skill** — Guides the AI on how to integrate the Mem0 SDK (Python & TypeScript) into your applications.
|
||||
- **Mem0 Skill** — Guides Claude on how to integrate the Mem0 SDK (Python & TypeScript) into your applications
|
||||
|
||||
## MCP Tools
|
||||
|
||||
|
||||
@@ -1,37 +0,0 @@
|
||||
{
|
||||
"hooks": {
|
||||
"sessionStart": [
|
||||
{
|
||||
"command": "${CURSOR_PLUGIN_ROOT}/scripts/on_session_start.sh",
|
||||
"matcher": "startup|resume|compact"
|
||||
}
|
||||
],
|
||||
"preToolUse": [
|
||||
{
|
||||
"command": "${CURSOR_PLUGIN_ROOT}/scripts/block_memory_write.sh",
|
||||
"matcher": "Write|Edit"
|
||||
}
|
||||
],
|
||||
"preCompact": [
|
||||
{
|
||||
"command": "${CURSOR_PLUGIN_ROOT}/scripts/on_pre_compact.sh"
|
||||
},
|
||||
{
|
||||
"command": "python3 ${CURSOR_PLUGIN_ROOT}/scripts/on_pre_compact.py",
|
||||
"timeout": 30
|
||||
}
|
||||
],
|
||||
"stop": [
|
||||
{
|
||||
"command": "${CURSOR_PLUGIN_ROOT}/scripts/on_stop.sh",
|
||||
"timeout": 10
|
||||
}
|
||||
],
|
||||
"beforeSubmitPrompt": [
|
||||
{
|
||||
"command": "${CURSOR_PLUGIN_ROOT}/scripts/on_user_prompt.sh",
|
||||
"timeout": 5
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -2,7 +2,6 @@
|
||||
module.exports = {
|
||||
...require("./jest.config"),
|
||||
testMatch: ["**/integration/**/*.test.ts"],
|
||||
globalSetup: "<rootDir>/src/client/tests/integration/global-setup.ts",
|
||||
globalTeardown: "<rootDir>/src/client/tests/integration/global-teardown.ts",
|
||||
// Run integration tests serially to avoid rate limiting and race conditions
|
||||
maxWorkers: 1,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "mem0ai",
|
||||
"version": "2.4.3",
|
||||
"version": "2.4.2",
|
||||
"description": "The Memory Layer For Your AI Apps",
|
||||
"main": "./dist/index.js",
|
||||
"module": "./dist/index.mjs",
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import type { Client as ClientType } from "pg";
|
||||
import pkg from "pg";
|
||||
const { Client } = pkg;
|
||||
import { Client } from "pg";
|
||||
import { VectorStore } from "./base";
|
||||
import { SearchFilters, VectorStoreConfig, VectorStoreResult } from "../types";
|
||||
|
||||
@@ -16,7 +14,7 @@ interface PGVectorConfig extends VectorStoreConfig {
|
||||
}
|
||||
|
||||
export class PGVector implements VectorStore {
|
||||
private client: ClientType;
|
||||
private client: Client;
|
||||
private collectionName: string;
|
||||
private useDiskann: boolean;
|
||||
private useHnsw: boolean;
|
||||
|
||||
@@ -21,7 +21,6 @@ class AzureOpenAIConfig(BaseLlmConfig):
|
||||
top_k: int = 1,
|
||||
enable_vision: bool = False,
|
||||
vision_details: Optional[str] = "auto",
|
||||
reasoning_effort: Optional[str] = None,
|
||||
http_client_proxies: Optional[dict] = None,
|
||||
# Azure OpenAI-specific parameters
|
||||
azure_kwargs: Optional[Dict[str, Any]] = None,
|
||||
@@ -38,7 +37,6 @@ class AzureOpenAIConfig(BaseLlmConfig):
|
||||
top_k: Top-k sampling parameter, defaults to 1
|
||||
enable_vision: Enable vision capabilities, defaults to False
|
||||
vision_details: Vision detail level, defaults to "auto"
|
||||
reasoning_effort: Effort level for reasoning models ("low", "medium", "high"), defaults to None
|
||||
http_client_proxies: HTTP client proxy settings, defaults to None
|
||||
azure_kwargs: Azure-specific configuration, defaults to None
|
||||
"""
|
||||
@@ -52,7 +50,6 @@ class AzureOpenAIConfig(BaseLlmConfig):
|
||||
top_k=top_k,
|
||||
enable_vision=enable_vision,
|
||||
vision_details=vision_details,
|
||||
reasoning_effort=reasoning_effort,
|
||||
http_client_proxies=http_client_proxies,
|
||||
)
|
||||
|
||||
|
||||
@@ -23,7 +23,6 @@ class BaseLlmConfig(ABC):
|
||||
top_k: int = 1,
|
||||
enable_vision: bool = False,
|
||||
vision_details: Optional[str] = "auto",
|
||||
reasoning_effort: Optional[str] = None,
|
||||
http_client_proxies: Optional[Union[Dict, str]] = None,
|
||||
):
|
||||
"""
|
||||
@@ -49,9 +48,6 @@ class BaseLlmConfig(ABC):
|
||||
Only applicable to vision-enabled models. Defaults to False
|
||||
vision_details: Level of detail for vision processing.
|
||||
Options: "low", "high", "auto". Defaults to "auto"
|
||||
reasoning_effort: Effort level for reasoning models (e.g., o1, o3, gpt-5).
|
||||
Options: "low", "medium", "high". Only applicable to reasoning models.
|
||||
Defaults to None (uses the model's default reasoning effort)
|
||||
http_client_proxies: Proxy settings for HTTP client.
|
||||
Can be a dict or string. Defaults to None
|
||||
"""
|
||||
@@ -63,5 +59,4 @@ class BaseLlmConfig(ABC):
|
||||
self.top_k = top_k
|
||||
self.enable_vision = enable_vision
|
||||
self.vision_details = vision_details
|
||||
self.reasoning_effort = reasoning_effort
|
||||
self.http_client = httpx.Client(proxies=http_client_proxies) if http_client_proxies else None
|
||||
|
||||
@@ -20,7 +20,6 @@ class OpenAIConfig(BaseLlmConfig):
|
||||
top_k: int = 1,
|
||||
enable_vision: bool = False,
|
||||
vision_details: Optional[str] = "auto",
|
||||
reasoning_effort: Optional[str] = None,
|
||||
http_client_proxies: Optional[dict] = None,
|
||||
# OpenAI-specific parameters
|
||||
openai_base_url: Optional[str] = None,
|
||||
@@ -45,7 +44,6 @@ class OpenAIConfig(BaseLlmConfig):
|
||||
top_k: Top-k sampling parameter, defaults to 1
|
||||
enable_vision: Enable vision capabilities, defaults to False
|
||||
vision_details: Vision detail level, defaults to "auto"
|
||||
reasoning_effort: Effort level for reasoning models ("low", "medium", "high"), defaults to None
|
||||
http_client_proxies: HTTP client proxy settings, defaults to None
|
||||
openai_base_url: OpenAI API base URL, defaults to None
|
||||
models: List of models for OpenRouter, defaults to None
|
||||
@@ -65,7 +63,6 @@ class OpenAIConfig(BaseLlmConfig):
|
||||
top_k=top_k,
|
||||
enable_vision=enable_vision,
|
||||
vision_details=vision_details,
|
||||
reasoning_effort=reasoning_effort,
|
||||
http_client_proxies=http_client_proxies,
|
||||
)
|
||||
|
||||
|
||||
@@ -31,7 +31,6 @@ class AzureOpenAILLM(LLMBase):
|
||||
top_k=config.top_k,
|
||||
enable_vision=config.enable_vision,
|
||||
vision_details=config.vision_details,
|
||||
reasoning_effort=getattr(config, 'reasoning_effort', None),
|
||||
http_client_proxies=config.http_client,
|
||||
)
|
||||
|
||||
|
||||
+1
-6
@@ -88,12 +88,7 @@ class LLMBase(ABC):
|
||||
supported_params["tools"] = kwargs["tools"]
|
||||
if "tool_choice" in kwargs:
|
||||
supported_params["tool_choice"] = kwargs["tool_choice"]
|
||||
|
||||
# Add reasoning_effort if configured
|
||||
reasoning_effort = getattr(self.config, 'reasoning_effort', None)
|
||||
if reasoning_effort:
|
||||
supported_params["reasoning_effort"] = reasoning_effort
|
||||
|
||||
|
||||
return supported_params
|
||||
else:
|
||||
# For regular models, include all common parameters
|
||||
|
||||
@@ -29,7 +29,6 @@ class OpenAILLM(LLMBase):
|
||||
top_k=config.top_k,
|
||||
enable_vision=config.enable_vision,
|
||||
vision_details=config.vision_details,
|
||||
reasoning_effort=getattr(config, 'reasoning_effort', None),
|
||||
http_client_proxies=config.http_client,
|
||||
)
|
||||
|
||||
|
||||
+23
-56
@@ -516,15 +516,15 @@ class Memory(MemoryBase):
|
||||
)
|
||||
|
||||
try:
|
||||
cleaned_response = remove_code_blocks(response)
|
||||
if not cleaned_response.strip():
|
||||
response = remove_code_blocks(response)
|
||||
if not response.strip():
|
||||
new_retrieved_facts = []
|
||||
else:
|
||||
try:
|
||||
# First try direct JSON parsing
|
||||
new_retrieved_facts = json.loads(cleaned_response, strict=False)["facts"]
|
||||
new_retrieved_facts = json.loads(response, strict=False)["facts"]
|
||||
except json.JSONDecodeError:
|
||||
# Try extracting JSON from response (handles chatty LLM output)
|
||||
# Try extracting JSON from response using built-in function
|
||||
extracted_json = extract_json(response)
|
||||
new_retrieved_facts = json.loads(extracted_json, strict=False)["facts"]
|
||||
new_retrieved_facts = normalize_facts(new_retrieved_facts)
|
||||
@@ -589,11 +589,8 @@ class Memory(MemoryBase):
|
||||
logger.warning("Empty response from LLM, no memories to extract")
|
||||
new_memories_with_actions = {}
|
||||
else:
|
||||
try:
|
||||
new_memories_with_actions = json.loads(remove_code_blocks(response), strict=False)
|
||||
except json.JSONDecodeError:
|
||||
extracted_json = extract_json(response)
|
||||
new_memories_with_actions = json.loads(extracted_json, strict=False)
|
||||
response = remove_code_blocks(response)
|
||||
new_memories_with_actions = json.loads(response, strict=False)
|
||||
except Exception as e:
|
||||
logger.error(f"Invalid JSON response: {e}")
|
||||
new_memories_with_actions = {}
|
||||
@@ -977,11 +974,11 @@ class Memory(MemoryBase):
|
||||
}
|
||||
|
||||
if operator in operator_map:
|
||||
result.setdefault(key, {})[operator_map[operator]] = value
|
||||
result[key] = {operator_map[operator]: value}
|
||||
else:
|
||||
raise ValueError(f"Unsupported metadata filter operator: {operator}")
|
||||
return result
|
||||
|
||||
|
||||
for key, value in metadata_filters.items():
|
||||
if key == "AND":
|
||||
# Logical AND: combine multiple conditions
|
||||
@@ -1081,15 +1078,13 @@ class Memory(MemoryBase):
|
||||
|
||||
return original_memories
|
||||
|
||||
def update(self, memory_id, data, metadata: Optional[Dict[str, Any]] = None):
|
||||
def update(self, memory_id, data):
|
||||
"""
|
||||
Update a memory by ID.
|
||||
|
||||
Args:
|
||||
memory_id (str): ID of the memory to update.
|
||||
data (str): New content to update the memory with.
|
||||
metadata (dict, optional): Additional metadata to update. Existing metadata fields
|
||||
not specified here will be preserved. Defaults to None.
|
||||
|
||||
Returns:
|
||||
dict: Success message indicating the memory was updated.
|
||||
@@ -1097,14 +1092,12 @@ class Memory(MemoryBase):
|
||||
Example:
|
||||
>>> m.update(memory_id="mem_123", data="Likes to play tennis on weekends")
|
||||
{'message': 'Memory updated successfully!'}
|
||||
>>> m.update(memory_id="mem_123", data="Likes tennis", metadata={"category": "sports"})
|
||||
{'message': 'Memory updated successfully!'}
|
||||
"""
|
||||
capture_event("mem0.update", self, {"memory_id": memory_id, "sync_type": "sync"})
|
||||
|
||||
existing_embeddings = {data: self.embedding_model.embed(data, "update")}
|
||||
|
||||
self._update_memory(memory_id, data, existing_embeddings, metadata)
|
||||
self._update_memory(memory_id, data, existing_embeddings)
|
||||
return {"message": "Memory updated successfully!"}
|
||||
|
||||
def delete(self, memory_id):
|
||||
@@ -1200,9 +1193,7 @@ class Memory(MemoryBase):
|
||||
new_metadata = deepcopy(metadata) if metadata is not None else {}
|
||||
new_metadata["data"] = data
|
||||
new_metadata["hash"] = hashlib.md5(data.encode()).hexdigest()
|
||||
if "created_at" not in new_metadata:
|
||||
new_metadata["created_at"] = datetime.now(timezone.utc).isoformat()
|
||||
new_metadata["updated_at"] = new_metadata["created_at"]
|
||||
new_metadata["created_at"] = datetime.now(timezone.utc).isoformat()
|
||||
|
||||
self.vector_store.insert(
|
||||
vectors=[embeddings],
|
||||
@@ -1215,7 +1206,6 @@ class Memory(MemoryBase):
|
||||
data,
|
||||
"ADD",
|
||||
created_at=new_metadata.get("created_at"),
|
||||
updated_at=new_metadata.get("updated_at"),
|
||||
actor_id=new_metadata.get("actor_id"),
|
||||
role=new_metadata.get("role"),
|
||||
)
|
||||
@@ -1289,7 +1279,7 @@ class Memory(MemoryBase):
|
||||
new_metadata["agent_id"] = existing_memory.payload["agent_id"]
|
||||
if "run_id" not in new_metadata and "run_id" in existing_memory.payload:
|
||||
new_metadata["run_id"] = existing_memory.payload["run_id"]
|
||||
if "actor_id" in existing_memory.payload:
|
||||
if "actor_id" not in new_metadata and "actor_id" in existing_memory.payload:
|
||||
new_metadata["actor_id"] = existing_memory.payload["actor_id"]
|
||||
if "role" not in new_metadata and "role" in existing_memory.payload:
|
||||
new_metadata["role"] = existing_memory.payload["role"]
|
||||
@@ -1327,19 +1317,12 @@ class Memory(MemoryBase):
|
||||
if existing_memory is None:
|
||||
raise ValueError(f"Memory with id {memory_id} not found")
|
||||
prev_value = existing_memory.payload.get("data", "")
|
||||
|
||||
# Preserve original created_at and record deletion time
|
||||
created_at = _normalize_iso_timestamp_to_utc(existing_memory.payload.get("created_at"))
|
||||
updated_at = datetime.now(timezone.utc).isoformat()
|
||||
|
||||
self.vector_store.delete(vector_id=memory_id)
|
||||
self.db.add_history(
|
||||
memory_id,
|
||||
prev_value,
|
||||
None,
|
||||
"DELETE",
|
||||
created_at=created_at,
|
||||
updated_at=updated_at,
|
||||
actor_id=existing_memory.payload.get("actor_id"),
|
||||
role=existing_memory.payload.get("role"),
|
||||
is_deleted=1,
|
||||
@@ -1607,15 +1590,15 @@ class AsyncMemory(MemoryBase):
|
||||
response_format={"type": "json_object"},
|
||||
)
|
||||
try:
|
||||
cleaned_response = remove_code_blocks(response)
|
||||
if not cleaned_response.strip():
|
||||
response = remove_code_blocks(response)
|
||||
if not response.strip():
|
||||
new_retrieved_facts = []
|
||||
else:
|
||||
try:
|
||||
# First try direct JSON parsing
|
||||
new_retrieved_facts = json.loads(cleaned_response, strict=False)["facts"]
|
||||
new_retrieved_facts = json.loads(response, strict=False)["facts"]
|
||||
except json.JSONDecodeError:
|
||||
# Try extracting JSON from response (handles chatty LLM output)
|
||||
# Try extracting JSON from response using built-in function
|
||||
extracted_json = extract_json(response)
|
||||
new_retrieved_facts = json.loads(extracted_json, strict=False)["facts"]
|
||||
new_retrieved_facts = normalize_facts(new_retrieved_facts)
|
||||
@@ -1683,11 +1666,8 @@ class AsyncMemory(MemoryBase):
|
||||
logger.warning("Empty response from LLM, no memories to extract")
|
||||
new_memories_with_actions = {}
|
||||
else:
|
||||
try:
|
||||
new_memories_with_actions = json.loads(remove_code_blocks(response), strict=False)
|
||||
except json.JSONDecodeError:
|
||||
extracted_json = extract_json(response)
|
||||
new_memories_with_actions = json.loads(extracted_json, strict=False)
|
||||
response = remove_code_blocks(response)
|
||||
new_memories_with_actions = json.loads(response, strict=False)
|
||||
except Exception as e:
|
||||
logger.error(f"Invalid JSON response: {e}")
|
||||
new_memories_with_actions = {}
|
||||
@@ -2102,7 +2082,7 @@ class AsyncMemory(MemoryBase):
|
||||
}
|
||||
|
||||
if operator in operator_map:
|
||||
result.setdefault(key, {})[operator_map[operator]] = value
|
||||
result[key] = {operator_map[operator]: value}
|
||||
else:
|
||||
raise ValueError(f"Unsupported metadata filter operator: {operator}")
|
||||
return result
|
||||
@@ -2208,15 +2188,13 @@ class AsyncMemory(MemoryBase):
|
||||
|
||||
return original_memories
|
||||
|
||||
async def update(self, memory_id, data, metadata: Optional[Dict[str, Any]] = None):
|
||||
async def update(self, memory_id, data):
|
||||
"""
|
||||
Update a memory by ID asynchronously.
|
||||
|
||||
Args:
|
||||
memory_id (str): ID of the memory to update.
|
||||
data (str): New content to update the memory with.
|
||||
metadata (dict, optional): Additional metadata to update. Existing metadata fields
|
||||
not specified here will be preserved. Defaults to None.
|
||||
|
||||
Returns:
|
||||
dict: Success message indicating the memory was updated.
|
||||
@@ -2224,15 +2202,13 @@ class AsyncMemory(MemoryBase):
|
||||
Example:
|
||||
>>> await m.update(memory_id="mem_123", data="Likes to play tennis on weekends")
|
||||
{'message': 'Memory updated successfully!'}
|
||||
>>> await m.update(memory_id="mem_123", data="Likes tennis", metadata={"category": "sports"})
|
||||
{'message': 'Memory updated successfully!'}
|
||||
"""
|
||||
capture_event("mem0.update", self, {"memory_id": memory_id, "sync_type": "async"})
|
||||
|
||||
embeddings = await asyncio.to_thread(self.embedding_model.embed, data, "update")
|
||||
existing_embeddings = {data: embeddings}
|
||||
|
||||
await self._update_memory(memory_id, data, existing_embeddings, metadata)
|
||||
await self._update_memory(memory_id, data, existing_embeddings)
|
||||
return {"message": "Memory updated successfully!"}
|
||||
|
||||
async def delete(self, memory_id):
|
||||
@@ -2332,9 +2308,7 @@ class AsyncMemory(MemoryBase):
|
||||
new_metadata = deepcopy(metadata) if metadata is not None else {}
|
||||
new_metadata["data"] = data
|
||||
new_metadata["hash"] = hashlib.md5(data.encode()).hexdigest()
|
||||
if "created_at" not in new_metadata:
|
||||
new_metadata["created_at"] = datetime.now(timezone.utc).isoformat()
|
||||
new_metadata["updated_at"] = new_metadata["created_at"]
|
||||
new_metadata["created_at"] = datetime.now(timezone.utc).isoformat()
|
||||
|
||||
await asyncio.to_thread(
|
||||
self.vector_store.insert,
|
||||
@@ -2350,7 +2324,6 @@ class AsyncMemory(MemoryBase):
|
||||
data,
|
||||
"ADD",
|
||||
created_at=new_metadata.get("created_at"),
|
||||
updated_at=new_metadata.get("updated_at"),
|
||||
actor_id=new_metadata.get("actor_id"),
|
||||
role=new_metadata.get("role"),
|
||||
)
|
||||
@@ -2440,7 +2413,7 @@ class AsyncMemory(MemoryBase):
|
||||
if "run_id" not in new_metadata and "run_id" in existing_memory.payload:
|
||||
new_metadata["run_id"] = existing_memory.payload["run_id"]
|
||||
|
||||
if "actor_id" in existing_memory.payload:
|
||||
if "actor_id" not in new_metadata and "actor_id" in existing_memory.payload:
|
||||
new_metadata["actor_id"] = existing_memory.payload["actor_id"]
|
||||
if "role" not in new_metadata and "role" in existing_memory.payload:
|
||||
new_metadata["role"] = existing_memory.payload["role"]
|
||||
@@ -2481,10 +2454,6 @@ class AsyncMemory(MemoryBase):
|
||||
raise ValueError(f"Memory with id {memory_id} not found")
|
||||
prev_value = existing_memory.payload.get("data", "")
|
||||
|
||||
# Preserve original created_at and record deletion time
|
||||
created_at = _normalize_iso_timestamp_to_utc(existing_memory.payload.get("created_at"))
|
||||
updated_at = datetime.now(timezone.utc).isoformat()
|
||||
|
||||
await asyncio.to_thread(self.vector_store.delete, vector_id=memory_id)
|
||||
await asyncio.to_thread(
|
||||
self.db.add_history,
|
||||
@@ -2492,8 +2461,6 @@ class AsyncMemory(MemoryBase):
|
||||
prev_value,
|
||||
None,
|
||||
"DELETE",
|
||||
created_at=created_at,
|
||||
updated_at=updated_at,
|
||||
actor_id=existing_memory.payload.get("actor_id"),
|
||||
role=existing_memory.payload.get("role"),
|
||||
is_deleted=1,
|
||||
|
||||
@@ -124,20 +124,14 @@ def remove_code_blocks(content: str) -> str:
|
||||
def extract_json(text):
|
||||
"""
|
||||
Extracts JSON content from a string, removing enclosing triple backticks and optional 'json' tag if present.
|
||||
If no code block is found, attempts to locate JSON by finding the first '{' and last '}'.
|
||||
If that also fails, returns the text as-is.
|
||||
If no code block is found, returns the text as-is.
|
||||
"""
|
||||
text = text.strip()
|
||||
match = re.search(r"```(?:json)?\s*(.*?)\s*```", text, re.DOTALL)
|
||||
if match:
|
||||
json_str = match.group(1)
|
||||
else:
|
||||
start_idx = text.find("{")
|
||||
end_idx = text.rfind("}")
|
||||
if start_idx != -1 and end_idx != -1 and end_idx > start_idx:
|
||||
json_str = text[start_idx : end_idx + 1]
|
||||
else:
|
||||
json_str = text
|
||||
json_str = text # assume it's raw JSON
|
||||
return json_str
|
||||
|
||||
|
||||
|
||||
@@ -1,13 +1,11 @@
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import uuid
|
||||
from typing import Optional, List
|
||||
from datetime import datetime, date
|
||||
from databricks.sdk.service.catalog import ColumnInfo, ColumnTypeName, TableType, DataSourceFormat
|
||||
from databricks.sdk.service.catalog import TableConstraint, PrimaryKeyConstraint
|
||||
from databricks.sdk import WorkspaceClient
|
||||
from databricks.sdk.service.sql import StatementParameterListItem
|
||||
from databricks.sdk.service.vectorsearch import (
|
||||
VectorIndexType,
|
||||
DeltaSyncVectorIndexSpecRequest,
|
||||
@@ -30,9 +28,6 @@ class MemoryResult(BaseModel):
|
||||
|
||||
excluded_keys = {"user_id", "agent_id", "run_id", "hash", "data", "created_at", "updated_at"}
|
||||
|
||||
# Pattern for valid SQL identifiers to prevent column name injection
|
||||
_VALID_SQL_IDENTIFIER = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
|
||||
|
||||
|
||||
class Databricks(VectorStoreBase):
|
||||
def __init__(
|
||||
@@ -393,46 +388,28 @@ class Databricks(VectorStoreBase):
|
||||
# Determine the number of items to process
|
||||
num_items = len(payloads) if payloads else len(vectors) if vectors else 0
|
||||
|
||||
params = []
|
||||
value_tuples = []
|
||||
for i in range(num_items):
|
||||
placeholders = []
|
||||
values = []
|
||||
for col in self.columns:
|
||||
param_name = f"{col.name}_{i}"
|
||||
if col.name == "memory_id":
|
||||
val = ids[i] if ids and i < len(ids) else str(uuid.uuid4())
|
||||
elif col.name == "embedding":
|
||||
val = vectors[i] if vectors and i < len(vectors) else []
|
||||
# Vectors are numeric arrays — ARRAY type not supported by StatementParameterListItem,
|
||||
# so we inline using _format_sql_value (values are floats from the embedding model).
|
||||
placeholders.append(self._format_sql_value(val))
|
||||
continue
|
||||
elif col.name == "memory":
|
||||
val = payloads[i].get("data") if payloads and i < len(payloads) else None
|
||||
else:
|
||||
val = payloads[i].get(col.name) if payloads and i < len(payloads) else None
|
||||
|
||||
if val is None:
|
||||
placeholders.append("NULL")
|
||||
else:
|
||||
placeholders.append(f":{param_name}")
|
||||
if isinstance(val, dict):
|
||||
val = json.dumps(val)
|
||||
# Use explicit type for TIMESTAMP columns so Databricks doesn't
|
||||
# rely on implicit STRING→TIMESTAMP casting.
|
||||
param_type = "TIMESTAMP" if col.type_name == ColumnTypeName.TIMESTAMP else None
|
||||
params.append(StatementParameterListItem(name=param_name, value=str(val), type=param_type))
|
||||
value_tuples.append(f"({', '.join(placeholders)})")
|
||||
values.append(val)
|
||||
formatted = [self._format_sql_value(v) for v in values]
|
||||
value_tuples.append(f"({', '.join(formatted)})")
|
||||
|
||||
insert_sql = f"INSERT INTO {self.fully_qualified_table_name} ({', '.join(self.column_names)}) VALUES {', '.join(value_tuples)}"
|
||||
|
||||
# Execute the insert
|
||||
try:
|
||||
response = self.client.statement_execution.execute_statement(
|
||||
statement=insert_sql,
|
||||
warehouse_id=self.warehouse_id,
|
||||
wait_timeout="30s",
|
||||
parameters=params,
|
||||
statement=insert_sql, warehouse_id=self.warehouse_id, wait_timeout="30s"
|
||||
)
|
||||
if response.status.state.value == "SUCCEEDED":
|
||||
logger.info(
|
||||
@@ -517,13 +494,10 @@ class Databricks(VectorStoreBase):
|
||||
try:
|
||||
logger.info(f"Deleting vector with ID {vector_id} from Delta table {self.fully_qualified_table_name}")
|
||||
|
||||
delete_sql = f"DELETE FROM {self.fully_qualified_table_name} WHERE memory_id = :vector_id"
|
||||
delete_sql = f"DELETE FROM {self.fully_qualified_table_name} WHERE memory_id = '{vector_id}'"
|
||||
|
||||
response = self.client.statement_execution.execute_statement(
|
||||
statement=delete_sql,
|
||||
warehouse_id=self.warehouse_id,
|
||||
wait_timeout="30s",
|
||||
parameters=[StatementParameterListItem(name="vector_id", value=str(vector_id))],
|
||||
statement=delete_sql, warehouse_id=self.warehouse_id, wait_timeout="30s"
|
||||
)
|
||||
|
||||
if response.status.state.value == "SUCCEEDED":
|
||||
@@ -545,8 +519,8 @@ class Databricks(VectorStoreBase):
|
||||
payload (dict, optional): New payload data.
|
||||
"""
|
||||
|
||||
update_sql = f"UPDATE {self.fully_qualified_table_name} SET "
|
||||
set_clauses = []
|
||||
params = []
|
||||
if not vector_id:
|
||||
logger.error("vector_id is required for update operation")
|
||||
return
|
||||
@@ -554,38 +528,25 @@ class Databricks(VectorStoreBase):
|
||||
if not isinstance(vector, list):
|
||||
logger.error("vector must be a list of float values")
|
||||
return
|
||||
# Vectors are numeric arrays — safe to inline since StatementParameterListItem
|
||||
# doesn't support ARRAY types, and values are validated as list of floats above.
|
||||
# Use array() SQL syntax, not Python list repr which is invalid Databricks SQL.
|
||||
set_clauses.append(f"embedding = {self._format_sql_value(vector)}")
|
||||
set_clauses.append(f"embedding = {vector}")
|
||||
if payload:
|
||||
if not isinstance(payload, dict):
|
||||
logger.error("payload must be a dictionary")
|
||||
return
|
||||
for key, value in payload.items():
|
||||
if key not in excluded_keys:
|
||||
if not _VALID_SQL_IDENTIFIER.match(key):
|
||||
logger.warning(f"Skipping invalid column name in payload: {key!r}")
|
||||
continue
|
||||
param_name = f"payload_{key}"
|
||||
set_clauses.append(f"{key} = :{param_name}")
|
||||
params.append(StatementParameterListItem(name=param_name, value=str(value)))
|
||||
set_clauses.append(f"{key} = '{value}'")
|
||||
|
||||
if not set_clauses:
|
||||
logger.error("No fields to update")
|
||||
return
|
||||
update_sql = f"UPDATE {self.fully_qualified_table_name} SET "
|
||||
update_sql += ", ".join(set_clauses)
|
||||
update_sql += " WHERE memory_id = :vector_id"
|
||||
params.append(StatementParameterListItem(name="vector_id", value=str(vector_id)))
|
||||
update_sql += f" WHERE memory_id = '{vector_id}'"
|
||||
try:
|
||||
logger.info(f"Updating vector with ID {vector_id} in Delta table {self.fully_qualified_table_name}")
|
||||
|
||||
response = self.client.statement_execution.execute_statement(
|
||||
statement=update_sql,
|
||||
warehouse_id=self.warehouse_id,
|
||||
wait_timeout="30s",
|
||||
parameters=params,
|
||||
statement=update_sql, warehouse_id=self.warehouse_id, wait_timeout="30s"
|
||||
)
|
||||
|
||||
if response.status.state.value == "SUCCEEDED":
|
||||
|
||||
@@ -2,15 +2,6 @@
|
||||
|
||||
All notable changes to the `@mem0/openclaw-mem0` plugin will be documented in this file.
|
||||
|
||||
## [0.4.1] - 2026-03-26
|
||||
|
||||
### Added
|
||||
- **Improved extraction quality**: Enhanced noise filtering, deduplication, and better extraction instructions for higher-quality memory capture (#4302)
|
||||
|
||||
### Fixed
|
||||
- **Credential detection in extraction**: Improved detection of credentials, API keys, and secrets in extraction instructions to prevent them from being stored as memories (#4552)
|
||||
- **Standalone timestamp extraction**: Prevented extraction of standalone timestamps as memories when no meaningful content accompanies them (#4550)
|
||||
|
||||
## [0.4.0] - 2026-03-16
|
||||
|
||||
### Added
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@mem0/openclaw-mem0",
|
||||
"version": "0.4.1",
|
||||
"version": "0.4.0",
|
||||
"type": "module",
|
||||
"description": "Mem0 memory backend for OpenClaw — platform or self-hosted open-source",
|
||||
"license": "Apache-2.0",
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "mem0ai"
|
||||
version = "1.0.8"
|
||||
version = "1.0.7"
|
||||
description = "Long-term memory for AI Agents"
|
||||
authors = [
|
||||
{ name = "Mem0", email = "support@mem0.ai" }
|
||||
|
||||
@@ -91,57 +91,6 @@ def test_generate_response_with_tools(mock_openai_client):
|
||||
assert response["tool_calls"][0]["arguments"] == {"data": "Today is a sunny day."}
|
||||
|
||||
|
||||
def test_reasoning_model_with_reasoning_effort(mock_openai_client):
|
||||
"""Test that reasoning_effort is passed to the API for Azure reasoning models."""
|
||||
config = AzureOpenAIConfig(model="o3-mini", reasoning_effort="low")
|
||||
llm = AzureOpenAILLM(config)
|
||||
messages = [
|
||||
{"role": "system", "content": "You are a helpful ai."},
|
||||
{"role": "user", "content": "Hello"},
|
||||
]
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.choices = [Mock(message=Mock(content="Response from o3-mini"))]
|
||||
mock_openai_client.chat.completions.create.return_value = mock_response
|
||||
|
||||
response = llm.generate_response(messages)
|
||||
|
||||
call_kwargs = mock_openai_client.chat.completions.create.call_args
|
||||
assert call_kwargs[1]["reasoning_effort"] == "low"
|
||||
assert "temperature" not in call_kwargs[1]
|
||||
assert response == "Response from o3-mini"
|
||||
|
||||
|
||||
def test_azure_reasoning_effort_not_passed_when_none(mock_openai_client):
|
||||
"""Test that reasoning_effort is not passed when not configured on Azure."""
|
||||
config = AzureOpenAIConfig(model="o3-mini")
|
||||
llm = AzureOpenAILLM(config)
|
||||
messages = [
|
||||
{"role": "system", "content": "You are a helpful ai."},
|
||||
{"role": "user", "content": "Hello"},
|
||||
]
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.choices = [Mock(message=Mock(content="Response"))]
|
||||
mock_openai_client.chat.completions.create.return_value = mock_response
|
||||
|
||||
llm.generate_response(messages)
|
||||
|
||||
call_kwargs = mock_openai_client.chat.completions.create.call_args
|
||||
assert "reasoning_effort" not in call_kwargs[1]
|
||||
|
||||
|
||||
def test_azure_config_accepts_reasoning_effort():
|
||||
"""Test that AzureOpenAIConfig accepts reasoning_effort without TypeError (issue #3651)."""
|
||||
config = AzureOpenAIConfig(
|
||||
model="o3-mini",
|
||||
reasoning_effort="low",
|
||||
azure_kwargs={"api_key": "test"},
|
||||
)
|
||||
assert config.reasoning_effort == "low"
|
||||
assert config.model == "o3-mini"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"default_headers",
|
||||
[None, {"Firstkey": "FirstVal", "SecondKey": "SecondVal"}],
|
||||
|
||||
@@ -170,67 +170,6 @@ def test_callback_exception_handling(mock_openai_client):
|
||||
assert llm.config.response_callback is faulty_callback
|
||||
|
||||
|
||||
def test_reasoning_model_with_reasoning_effort(mock_openai_client):
|
||||
"""Test that reasoning_effort is passed to the API for reasoning models."""
|
||||
config = OpenAIConfig(model="o3-mini", reasoning_effort="low")
|
||||
llm = OpenAILLM(config)
|
||||
messages = [{"role": "user", "content": "Hello"}]
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.choices = [Mock(message=Mock(content="Response from o3-mini"))]
|
||||
mock_openai_client.chat.completions.create.return_value = mock_response
|
||||
|
||||
response = llm.generate_response(messages)
|
||||
|
||||
call_kwargs = mock_openai_client.chat.completions.create.call_args
|
||||
assert call_kwargs[1]["reasoning_effort"] == "low"
|
||||
assert "temperature" not in call_kwargs[1] # reasoning models don't get temperature
|
||||
assert response == "Response from o3-mini"
|
||||
|
||||
|
||||
def test_reasoning_model_without_reasoning_effort(mock_openai_client):
|
||||
"""Test that reasoning_effort is not passed when not configured."""
|
||||
config = OpenAIConfig(model="o3-mini")
|
||||
llm = OpenAILLM(config)
|
||||
messages = [{"role": "user", "content": "Hello"}]
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.choices = [Mock(message=Mock(content="Response"))]
|
||||
mock_openai_client.chat.completions.create.return_value = mock_response
|
||||
|
||||
llm.generate_response(messages)
|
||||
|
||||
call_kwargs = mock_openai_client.chat.completions.create.call_args
|
||||
assert "reasoning_effort" not in call_kwargs[1]
|
||||
|
||||
|
||||
def test_non_reasoning_model_ignores_reasoning_effort(mock_openai_client):
|
||||
"""Test that reasoning_effort is not passed for non-reasoning models."""
|
||||
config = OpenAIConfig(model="gpt-4.1-nano-2025-04-14", reasoning_effort="high")
|
||||
llm = OpenAILLM(config)
|
||||
messages = [{"role": "user", "content": "Hello"}]
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.choices = [Mock(message=Mock(content="Response"))]
|
||||
mock_openai_client.chat.completions.create.return_value = mock_response
|
||||
|
||||
llm.generate_response(messages)
|
||||
|
||||
call_kwargs = mock_openai_client.chat.completions.create.call_args
|
||||
# Non-reasoning models use common params path, reasoning_effort not added there
|
||||
assert "reasoning_effort" not in call_kwargs[1]
|
||||
|
||||
|
||||
def test_reasoning_effort_config_values():
|
||||
"""Test that reasoning_effort can be set to all valid values."""
|
||||
for effort in ["low", "medium", "high"]:
|
||||
config = OpenAIConfig(model="o3", reasoning_effort=effort)
|
||||
assert config.reasoning_effort == effort
|
||||
|
||||
config = OpenAIConfig(model="o3")
|
||||
assert config.reasoning_effort is None
|
||||
|
||||
|
||||
def test_callback_with_tools(mock_openai_client):
|
||||
mock_callback = Mock()
|
||||
config = OpenAIConfig(model="gpt-4.1-nano-2025-04-14", response_callback=mock_callback)
|
||||
|
||||
+1
-245
@@ -1,6 +1,6 @@
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from unittest.mock import MagicMock, Mock
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -79,57 +79,6 @@ class TestAddToVectorStoreErrors:
|
||||
assert "Empty response from LLM, no memories to extract" in caplog.text
|
||||
|
||||
|
||||
class TestAsyncUpdate:
|
||||
@pytest.fixture
|
||||
def mock_async_memory(self, mocker):
|
||||
"""Fixture for AsyncMemory with mocker-based mocks"""
|
||||
_setup_mocks(mocker)
|
||||
memory = AsyncMemory()
|
||||
return memory
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_update_without_metadata(self, mock_async_memory, mocker):
|
||||
"""Test async update passes None metadata by default"""
|
||||
mock_async_memory.embedding_model = Mock()
|
||||
mock_async_memory.embedding_model.embed = Mock(return_value=[0.1, 0.2, 0.3])
|
||||
mock_async_memory._update_memory = mocker.AsyncMock()
|
||||
|
||||
result = await mock_async_memory.update("test_id", "Updated memory")
|
||||
|
||||
mock_async_memory._update_memory.assert_called_once_with(
|
||||
"test_id", "Updated memory", {"Updated memory": [0.1, 0.2, 0.3]}, None
|
||||
)
|
||||
assert result["message"] == "Memory updated successfully!"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_update_with_metadata(self, mock_async_memory, mocker):
|
||||
"""Test async update correctly forwards metadata"""
|
||||
mock_async_memory.embedding_model = Mock()
|
||||
mock_async_memory.embedding_model.embed = Mock(return_value=[0.1, 0.2, 0.3])
|
||||
mock_async_memory._update_memory = mocker.AsyncMock()
|
||||
metadata = {"category": "sports", "priority": "high"}
|
||||
|
||||
result = await mock_async_memory.update("test_id", "Updated memory", metadata=metadata)
|
||||
|
||||
mock_async_memory._update_memory.assert_called_once_with(
|
||||
"test_id", "Updated memory", {"Updated memory": [0.1, 0.2, 0.3]}, metadata
|
||||
)
|
||||
assert result["message"] == "Memory updated successfully!"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_update_with_empty_metadata(self, mock_async_memory, mocker):
|
||||
"""Test async update with empty metadata dict"""
|
||||
mock_async_memory.embedding_model = Mock()
|
||||
mock_async_memory.embedding_model.embed = Mock(return_value=[0.1, 0.2, 0.3])
|
||||
mock_async_memory._update_memory = mocker.AsyncMock()
|
||||
|
||||
await mock_async_memory.update("test_id", "Updated memory", metadata={})
|
||||
|
||||
mock_async_memory._update_memory.assert_called_once_with(
|
||||
"test_id", "Updated memory", {"Updated memory": [0.1, 0.2, 0.3]}, {}
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
class TestAsyncAddToVectorStoreErrors:
|
||||
@pytest.fixture
|
||||
@@ -208,28 +157,6 @@ def test_create_memory_uses_utc_timestamps(mocker):
|
||||
_assert_utc_timestamp(payload["created_at"])
|
||||
|
||||
|
||||
def test_create_memory_sets_updated_at(mocker):
|
||||
memory = _build_memory_instance(mocker, Memory)
|
||||
memory._create_memory("new memory", {"new memory": [0.1, 0.2, 0.3]}, metadata={})
|
||||
payload = memory.vector_store.insert.call_args.kwargs["payloads"][0]
|
||||
assert "updated_at" in payload
|
||||
assert payload["updated_at"] == payload["created_at"]
|
||||
_assert_utc_timestamp(payload["updated_at"])
|
||||
|
||||
# History should also receive updated_at
|
||||
history_kwargs = memory.db.add_history.call_args
|
||||
assert history_kwargs.kwargs["updated_at"] == payload["updated_at"]
|
||||
|
||||
|
||||
def test_create_memory_preserves_existing_created_at(mocker):
|
||||
memory = _build_memory_instance(mocker, Memory)
|
||||
custom_ts = "2023-05-06T09:19:20+00:00"
|
||||
memory._create_memory("new memory", {"new memory": [0.1, 0.2, 0.3]}, metadata={"created_at": custom_ts})
|
||||
payload = memory.vector_store.insert.call_args.kwargs["payloads"][0]
|
||||
assert payload["created_at"] == custom_ts
|
||||
assert payload["updated_at"] == custom_ts
|
||||
|
||||
|
||||
def test_update_memory_uses_utc_timestamps(mocker):
|
||||
memory = _build_memory_instance(mocker, Memory)
|
||||
memory.vector_store.get.return_value = MagicMock(
|
||||
@@ -249,30 +176,6 @@ async def test_async_create_memory_uses_utc_timestamps(mocker):
|
||||
_assert_utc_timestamp(payload["created_at"])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_create_memory_sets_updated_at(mocker):
|
||||
memory = _build_memory_instance(mocker, AsyncMemory)
|
||||
await memory._create_memory("new memory", {"new memory": [0.1, 0.2, 0.3]}, metadata={})
|
||||
payload = memory.vector_store.insert.call_args.kwargs["payloads"][0]
|
||||
assert "updated_at" in payload
|
||||
assert payload["updated_at"] == payload["created_at"]
|
||||
_assert_utc_timestamp(payload["updated_at"])
|
||||
|
||||
# History should also receive updated_at
|
||||
history_kwargs = memory.db.add_history.call_args
|
||||
assert history_kwargs.kwargs["updated_at"] == payload["updated_at"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_create_memory_preserves_existing_created_at(mocker):
|
||||
memory = _build_memory_instance(mocker, AsyncMemory)
|
||||
custom_ts = "2023-05-06T09:19:20+00:00"
|
||||
await memory._create_memory("new memory", {"new memory": [0.1, 0.2, 0.3]}, metadata={"created_at": custom_ts})
|
||||
payload = memory.vector_store.insert.call_args.kwargs["payloads"][0]
|
||||
assert payload["created_at"] == custom_ts
|
||||
assert payload["updated_at"] == custom_ts
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_update_memory_uses_utc_timestamps(mocker):
|
||||
memory = _build_memory_instance(mocker, AsyncMemory)
|
||||
@@ -285,107 +188,6 @@ async def test_async_update_memory_uses_utc_timestamps(mocker):
|
||||
_assert_utc_timestamp(payload["updated_at"])
|
||||
|
||||
|
||||
def test_create_then_search_and_get_all_return_same_timestamps(mocker):
|
||||
"""Reproduces issue #3720: created_at must be identical in search() and get_all()."""
|
||||
memory = _build_memory_instance(mocker, Memory)
|
||||
|
||||
# Step 1: Create a memory — capture the payload stored in the vector store
|
||||
memory._create_memory("Likes pizza", {"Likes pizza": [0.1, 0.2, 0.3]}, metadata={"user_id": "alice"})
|
||||
stored_payload = memory.vector_store.insert.call_args.kwargs["payloads"][0]
|
||||
stored_id = memory.vector_store.insert.call_args.kwargs["ids"][0]
|
||||
|
||||
# Verify both timestamps were stored
|
||||
assert stored_payload["created_at"] is not None
|
||||
assert stored_payload["updated_at"] is not None
|
||||
assert stored_payload["updated_at"] == stored_payload["created_at"]
|
||||
|
||||
# Step 2: Simulate the vector store returning this memory for both search and get_all
|
||||
mem_result = MagicMock()
|
||||
mem_result.id = stored_id
|
||||
mem_result.payload = stored_payload
|
||||
mem_result.score = 0.95
|
||||
|
||||
memory.vector_store.search.return_value = [mem_result]
|
||||
memory.vector_store.list.return_value = [[mem_result]]
|
||||
|
||||
# Step 3: Call search and get_all, compare timestamps
|
||||
search_results = memory._search_vector_store("pizza", filters={"user_id": "alice"}, limit=10, threshold=None)
|
||||
get_all_results = memory._get_all_from_vector_store(filters={"user_id": "alice"}, limit=100)
|
||||
|
||||
search_item = search_results[0]
|
||||
get_all_item = get_all_results[0]
|
||||
|
||||
# The core assertion from issue #3720: created_at must be the same
|
||||
assert search_item["created_at"] == get_all_item["created_at"], (
|
||||
f"created_at mismatch: search={search_item['created_at']}, get_all={get_all_item['created_at']}"
|
||||
)
|
||||
assert search_item["updated_at"] == get_all_item["updated_at"], (
|
||||
f"updated_at mismatch: search={search_item['updated_at']}, get_all={get_all_item['updated_at']}"
|
||||
)
|
||||
|
||||
# Neither should be None
|
||||
assert search_item["created_at"] is not None
|
||||
assert search_item["updated_at"] is not None
|
||||
assert get_all_item["created_at"] is not None
|
||||
assert get_all_item["updated_at"] is not None
|
||||
|
||||
|
||||
def test_update_preserves_created_at_and_updates_updated_at(mocker):
|
||||
"""After an update, created_at must stay the same and updated_at must change."""
|
||||
memory = _build_memory_instance(mocker, Memory)
|
||||
|
||||
# Create a memory
|
||||
memory._create_memory("Likes pizza", {"Likes pizza": [0.1, 0.2, 0.3]}, metadata={"user_id": "alice"})
|
||||
created_payload = memory.vector_store.insert.call_args.kwargs["payloads"][0]
|
||||
created_id = memory.vector_store.insert.call_args.kwargs["ids"][0]
|
||||
original_created_at = created_payload["created_at"]
|
||||
|
||||
# Update the memory — simulate existing memory in vector store
|
||||
memory.vector_store.get.return_value = MagicMock(
|
||||
id=created_id,
|
||||
payload=created_payload,
|
||||
)
|
||||
memory._update_memory(created_id, "Loves pizza", {"Loves pizza": [0.2, 0.3, 0.4]}, metadata={})
|
||||
updated_payload = memory.vector_store.update.call_args.kwargs["payload"]
|
||||
|
||||
# created_at must be preserved
|
||||
assert updated_payload["created_at"] == original_created_at
|
||||
# updated_at must be set and different from creation time (or at least present)
|
||||
assert updated_payload["updated_at"] is not None
|
||||
_assert_utc_timestamp(updated_payload["updated_at"])
|
||||
|
||||
|
||||
def test_search_and_get_all_consistent_after_update(mocker):
|
||||
"""After update, search and get_all must still return the same timestamps."""
|
||||
memory = _build_memory_instance(mocker, Memory)
|
||||
|
||||
# Simulate a memory that was created then updated
|
||||
updated_payload = {
|
||||
"data": "Loves pizza",
|
||||
"hash": "abc123",
|
||||
"user_id": "alice",
|
||||
"created_at": "2023-05-06T09:19:20+00:00",
|
||||
"updated_at": "2026-03-23T10:00:00+00:00",
|
||||
}
|
||||
|
||||
mem_result = MagicMock()
|
||||
mem_result.id = "mem-1"
|
||||
mem_result.payload = updated_payload
|
||||
mem_result.score = 0.9
|
||||
|
||||
memory.vector_store.search.return_value = [mem_result]
|
||||
memory.vector_store.list.return_value = [[mem_result]]
|
||||
|
||||
search_results = memory._search_vector_store("pizza", filters={"user_id": "alice"}, limit=10, threshold=None)
|
||||
get_all_results = memory._get_all_from_vector_store(filters={"user_id": "alice"}, limit=100)
|
||||
|
||||
assert search_results[0]["created_at"] == get_all_results[0]["created_at"]
|
||||
assert search_results[0]["updated_at"] == get_all_results[0]["updated_at"]
|
||||
# created_at should be the original, not the updated time
|
||||
assert search_results[0]["created_at"] == "2023-05-06T09:19:20+00:00"
|
||||
assert search_results[0]["updated_at"] == "2026-03-23T10:00:00+00:00"
|
||||
|
||||
|
||||
class TestMetadataNotMutated:
|
||||
"""Tests that metadata dicts passed to memory methods are not mutated in-place (issue #2648)."""
|
||||
|
||||
@@ -581,52 +383,6 @@ class TestMetadataNotMutated:
|
||||
)
|
||||
|
||||
|
||||
def test_update_preserves_actor_id_when_different_actor_updates(mocker):
|
||||
"""actor_id must be preserved from the original memory even when the
|
||||
updating caller passes a different actor_id in metadata (issue #4490)."""
|
||||
memory = _build_memory_instance(mocker, Memory)
|
||||
memory.vector_store.get.return_value = MagicMock(
|
||||
payload={
|
||||
"data": "I am player #1",
|
||||
"user_id": "team",
|
||||
"actor_id": "Alice",
|
||||
"created_at": "2026-01-01T00:00:00+00:00",
|
||||
}
|
||||
)
|
||||
|
||||
memory._update_memory(
|
||||
"mem-id", "Player #1 is a good person",
|
||||
{"Player #1 is a good person": [0.1, 0.2, 0.3]},
|
||||
metadata={"user_id": "team", "actor_id": "Bob"},
|
||||
)
|
||||
|
||||
stored = memory.vector_store.update.call_args.kwargs["payload"]
|
||||
assert stored["actor_id"] == "Alice"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_update_preserves_actor_id_when_different_actor_updates(mocker):
|
||||
"""Async variant: actor_id must be preserved from the original memory (issue #4490)."""
|
||||
memory = _build_memory_instance(mocker, AsyncMemory)
|
||||
memory.vector_store.get.return_value = MagicMock(
|
||||
payload={
|
||||
"data": "I am player #1",
|
||||
"user_id": "team",
|
||||
"actor_id": "Alice",
|
||||
"created_at": "2026-01-01T00:00:00+00:00",
|
||||
}
|
||||
)
|
||||
|
||||
await memory._update_memory(
|
||||
"mem-id", "Player #1 is a good person",
|
||||
{"Player #1 is a good person": [0.1, 0.2, 0.3]},
|
||||
metadata={"user_id": "team", "actor_id": "Bob"},
|
||||
)
|
||||
|
||||
stored = memory.vector_store.update.call_args.kwargs["payload"]
|
||||
assert stored["actor_id"] == "Alice"
|
||||
|
||||
|
||||
def test_normalize_iso_timestamp_to_utc_preserves_naive_values():
|
||||
assert _normalize_iso_timestamp_to_utc("2026-03-18T00:00:00") == "2026-03-18T00:00:00"
|
||||
|
||||
|
||||
@@ -1,232 +0,0 @@
|
||||
"""Tests for chatty LLM JSON parsing robustness.
|
||||
|
||||
Validates that extract_json and the full fallback chain correctly handle
|
||||
responses from local LLMs (LM Studio, Ollama) that wrap JSON in
|
||||
conversational text, markdown code blocks, or both.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from mem0.memory.utils import extract_json, remove_code_blocks
|
||||
|
||||
|
||||
# --- Test extract_json ---
|
||||
|
||||
class TestExtractJson:
|
||||
"""Tests for extract_json utility."""
|
||||
|
||||
def test_pure_json(self):
|
||||
text = '{"memory": [{"id": "0", "text": "likes basketball", "event": "ADD"}]}'
|
||||
result = extract_json(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed["memory"][0]["text"] == "likes basketball"
|
||||
|
||||
def test_json_in_markdown_code_block(self):
|
||||
text = '```json\n{"memory": [{"id": "0", "text": "likes basketball", "event": "ADD"}]}\n```'
|
||||
result = extract_json(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed["memory"][0]["text"] == "likes basketball"
|
||||
|
||||
def test_json_in_plain_code_block(self):
|
||||
text = '```\n{"memory": [{"id": "0", "text": "likes basketball", "event": "ADD"}]}\n```'
|
||||
result = extract_json(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed["memory"][0]["text"] == "likes basketball"
|
||||
|
||||
def test_chatty_with_markdown(self):
|
||||
"""extract_json can handle embedded code blocks via re.search."""
|
||||
text = """Here is the extracted memory:
|
||||
```json
|
||||
{"memory": [{"id": "0", "text": "likes basketball", "event": "ADD"}]}
|
||||
```
|
||||
I hope this helps!"""
|
||||
result = extract_json(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed["memory"][0]["text"] == "likes basketball"
|
||||
|
||||
def test_chatty_without_markdown(self):
|
||||
"""extract_json finds JSON by {/} boundaries when no code block is present."""
|
||||
text = """Here is the memory update you requested:
|
||||
{"memory": [{"id": "0", "text": "likes gaming", "event": "ADD"}]}
|
||||
That's the result."""
|
||||
result = extract_json(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed["memory"][0]["text"] == "likes gaming"
|
||||
|
||||
def test_chatty_multiline_json_without_markdown(self):
|
||||
"""extract_json finds multi-line JSON by {/} boundaries."""
|
||||
text = """Here is the memory update:
|
||||
{
|
||||
"memory": [
|
||||
{
|
||||
"id": "0",
|
||||
"text": "User likes gaming",
|
||||
"event": "ADD"
|
||||
}
|
||||
]
|
||||
}
|
||||
"""
|
||||
result = extract_json(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed["memory"][0]["text"] == "User likes gaming"
|
||||
|
||||
def test_no_json_at_all(self):
|
||||
"""No JSON in the text — should return as-is."""
|
||||
text = "I don't have any memory updates."
|
||||
result = extract_json(text)
|
||||
assert result == text
|
||||
|
||||
def test_whitespace_padding(self):
|
||||
text = ' \n {"memory": []} \n '
|
||||
result = extract_json(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed == {"memory": []}
|
||||
|
||||
def test_nested_json_objects(self):
|
||||
"""JSON with nested objects should find outermost braces."""
|
||||
text = 'Sure! {"memory": [{"id": "0", "text": "test", "event": "ADD"}]} Done.'
|
||||
result = extract_json(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed["memory"][0]["id"] == "0"
|
||||
|
||||
|
||||
# --- Test remove_code_blocks ---
|
||||
|
||||
class TestRemoveCodeBlocks:
|
||||
"""Tests for remove_code_blocks — verify it does NOT handle chatty text."""
|
||||
|
||||
def test_clean_code_block(self):
|
||||
text = '```json\n{"memory": []}\n```'
|
||||
result = remove_code_blocks(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed == {"memory": []}
|
||||
|
||||
def test_no_code_block(self):
|
||||
"""Without code blocks, returns content as-is (may not be valid JSON)."""
|
||||
text = 'Here is the result: {"memory": []}'
|
||||
result = remove_code_blocks(text)
|
||||
# This should NOT be parseable as JSON because it has leading text
|
||||
with pytest.raises(json.JSONDecodeError):
|
||||
json.loads(result)
|
||||
|
||||
def test_think_tags_removed_inside_code_block(self):
|
||||
"""Think tags inside a code block response are removed."""
|
||||
text = '```json\n<think>reasoning here</think>\n{"memory": []}\n```'
|
||||
result = remove_code_blocks(text)
|
||||
parsed = json.loads(result)
|
||||
assert parsed == {"memory": []}
|
||||
|
||||
def test_think_tags_before_code_block_not_handled(self):
|
||||
"""Think tags before code block cause regex to miss — fallback needed.
|
||||
|
||||
This is a known limitation of remove_code_blocks: it matches the code
|
||||
block regex first, and only strips think tags from the result. When
|
||||
think tags come before the code block, the regex fails entirely.
|
||||
The fallback chain (extract_json) handles this case.
|
||||
"""
|
||||
text = '<think>reasoning here</think>\n```json\n{"memory": []}\n```'
|
||||
result = remove_code_blocks(text)
|
||||
with pytest.raises(json.JSONDecodeError):
|
||||
json.loads(result)
|
||||
|
||||
|
||||
# --- Test the full fallback chain (remove_code_blocks -> extract_json) ---
|
||||
|
||||
class TestFallbackChain:
|
||||
"""Tests the actual fallback pattern used in _add_to_vector_store:
|
||||
try json.loads(remove_code_blocks(response))
|
||||
except JSONDecodeError: json.loads(extract_json(response))
|
||||
"""
|
||||
|
||||
def _parse_with_fallback(self, response):
|
||||
"""Mimics the parsing logic in _add_to_vector_store."""
|
||||
try:
|
||||
return json.loads(remove_code_blocks(response), strict=False)
|
||||
except json.JSONDecodeError:
|
||||
extracted = extract_json(response)
|
||||
return json.loads(extracted, strict=False)
|
||||
|
||||
def test_clean_json(self):
|
||||
response = '{"memory": [{"id": "0", "text": "Name is Alex", "event": "ADD"}]}'
|
||||
result = self._parse_with_fallback(response)
|
||||
assert result["memory"][0]["text"] == "Name is Alex"
|
||||
|
||||
def test_markdown_wrapped(self):
|
||||
response = '```json\n{"memory": [{"id": "0", "text": "Name is Alex", "event": "ADD"}]}\n```'
|
||||
result = self._parse_with_fallback(response)
|
||||
assert result["memory"][0]["text"] == "Name is Alex"
|
||||
|
||||
def test_chatty_markdown(self):
|
||||
"""Issue #3788: LLM wraps in conversational text + markdown."""
|
||||
response = """Here is the extracted memory:
|
||||
```json
|
||||
{
|
||||
"memory": [
|
||||
{"id": "0", "text": "Name is Alex", "event": "ADD"},
|
||||
{"id": "1", "text": "Love basketball", "event": "ADD"},
|
||||
{"id": "2", "text": "Love gaming", "event": "ADD"}
|
||||
]
|
||||
}
|
||||
```
|
||||
I hope this helps!"""
|
||||
result = self._parse_with_fallback(response)
|
||||
assert len(result["memory"]) == 3
|
||||
assert result["memory"][0]["text"] == "Name is Alex"
|
||||
assert result["memory"][1]["text"] == "Love basketball"
|
||||
assert result["memory"][2]["text"] == "Love gaming"
|
||||
|
||||
def test_chatty_no_markdown(self):
|
||||
"""Issue #3788: LLM wraps in conversational text without markdown."""
|
||||
response = """Here is the memory update you requested:
|
||||
{
|
||||
"memory": [
|
||||
{"id": "0", "text": "Name is Alex", "event": "ADD"},
|
||||
{"id": "1", "text": "Love basketball", "event": "ADD"}
|
||||
]
|
||||
}"""
|
||||
result = self._parse_with_fallback(response)
|
||||
assert len(result["memory"]) == 2
|
||||
|
||||
def test_think_tags_with_json(self):
|
||||
"""Reasoning model with <think> tags."""
|
||||
response = '<think>Let me process this...</think>\n```json\n{"memory": [{"id": "0", "text": "test", "event": "ADD"}]}\n```'
|
||||
result = self._parse_with_fallback(response)
|
||||
assert result["memory"][0]["text"] == "test"
|
||||
|
||||
def test_lmstudio_style_response(self):
|
||||
"""Mimics the actual LM Studio response format from the issue."""
|
||||
response = """{
|
||||
"memory": [
|
||||
{
|
||||
"id": "0",
|
||||
"text": "Name is Alex",
|
||||
"event": "ADD"
|
||||
},
|
||||
{
|
||||
"id": "1",
|
||||
"text": "Love basketball",
|
||||
"event": "ADD"
|
||||
},
|
||||
{
|
||||
"id": "2",
|
||||
"text": "Love gaming",
|
||||
"event": "ADD"
|
||||
}
|
||||
]
|
||||
}"""
|
||||
result = self._parse_with_fallback(response)
|
||||
assert len(result["memory"]) == 3
|
||||
|
||||
def test_completely_invalid_response(self):
|
||||
"""If both attempts fail, the outer except in main.py catches it."""
|
||||
response = "I don't understand the question"
|
||||
with pytest.raises(json.JSONDecodeError):
|
||||
self._parse_with_fallback(response)
|
||||
|
||||
def test_facts_extraction_fallback(self):
|
||||
"""Also verify the pattern works for facts extraction responses."""
|
||||
response = 'Sure! Here are the facts:\n{"facts": ["Name is Alex", "Loves basketball"]}\nHope that helps!'
|
||||
result = self._parse_with_fallback(response)
|
||||
assert result["facts"] == ["Name is Alex", "Loves basketball"]
|
||||
+1
-30
@@ -175,41 +175,12 @@ def test_update(memory_instance):
|
||||
result = memory_instance.update("test_id", "Updated memory")
|
||||
|
||||
memory_instance._update_memory.assert_called_once_with(
|
||||
"test_id", "Updated memory", {"Updated memory": [0.1, 0.2, 0.3]}, None
|
||||
"test_id", "Updated memory", {"Updated memory": [0.1, 0.2, 0.3]}
|
||||
)
|
||||
|
||||
assert result["message"] == "Memory updated successfully!"
|
||||
|
||||
|
||||
def test_update_with_metadata(memory_instance):
|
||||
memory_instance.embedding_model = Mock()
|
||||
memory_instance.embedding_model.embed = Mock(return_value=[0.1, 0.2, 0.3])
|
||||
|
||||
memory_instance._update_memory = Mock()
|
||||
metadata = {"category": "sports", "priority": "high"}
|
||||
|
||||
result = memory_instance.update("test_id", "Updated memory", metadata=metadata)
|
||||
|
||||
memory_instance._update_memory.assert_called_once_with(
|
||||
"test_id", "Updated memory", {"Updated memory": [0.1, 0.2, 0.3]}, metadata
|
||||
)
|
||||
|
||||
assert result["message"] == "Memory updated successfully!"
|
||||
|
||||
|
||||
def test_update_with_empty_metadata(memory_instance):
|
||||
memory_instance.embedding_model = Mock()
|
||||
memory_instance.embedding_model.embed = Mock(return_value=[0.1, 0.2, 0.3])
|
||||
|
||||
memory_instance._update_memory = Mock()
|
||||
|
||||
memory_instance.update("test_id", "Updated memory", metadata={})
|
||||
|
||||
memory_instance._update_memory.assert_called_once_with(
|
||||
"test_id", "Updated memory", {"Updated memory": [0.1, 0.2, 0.3]}, {}
|
||||
)
|
||||
|
||||
|
||||
def test_delete(memory_instance):
|
||||
memory_instance._delete_memory = Mock()
|
||||
|
||||
|
||||
+4
-189
@@ -1,5 +1,4 @@
|
||||
import json
|
||||
from datetime import datetime
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
@@ -11,7 +10,7 @@ from mem0.memory.utils import normalize_facts
|
||||
|
||||
class MockVectorMemory:
|
||||
"""Mock memory object for testing incomplete payloads."""
|
||||
|
||||
|
||||
def __init__(self, memory_id: str, payload: dict, score: float = 0.8):
|
||||
self.id = memory_id
|
||||
self.payload = payload
|
||||
@@ -105,7 +104,7 @@ def test_collection_name_preserved_after_reset(mock_sqlite, mock_llm_factory, mo
|
||||
|
||||
reset_calls = [call for call in mock_vector_factory.call_args_list if len(mock_vector_factory.call_args_list) > 2]
|
||||
if reset_calls:
|
||||
reset_config = reset_calls[-1][0][1]
|
||||
reset_config = reset_calls[-1][0][1]
|
||||
assert reset_config.collection_name == test_collection_name, f"Reset used wrong collection name: {reset_config.collection_name}"
|
||||
|
||||
|
||||
@@ -130,13 +129,13 @@ def test_search_handles_incomplete_payloads(mock_sqlite, mock_llm_factory, mock_
|
||||
complete_memory = MockVectorMemory("mem_2", {"data": "content", "hash": "def456"})
|
||||
|
||||
mock_vector_store.search.return_value = [incomplete_memory, complete_memory]
|
||||
|
||||
|
||||
mock_embedder = MagicMock()
|
||||
mock_embedder.embed.return_value = [0.1, 0.2, 0.3]
|
||||
memory.embedding_model = mock_embedder
|
||||
|
||||
result = memory._search_vector_store("test", {"user_id": "test"}, 10)
|
||||
|
||||
|
||||
assert len(result) == 2
|
||||
memories_by_id = {mem["id"]: mem for mem in result}
|
||||
|
||||
@@ -573,187 +572,3 @@ def test_update_infer_true_caches_embedding_on_llm_rewrite(mock_sqlite, mock_llm
|
||||
# It should NOT be called a 3rd time inside _update_memory
|
||||
assert embedder.embed.call_count == 2
|
||||
mock_vector_store.update.assert_called_once()
|
||||
|
||||
|
||||
@patch('mem0.utils.factory.EmbedderFactory.create')
|
||||
@patch('mem0.utils.factory.VectorStoreFactory.create')
|
||||
@patch('mem0.utils.factory.LlmFactory.create')
|
||||
@patch('mem0.memory.main.SQLiteManager')
|
||||
def test_delete_memory_history_has_timestamps(mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory):
|
||||
"""
|
||||
Test that deleting a memory records created_at and updated_at in history.
|
||||
|
||||
Ensures DELETE operations preserve the original creation timestamp
|
||||
and record the deletion time for proper audit trails.
|
||||
"""
|
||||
mock_embedder_factory.return_value = MagicMock()
|
||||
mock_vector_store = MagicMock()
|
||||
mock_vector_factory.return_value = mock_vector_store
|
||||
mock_llm_factory.return_value = MagicMock()
|
||||
mock_sqlite.return_value = MagicMock()
|
||||
|
||||
from mem0.memory.main import Memory as MemoryClass
|
||||
config = MemoryConfig()
|
||||
memory = MemoryClass(config)
|
||||
|
||||
existing_memory = MagicMock()
|
||||
existing_memory.payload = {
|
||||
"data": "I like Python.",
|
||||
"created_at": "2024-01-01T00:00:00+00:00",
|
||||
"actor_id": None,
|
||||
"role": None,
|
||||
}
|
||||
mock_vector_store.get.return_value = existing_memory
|
||||
|
||||
memory.delete("mem-123")
|
||||
|
||||
call_kwargs = memory.db.add_history.call_args.kwargs
|
||||
assert call_kwargs["created_at"] == "2024-01-01T00:00:00+00:00"
|
||||
assert call_kwargs["updated_at"] is not None
|
||||
datetime.fromisoformat(call_kwargs["updated_at"]) # verify valid ISO timestamp
|
||||
|
||||
|
||||
@patch('mem0.utils.factory.EmbedderFactory.create')
|
||||
@patch('mem0.utils.factory.VectorStoreFactory.create')
|
||||
@patch('mem0.utils.factory.LlmFactory.create')
|
||||
@patch('mem0.memory.main.SQLiteManager')
|
||||
def test_delete_memory_normalizes_non_utc_created_at(mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory):
|
||||
"""Test that non-UTC created_at timestamps are normalized to UTC on delete."""
|
||||
mock_embedder_factory.return_value = MagicMock()
|
||||
mock_vector_store = MagicMock()
|
||||
mock_vector_factory.return_value = mock_vector_store
|
||||
mock_llm_factory.return_value = MagicMock()
|
||||
mock_sqlite.return_value = MagicMock()
|
||||
|
||||
from mem0.memory.main import Memory as MemoryClass
|
||||
config = MemoryConfig()
|
||||
memory = MemoryClass(config)
|
||||
|
||||
existing_memory = MagicMock()
|
||||
existing_memory.payload = {
|
||||
"data": "I like Python.",
|
||||
"created_at": "2024-01-01T05:00:00+05:00", # UTC+5
|
||||
"actor_id": None,
|
||||
"role": None,
|
||||
}
|
||||
mock_vector_store.get.return_value = existing_memory
|
||||
|
||||
memory.delete("mem-123")
|
||||
|
||||
call_kwargs = memory.db.add_history.call_args.kwargs
|
||||
assert call_kwargs["created_at"] == "2024-01-01T00:00:00+00:00" # normalized to UTC
|
||||
|
||||
|
||||
@patch('mem0.utils.factory.EmbedderFactory.create')
|
||||
@patch('mem0.utils.factory.VectorStoreFactory.create')
|
||||
@patch('mem0.utils.factory.LlmFactory.create')
|
||||
@patch('mem0.memory.main.SQLiteManager')
|
||||
def test_delete_memory_missing_created_at(mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory):
|
||||
"""Test that delete works when created_at is absent from the payload (pre-existing memories)."""
|
||||
mock_embedder_factory.return_value = MagicMock()
|
||||
mock_vector_store = MagicMock()
|
||||
mock_vector_factory.return_value = mock_vector_store
|
||||
mock_llm_factory.return_value = MagicMock()
|
||||
mock_sqlite.return_value = MagicMock()
|
||||
|
||||
from mem0.memory.main import Memory as MemoryClass
|
||||
config = MemoryConfig()
|
||||
memory = MemoryClass(config)
|
||||
|
||||
existing_memory = MagicMock()
|
||||
existing_memory.payload = {
|
||||
"data": "I like Python.",
|
||||
"actor_id": None,
|
||||
"role": None,
|
||||
}
|
||||
mock_vector_store.get.return_value = existing_memory
|
||||
|
||||
memory.delete("mem-123")
|
||||
|
||||
call_kwargs = memory.db.add_history.call_args.kwargs
|
||||
assert call_kwargs["created_at"] is None
|
||||
assert call_kwargs["updated_at"] is not None
|
||||
datetime.fromisoformat(call_kwargs["updated_at"]) # verify valid ISO timestamp
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@patch('mem0.utils.factory.EmbedderFactory.create')
|
||||
@patch('mem0.utils.factory.VectorStoreFactory.create')
|
||||
@patch('mem0.utils.factory.LlmFactory.create')
|
||||
@patch('mem0.memory.main.SQLiteManager')
|
||||
async def test_async_delete_memory_history_has_timestamps(mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory):
|
||||
"""
|
||||
Test that async deleting a memory records created_at and updated_at in history.
|
||||
|
||||
Ensures async DELETE operations preserve the original creation timestamp
|
||||
and record the deletion time for proper audit trails.
|
||||
"""
|
||||
mock_embedder_factory.return_value = MagicMock()
|
||||
mock_vector_store = MagicMock()
|
||||
mock_vector_factory.return_value = mock_vector_store
|
||||
mock_llm_factory.return_value = MagicMock()
|
||||
mock_sqlite.return_value = MagicMock()
|
||||
|
||||
from mem0.memory.main import AsyncMemory
|
||||
config = MemoryConfig()
|
||||
memory = AsyncMemory(config)
|
||||
|
||||
existing_memory = MagicMock()
|
||||
existing_memory.payload = {
|
||||
"data": "I like Python.",
|
||||
"created_at": "2024-01-01T00:00:00+00:00",
|
||||
"actor_id": None,
|
||||
"role": None,
|
||||
}
|
||||
mock_vector_store.get.return_value = existing_memory
|
||||
|
||||
await memory.delete("mem-123")
|
||||
|
||||
call_kwargs = memory.db.add_history.call_args.kwargs
|
||||
assert call_kwargs["created_at"] == "2024-01-01T00:00:00+00:00"
|
||||
assert call_kwargs["updated_at"] is not None
|
||||
datetime.fromisoformat(call_kwargs["updated_at"]) # verify valid ISO timestamp
|
||||
|
||||
|
||||
@patch('mem0.utils.factory.EmbedderFactory.create')
|
||||
@patch('mem0.utils.factory.VectorStoreFactory.create')
|
||||
@patch('mem0.utils.factory.LlmFactory.create')
|
||||
@patch('mem0.memory.storage.SQLiteManager')
|
||||
class TestProcessMetadataFiltersMerge:
|
||||
"""Regression tests for issue #3952: multiple operators on the same key must be merged."""
|
||||
|
||||
def _make_memory(self, mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory):
|
||||
mock_embedder_factory.return_value = MagicMock()
|
||||
mock_vector_factory.return_value = MagicMock()
|
||||
mock_llm_factory.return_value = MagicMock()
|
||||
mock_sqlite.return_value = MagicMock()
|
||||
from mem0.memory.main import Memory as MemoryClass
|
||||
return MemoryClass(MemoryConfig())
|
||||
|
||||
def test_multiple_operators_same_key_merged(self, mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory):
|
||||
"""Filters like created_at: {gte: X, lte: Y} must preserve both operators."""
|
||||
memory = self._make_memory(mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory)
|
||||
result = memory._process_metadata_filters({
|
||||
"created_at": {"gte": 1000, "lte": 2000}
|
||||
})
|
||||
assert result == {"created_at": {"gte": 1000, "lte": 2000}}
|
||||
|
||||
def test_single_operator_still_works(self, mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory):
|
||||
"""Single operator filters must continue to work."""
|
||||
memory = self._make_memory(mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory)
|
||||
result = memory._process_metadata_filters({
|
||||
"created_at": {"gte": 1000}
|
||||
})
|
||||
assert result == {"created_at": {"gte": 1000}}
|
||||
|
||||
def test_multiple_keys_with_multiple_operators(self, mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory):
|
||||
"""Multiple keys each with multiple operators."""
|
||||
memory = self._make_memory(mock_sqlite, mock_llm_factory, mock_vector_factory, mock_embedder_factory)
|
||||
result = memory._process_metadata_filters({
|
||||
"created_at": {"gte": 1000, "lte": 2000},
|
||||
"score": {"gt": 0.5, "lt": 0.9},
|
||||
})
|
||||
assert result == {
|
||||
"created_at": {"gte": 1000, "lte": 2000},
|
||||
"score": {"gt": 0.5, "lt": 0.9},
|
||||
}
|
||||
|
||||
@@ -153,16 +153,9 @@ def test_insert_generates_sql(db_instance_direct, mock_workspace_client):
|
||||
sql = kwargs["statement"] if "statement" in kwargs else args[0]
|
||||
assert "INSERT INTO" in sql
|
||||
assert "catalog.schema.table" in sql
|
||||
# Embedding list rendered inline (ARRAY type not supported by parameterized queries)
|
||||
assert "id1" in sql
|
||||
# Embedding list rendered
|
||||
assert "array(0.1, 0.2, 0.3, 0.4)" in sql
|
||||
# User-supplied values should use parameterized queries, not inline values
|
||||
assert ":memory_id_0" in sql
|
||||
assert "id1" not in sql # id should be in params, not in SQL
|
||||
params = kwargs["parameters"]
|
||||
param_names = {p.name for p in params}
|
||||
assert "memory_id_0" in param_names
|
||||
id_param = next(p for p in params if p.name == "memory_id_0")
|
||||
assert id_param.value == "id1"
|
||||
|
||||
|
||||
# ---------------------- Search Tests ---------------------- #
|
||||
@@ -253,13 +246,7 @@ def test_delete_vector(db_instance_delta, mock_workspace_client):
|
||||
db_instance_delta.delete("id-delete")
|
||||
args, kwargs = mock_workspace_client.statement_execution.execute_statement.call_args
|
||||
sql = kwargs.get("statement") or args[0]
|
||||
assert "DELETE FROM" in sql
|
||||
assert ":vector_id" in sql
|
||||
assert "id-delete" not in sql # value should be in params, not in SQL
|
||||
params = kwargs["parameters"]
|
||||
assert len(params) == 1
|
||||
assert params[0].name == "vector_id"
|
||||
assert params[0].value == "id-delete"
|
||||
assert "DELETE FROM" in sql and "id-delete" in sql
|
||||
|
||||
|
||||
# ---------------------- Update Tests ---------------------- #
|
||||
@@ -273,17 +260,10 @@ def test_update_vector(db_instance_direct, mock_workspace_client):
|
||||
)
|
||||
args, kwargs = mock_workspace_client.statement_execution.execute_statement.call_args
|
||||
sql = kwargs.get("statement") or args[0]
|
||||
assert "UPDATE" in sql
|
||||
assert "embedding = array(0.4, 0.5, 0.6, 0.7)" in sql
|
||||
assert "custom = :payload_custom" in sql
|
||||
assert ":vector_id" in sql
|
||||
assert "id-upd" not in sql # value should be in params, not in SQL
|
||||
assert "'val'" not in sql # value should be in params, not in SQL
|
||||
assert "UPDATE" in sql and "id-upd" in sql
|
||||
assert "embedding = [0.4, 0.5, 0.6, 0.7]" in sql
|
||||
assert "custom = 'val'" in sql
|
||||
assert "user_id" not in sql # excluded
|
||||
params = kwargs["parameters"]
|
||||
param_map = {p.name: p.value for p in params}
|
||||
assert param_map["payload_custom"] == "val"
|
||||
assert param_map["vector_id"] == "id-upd"
|
||||
|
||||
|
||||
# ---------------------- Get Tests ---------------------- #
|
||||
@@ -723,12 +703,9 @@ def test_e2e_crud_lifecycle_delta_sync(mock_workspace_client):
|
||||
payloads=[{"data": "test memory", "user_id": "u1", "hash": "h1"}],
|
||||
ids=["mem-001"],
|
||||
)
|
||||
insert_call = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
insert_sql = insert_call["statement"]
|
||||
insert_sql = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs["statement"]
|
||||
assert "INSERT INTO cat.sch.tbl" in insert_sql
|
||||
assert ":memory_id_0" in insert_sql # parameterized, not inline
|
||||
insert_params = {p.name: p.value for p in insert_call["parameters"]}
|
||||
assert insert_params["memory_id_0"] == "mem-001"
|
||||
assert "mem-001" in insert_sql
|
||||
|
||||
# SEARCH
|
||||
mock_workspace_client.vector_search_indexes.query_index.return_value = SimpleNamespace(
|
||||
@@ -776,23 +753,16 @@ def test_e2e_crud_lifecycle_delta_sync(mock_workspace_client):
|
||||
|
||||
# UPDATE
|
||||
db.update(vector_id="mem-001", payload={"memory": "updated memory"})
|
||||
update_call = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
update_sql = update_call["statement"]
|
||||
update_sql = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs["statement"]
|
||||
assert "UPDATE cat.sch.tbl" in update_sql
|
||||
assert ":vector_id" in update_sql
|
||||
assert ":payload_memory" in update_sql
|
||||
update_params = {p.name: p.value for p in update_call["parameters"]}
|
||||
assert update_params["vector_id"] == "mem-001"
|
||||
assert update_params["payload_memory"] == "updated memory"
|
||||
assert "mem-001" in update_sql
|
||||
assert "updated memory" in update_sql
|
||||
|
||||
# DELETE
|
||||
db.delete("mem-001")
|
||||
delete_call = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
delete_sql = delete_call["statement"]
|
||||
delete_sql = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs["statement"]
|
||||
assert "DELETE FROM cat.sch.tbl" in delete_sql
|
||||
assert ":vector_id" in delete_sql
|
||||
delete_params = {p.name: p.value for p in delete_call["parameters"]}
|
||||
assert delete_params["vector_id"] == "mem-001"
|
||||
assert "mem-001" in delete_sql
|
||||
|
||||
|
||||
def test_e2e_crud_lifecycle_direct_access(mock_workspace_client):
|
||||
@@ -826,11 +796,8 @@ def test_e2e_crud_lifecycle_direct_access(mock_workspace_client):
|
||||
payloads=[{"data": "direct memory", "user_id": "u1", "hash": "h1"}],
|
||||
ids=["mem-da-001"],
|
||||
)
|
||||
insert_call = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
insert_sql = insert_call["statement"]
|
||||
insert_sql = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs["statement"]
|
||||
assert "array(0.1, 0.2, 0.3, 0.4)" in insert_sql
|
||||
insert_params = {p.name: p.value for p in insert_call["parameters"]}
|
||||
assert insert_params["memory_id_0"] == "mem-da-001"
|
||||
|
||||
# SEARCH with vector
|
||||
mock_workspace_client.vector_search_indexes.query_index.return_value = SimpleNamespace(
|
||||
@@ -871,191 +838,3 @@ def test_e2e_crud_lifecycle_direct_access(mock_workspace_client):
|
||||
list_kwargs = mock_workspace_client.vector_search_indexes.query_index.call_args.kwargs
|
||||
assert "query_vector" in list_kwargs
|
||||
assert "query_text" not in list_kwargs
|
||||
|
||||
|
||||
# ---------------------- SQL Injection Prevention Tests (Issue #4073) ---------------------- #
|
||||
|
||||
|
||||
SQLI_PAYLOADS = [
|
||||
"'; DELETE FROM table; --",
|
||||
"' OR '1'='1",
|
||||
"'; DROP TABLE memories; --",
|
||||
"1' UNION SELECT * FROM secrets --",
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("malicious_id", SQLI_PAYLOADS)
|
||||
def test_delete_prevents_sql_injection(db_instance_delta, mock_workspace_client, malicious_id):
|
||||
"""Verify delete() uses parameterized queries so SQL injection payloads are never in the SQL string."""
|
||||
db_instance_delta.delete(malicious_id)
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
# The malicious payload must NOT appear in the SQL statement itself
|
||||
assert malicious_id not in sql
|
||||
assert ":vector_id" in sql
|
||||
# It should be safely passed as a parameter
|
||||
assert kwargs["parameters"][0].value == malicious_id
|
||||
|
||||
|
||||
@pytest.mark.parametrize("malicious_id", SQLI_PAYLOADS)
|
||||
def test_update_prevents_sql_injection_in_vector_id(db_instance_delta, mock_workspace_client, malicious_id):
|
||||
"""Verify update() parameterizes vector_id."""
|
||||
db_instance_delta.update(vector_id=malicious_id, payload={"memory": "safe value"})
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
assert malicious_id not in sql
|
||||
assert ":vector_id" in sql
|
||||
param_map = {p.name: p.value for p in kwargs["parameters"]}
|
||||
assert param_map["vector_id"] == malicious_id
|
||||
|
||||
|
||||
@pytest.mark.parametrize("malicious_value", SQLI_PAYLOADS)
|
||||
def test_update_prevents_sql_injection_in_payload(db_instance_delta, mock_workspace_client, malicious_value):
|
||||
"""Verify update() parameterizes payload values."""
|
||||
db_instance_delta.update(vector_id="safe-id", payload={"memory": malicious_value})
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
assert malicious_value not in sql
|
||||
assert ":payload_memory" in sql
|
||||
param_map = {p.name: p.value for p in kwargs["parameters"]}
|
||||
assert param_map["payload_memory"] == malicious_value
|
||||
|
||||
|
||||
@pytest.mark.parametrize("malicious_id", SQLI_PAYLOADS)
|
||||
def test_insert_prevents_sql_injection_in_ids(db_instance_delta, mock_workspace_client, malicious_id):
|
||||
"""Verify insert() parameterizes memory IDs."""
|
||||
db_instance_delta.insert(
|
||||
vectors=[[0.1, 0.2]],
|
||||
payloads=[{"data": "test", "hash": "h1"}],
|
||||
ids=[malicious_id],
|
||||
)
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
assert malicious_id not in sql
|
||||
assert ":memory_id_0" in sql
|
||||
param_map = {p.name: p.value for p in kwargs["parameters"]}
|
||||
assert param_map["memory_id_0"] == malicious_id
|
||||
|
||||
|
||||
@pytest.mark.parametrize("malicious_data", SQLI_PAYLOADS)
|
||||
def test_insert_prevents_sql_injection_in_payload_data(db_instance_delta, mock_workspace_client, malicious_data):
|
||||
"""Verify insert() parameterizes payload data values."""
|
||||
db_instance_delta.insert(
|
||||
vectors=[[0.1, 0.2]],
|
||||
payloads=[{"data": malicious_data, "hash": "h1"}],
|
||||
ids=["safe-id"],
|
||||
)
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
assert malicious_data not in sql
|
||||
param_map = {p.name: p.value for p in kwargs["parameters"]}
|
||||
assert param_map["memory_0"] == malicious_data
|
||||
|
||||
|
||||
@pytest.mark.parametrize("malicious_key", [
|
||||
"memory; DROP TABLE x--",
|
||||
"col' OR '1'='1",
|
||||
"valid_col; DELETE FROM t",
|
||||
"col\nname",
|
||||
])
|
||||
def test_update_rejects_malicious_column_names(db_instance_delta, mock_workspace_client, malicious_key):
|
||||
"""Verify update() skips payload keys that are not valid SQL identifiers."""
|
||||
db_instance_delta.update(
|
||||
vector_id="safe-id",
|
||||
payload={malicious_key: "some value", "memory": "legit"},
|
||||
)
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
# The malicious key must NOT appear as a column name in the SQL
|
||||
assert malicious_key not in sql
|
||||
# The legitimate key should still be present
|
||||
assert ":payload_memory" in sql
|
||||
|
||||
|
||||
def test_insert_multi_row(db_instance_delta, mock_workspace_client):
|
||||
"""Verify multi-row insert generates unique parameter names per row."""
|
||||
db_instance_delta.insert(
|
||||
vectors=[[0.1, 0.2], [0.3, 0.4]],
|
||||
payloads=[
|
||||
{"data": "first memory", "hash": "h1"},
|
||||
{"data": "second memory", "hash": "h2"},
|
||||
],
|
||||
ids=["id-1", "id-2"],
|
||||
)
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
params = kwargs["parameters"]
|
||||
param_map = {p.name: p.value for p in params}
|
||||
# Each row should have distinct parameter names
|
||||
assert ":memory_id_0" in sql
|
||||
assert ":memory_id_1" in sql
|
||||
assert param_map["memory_id_0"] == "id-1"
|
||||
assert param_map["memory_id_1"] == "id-2"
|
||||
assert param_map["memory_0"] == "first memory"
|
||||
assert param_map["memory_1"] == "second memory"
|
||||
|
||||
|
||||
def test_insert_with_none_values(db_instance_delta, mock_workspace_client):
|
||||
"""Verify insert() uses NULL literal for None values, not parameters."""
|
||||
db_instance_delta.insert(
|
||||
vectors=[[0.1, 0.2]],
|
||||
payloads=[{"data": "test", "hash": "h1"}], # agent_id, run_id etc will be None
|
||||
ids=["id-1"],
|
||||
)
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
assert "NULL" in sql # None values should be literal NULL
|
||||
# Only non-None values should have parameters
|
||||
param_names = {p.name for p in kwargs["parameters"]}
|
||||
assert "memory_id_0" in param_names
|
||||
assert "memory_0" in param_names
|
||||
assert "hash_0" in param_names
|
||||
|
||||
|
||||
def test_update_payload_only(db_instance_delta, mock_workspace_client):
|
||||
"""Verify update() works with payload only (no vector)."""
|
||||
db_instance_delta.update(vector_id="id-1", payload={"memory": "new text"})
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
assert "UPDATE" in sql
|
||||
assert "embedding" not in sql
|
||||
assert ":payload_memory" in sql
|
||||
assert ":vector_id" in sql
|
||||
|
||||
|
||||
def test_update_vector_only(db_instance_direct, mock_workspace_client):
|
||||
"""Verify update() works with vector only (no payload)."""
|
||||
db_instance_direct.update(vector_id="id-1", vector=[0.1, 0.2, 0.3, 0.4])
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
sql = kwargs["statement"]
|
||||
assert "UPDATE" in sql
|
||||
assert "embedding = array(0.1, 0.2, 0.3, 0.4)" in sql
|
||||
assert ":vector_id" in sql
|
||||
param_map = {p.name: p.value for p in kwargs["parameters"]}
|
||||
assert param_map["vector_id"] == "id-1"
|
||||
assert len(kwargs["parameters"]) == 1 # only vector_id param
|
||||
|
||||
|
||||
def test_insert_timestamp_params_have_explicit_type(db_instance_delta, mock_workspace_client):
|
||||
"""Verify insert() sets type='TIMESTAMP' on created_at/updated_at parameters
|
||||
so Databricks doesn't rely on implicit STRING->TIMESTAMP casting."""
|
||||
db_instance_delta.insert(
|
||||
vectors=[[0.1, 0.2]],
|
||||
payloads=[{
|
||||
"data": "test",
|
||||
"hash": "h1",
|
||||
"created_at": "2024-01-01T00:00:00+00:00",
|
||||
"updated_at": "2024-01-02T00:00:00+00:00",
|
||||
}],
|
||||
ids=["id-1"],
|
||||
)
|
||||
kwargs = mock_workspace_client.statement_execution.execute_statement.call_args.kwargs
|
||||
params = kwargs["parameters"]
|
||||
ts_params = [p for p in params if "created_at" in p.name or "updated_at" in p.name]
|
||||
assert len(ts_params) == 2
|
||||
for p in ts_params:
|
||||
assert p.type == "TIMESTAMP", f"Parameter {p.name} should have type=TIMESTAMP, got {p.type}"
|
||||
# Non-timestamp params should not have a type set (defaults to STRING)
|
||||
non_ts_params = [p for p in params if "created_at" not in p.name and "updated_at" not in p.name]
|
||||
for p in non_ts_params:
|
||||
assert p.type is None, f"Parameter {p.name} should not have explicit type, got {p.type}"
|
||||
|
||||
Reference in New Issue
Block a user