feat(openclaw): improve extraction quality with noise filtering, deduplication, and better instructions (#4302)
Co-authored-by: utkarsh240799 <utkarsh240799@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Utkarsh
@@ -2,8 +2,46 @@
|
||||
|
||||
All notable changes to the `@mem0/openclaw-mem0` plugin will be documented in this file.
|
||||
|
||||
## [0.4.0] - 2026-03-16
|
||||
|
||||
### Added
|
||||
- **Non-interactive trigger filtering**: Skips recall and capture for `cron`, `heartbeat`, `automation`, and `schedule` triggers — prevents system-generated noise from polluting memory
|
||||
- **Subagent hallucination prevention**: `isSubagentSession()` detects ephemeral subagent sessions and routes recall to the parent (main user) namespace instead of empty ephemeral namespaces; skips capture to prevent orphaned memories
|
||||
- **Subagent-specific preamble**: Subagents receive "You are a subagent — use these memories for context but do not assume you are this user" to prevent identity assumption
|
||||
- **User identity in recall preamble**: Recalled memories now include `userId` attribution for better context
|
||||
- **User identity in extraction preamble**: Extraction context includes user identity and current date for accurate attribution and temporal anchoring
|
||||
- **User-content guard**: Skips extraction when no meaningful user messages remain after filtering
|
||||
- **Dynamic recall thresholding**: Memories scoring less than 50% of the top result are dropped to filter out the long tail of weak matches
|
||||
- **SQLite resilience for OSS mode**: Init error recovery with automatic retry (history disabled) when native SQLite bindings fail under jiti
|
||||
- **`disableHistory` config option**: New `oss.disableHistory` flag to explicitly skip history DB initialization
|
||||
- **Updated minimum package version of mem0ai package**: Updated minimum package version of mem0ai package to ^2.3.0 to force old users to migrate to better-sqlite3
|
||||
- 78 unit tests covering filtering, isolation, trigger filtering, subagent detection, and SQLite resilience
|
||||
|
||||
### Changed
|
||||
- Auto-recall threshold raised from 0.5 to 0.6 for stricter precision during automatic injection (explicit tool searches remain at 0.5)
|
||||
- Recall candidate pool increased to `topK * 2` for better filtering headroom
|
||||
- Provider init promises now reset on failure, allowing retry on subsequent calls
|
||||
- Relaxed extraction instructions: related facts are kept together to preserve context (removed atomic memory requirement)
|
||||
|
||||
### Fixed
|
||||
- **Concurrent session race condition**: Lifecycle hooks (`before_agent_start`, `agent_end`) now use `ctx.sessionKey` directly from the event context instead of a shared mutable `currentSessionId` variable, preventing cross-session data leaks when multiple sessions run simultaneously
|
||||
|
||||
## [0.3.1] - 2026-03-12
|
||||
|
||||
### Added
|
||||
- **Message filtering pipeline**: Multi-stage noise removal before extraction — drops heartbeats, timestamps, single-word acks, system routing metadata, compaction audit logs, and generic assistant acknowledgments
|
||||
- **Broad recall for new sessions**: Short or new-session prompts trigger a secondary broad search to avoid cold-start blindness
|
||||
- **Client-side threshold filtering**: Safety net that drops low-relevance results even if the API doesn't honor the threshold parameter
|
||||
- **Temporal anchoring**: Extraction instructions now include current date so memories are prefixed with "As of YYYY-MM-DD, ..."
|
||||
- **Summary message inclusion**: Earlier assistant messages containing work summaries are included in extraction context even if outside the recent-message window
|
||||
- 55 unit tests covering filtering and isolation helpers
|
||||
|
||||
### Changed
|
||||
- Default `searchThreshold` remains at 0.5, with client-side filtering as a safety net
|
||||
- Extraction window expanded from last 10 → last 20 messages for richer context
|
||||
- Rewritten custom extraction instructions: conciseness, outcome-over-intent, deduplication guidance, language preservation
|
||||
- **Refactored** monolithic `index.ts` (1772 lines) into 6 focused modules: `types.ts`, `providers.ts`, `config.ts`, `filtering.ts`, `isolation.ts`, `index.ts`
|
||||
|
||||
### Fixed
|
||||
- **README image on npmjs.com**: Changed architecture diagram from relative path to absolute GitHub URL so it renders correctly on the npm registry
|
||||
|
||||
|
||||
+37
-8
@@ -12,10 +12,19 @@ Your agent forgets everything between sessions. This plugin fixes that. It watch
|
||||
|
||||
**Auto-Recall** — Before the agent responds, the plugin searches Mem0 for memories that match the current message and injects them into context.
|
||||
|
||||
**Auto-Capture** — After the agent responds, the plugin sends the exchange to Mem0. Mem0 decides what's worth keeping — new facts get stored, stale ones updated, duplicates merged.
|
||||
**Auto-Capture** — After the agent responds, the plugin filters the conversation through a noise-removal pipeline, then sends the cleaned exchange to Mem0. Mem0 decides what's worth keeping — new facts get stored, stale ones updated, duplicates merged.
|
||||
|
||||
Both run silently. No prompting, no configuration, no manual calls.
|
||||
|
||||
### Message filtering
|
||||
|
||||
Before extraction, messages pass through a multi-stage filtering pipeline:
|
||||
|
||||
1. **Noise detection** — Drops entire messages that are system noise: heartbeats (`HEARTBEAT_OK`, `NO_REPLY`), timestamps, single-word acknowledgments (`ok`, `sure`, `done`), system routing metadata, and compaction audit logs.
|
||||
2. **Generic assistant detection** — Drops short assistant messages that are boilerplate acknowledgments with no extractable facts (e.g. "I see you've shared an update. How can I help?").
|
||||
3. **Content stripping** — Removes embedded noise fragments (media boilerplate, routing metadata, compaction blocks) from otherwise useful messages.
|
||||
4. **Truncation** — Caps messages at 2000 characters to avoid sending excessive context.
|
||||
|
||||
### Short-term vs long-term memory
|
||||
|
||||
Memories are organized into two scopes:
|
||||
@@ -40,6 +49,13 @@ In multi-agent setups, each agent automatically gets its own memory namespace. S
|
||||
- If the key matches `agent:<name>:<uuid>`, memories are stored under `userId:agent:<name>`
|
||||
- Different agents never see each other's memories unless explicitly queried
|
||||
|
||||
**Subagent handling:**
|
||||
|
||||
Ephemeral subagents (session keys like `agent:main:subagent:<uuid>`) are handled specially:
|
||||
- **Recall** is routed to the parent (main user) namespace — subagents get the user's long-term context instead of searching their empty ephemeral namespace
|
||||
- **Capture** is skipped entirely — the main agent's `agent_end` hook captures the consolidated result including subagent output, preventing orphaned memories
|
||||
- A **subagent-specific preamble** is used: "You are a subagent — use these memories for context but do not assume you are this user"
|
||||
|
||||
**Explicit cross-agent queries:**
|
||||
|
||||
All memory tools (`memory_search`, `memory_store`, `memory_list`, `memory_forget`) accept an optional `agentId` parameter to query another agent's namespace:
|
||||
@@ -48,7 +64,17 @@ All memory tools (`memory_search`, `memory_store`, `memory_list`, `memory_forget
|
||||
memory_search({ query: "user's tech stack", agentId: "researcher" })
|
||||
```
|
||||
|
||||
Resolution priority: explicit `agentId` > explicit `userId` > session-derived > configured default.
|
||||
The `agentId` is always namespaced under the configured `userId` (e.g. `agentId: "researcher"` → `utkarsh:agent:researcher`), so it cannot be used to access other users' namespaces.
|
||||
|
||||
### Concurrency safety
|
||||
|
||||
Lifecycle hooks (`before_agent_start`, `agent_end`) use `ctx.sessionKey` directly from the event context rather than shared mutable state. This prevents race conditions when multiple sessions run concurrently (e.g. multiple Telegram users chatting simultaneously).
|
||||
|
||||
Tools still read from a best-effort `currentSessionId` variable (since tools don't receive `ctx`), but hooks — where the critical recall and capture logic runs — are fully concurrency-safe.
|
||||
|
||||
### Non-interactive trigger filtering
|
||||
|
||||
The plugin automatically skips recall and capture for non-interactive triggers: `cron`, `heartbeat`, `automation`, and `schedule`. Detection works via both `ctx.trigger` and session key patterns (`:cron:`, `:heartbeat:`). This prevents system-generated noise from polluting long-term memory.
|
||||
|
||||
## Setup
|
||||
|
||||
@@ -121,10 +147,10 @@ The agent gets five tools it can call during conversations:
|
||||
|
||||
| Tool | Description |
|
||||
|------|-------------|
|
||||
| `memory_search` | Search memories by natural language. Optional `agentId` to scope to a specific agent. |
|
||||
| `memory_list` | List all stored memories for a user. Optional `agentId` to scope to a specific agent. |
|
||||
| `memory_store` | Explicitly save a fact. Optional `agentId` to store under a specific agent's namespace. |
|
||||
| `memory_get` | Retrieve a memory by ID |
|
||||
| `memory_search` | Search memories by natural language. Optional `agentId` to scope to a specific agent, `scope` to filter by session/long-term. |
|
||||
| `memory_list` | List all stored memories. Optional `agentId` to scope to a specific agent, `scope` to filter. |
|
||||
| `memory_store` | Explicitly save a fact. Optional `agentId` to store under a specific agent's namespace, `longTerm` to choose scope. |
|
||||
| `memory_get` | Retrieve a memory by ID. |
|
||||
| `memory_forget` | Delete by ID or by query. Optional `agentId` to scope deletion to a specific agent. |
|
||||
|
||||
## CLI
|
||||
@@ -160,7 +186,7 @@ openclaw mem0 stats --agent researcher
|
||||
| `autoRecall` | `boolean` | `true` | Inject memories before each turn |
|
||||
| `autoCapture` | `boolean` | `true` | Store facts after each turn |
|
||||
| `topK` | `number` | `5` | Max memories per recall |
|
||||
| `searchThreshold` | `number` | `0.3` | Min similarity (0–1) |
|
||||
| `searchThreshold` | `number` | `0.5` | Min similarity (0–1) |
|
||||
|
||||
### Platform mode
|
||||
|
||||
@@ -170,7 +196,7 @@ openclaw mem0 stats --agent researcher
|
||||
| `orgId` | `string` | — | Organization ID |
|
||||
| `projectId` | `string` | — | Project ID |
|
||||
| `enableGraph` | `boolean` | `false` | Entity graph for relationships |
|
||||
| `customInstructions` | `string` | *(built-in)* | Extraction rules — what to store, how to format |
|
||||
| `customInstructions` | `string` | *(built-in)* | Extraction rules — what to store, how to format. Built-in instructions include temporal anchoring, conciseness, outcome-over-intent, deduplication, and language preservation guidelines. |
|
||||
| `customCategories` | `object` | *(12 defaults)* | Category name → description map for tagging |
|
||||
|
||||
### Open-source mode
|
||||
@@ -187,9 +213,12 @@ Works with zero extra config. The `oss` block lets you swap out any component:
|
||||
| `oss.llm.provider` | `string` | `"openai"` | LLM provider (`"openai"`, `"anthropic"`, `"ollama"`, `"lmstudio"`, etc.) |
|
||||
| `oss.llm.config` | `object` | — | Provider config: `apiKey`, `model`, `baseURL`, `temperature` |
|
||||
| `oss.historyDbPath` | `string` | — | SQLite path for memory edit history |
|
||||
| `oss.disableHistory` | `boolean` | `false` | Skip history DB initialization (useful when native SQLite bindings fail) |
|
||||
|
||||
Everything inside `oss` is optional — defaults use OpenAI embeddings (`text-embedding-3-small`), in-memory vector store, and OpenAI LLM. Override only what you need.
|
||||
|
||||
> **SQLite resilience:** If the history DB fails to initialize (e.g. native binding resolution under jiti), the plugin automatically retries with history disabled. Core memory operations (add, search, get, delete) work without the history DB.
|
||||
|
||||
## License
|
||||
|
||||
Apache 2.0
|
||||
|
||||
@@ -0,0 +1,243 @@
|
||||
/**
|
||||
* Configuration parsing, env var resolution, and default instructions/categories.
|
||||
*/
|
||||
|
||||
import type { Mem0Config, Mem0Mode } from "./types.ts";
|
||||
|
||||
// ============================================================================
|
||||
// Env Var Resolution
|
||||
// ============================================================================
|
||||
|
||||
function resolveEnvVars(value: string): string {
|
||||
return value.replace(/\$\{([^}]+)\}/g, (_, envVar) => {
|
||||
const envValue = process.env[envVar];
|
||||
if (!envValue) {
|
||||
throw new Error(`Environment variable ${envVar} is not set`);
|
||||
}
|
||||
return envValue;
|
||||
});
|
||||
}
|
||||
|
||||
function resolveEnvVarsDeep(obj: Record<string, unknown>): Record<string, unknown> {
|
||||
const result: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(obj)) {
|
||||
if (typeof value === "string") {
|
||||
result[key] = resolveEnvVars(value);
|
||||
} else if (value && typeof value === "object" && !Array.isArray(value)) {
|
||||
result[key] = resolveEnvVarsDeep(value as Record<string, unknown>);
|
||||
} else {
|
||||
result[key] = value;
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Default Custom Instructions & Categories
|
||||
// ============================================================================
|
||||
|
||||
export const DEFAULT_CUSTOM_INSTRUCTIONS = `Your Task: Extract durable, actionable facts from conversations between a user and an AI assistant. Only store information that would be useful to an agent in a FUTURE session, days or weeks later.
|
||||
|
||||
Before storing any fact, ask: "Would a new agent — with no prior context — benefit from knowing this?" If the answer is no, do not store it.
|
||||
|
||||
Information to Extract (in priority order):
|
||||
|
||||
1. Configuration & System State Changes:
|
||||
- Tools/services configured, installed, or removed (with versions/dates)
|
||||
- Model assignments for agents, API keys configured (NEVER the key itself — see Exclude)
|
||||
- Cron schedules, automation pipelines, deployment configurations
|
||||
- Architecture decisions (agent hierarchy, system design, deployment strategy)
|
||||
- Specific identifiers: file paths, sheet IDs, channel IDs, user IDs, folder IDs
|
||||
|
||||
2. Standing Rules & Policies:
|
||||
- Explicit user directives about behavior ("never create accounts without consent")
|
||||
- Workflow policies ("each agent must review model selection before completing a task")
|
||||
- Security constraints, permission boundaries, access patterns
|
||||
|
||||
3. Identity & Demographics:
|
||||
- Name, location, timezone, language preferences
|
||||
- Occupation, employer, job role, industry
|
||||
|
||||
4. Preferences & Opinions:
|
||||
- Communication style preferences
|
||||
- Tool and technology preferences (with specifics: versions, configs)
|
||||
- Strong opinions or values explicitly stated
|
||||
- The WHY behind preferences when stated
|
||||
|
||||
5. Goals, Projects & Milestones:
|
||||
- Active projects (name, description, current status)
|
||||
- Completed setup milestones ("ElevenLabs fully configured as of 2026-02-20")
|
||||
- Deadlines, roadmaps, and progress tracking
|
||||
- Problems actively being solved
|
||||
|
||||
6. Technical Context:
|
||||
- Tech stack, tools, development environment
|
||||
- Agent ecosystem structure (names, roles, relationships)
|
||||
- Skill levels in different areas
|
||||
|
||||
7. Relationships & People:
|
||||
- Names and roles of people mentioned (colleagues, family, clients)
|
||||
- Team structure, key contacts
|
||||
|
||||
8. Decisions & Lessons:
|
||||
- Important decisions made and their reasoning
|
||||
- Lessons learned, strategies that worked or failed
|
||||
|
||||
Guidelines:
|
||||
|
||||
TEMPORAL ANCHORING (critical):
|
||||
- ALWAYS include temporal context for time-sensitive facts using "As of YYYY-MM-DD, ..."
|
||||
- Extract dates from message timestamps, dates mentioned in the text, or the system-provided current date
|
||||
- If no date is available, note "date unknown" rather than omitting temporal context
|
||||
- Examples: "As of 2026-02-20, ElevenLabs setup is complete" NOT "ElevenLabs setup is complete"
|
||||
|
||||
CONCISENESS:
|
||||
- Use third person ("User prefers..." not "I prefer...")
|
||||
- Keep related facts together in a single memory to preserve context
|
||||
- "User's Tailscale machine 'mac' (IP 100.71.135.41) is configured under beau@rizedigital.io (as of 2026-02-20)"
|
||||
- NOT a paragraph retelling the whole conversation
|
||||
|
||||
OUTCOMES OVER INTENT:
|
||||
- When an assistant message summarizes completed work, extract the durable OUTCOMES
|
||||
- "Call scripts sheet (ID: 146Qbb...) was updated with truth-based templates" NOT "User wants to update call scripts"
|
||||
- Extract what WAS DONE, not what was requested
|
||||
|
||||
DEDUPLICATION:
|
||||
- Before creating a new memory, check if a substantially similar fact already exists
|
||||
- If so, UPDATE the existing memory with any new details rather than creating a duplicate
|
||||
|
||||
LANGUAGE:
|
||||
- ALWAYS preserve the original language of the conversation
|
||||
- If the user speaks Spanish, store the memory in Spanish; do not translate
|
||||
|
||||
Exclude (NEVER store):
|
||||
- Passwords, API keys, tokens, secrets, or any credentials — even if shared in conversation. Instead store: "Tavily API key was configured and saved to .env (as of 2026-02-20)"
|
||||
- One-time commands or instructions ("stop the script", "continue where you left off")
|
||||
- Acknowledgments or emotional reactions ("ok", "sounds good", "you're right", "sir")
|
||||
- Transient UI/navigation states ("user is in the admin panel", "relay is attached")
|
||||
- Ephemeral process status ("download at 50%", "daemon not running", "still syncing")
|
||||
- Cron heartbeat outputs, NO_REPLY responses, compaction flush directives
|
||||
- System routing metadata (message IDs, sender IDs, channel routing info)
|
||||
- Generic small talk with no informational content
|
||||
- Raw code snippets (capture the intent/decision, not the code itself)
|
||||
- Information the user explicitly asks not to remember`;
|
||||
|
||||
export const DEFAULT_CUSTOM_CATEGORIES: Record<string, string> = {
|
||||
identity:
|
||||
"Personal identity information: name, age, location, timezone, occupation, employer, education, demographics",
|
||||
preferences:
|
||||
"Explicitly stated likes, dislikes, preferences, opinions, and values across any domain",
|
||||
goals:
|
||||
"Current and future goals, aspirations, objectives, targets the user is working toward",
|
||||
projects:
|
||||
"Specific projects, initiatives, or endeavors the user is working on, including status and details",
|
||||
technical:
|
||||
"Technical skills, tools, tech stack, development environment, programming languages, frameworks",
|
||||
decisions:
|
||||
"Important decisions made, reasoning behind choices, strategy changes, and their outcomes",
|
||||
relationships:
|
||||
"People mentioned by the user: colleagues, family, friends, their roles and relevance",
|
||||
routines:
|
||||
"Daily habits, work patterns, schedules, productivity routines, health and wellness habits",
|
||||
life_events:
|
||||
"Significant life events, milestones, transitions, upcoming plans and changes",
|
||||
lessons:
|
||||
"Lessons learned, insights gained, mistakes acknowledged, changed opinions or beliefs",
|
||||
work:
|
||||
"Work-related context: job responsibilities, workplace dynamics, career progression, professional challenges",
|
||||
health:
|
||||
"Health-related information voluntarily shared: conditions, medications, fitness, wellness goals",
|
||||
};
|
||||
|
||||
// ============================================================================
|
||||
// Config Schema
|
||||
// ============================================================================
|
||||
|
||||
const ALLOWED_KEYS = [
|
||||
"mode",
|
||||
"apiKey",
|
||||
"userId",
|
||||
"orgId",
|
||||
"projectId",
|
||||
"autoCapture",
|
||||
"autoRecall",
|
||||
"customInstructions",
|
||||
"customCategories",
|
||||
"customPrompt",
|
||||
"enableGraph",
|
||||
"searchThreshold",
|
||||
"topK",
|
||||
"oss",
|
||||
];
|
||||
|
||||
function assertAllowedKeys(
|
||||
value: Record<string, unknown>,
|
||||
allowed: string[],
|
||||
label: string,
|
||||
) {
|
||||
const unknown = Object.keys(value).filter((key) => !allowed.includes(key));
|
||||
if (unknown.length === 0) return;
|
||||
throw new Error(`${label} has unknown keys: ${unknown.join(", ")}`);
|
||||
}
|
||||
|
||||
export const mem0ConfigSchema = {
|
||||
parse(value: unknown): Mem0Config {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
||||
throw new Error("openclaw-mem0 config required");
|
||||
}
|
||||
const cfg = value as Record<string, unknown>;
|
||||
assertAllowedKeys(cfg, ALLOWED_KEYS, "openclaw-mem0 config");
|
||||
|
||||
// Accept both "open-source" and legacy "oss" as open-source mode; everything else is platform
|
||||
const mode: Mem0Mode =
|
||||
cfg.mode === "oss" || cfg.mode === "open-source" ? "open-source" : "platform";
|
||||
|
||||
// Platform mode requires apiKey
|
||||
if (mode === "platform") {
|
||||
if (typeof cfg.apiKey !== "string" || !cfg.apiKey) {
|
||||
throw new Error(
|
||||
"apiKey is required for platform mode (set mode: \"open-source\" for self-hosted)",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Resolve env vars in oss config
|
||||
let ossConfig: Mem0Config["oss"];
|
||||
if (cfg.oss && typeof cfg.oss === "object" && !Array.isArray(cfg.oss)) {
|
||||
ossConfig = resolveEnvVarsDeep(
|
||||
cfg.oss as Record<string, unknown>,
|
||||
) as unknown as Mem0Config["oss"];
|
||||
}
|
||||
|
||||
return {
|
||||
mode,
|
||||
apiKey:
|
||||
typeof cfg.apiKey === "string" ? resolveEnvVars(cfg.apiKey) : undefined,
|
||||
userId:
|
||||
typeof cfg.userId === "string" && cfg.userId ? cfg.userId : "default",
|
||||
orgId: typeof cfg.orgId === "string" ? cfg.orgId : undefined,
|
||||
projectId: typeof cfg.projectId === "string" ? cfg.projectId : undefined,
|
||||
autoCapture: cfg.autoCapture !== false,
|
||||
autoRecall: cfg.autoRecall !== false,
|
||||
customInstructions:
|
||||
typeof cfg.customInstructions === "string"
|
||||
? cfg.customInstructions
|
||||
: DEFAULT_CUSTOM_INSTRUCTIONS,
|
||||
customCategories:
|
||||
cfg.customCategories &&
|
||||
typeof cfg.customCategories === "object" &&
|
||||
!Array.isArray(cfg.customCategories)
|
||||
? (cfg.customCategories as Record<string, string>)
|
||||
: DEFAULT_CUSTOM_CATEGORIES,
|
||||
customPrompt:
|
||||
typeof cfg.customPrompt === "string"
|
||||
? cfg.customPrompt
|
||||
: DEFAULT_CUSTOM_INSTRUCTIONS,
|
||||
enableGraph: cfg.enableGraph === true,
|
||||
searchThreshold:
|
||||
typeof cfg.searchThreshold === "number" ? cfg.searchThreshold : 0.5,
|
||||
topK: typeof cfg.topK === "number" ? cfg.topK : 5,
|
||||
oss: ossConfig,
|
||||
};
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,115 @@
|
||||
/**
|
||||
* Pre-extraction message filtering: noise detection, content stripping,
|
||||
* generic assistant detection, truncation, and deduplication.
|
||||
*/
|
||||
|
||||
import type { MemoryItem } from "./types.ts";
|
||||
|
||||
// ============================================================================
|
||||
// Noise Detection
|
||||
// ============================================================================
|
||||
|
||||
/** Patterns that indicate an entire message is noise and should be dropped. */
|
||||
const NOISE_MESSAGE_PATTERNS: RegExp[] = [
|
||||
/^(HEARTBEAT_OK|NO_REPLY)$/i,
|
||||
/^Current time:.*\d{4}/,
|
||||
/^Pre-compaction memory flush/i,
|
||||
/^(ok|yes|no|sir|sure|thanks|done|good|nice|cool|got it|it's on|continue)$/i,
|
||||
/^System: \[.*\] (Slack message edited|Gateway restart|Exec (failed|completed))/,
|
||||
/^System: \[.*\] ⚠️ Post-Compaction Audit:/,
|
||||
];
|
||||
|
||||
/** Content fragments that should be stripped from otherwise-valid messages. */
|
||||
const NOISE_CONTENT_PATTERNS: Array<{ pattern: RegExp; replacement: string }> = [
|
||||
{ pattern: /Conversation info \(untrusted metadata\):\s*```json\s*\{[\s\S]*?\}\s*```/g, replacement: "" },
|
||||
{ pattern: /\[media attached:.*?\]/g, replacement: "" },
|
||||
{ pattern: /To send an image back, prefer the message tool[\s\S]*?Keep caption in the text body\./g, replacement: "" },
|
||||
{ pattern: /System: \[\d{4}-\d{2}-\d{2}.*?\] ⚠️ Post-Compaction Audit:[\s\S]*?after memory compaction\./g, replacement: "" },
|
||||
{ pattern: /Replied message \(untrusted, for context\):\s*```json[\s\S]*?```/g, replacement: "" },
|
||||
];
|
||||
|
||||
const MAX_MESSAGE_LENGTH = 2000;
|
||||
|
||||
/**
|
||||
* Patterns indicating an assistant message is a generic acknowledgment with
|
||||
* no extractable facts. These are produced when the agent receives a
|
||||
* transcript dump or forwarded message and responds with a boilerplate reply.
|
||||
*/
|
||||
const GENERIC_ASSISTANT_PATTERNS: RegExp[] = [
|
||||
/^(I see you'?ve shared|Thanks for sharing|Got it[.!]?\s*(I see|Let me|How can)|I understand[.!]?\s*(How can|Is there|Would you))/i,
|
||||
/^(How can I help|Is there anything|Would you like me to|Let me know (if|how|what))/i,
|
||||
/^(I('?ll| will) (help|assist|look into|review|take a look))/i,
|
||||
/^(Sure[.!]?\s*(How|What|Is)|Understood[.!]?\s*(How|What|Is))/i,
|
||||
/^(That('?s| is) (noted|understood|clear))/i,
|
||||
];
|
||||
|
||||
// ============================================================================
|
||||
// Public Functions
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Check whether a message's content is entirely noise (cron heartbeats,
|
||||
* single-word acknowledgments, system routing metadata, etc.).
|
||||
*/
|
||||
export function isNoiseMessage(content: string): boolean {
|
||||
const trimmed = content.trim();
|
||||
if (!trimmed) return true;
|
||||
return NOISE_MESSAGE_PATTERNS.some((p) => p.test(trimmed));
|
||||
}
|
||||
|
||||
/**
|
||||
* Check whether an assistant message is a generic acknowledgment with no
|
||||
* extractable facts (e.g. "I see you've shared an update. How can I help?").
|
||||
* Only applies to short assistant messages — longer responses likely contain
|
||||
* substantive content even if they start with a generic opener.
|
||||
*/
|
||||
export function isGenericAssistantMessage(content: string): boolean {
|
||||
const trimmed = content.trim();
|
||||
// Only flag short messages — longer ones likely have substance after the opener
|
||||
if (trimmed.length > 300) return false;
|
||||
return GENERIC_ASSISTANT_PATTERNS.some((p) => p.test(trimmed));
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove embedded noise fragments (routing metadata, media boilerplate,
|
||||
* compaction audit blocks) from a message while preserving the useful content.
|
||||
*/
|
||||
export function stripNoiseFromContent(content: string): string {
|
||||
let cleaned = content;
|
||||
for (const { pattern, replacement } of NOISE_CONTENT_PATTERNS) {
|
||||
cleaned = cleaned.replace(pattern, replacement);
|
||||
}
|
||||
// Collapse excessive whitespace left behind after stripping
|
||||
cleaned = cleaned.replace(/\n{3,}/g, "\n\n").trim();
|
||||
return cleaned;
|
||||
}
|
||||
|
||||
/**
|
||||
* Truncate a message to `MAX_MESSAGE_LENGTH` characters, preserving the
|
||||
* opening (which typically contains the summary/conclusion) and appending
|
||||
* a truncation marker so the extraction model knows content was cut.
|
||||
*/
|
||||
function truncateMessage(content: string): string {
|
||||
if (content.length <= MAX_MESSAGE_LENGTH) return content;
|
||||
return content.slice(0, MAX_MESSAGE_LENGTH) + "\n[...truncated]";
|
||||
}
|
||||
|
||||
/**
|
||||
* Full pre-extraction pipeline: drop noise messages, strip noise fragments,
|
||||
* and truncate remaining messages to a reasonable length.
|
||||
*/
|
||||
export function filterMessagesForExtraction(
|
||||
messages: Array<{ role: string; content: string }>,
|
||||
): Array<{ role: string; content: string }> {
|
||||
const filtered: Array<{ role: string; content: string }> = [];
|
||||
for (const msg of messages) {
|
||||
if (isNoiseMessage(msg.content)) continue;
|
||||
// Drop generic assistant acknowledgments that contain no facts
|
||||
if (msg.role === "assistant" && isGenericAssistantMessage(msg.content)) continue;
|
||||
const cleaned = stripNoiseFromContent(msg.content);
|
||||
if (!cleaned) continue;
|
||||
filtered.push({ role: msg.role, content: truncateMessage(cleaned) });
|
||||
}
|
||||
return filtered;
|
||||
}
|
||||
|
||||
+329
-5
@@ -1,8 +1,6 @@
|
||||
/**
|
||||
* Regression tests for per-agent memory isolation helpers.
|
||||
*
|
||||
* Addresses review feedback: targeted coverage for auth/session state,
|
||||
* malformed input, and the resolveUserId priority chain.
|
||||
* Regression tests for per-agent memory isolation helpers and
|
||||
* message filtering logic.
|
||||
*/
|
||||
import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
@@ -10,16 +8,33 @@ import {
|
||||
effectiveUserId,
|
||||
agentUserId,
|
||||
resolveUserId,
|
||||
isNonInteractiveTrigger,
|
||||
isSubagentSession,
|
||||
isNoiseMessage,
|
||||
isGenericAssistantMessage,
|
||||
stripNoiseFromContent,
|
||||
filterMessagesForExtraction,
|
||||
} from "./index.ts";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// extractAgentId
|
||||
// ---------------------------------------------------------------------------
|
||||
describe("extractAgentId", () => {
|
||||
it("returns agentId from a well-formed session key", () => {
|
||||
it("returns agentId from a named agent session key", () => {
|
||||
expect(extractAgentId("agent:researcher:550e8400-e29b")).toBe("researcher");
|
||||
});
|
||||
|
||||
it("returns subagent namespace from subagent session key", () => {
|
||||
// OpenClaw subagent format: agent:main:subagent:<uuid>
|
||||
expect(extractAgentId("agent:main:subagent:3b85177f-69e0-412d-8ecd-fbe542f362ce")).toBe(
|
||||
"subagent-3b85177f-69e0-412d-8ecd-fbe542f362ce",
|
||||
);
|
||||
});
|
||||
|
||||
it("returns undefined for the main agent session (agent:main:main)", () => {
|
||||
expect(extractAgentId("agent:main:main")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("returns undefined for the 'main' sentinel", () => {
|
||||
expect(extractAgentId("agent:main:abc-123")).toBeUndefined();
|
||||
});
|
||||
@@ -163,3 +178,312 @@ describe("multi-agent isolation", () => {
|
||||
expect(mainId).toBe(base);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// isNonInteractiveTrigger
|
||||
// ---------------------------------------------------------------------------
|
||||
describe("isNonInteractiveTrigger", () => {
|
||||
it("returns true for cron trigger", () => {
|
||||
expect(isNonInteractiveTrigger("cron", undefined)).toBe(true);
|
||||
});
|
||||
|
||||
it("returns true for heartbeat trigger", () => {
|
||||
expect(isNonInteractiveTrigger("heartbeat", undefined)).toBe(true);
|
||||
});
|
||||
|
||||
it("returns true for automation trigger", () => {
|
||||
expect(isNonInteractiveTrigger("automation", undefined)).toBe(true);
|
||||
});
|
||||
|
||||
it("returns true for schedule trigger", () => {
|
||||
expect(isNonInteractiveTrigger("schedule", undefined)).toBe(true);
|
||||
});
|
||||
|
||||
it("is case-insensitive for trigger", () => {
|
||||
expect(isNonInteractiveTrigger("CRON", undefined)).toBe(true);
|
||||
expect(isNonInteractiveTrigger("Heartbeat", undefined)).toBe(true);
|
||||
});
|
||||
|
||||
it("returns false for user-initiated triggers", () => {
|
||||
expect(isNonInteractiveTrigger("user", undefined)).toBe(false);
|
||||
expect(isNonInteractiveTrigger("webchat", undefined)).toBe(false);
|
||||
expect(isNonInteractiveTrigger("telegram", undefined)).toBe(false);
|
||||
});
|
||||
|
||||
it("returns false when trigger is undefined and session key is normal", () => {
|
||||
expect(isNonInteractiveTrigger(undefined, "agent:main:main")).toBe(false);
|
||||
});
|
||||
|
||||
it("detects cron from session key as fallback", () => {
|
||||
expect(isNonInteractiveTrigger(undefined, "agent:main:cron:c85abdb2-d900-4cd8-8601-9dd960c560c9")).toBe(true);
|
||||
});
|
||||
|
||||
it("detects heartbeat from session key as fallback", () => {
|
||||
expect(isNonInteractiveTrigger(undefined, "agent:main:heartbeat:abc123")).toBe(true);
|
||||
});
|
||||
|
||||
it("returns false when both trigger and sessionKey are undefined", () => {
|
||||
expect(isNonInteractiveTrigger(undefined, undefined)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// isSubagentSession
|
||||
// ---------------------------------------------------------------------------
|
||||
describe("isSubagentSession", () => {
|
||||
it("returns true for subagent session keys", () => {
|
||||
expect(isSubagentSession("agent:main:subagent:3b85177f-69e0-412d-8ecd-fbe542f362ce")).toBe(true);
|
||||
});
|
||||
|
||||
it("returns false for main agent session", () => {
|
||||
expect(isSubagentSession("agent:main:main")).toBe(false);
|
||||
});
|
||||
|
||||
it("returns false for named agent session", () => {
|
||||
expect(isSubagentSession("agent:researcher:550e8400-e29b")).toBe(false);
|
||||
});
|
||||
|
||||
it("returns false for undefined", () => {
|
||||
expect(isSubagentSession(undefined)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// isNoiseMessage
|
||||
// ---------------------------------------------------------------------------
|
||||
describe("isNoiseMessage", () => {
|
||||
it("detects HEARTBEAT_OK", () => {
|
||||
expect(isNoiseMessage("HEARTBEAT_OK")).toBe(true);
|
||||
expect(isNoiseMessage("heartbeat_ok")).toBe(true);
|
||||
});
|
||||
|
||||
it("detects NO_REPLY", () => {
|
||||
expect(isNoiseMessage("NO_REPLY")).toBe(true);
|
||||
});
|
||||
|
||||
it("detects current-time stamps", () => {
|
||||
expect(
|
||||
isNoiseMessage("Current time: Friday, February 20th, 2026 — 3:58 AM (America/New_York)"),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("detects single-word acknowledgments", () => {
|
||||
for (const word of ["ok", "yes", "sir", "done", "cool", "Got it", "it's on"]) {
|
||||
expect(isNoiseMessage(word)).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("detects system routing messages", () => {
|
||||
expect(
|
||||
isNoiseMessage("System: [2026-02-19 19:51:31 PST] Slack message edited in #D0AFV2LDGDS."),
|
||||
).toBe(true);
|
||||
expect(
|
||||
isNoiseMessage("System: [2026-02-19 22:15:42 PST] Exec failed (gentle-b, signal 15)"),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("detects compaction audit messages", () => {
|
||||
expect(
|
||||
isNoiseMessage(
|
||||
"System: [2026-02-20 16:12:04 EST] ⚠️ Post-Compaction Audit: The following required startup files were not read",
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("preserves real content", () => {
|
||||
expect(isNoiseMessage("Beau runs Rize Digital LLC")).toBe(false);
|
||||
expect(isNoiseMessage("Can you check the lovable discord?")).toBe(false);
|
||||
expect(isNoiseMessage("I approve the Tailscale installation")).toBe(false);
|
||||
});
|
||||
|
||||
it("treats empty/whitespace as noise", () => {
|
||||
expect(isNoiseMessage("")).toBe(true);
|
||||
expect(isNoiseMessage(" ")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// isGenericAssistantMessage
|
||||
// ---------------------------------------------------------------------------
|
||||
describe("isGenericAssistantMessage", () => {
|
||||
it("detects 'I see you've shared' openers", () => {
|
||||
expect(isGenericAssistantMessage("I see you've shared an update. How can I help?")).toBe(true);
|
||||
expect(isGenericAssistantMessage("I see you've shared a summary of the Atlas configuration update. Is there anything specific you'd like me to help with?")).toBe(true);
|
||||
});
|
||||
|
||||
it("detects 'Thanks for sharing' openers", () => {
|
||||
expect(isGenericAssistantMessage("Thanks for sharing that update! Would you like me to review the changes?")).toBe(true);
|
||||
});
|
||||
|
||||
it("detects 'How can I help' standalone", () => {
|
||||
expect(isGenericAssistantMessage("How can I help you with this?")).toBe(true);
|
||||
});
|
||||
|
||||
it("detects 'Got it' + follow-up", () => {
|
||||
expect(isGenericAssistantMessage("Got it! How can I assist?")).toBe(true);
|
||||
expect(isGenericAssistantMessage("Got it. Let me know what you need.")).toBe(true);
|
||||
});
|
||||
|
||||
it("detects 'I'll help/review/look into'", () => {
|
||||
expect(isGenericAssistantMessage("I'll review that for you.")).toBe(true);
|
||||
expect(isGenericAssistantMessage("I'll look into this right away.")).toBe(true);
|
||||
});
|
||||
|
||||
it("preserves substantive assistant content", () => {
|
||||
expect(isGenericAssistantMessage("## What I Accomplished\n\nDeployed the API to production with Vercel.")).toBe(false);
|
||||
expect(isGenericAssistantMessage("The ElevenLabs SDK has been installed and configured. Voice skill is ready.")).toBe(false);
|
||||
expect(isGenericAssistantMessage("Updated the call scripts sheet with truth-based messaging templates.")).toBe(false);
|
||||
});
|
||||
|
||||
it("preserves long messages even with generic openers", () => {
|
||||
const longMsg = "I see you've shared an update. " + "Here are the detailed changes I made to the configuration. ".repeat(10);
|
||||
expect(isGenericAssistantMessage(longMsg)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// stripNoiseFromContent
|
||||
// ---------------------------------------------------------------------------
|
||||
describe("stripNoiseFromContent", () => {
|
||||
it("removes conversation metadata JSON blocks", () => {
|
||||
const input = `Conversation info (untrusted metadata):
|
||||
\`\`\`json
|
||||
{
|
||||
"message_id": "499",
|
||||
"sender": "6039555582"
|
||||
}
|
||||
\`\`\`
|
||||
|
||||
What models are you currently using?`;
|
||||
const result = stripNoiseFromContent(input);
|
||||
expect(result).toBe("What models are you currently using?");
|
||||
});
|
||||
|
||||
it("removes media attachment lines", () => {
|
||||
const input = "[media attached: /path/to/file.jpg (image/jpeg) | /path/to/file.jpg]\nActual question here";
|
||||
const result = stripNoiseFromContent(input);
|
||||
expect(result).toContain("Actual question here");
|
||||
expect(result).not.toContain("[media attached:");
|
||||
});
|
||||
|
||||
it("removes image sending boilerplate", () => {
|
||||
const input =
|
||||
"To send an image back, prefer the message tool (media/path/filePath). If you must inline, use MEDIA:https://example.com/image.jpg. Keep caption in the text body.\nReal content here";
|
||||
const result = stripNoiseFromContent(input);
|
||||
expect(result).toContain("Real content here");
|
||||
expect(result).not.toContain("prefer the message tool");
|
||||
});
|
||||
|
||||
it("preserves content when no noise is present", () => {
|
||||
const input = "User wants to deploy to production via Vercel.";
|
||||
expect(stripNoiseFromContent(input)).toBe(input);
|
||||
});
|
||||
|
||||
it("collapses excessive blank lines after stripping", () => {
|
||||
const input = "Line one\n\n\n\n\nLine two";
|
||||
expect(stripNoiseFromContent(input)).toBe("Line one\n\nLine two");
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// filterMessagesForExtraction
|
||||
// ---------------------------------------------------------------------------
|
||||
describe("filterMessagesForExtraction", () => {
|
||||
it("drops noise messages entirely", () => {
|
||||
const messages = [
|
||||
{ role: "user", content: "HEARTBEAT_OK" },
|
||||
{ role: "assistant", content: "Real response with durable facts." },
|
||||
{ role: "user", content: "ok" },
|
||||
];
|
||||
const result = filterMessagesForExtraction(messages);
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].content).toBe("Real response with durable facts.");
|
||||
});
|
||||
|
||||
it("strips noise fragments but keeps the rest", () => {
|
||||
const messages = [
|
||||
{
|
||||
role: "user",
|
||||
content: `Conversation info (untrusted metadata):
|
||||
\`\`\`json
|
||||
{
|
||||
"message_id": "123",
|
||||
"sender": "456"
|
||||
}
|
||||
\`\`\`
|
||||
|
||||
What is the deployment plan?`,
|
||||
},
|
||||
];
|
||||
const result = filterMessagesForExtraction(messages);
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].content).toBe("What is the deployment plan?");
|
||||
});
|
||||
|
||||
it("truncates long messages", () => {
|
||||
const longContent = "A".repeat(3000);
|
||||
const messages = [{ role: "assistant", content: longContent }];
|
||||
const result = filterMessagesForExtraction(messages);
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].content.length).toBeLessThan(2100);
|
||||
expect(result[0].content).toContain("[...truncated]");
|
||||
});
|
||||
|
||||
it("returns empty array when all messages are noise", () => {
|
||||
const messages = [
|
||||
{ role: "user", content: "NO_REPLY" },
|
||||
{ role: "user", content: "ok" },
|
||||
{ role: "user", content: "Current time: Friday, February 20th, 2026" },
|
||||
];
|
||||
expect(filterMessagesForExtraction(messages)).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("handles a realistic mixed payload", () => {
|
||||
const messages = [
|
||||
{ role: "user", content: "Pre-compaction memory flush. Store durable memories now." },
|
||||
{
|
||||
role: "assistant",
|
||||
content: "## What I Accomplished\n\nDeployed the API to production with Vercel.",
|
||||
},
|
||||
{ role: "user", content: "sir" },
|
||||
];
|
||||
const result = filterMessagesForExtraction(messages);
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].content).toContain("Deployed the API");
|
||||
});
|
||||
|
||||
it("drops generic assistant acknowledgments", () => {
|
||||
const messages = [
|
||||
{ role: "user", content: "[ASSISTANT]: Updated the Google Sheet with truth-based scripts." },
|
||||
{ role: "assistant", content: "I see you've shared an update. How can I help?" },
|
||||
];
|
||||
const result = filterMessagesForExtraction(messages);
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].role).toBe("user");
|
||||
expect(result[0].content).toContain("Google Sheet");
|
||||
});
|
||||
|
||||
it("returns only assistant messages when all user messages are noise", () => {
|
||||
// This scenario triggers the #2 guard: no user content remains
|
||||
const messages = [
|
||||
{ role: "user", content: "ok" },
|
||||
{ role: "user", content: "HEARTBEAT_OK" },
|
||||
{ role: "assistant", content: "I deployed the API to production." },
|
||||
];
|
||||
const result = filterMessagesForExtraction(messages);
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].role).toBe("assistant");
|
||||
// The capture hook checks: if no user messages remain, skip add()
|
||||
expect(result.some((m) => m.role === "user")).toBe(false);
|
||||
});
|
||||
|
||||
it("keeps substantive assistant messages even with generic opener", () => {
|
||||
const messages = [
|
||||
{ role: "user", content: "What did you do?" },
|
||||
{ role: "assistant", content: "I deployed the API to production and configured the webhook endpoints for Stripe integration." },
|
||||
];
|
||||
const result = filterMessagesForExtraction(messages);
|
||||
expect(result).toHaveLength(2);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
+992
-1388
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,101 @@
|
||||
/**
|
||||
* Per-agent memory isolation helpers.
|
||||
*
|
||||
* Multi-agent setups write/read from separate userId namespaces
|
||||
* automatically via sessionKey routing.
|
||||
*/
|
||||
|
||||
// ============================================================================
|
||||
// Trigger filtering — skip non-interactive sessions
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Triggers that should NOT run autocapture/autorecall.
|
||||
* These are system-initiated sessions (cron jobs, heartbeats, automation
|
||||
* pipelines) whose prompts would pollute the user's memory store.
|
||||
*/
|
||||
const SKIP_TRIGGERS = new Set(["cron", "heartbeat", "automation", "schedule"]);
|
||||
|
||||
/**
|
||||
* Returns true if the session trigger is non-interactive and memory
|
||||
* hooks should be skipped entirely.
|
||||
*
|
||||
* Also detects cron-style session keys (e.g. "agent:main:cron:<id>")
|
||||
* as a fallback when the trigger field is not set.
|
||||
*/
|
||||
export function isNonInteractiveTrigger(
|
||||
trigger: string | undefined,
|
||||
sessionKey: string | undefined,
|
||||
): boolean {
|
||||
if (trigger && SKIP_TRIGGERS.has(trigger.toLowerCase())) return true;
|
||||
|
||||
// Fallback: detect cron/heartbeat from the session key pattern
|
||||
if (sessionKey) {
|
||||
if (/:cron:/i.test(sessionKey) || /:heartbeat:/i.test(sessionKey)) return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns true if the session key indicates a subagent (ephemeral) session.
|
||||
* Subagent UUIDs are random per-spawn, so their namespaces are always empty
|
||||
* on recall and orphaned after capture.
|
||||
*/
|
||||
export function isSubagentSession(sessionKey: string | undefined): boolean {
|
||||
if (!sessionKey) return false;
|
||||
return /:subagent:/i.test(sessionKey);
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse an agent ID from a session key.
|
||||
*
|
||||
* OpenClaw session key formats:
|
||||
* - Main agent: "agent:main:main"
|
||||
* - Subagent: "agent:main:subagent:<uuid>"
|
||||
* - Named agent: "agent:<agentId>:<session>"
|
||||
*
|
||||
* Returns the subagent UUID for subagent sessions, the agentId for
|
||||
* non-"main" named agents, or undefined for the main agent session.
|
||||
*/
|
||||
export function extractAgentId(sessionKey: string | undefined): string | undefined {
|
||||
if (!sessionKey) return undefined;
|
||||
|
||||
// Check for subagent pattern: "agent:<parent>:subagent:<uuid>"
|
||||
const subagentMatch = sessionKey.match(/:subagent:([^:]+)$/);
|
||||
if (subagentMatch?.[1]) return `subagent-${subagentMatch[1]}`;
|
||||
|
||||
// Check for named agent pattern: "agent:<agentId>:<session>"
|
||||
const match = sessionKey.match(/^agent:([^:]+):/);
|
||||
const agentId = match?.[1];
|
||||
// "main" is the primary session — fall back to configured userId
|
||||
if (!agentId || agentId === "main") return undefined;
|
||||
return agentId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Derive the effective user_id from a session key, namespacing per-agent.
|
||||
* Falls back to baseUserId when the session is not agent-scoped.
|
||||
*/
|
||||
export function effectiveUserId(baseUserId: string, sessionKey?: string): string {
|
||||
const agentId = extractAgentId(sessionKey);
|
||||
return agentId ? `${baseUserId}:agent:${agentId}` : baseUserId;
|
||||
}
|
||||
|
||||
/** Build a user_id for an explicit agentId (e.g. from tool params). */
|
||||
export function agentUserId(baseUserId: string, agentId: string): string {
|
||||
return `${baseUserId}:agent:${agentId}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve user_id with priority: explicit agentId > explicit userId > session-derived > configured.
|
||||
*/
|
||||
export function resolveUserId(
|
||||
baseUserId: string,
|
||||
opts: { agentId?: string; userId?: string },
|
||||
currentSessionId?: string,
|
||||
): string {
|
||||
if (opts.agentId) return agentUserId(baseUserId, opts.agentId);
|
||||
if (opts.userId) return opts.userId;
|
||||
return effectiveUserId(baseUserId, currentSessionId);
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@mem0/openclaw-mem0",
|
||||
"version": "0.3.3",
|
||||
"version": "0.4.0",
|
||||
"type": "module",
|
||||
"description": "Mem0 memory backend for OpenClaw — platform or self-hosted open-source",
|
||||
"license": "Apache-2.0",
|
||||
@@ -29,7 +29,7 @@
|
||||
},
|
||||
"dependencies": {
|
||||
"@sinclair/typebox": "0.34.47",
|
||||
"mem0ai": "^2.2.1"
|
||||
"mem0ai": "^2.3.0"
|
||||
},
|
||||
"openclaw": {
|
||||
"extensions": [
|
||||
|
||||
Generated
+1
-1
@@ -12,7 +12,7 @@ importers:
|
||||
specifier: 0.34.47
|
||||
version: 0.34.47
|
||||
mem0ai:
|
||||
specifier: ^2.2.1
|
||||
specifier: ^2.3.0
|
||||
version: 2.4.0(@anthropic-ai/sdk@0.40.1)(@azure/identity@4.13.0)(@azure/search-documents@12.2.0)(@cloudflare/workers-types@4.20260313.1)(@google/genai@1.45.0)(@langchain/core@0.3.80(openai@4.104.0(ws@8.19.0)(zod@3.25.76)))(@mistralai/mistralai@1.15.1)(@qdrant/js-client-rest@1.13.0(typescript@5.9.3))(@supabase/supabase-js@2.99.1)(@types/jest@29.5.14)(@types/pg@8.11.0)(better-sqlite3@12.8.0)(cloudflare@4.5.0)(groq-sdk@0.3.0)(neo4j-driver@5.28.3)(ollama@0.5.18)(pg@8.11.3)(redis@4.7.1)(ws@8.19.0)
|
||||
devDependencies:
|
||||
'@types/node':
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
/**
|
||||
* Mem0 provider implementations: Platform (cloud) and OSS (self-hosted).
|
||||
*/
|
||||
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk";
|
||||
import type {
|
||||
Mem0Config,
|
||||
Mem0Provider,
|
||||
AddOptions,
|
||||
SearchOptions,
|
||||
ListOptions,
|
||||
MemoryItem,
|
||||
AddResult,
|
||||
} from "./types.ts";
|
||||
|
||||
// ============================================================================
|
||||
// Result Normalizers
|
||||
// ============================================================================
|
||||
|
||||
function normalizeMemoryItem(raw: any): MemoryItem {
|
||||
return {
|
||||
id: raw.id ?? raw.memory_id ?? "",
|
||||
memory: raw.memory ?? raw.text ?? raw.content ?? "",
|
||||
// Handle both platform (user_id, created_at) and OSS (userId, createdAt) field names
|
||||
user_id: raw.user_id ?? raw.userId,
|
||||
score: raw.score,
|
||||
categories: raw.categories,
|
||||
metadata: raw.metadata,
|
||||
created_at: raw.created_at ?? raw.createdAt,
|
||||
updated_at: raw.updated_at ?? raw.updatedAt,
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeSearchResults(raw: any): MemoryItem[] {
|
||||
// Platform API returns flat array, OSS returns { results: [...] }
|
||||
if (Array.isArray(raw)) return raw.map(normalizeMemoryItem);
|
||||
if (raw?.results && Array.isArray(raw.results))
|
||||
return raw.results.map(normalizeMemoryItem);
|
||||
return [];
|
||||
}
|
||||
|
||||
function normalizeAddResult(raw: any): AddResult {
|
||||
// Handle { results: [...] } shape (both platform and OSS)
|
||||
if (raw?.results && Array.isArray(raw.results)) {
|
||||
return {
|
||||
results: raw.results.map((r: any) => ({
|
||||
id: r.id ?? r.memory_id ?? "",
|
||||
memory: r.memory ?? r.text ?? "",
|
||||
// Platform API may return PENDING status (async processing)
|
||||
// OSS stores event in metadata.event
|
||||
event: r.event ?? r.metadata?.event ?? (r.status === "PENDING" ? "ADD" : "ADD"),
|
||||
})),
|
||||
};
|
||||
}
|
||||
// Platform API without output_format returns flat array
|
||||
if (Array.isArray(raw)) {
|
||||
return {
|
||||
results: raw.map((r: any) => ({
|
||||
id: r.id ?? r.memory_id ?? "",
|
||||
memory: r.memory ?? r.text ?? "",
|
||||
event: r.event ?? r.metadata?.event ?? (r.status === "PENDING" ? "ADD" : "ADD"),
|
||||
})),
|
||||
};
|
||||
}
|
||||
return { results: [] };
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Platform Provider (Mem0 Cloud)
|
||||
// ============================================================================
|
||||
|
||||
class PlatformProvider implements Mem0Provider {
|
||||
private client: any; // MemoryClient from mem0ai
|
||||
private initPromise: Promise<void> | null = null;
|
||||
|
||||
constructor(
|
||||
private readonly apiKey: string,
|
||||
private readonly orgId?: string,
|
||||
private readonly projectId?: string,
|
||||
) { }
|
||||
|
||||
private async ensureClient(): Promise<void> {
|
||||
if (this.client) return;
|
||||
if (this.initPromise) return this.initPromise;
|
||||
this.initPromise = this._init().catch((err) => {
|
||||
this.initPromise = null;
|
||||
throw err;
|
||||
});
|
||||
return this.initPromise;
|
||||
}
|
||||
|
||||
private async _init(): Promise<void> {
|
||||
const { default: MemoryClient } = await import("mem0ai");
|
||||
const opts: { apiKey: string; org_id?: string; project_id?: string } = { apiKey: this.apiKey };
|
||||
if (this.orgId) opts.org_id = this.orgId;
|
||||
if (this.projectId) opts.project_id = this.projectId;
|
||||
this.client = new MemoryClient(opts);
|
||||
}
|
||||
|
||||
async add(
|
||||
messages: Array<{ role: string; content: string }>,
|
||||
options: AddOptions,
|
||||
): Promise<AddResult> {
|
||||
await this.ensureClient();
|
||||
const opts: Record<string, unknown> = { user_id: options.user_id };
|
||||
if (options.run_id) opts.run_id = options.run_id;
|
||||
if (options.custom_instructions)
|
||||
opts.custom_instructions = options.custom_instructions;
|
||||
if (options.custom_categories)
|
||||
opts.custom_categories = options.custom_categories;
|
||||
if (options.enable_graph) opts.enable_graph = options.enable_graph;
|
||||
if (options.output_format) opts.output_format = options.output_format;
|
||||
if (options.source) opts.source = options.source;
|
||||
|
||||
const result = await this.client.add(messages, opts);
|
||||
return normalizeAddResult(result);
|
||||
}
|
||||
|
||||
async search(query: string, options: SearchOptions): Promise<MemoryItem[]> {
|
||||
await this.ensureClient();
|
||||
const filters: Record<string, unknown> = { user_id: options.user_id };
|
||||
if (options.run_id) filters.run_id = options.run_id;
|
||||
|
||||
const opts: Record<string, unknown> = {
|
||||
api_version: "v2",
|
||||
filters,
|
||||
};
|
||||
if (options.top_k != null) opts.top_k = options.top_k;
|
||||
if (options.threshold != null) opts.threshold = options.threshold;
|
||||
if (options.keyword_search != null) opts.keyword_search = options.keyword_search;
|
||||
if (options.reranking != null) opts.rerank = options.reranking;
|
||||
|
||||
const results = await this.client.search(query, opts);
|
||||
return normalizeSearchResults(results);
|
||||
}
|
||||
|
||||
async get(memoryId: string): Promise<MemoryItem> {
|
||||
await this.ensureClient();
|
||||
const result = await this.client.get(memoryId);
|
||||
return normalizeMemoryItem(result);
|
||||
}
|
||||
|
||||
async getAll(options: ListOptions): Promise<MemoryItem[]> {
|
||||
await this.ensureClient();
|
||||
const opts: Record<string, unknown> = { user_id: options.user_id };
|
||||
if (options.run_id) opts.run_id = options.run_id;
|
||||
if (options.page_size != null) opts.page_size = options.page_size;
|
||||
if (options.source) opts.source = options.source;
|
||||
|
||||
const results = await this.client.getAll(opts);
|
||||
if (Array.isArray(results)) return results.map(normalizeMemoryItem);
|
||||
// Some versions return { results: [...] }
|
||||
if (results?.results && Array.isArray(results.results))
|
||||
return results.results.map(normalizeMemoryItem);
|
||||
return [];
|
||||
}
|
||||
|
||||
async delete(memoryId: string): Promise<void> {
|
||||
await this.ensureClient();
|
||||
await this.client.delete(memoryId);
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Open-Source Provider (Self-hosted)
|
||||
// ============================================================================
|
||||
|
||||
class OSSProvider implements Mem0Provider {
|
||||
private memory: any; // Memory from mem0ai/oss
|
||||
private initPromise: Promise<void> | null = null;
|
||||
|
||||
constructor(
|
||||
private readonly ossConfig?: Mem0Config["oss"],
|
||||
private readonly customPrompt?: string,
|
||||
private readonly resolvePath?: (p: string) => string,
|
||||
) { }
|
||||
|
||||
private async ensureMemory(): Promise<void> {
|
||||
if (this.memory) return;
|
||||
if (this.initPromise) return this.initPromise;
|
||||
this.initPromise = this._init().catch((err) => {
|
||||
this.initPromise = null;
|
||||
throw err;
|
||||
});
|
||||
return this.initPromise;
|
||||
}
|
||||
|
||||
private async _init(): Promise<void> {
|
||||
const { Memory } = await import("mem0ai/oss");
|
||||
|
||||
const config: Record<string, unknown> = { version: "v1.1" };
|
||||
|
||||
if (this.ossConfig?.embedder) config.embedder = this.ossConfig.embedder;
|
||||
if (this.ossConfig?.vectorStore)
|
||||
config.vectorStore = this.ossConfig.vectorStore;
|
||||
if (this.ossConfig?.llm) config.llm = this.ossConfig.llm;
|
||||
|
||||
if (this.ossConfig?.historyDbPath) {
|
||||
const dbPath = this.resolvePath
|
||||
? this.resolvePath(this.ossConfig.historyDbPath)
|
||||
: this.ossConfig.historyDbPath;
|
||||
config.historyDbPath = dbPath;
|
||||
}
|
||||
|
||||
if (this.ossConfig?.disableHistory) {
|
||||
config.disableHistory = true;
|
||||
}
|
||||
|
||||
if (this.customPrompt) config.customPrompt = this.customPrompt;
|
||||
|
||||
try {
|
||||
this.memory = new Memory(config);
|
||||
} catch (err) {
|
||||
// If initialization fails (e.g. native SQLite binding resolution under
|
||||
// jiti), retry with history disabled — the history DB is the most common
|
||||
// source of native-binding failures and is not required for core
|
||||
// memory operations.
|
||||
if (!config.disableHistory) {
|
||||
console.warn(
|
||||
"[mem0] Memory initialization failed, retrying with history disabled:",
|
||||
err instanceof Error ? err.message : err,
|
||||
);
|
||||
config.disableHistory = true;
|
||||
this.memory = new Memory(config);
|
||||
} else {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async add(
|
||||
messages: Array<{ role: string; content: string }>,
|
||||
options: AddOptions,
|
||||
): Promise<AddResult> {
|
||||
await this.ensureMemory();
|
||||
// OSS SDK uses camelCase: userId/runId, not user_id/run_id
|
||||
const addOpts: Record<string, unknown> = { userId: options.user_id };
|
||||
if (options.run_id) addOpts.runId = options.run_id;
|
||||
if (options.source) addOpts.source = options.source;
|
||||
const result = await this.memory.add(messages, addOpts);
|
||||
return normalizeAddResult(result);
|
||||
}
|
||||
|
||||
async search(query: string, options: SearchOptions): Promise<MemoryItem[]> {
|
||||
await this.ensureMemory();
|
||||
// OSS SDK uses camelCase: userId/runId, not user_id/run_id
|
||||
const opts: Record<string, unknown> = { userId: options.user_id };
|
||||
if (options.run_id) opts.runId = options.run_id;
|
||||
if (options.limit != null) opts.limit = options.limit;
|
||||
else if (options.top_k != null) opts.limit = options.top_k;
|
||||
if (options.keyword_search != null) opts.keyword_search = options.keyword_search;
|
||||
if (options.reranking != null) opts.reranking = options.reranking;
|
||||
if (options.source) opts.source = options.source;
|
||||
if (options.threshold != null) opts.threshold = options.threshold;
|
||||
|
||||
const results = await this.memory.search(query, opts);
|
||||
const normalized = normalizeSearchResults(results);
|
||||
|
||||
// Filter results by threshold if specified (client-side filtering as fallback)
|
||||
if (options.threshold != null) {
|
||||
return normalized.filter(item => (item.score ?? 0) >= options.threshold!);
|
||||
}
|
||||
|
||||
return normalized;
|
||||
}
|
||||
|
||||
async get(memoryId: string): Promise<MemoryItem> {
|
||||
await this.ensureMemory();
|
||||
const result = await this.memory.get(memoryId);
|
||||
return normalizeMemoryItem(result);
|
||||
}
|
||||
|
||||
async getAll(options: ListOptions): Promise<MemoryItem[]> {
|
||||
await this.ensureMemory();
|
||||
// OSS SDK uses camelCase: userId/runId, not user_id/run_id
|
||||
const getAllOpts: Record<string, unknown> = { userId: options.user_id };
|
||||
if (options.run_id) getAllOpts.runId = options.run_id;
|
||||
if (options.source) getAllOpts.source = options.source;
|
||||
const results = await this.memory.getAll(getAllOpts);
|
||||
if (Array.isArray(results)) return results.map(normalizeMemoryItem);
|
||||
if (results?.results && Array.isArray(results.results))
|
||||
return results.results.map(normalizeMemoryItem);
|
||||
return [];
|
||||
}
|
||||
|
||||
async delete(memoryId: string): Promise<void> {
|
||||
await this.ensureMemory();
|
||||
await this.memory.delete(memoryId);
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Provider Factory
|
||||
// ============================================================================
|
||||
|
||||
export function createProvider(
|
||||
cfg: Mem0Config,
|
||||
api: OpenClawPluginApi,
|
||||
): Mem0Provider {
|
||||
if (cfg.mode === "open-source") {
|
||||
return new OSSProvider(cfg.oss, cfg.customPrompt, (p) =>
|
||||
api.resolvePath(p),
|
||||
);
|
||||
}
|
||||
|
||||
return new PlatformProvider(cfg.apiKey!, cfg.orgId, cfg.projectId);
|
||||
}
|
||||
@@ -15,8 +15,10 @@
|
||||
"skipLibCheck": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"isolatedModules": true,
|
||||
"verbatimModuleSyntax": true
|
||||
"verbatimModuleSyntax": true,
|
||||
"allowImportingTsExtensions": true,
|
||||
"noEmit": true
|
||||
},
|
||||
"include": ["index.ts", "openclaw-plugin-sdk.d.ts"],
|
||||
"include": ["index.ts", "types.ts", "providers.ts", "config.ts", "filtering.ts", "isolation.ts", "openclaw-plugin-sdk.d.ts"],
|
||||
"exclude": ["node_modules", "dist", "**/*.test.ts"]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
/**
|
||||
* Shared type definitions for the OpenClaw Mem0 plugin.
|
||||
*/
|
||||
|
||||
export type Mem0Mode = "platform" | "open-source";
|
||||
|
||||
export type Mem0Config = {
|
||||
mode: Mem0Mode;
|
||||
// Platform-specific
|
||||
apiKey?: string;
|
||||
orgId?: string;
|
||||
projectId?: string;
|
||||
customInstructions: string;
|
||||
customCategories: Record<string, string>;
|
||||
enableGraph: boolean;
|
||||
// OSS-specific
|
||||
customPrompt?: string;
|
||||
oss?: {
|
||||
embedder?: { provider: string; config: Record<string, unknown> };
|
||||
vectorStore?: { provider: string; config: Record<string, unknown> };
|
||||
llm?: { provider: string; config: Record<string, unknown> };
|
||||
historyDbPath?: string;
|
||||
disableHistory?: boolean;
|
||||
};
|
||||
// Shared
|
||||
userId: string;
|
||||
autoCapture: boolean;
|
||||
autoRecall: boolean;
|
||||
searchThreshold: number;
|
||||
topK: number;
|
||||
};
|
||||
|
||||
export interface AddOptions {
|
||||
user_id: string;
|
||||
run_id?: string;
|
||||
custom_instructions?: string;
|
||||
custom_categories?: Array<Record<string, string>>;
|
||||
enable_graph?: boolean;
|
||||
output_format?: string;
|
||||
source?: string;
|
||||
}
|
||||
|
||||
export interface SearchOptions {
|
||||
user_id: string;
|
||||
run_id?: string;
|
||||
top_k?: number;
|
||||
threshold?: number;
|
||||
limit?: number;
|
||||
keyword_search?: boolean;
|
||||
reranking?: boolean;
|
||||
source?: string;
|
||||
}
|
||||
|
||||
export interface ListOptions {
|
||||
user_id: string;
|
||||
run_id?: string;
|
||||
page_size?: number;
|
||||
source?: string;
|
||||
}
|
||||
|
||||
export interface MemoryItem {
|
||||
id: string;
|
||||
memory: string;
|
||||
user_id?: string;
|
||||
score?: number;
|
||||
categories?: string[];
|
||||
metadata?: Record<string, unknown>;
|
||||
created_at?: string;
|
||||
updated_at?: string;
|
||||
}
|
||||
|
||||
export interface AddResultItem {
|
||||
id: string;
|
||||
memory: string;
|
||||
event: "ADD" | "UPDATE" | "DELETE" | "NOOP";
|
||||
}
|
||||
|
||||
export interface AddResult {
|
||||
results: AddResultItem[];
|
||||
}
|
||||
|
||||
export interface Mem0Provider {
|
||||
add(
|
||||
messages: Array<{ role: string; content: string }>,
|
||||
options: AddOptions,
|
||||
): Promise<AddResult>;
|
||||
search(query: string, options: SearchOptions): Promise<MemoryItem[]>;
|
||||
get(memoryId: string): Promise<MemoryItem>;
|
||||
getAll(options: ListOptions): Promise<MemoryItem[]>;
|
||||
delete(memoryId: string): Promise<void>;
|
||||
}
|
||||
Reference in New Issue
Block a user