JDSS/mem0
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(skills): add mem0-integrate + mem0-test-integration pipeline skills (#4961)

Browse Source
This commit is contained in:
Saket Aryan
2026-05-05 18:52:22 +05:30
committed by GitHub
parent 6d3486ca56
commit 0fdaa29b4a
10 changed files with 1634 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/mem0-test-integration/LICENSE
+189
View File
@@ -0,0 +1,189 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but not
limited to compiled object code, generated documentation, and
conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work.
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to the Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by the Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding any notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
Copyright 2024 Mem0.ai
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
skills/mem0-test-integration/README.md
+81
View File
@@ -0,0 +1,81 @@
# mem0-test-integration — Pipeline Skill
Verify a Mem0 integration produced by [`/mem0-integrate`](../mem0-integrate/SKILL.md). Runs in the same workspace on the same branch — installs dependencies, runs the repo's native test suite, then exercises a real end-to-end smoke flow against the user's API key.
> **This is a pipeline skill, not a reference skill.** Invoke it as `/mem0-test-integration` after `/mem0-integrate` has produced a branch to verify. It catches compile and runtime bugs by design — logical integration errors (wrong data stored, wrong scoping) are for human review.
>
> **Part of the Mem0 Skill Graph:**
> - Reference: [mem0](../mem0/SKILL.md) · [mem0-cli](../mem0-cli/SKILL.md) · [mem0-vercel-ai-sdk](../mem0-vercel-ai-sdk/SKILL.md)
> - Pipeline: [mem0-integrate](../mem0-integrate/SKILL.md) → **mem0-test-integration** (this skill)
## What This Skill Does
When invoked, your assistant will:
- **Refuse to start** unless the branch has `.mem0-integration/` artifacts, the working tree is clean, and the right API key is in the environment
- **Install** the repo's dependencies using its native tooling (pip, pnpm, npm, hatch, etc.)
- **Run the native test suite** in two passes: flag-unset (must behave like `main`) and flag-set (new tests run)
- **Execute a real end-to-end smoke flow** against Mem0 Platform (`MEM0_API_KEY`) or OSS (`OPENAI_API_KEY`)
- **Produce a scorecard** — `overall: pass | fail`, per-check reasons, and the reproduction command for each failure
## When to Use
Trigger phrases:
- "Verify the integration"
- "Test the Mem0 integration"
- "Run `/mem0-test-integration`"
Do **not** use this skill to run general project tests (defer to the repo's native test command) or before `/mem0-integrate` has produced a branch on the current workspace.
## Installation
### CLI (Claude Code, Codex, OpenCode, OpenClaw, or any tool that supports skills)
```bash
npx skills add https://github.com/mem0ai/mem0 --skill mem0-test-integration
```
Typically installed alongside the companion pipeline skill:
```bash
npx skills add https://github.com/mem0ai/mem0 --skill mem0-integrate
```
### Claude.ai
1. Download this `skills/mem0-test-integration` folder as a ZIP
2. Go to **Settings > Capabilities > Skills**
3. Click **Upload skill** and select the ZIP
### Claude API (Skills API)
```bash
curl -X POST https://api.anthropic.com/v1/skills \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "mem0-test-integration", "source": "https://github.com/mem0ai/mem0/tree/main/skills/mem0-test-integration"}'
```
### Preconditions
The skill refuses to start unless all of the following are true:
- `.mem0-integration/` directory exists in the repo root
- Current branch starts with `mem0-integrate/`
- Working tree is clean
- The same API key used during `/mem0-integrate` is exported in the environment
## What This Skill Does *Not* Catch
By design, this skill only catches compile and runtime bugs. Logical errors — memories stored with the wrong scoping, retrieval returning the wrong user's data, filter mismatches — are the human reviewer's responsibility.
## Links
- [Mem0 Documentation](https://docs.mem0.ai)
- [Mem0 GitHub](https://github.com/mem0ai/mem0)
- [API Reference](https://docs.mem0.ai/api-reference)
## License
Apache-2.0
skills/mem0-test-integration/SKILL.md
+368
View File
@@ -0,0 +1,368 @@
---
name: mem0-test-integration
description: >
Verify a Mem0 integration produced by /mem0-integrate. Runs in the same
workspace on the same branch (loose coupling) — installs dependencies,
runs the repo's native test suite, then exercises a real end-to-end
smoke flow against the user's API key. Produces a scorecard.
TRIGGER when: user has just run /mem0-integrate and says "verify",
"test the integration", "run /mem0-test-integration", or when a
.mem0-integration/ directory exists and tests have not been run yet
on the current branch.
DO NOT TRIGGER when: the user wants to run general project tests
(defer to the repo's native test command), or when no prior /mem0-integrate
run exists in the current branch (ask them to run /mem0-integrate first).
This skill ONLY catches compile and runtime bugs by design. Logical
integration errors — wrong data stored, wrong time retrieved, wrong
user scoping — are on the human reviewer.
license: Apache-2.0
metadata:
author: mem0ai
version: "0.1.0"
category: ai-memory
tags: "memory, integration, testing, tdd, platform, oss"
coupling: loose
mem0_tested_versions: "mem0ai (PyPI) >=2.0.0,<3.0.0; mem0ai (npm) >=3.0.0,<4.0.0"
---
# mem0-test-integration
Verifies what `/mem0-integrate` produced. Runs in the same workspace,
on the same feature branch. Loose coupling — fast, catches compile and
runtime bugs, does not catch logical errors.
## Canonical sources (use these, not ambient knowledge)
All static checks and smoke-test shapes validate against these URLs.
`WebFetch` each before running step 3.
- Scope-tagged docs index: https://docs.mem0.ai/llms.txt
- OpenAPI (Platform REST): https://docs.mem0.ai/openapi.json
- Published SDK skill (canonical call patterns): https://raw.githubusercontent.com/mem0ai/mem0/main/skills/mem0/SKILL.md
- Vercel AI SDK skill (if the target repo uses `@ai-sdk/*`): https://raw.githubusercontent.com/mem0ai/mem0/main/skills/mem0-vercel-ai-sdk/SKILL.md
- SDK source (cross-check version against frontmatter `mem0_tested_versions`):
- Repo root: https://github.com/mem0ai/mem0
- Python: https://github.com/mem0ai/mem0/tree/main/mem0
- TypeScript: https://github.com/mem0ai/mem0/tree/main/mem0-ts
Read the `Delegated skill:` field in `.mem0-integration/plan.md` — if it
names a skill URL, fetch that skill and use its example blocks as the
reference for both static checks (step 3) and the smoke test (step 5).
## Non-invasiveness contract
Every check in this skill assumes the integration is **additive and
feature-flagged** (see `/mem0-integrate` "Integration principles").
Specifically:
- `product.json` must contain a `feature_flag` field.
- Steps 4–6 run in two passes:
- **Pass A — flag unset.** All pre-existing tests must pass, smoke/E2E
skip. The repo must behave like `main`. Any failure here is a
**hard fail** — do not let the self-heal loop attempt a patch.
- **Pass B — flag set.** New tests must pass, smoke and E2E run.
- If Pass A fails, the scorecard marks `non_invasive: false` and sets
`overall: fail` with a distinct reason code the integrator's heal
loop refuses to touch.
## Preconditions
Refuse to start unless ALL of the following are true:
- `.mem0-integration/` directory exists in the repo root.
- `.mem0-integration/product.json`, `goal.md`, and `plan.md` are readable
and internally consistent (JSON parses, docs non-empty).
- Current branch name begins with `mem0-integrate/` (set by the companion
skill). Prevents accidental runs on unrelated branches.
- Working tree is clean. The skill never modifies source files; any dirty
state means the integration is mid-edit and not ready to verify.
- The same API key the integration used is available in the environment
(`MEM0_API_KEY` for Platform, `OPENAI_API_KEY` for OSS — read which from
`product.json`). Interactive mode asks if missing; CI mode exits 2.
Exit with a written rationale on any precondition failure. Never attempt
to "fix up" state.
## Pipeline
### 1. Read the contract
Load:
- `product.json` → which language, which product (Platform vs OSS), which
mem0 version, `write_site`, `read_site`.
- `plan.md` → the mechanical contract (write pattern, read pattern,
preserved behavior).
- `goal.md` → the intent (displayed in the scorecard only; not tested).
### 2. Install dependencies
Route by language from `product.json`:
| Language | Command |
|---|---|
| Python | `pip install -e .` if editable, else `pip install -r requirements.txt`. Then `pip install mem0ai` if not already present at the pinned version. |
| TypeScript / JavaScript | `npm install` (or `pnpm install` / `yarn install` if detected by lockfile). |
If install fails → exit code 2 with stderr tail. Never move to testing
if dependencies don't resolve.
### 3. Static sanity checks (fast, local, no API calls)
- **Import check**: does the write-site file import the expected Mem0
surface? Authoritative list comes from `## Identify the User's Setup`
in `https://docs.mem0.ai/llms.txt`:
- Platform Python → `from mem0 import MemoryClient`
- Platform TS → `import MemoryClient from "mem0ai"`
- OSS Python → `from mem0 import Memory`
- OSS TS → `import { Memory } from "mem0ai/oss"`
If `plan.md` names a delegated skill (e.g., Vercel AI), use *that*
skill's import signature instead of the list above. Mismatch → fail
with line number.
- **Version check**: installed `mem0ai` version falls in the range from
this skill's `mem0_tested_versions`. Out of range → warn but continue.
- **Type check** (TS tracks only): run `tsc --noEmit` or `tsup --dts`.
Non-zero → fail.
- **Lint** (if the repo has a linter configured): run the repo's own
lint command. Lint failures from this skill's changes → fail; pre-existing
lint failures → surface as a warning.
- **Eager-init check**: grep the `write_site` and `read_site` files (paths
from `product.json`) for `MemoryClient(` or `Memory(` at module scope —
i.e., not inside a function, method, or class body. `MemoryClient()`
validates the API key in `__init__` (network call) and OSS `Memory()`
can eagerly initialize embedding/LLM providers — module-level
instantiation hits the wire on import and breaks Pass A's test
collection whenever the key is unset. Hit → fail with `file:line` and
the lazy-init guidance from `/mem0-integrate` step 8 constraint #7.
### 4. Run the repo's native test suite (two passes)
| Language | Test command (in priority order) |
|---|---|
| Python | `pytest` with the test files from step 5 of the companion skill, else `python -m unittest discover`. |
| TypeScript / JavaScript | `npm test` if defined in package.json; else auto-detect `vitest` or `jest`. |
**Pass A — `feature_flag` unset.** Run the *entire* pre-existing suite
(excluding the new `test_mem0_*` files). **Must be 100% green.** Any
failure here marks `non_invasive: false` in the scorecard and is
a **hard fail** — the integrator's self-heal loop refuses to touch it.
**Pass B — `feature_flag` set** (value from `product.json`). Run the
full suite including the new tests. All must pass.
Isolate integration-introduced failures using `git diff main..HEAD
--name-only`. A test file that exists on `main` and fails only under
the integration branch (flag set *or* unset) counts against the
scorecard regardless of pass. A test file that already failed on `main`
is surfaced as `pre_existing_unrelated` and does not count — but is
still reported so the user can clean it up.
Capture output to `.mem0-integration/test-stdout-flag-off.log` and
`.mem0-integration/test-stdout-flag-on.log`. Scorecard reports pass/fail
per pass.
### 5. Smoke test (real API call, shortest round-trip)
Scripted end-to-end flow tailored to `product.json`. The call shapes
below are the minimal ones; if `plan.md` names a delegated skill, use
*that skill's* minimal example verbatim instead — it is the canonical
shape for the detected stack.
**Platform (Python):**
from mem0 import MemoryClient
c = MemoryClient() # uses MEM0_API_KEY
uid = f"mem0-test-integration-{os.urandom(4).hex()}"
c.add([{"role": "user", "content": "I prefer aisle seats"}], user_id=uid)
hits = c.search("seat preference", user_id=uid)
assert any("aisle" in h.get("memory", "") for h in hits), hits
c.delete_all(user_id=uid) # clean up
**Platform (TS):** same shape with `MemoryClient` from `"mem0ai"`.
**OSS (Python / TS):** uses `Memory()` / `new Memory()` with default config
(OpenAI LLM via `OPENAI_API_KEY`, local Qdrant). If the repo ships a
`docker-compose.yml` with a Qdrant service, the skill starts it first and
tears it down after. If no backing store is reachable → fail with a
clear message naming the fix.
The smoke test always uses a **disposable random user_id** prefixed with
`mem0-test-integration-` so a failed cleanup doesn't pollute the user's
real data. A background tidy step deletes any prefix-matching entries
older than 24 hours on the next run.
Capture output to `.mem0-integration/smoke-stdout.log`.
### 6. E2E integration test (run the app, exercise the flow)
Unit tests + smoke prove the SDK works in isolation. This step is the
real signal: **does memory actually appear in the app's user-visible
output when the integration runs end-to-end?**
Requires `plan.md` to contain an `E2E recipe:` section (authored by
`/mem0-integrate` step 5). If absent → status `skipped` (not `fail`),
note in scorecard that the repo has no runnable entry point.
Recipe fields the skill reads:
- `start` — shell command to launch the app using `$PORT` for any network
port. Run in background with stdout/stderr teed to
`.mem0-integration/e2e-app.log`.
- `ready_probe` — how to detect readiness. `url=... status=...` polls an
HTTP endpoint; `log="..."` waits for a substring in `e2e-app.log`;
`sleep=N` waits N seconds (last resort). 60-second hard timeout.
- `compose_services` — optional. If set, bring them up via
`docker compose up -d <services>` before `start`, tear them down with
`docker compose down` at the end.
- `write_call` — triggers the Mem0 write path exactly once. Output is
captured and surfaced on failure. 60-second hard timeout.
- `write_async_wait_ms` — pause after `write_call` to let async memory
flushes land. Default 0.
- `read_call` — triggers the Mem0 read path. Typically a fresh session
or new request that should surface the stored memory.
- `read_assert` — substring, `regex=...`, or `jsonpath=<expr>=<value>`
that must appear in `read_call`'s stdout. This is the E2E pass gate.
Execution order:
1. Allocate an ephemeral TCP port; export as `PORT`.
2. Set `MEM0_USER_ID` to a disposable `mem0-test-integration-<rand>` value
and export it, so the app can use the same scoping the smoke test does
if the recipe wants cleanup.
3. Bring up `compose_services` if named.
4. Run `start` in the background.
5. Poll `ready_probe` until success or 60s timeout. Timeout → fail.
6. Run `write_call`. Non-zero exit → fail (but continue to cleanup).
7. Sleep `write_async_wait_ms`.
8. Run `read_call`.
9. Evaluate `read_assert` against `read_call`'s stdout. Miss → fail.
10. Cleanup (always, even on failure): SIGTERM the app, SIGKILL after
5s, `docker compose down` if services were started, `delete_all`
memories matching `mem0-test-integration-*` on Platform scenarios.
On any failure, the scorecard includes:
- Last 40 lines of `e2e-app.log`
- Full `write_call` output
- Full `read_call` output
- The expected vs actual for `read_assert`
### 7. Scorecard
Write `.mem0-integration/scorecard.md` and `.mem0-integration/scorecard.json`:
{
"timestamp": "2026-04-20T14:03:11Z",
"branch": "mem0-integrate/remember-user-preferences",
"product": "platform",
"language": "python",
"mem0_version": "2.0.0",
"non_invasive": true,
"feature_flag": "MEM0_ENABLED",
"results": {
"install": {"status": "pass", "duration_ms": 12043},
"static_checks":{"status": "pass", "duration_ms": 812},
"unit_tests_flag_off": {"status": "pass", "duration_ms": 3920, "count": 47,
"reason": "all pre-existing tests green with flag unset"},
"unit_tests_flag_on": {"status": "pass", "duration_ms": 4321, "count": 49},
"smoke_test": {"status": "pass", "duration_ms": 2890, "memory_id": "mem_..."},
"e2e_test": {"status": "pass", "duration_ms": 14200,
"ready_probe_ms": 3100, "write_exit": 0,
"read_assert_matched": true}
},
"friction": {
"dependency_install_retries": 0,
"pre_existing_test_failures": 0,
"warnings": ["mem0ai 2.0.0 pinned; consider 2.0.1 for fix X"]
},
"overall": "pass"
}
The markdown version is human-readable and includes:
- Goal doc + plan doc reprinted at top (so reviewers don't have to hunt).
- Each check with pass/fail + log excerpt.
- Friction summary.
- Verbatim warnings from mem0 SDK (if any — e.g., deprecated field usage).
- **Explicit "NOT checked" section** listing what loose coupling misses:
"Whether the stored data is what the user wants stored. Whether search
runs at the right moment. Whether user_id matches the actual session
scope. Human review required."
### 8. Report + exit
- Print the scorecard path + overall pass/fail to stdout.
- **Do not commit the scorecard files.** They live in `.mem0-integration/`,
which is gitignored. The user can inspect and optionally pin.
- On fail: print the first failing step's log tail (last 40 lines) and
stop. Do not attempt to fix anything.
## Artifacts (all under `.mem0-integration/`)
| File | Purpose | Retention |
|---|---|---|
| `scorecard.md` | Human-readable verdict. | Overwritten per run. |
| `scorecard.json` | Machine-readable verdict. Consumed by the CI scorecard workflow later. | Overwritten per run. |
| `test-stdout-flag-off.log` | Step 4 Pass A (pre-existing suite, flag unset). | Overwritten per run. |
| `test-stdout-flag-on.log` | Step 4 Pass B (full suite, flag set). | Overwritten per run. |
| `smoke-stdout.log` | Full output from step 5. | Overwritten per run. |
| `e2e-app.log` | Background app stdout/stderr from step 6. | Overwritten per run. |
| `e2e-calls.log` | write_call + read_call invocations and outputs. | Overwritten per run. |
## Modes
| Mode | Trigger | Behavior |
|---|---|---|
| Interactive (default) | TTY present, `MEM0_TEST_CI` unset | Asks for missing keys, prints friendly summaries. |
| CI | `MEM0_TEST_CI=1` | Keys must be in env, no prompts, non-zero exit on any fail. JSON scorecard goes to stdout's tail for workflow parsing. |
## Invocation
/mem0-test-integration # interactive, all steps
/mem0-test-integration --ci # non-interactive
/mem0-test-integration --skip-smoke # no API calls, no E2E
/mem0-test-integration --skip-e2e # unit + smoke only (faster CI)
/mem0-test-integration --only-smoke # just smoke
/mem0-test-integration --only-e2e # just E2E (assumes deps installed)
Composition: `--skip-*` can stack (`--skip-smoke --skip-e2e` = static +
unit only, zero API cost). `--only-*` is mutually exclusive with all
other flags.
## Exit codes
| Code | Meaning |
|---|---|
| 0 | All checks passed. |
| 1 | Precondition failed (no `.mem0-integration/`, wrong branch, dirty tree). |
| 2 | Missing env key (CI mode) or dependency install failure. |
| 3 | Static sanity check failed (wrong import, type error). |
| 4 | Unit tests failed (Pass B — integration itself broken). |
| 5 | Smoke test failed. |
| 6 | E2E test failed (ready_probe timeout, write/read call failed, or read_assert miss). |
| 7 | Non-invasiveness violation: Pass A failed (pre-existing tests broke). Integrator's heal loop refuses to touch this. |
| 8 | Internal error (skill bug — report it). |
## Explicitly out of scope
- **Modifying source files.** The skill is read-only against the repo.
If verification exposes a bug, re-run `/mem0-integrate` on the same
goal + plan; do not hand-patch.
- **Fixing broken tests.** Failing unit tests are a signal that the
integration is wrong, not that the tests are wrong. The skill does
not "try a different test."
- **Deep logical correctness.** The E2E step proves "something the user
said earlier comes back later," which is a useful but shallow signal.
It does NOT prove the integration picks the *right* facts to store,
scopes `user_id` correctly across real users, or handles conflict
resolution well. That's human review territory.
- **Self-healing.** This skill never modifies source files. The paired
`/mem0-integrate` skill in its default `--heal` mode consumes the
scorecard produced here and drives its own remediation loop. Exit
code 7 (non-invasiveness violation) is the explicit signal the heal
loop must stop and surface to the user.
- **Cross-branch comparisons.** No `main` baseline diffing. The
scorecard reflects this branch only.
- **Running against production data.** Every smoke test uses a disposable
random user_id and cleans up after. Never touches any other user's data.