fix(plugins): stamp surface identity at record time, not send time
Six defects in 0.3.x plugin telemetry. Defects 1, 2 and 6 were not three bugs: they were one spool protocol getting three properties wrong. Identity was decided by the wrong process. `harness` was stamped in record(), correctly, but `source` was read from a module global in flush() — so whichever process drained the spool named every event in it. Two processes never call init(): mcp_server.py, and the detached `python3 telemetry.py` sender that spawn_flush() starts. record() now stamps source beside harness, and the build generates core/_harness_id.py per host so identity resolves with no init() call at all. That also unifies two defaults that disagreed (`<host>_plugin` vs `MEM0_<HOST>_PLUGIN`), which could yield three source values for one plugin. Ownership was inferred, not held. Path.replace is os.rename, which preserves mtime, so a claim made after a quiet minute inherited the spool's age and was stealable the instant it existed. Claims are touched on creation and the per-batch rewrite doubles as a lease heartbeat. Progress was not durable. flush() returned on the first failed batch without truncating, so the retry re-posted from index 0 — 150 events delivered 250 times. It now rewrites the claim with the unsent remainder after every batch, bounding a crash to one repeated batch, and each event carries a uuid. Parked batches starved. They were only reachable when no spool existed, and because sessions keep recording there usually was one, so a batch parked by a failed send waited until the 7-day expiry deleted it unsent — despite its own presence being what starts the sender. flush() drains them in the same run, and expiry now applies only after a genuine retry has failed. code.install counted upgrades and repeat sessions. is_first_run() read the identity file, which only a successful flush writes, so an offline user recorded an install every session forever. A dedicated install-state.json is claimed atomically at record time; a non-empty data directory reads as an upgrade. The docs called this anonymous. Every event carries the account email, and the hashes were unsalted SHA-256 over a git remote URL or an absolute path containing the username. READMEs, the module docstring and a new docs section now say what the code does, and repo/session digests are salted per install. A cached email outlived an API key change. It is now re-resolved when the key's fingerprint differs, and $identify aliases anonymous->email only — aliasing one account to another merges person profiles irreversibly. All six shipped green because the shared core's only tests lived under one host, behind a conftest that calls init() at import. Core behaviour was never exercised uninitialised. Adds agent-plugin-core/tests with no init, including subprocess tests and coverage for the portable plugin, which has no flush worker and would pass a native-only test vacuously. Also puts the three surface headers on the SDKs, CLIs and integrations, and corrects a README claiming ZAPIER/STRANDS were already in the platform allowlist. Verified: 59 core tests, 203 claude-code, 11 cursor, 5 codex, 2 kimi, 6 antigravity. ruff and compileall clean. --check clean for all six hosts. TypeScript changes are not typechecked locally (deps not installed). Claude-Session: https://claude.ai/code/session_01C7tEmH86HAr7GoAAKCEHZb
This commit is contained in:
Saket Aryan
@@ -0,0 +1,179 @@
|
||||
"""Core telemetry behaviour with NO telemetry.init(), in a real subprocess.
|
||||
|
||||
Why this file exists
|
||||
--------------------
|
||||
``telemetry.py`` lives in ``agent-plugin-core/python/`` but its only tests lived
|
||||
under ``claude-code-plugin/tests/``, behind a ``conftest.py`` that calls
|
||||
``configure_harness()`` and ``telemetry.init()`` at import. Core behaviour was
|
||||
therefore only ever exercised inside an already-configured module.
|
||||
|
||||
Two processes in the real pipeline never call ``init()``:
|
||||
|
||||
- ``mcp_server.py``, which records every manual search;
|
||||
- the detached ``python3 telemetry.py`` sender that ``spawn_flush()`` starts at
|
||||
session start, after every skill command, and when the MCP server exits.
|
||||
|
||||
Both fell back to module defaults, so MCP searches reported ``harness=generic``
|
||||
and everything that sender delivered was labelled ``MEM0_PLUGIN`` regardless of
|
||||
which of the six plugins produced it. The suite stayed green throughout.
|
||||
|
||||
These tests run in a fresh interpreter with no conftest, against a built host
|
||||
bundle, which is the only arrangement that can catch that class of bug.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
CORE_ROOT = Path(__file__).resolve().parents[1]
|
||||
REPOSITORY_ROOT = CORE_ROOT.parents[1]
|
||||
HOSTS = {
|
||||
"claude-code": ("claude-code-plugin", "CLAUDE_CODE_PLUGIN"),
|
||||
"cursor": ("cursor-plugin", "CURSOR_PLUGIN"),
|
||||
"codex": ("codex-plugin", "CODEX_PLUGIN"),
|
||||
"kimi": ("kimi-plugin", "KIMI_PLUGIN"),
|
||||
"antigravity": ("antigravity-plugin", "ANTIGRAVITY_PLUGIN"),
|
||||
# Portable: no flush_worker and no hook_runner, so its ONLY sender is the
|
||||
# uninitialised telemetry.py. A native-only test passes here vacuously.
|
||||
"coding-agent": ("mem0-agent-plugin", "CODING_AGENT_PLUGIN"),
|
||||
}
|
||||
|
||||
|
||||
def _core_dir(directory: str) -> Path:
|
||||
return REPOSITORY_ROOT / "integrations" / directory / "core"
|
||||
|
||||
|
||||
def _run(core: Path, data_dir: Path, body: str) -> str:
|
||||
"""Execute `body` in a fresh interpreter with only the host's core on sys.path."""
|
||||
script = f"import sys; sys.path.insert(0, {str(core)!r})\n{body}"
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-c", script],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={
|
||||
"MEM0_CODE_DATA_DIR": str(data_dir),
|
||||
"PATH": "/usr/bin:/bin",
|
||||
"HOME": str(data_dir),
|
||||
},
|
||||
)
|
||||
assert result.returncode == 0, result.stderr
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("harness,spec", sorted(HOSTS.items()))
|
||||
def test_identity_resolves_without_init(harness, spec):
|
||||
"""Every built host knows what it is with no configuration call at all."""
|
||||
directory, source_tag = spec
|
||||
core = _core_dir(directory)
|
||||
if not core.exists():
|
||||
pytest.skip(f"{directory} is not built in this tree")
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
out = _run(
|
||||
core,
|
||||
Path(tmp),
|
||||
"import telemetry; print(telemetry._harness, telemetry._source_tag)",
|
||||
)
|
||||
assert out == f"{harness} {source_tag}"
|
||||
|
||||
|
||||
def test_mcp_server_records_the_real_harness():
|
||||
"""mcp_server imports telemetry and never initialises it (server.py has no init).
|
||||
|
||||
Its recorded events used to carry harness=generic for every plugin.
|
||||
"""
|
||||
core = _core_dir("claude-code-plugin")
|
||||
if not core.exists():
|
||||
pytest.skip("claude-code-plugin is not built in this tree")
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
data_dir = Path(tmp)
|
||||
_run(
|
||||
core,
|
||||
data_dir,
|
||||
"import mcp_server, telemetry; telemetry.record('search', trigger='mcp-search')",
|
||||
)
|
||||
spooled = (data_dir / "telemetry.jsonl").read_text(encoding="utf-8").strip()
|
||||
|
||||
event = json.loads(spooled)
|
||||
assert event["properties"]["harness"] == "claude-code"
|
||||
assert event["properties"]["source"] == "CLAUDE_CODE_PLUGIN"
|
||||
|
||||
|
||||
def test_the_detached_sender_does_not_relabel_events():
|
||||
"""`python3 telemetry.py` is the sender spawn_flush() starts, and never inits.
|
||||
|
||||
source is stamped at record time now, so which process sends is irrelevant.
|
||||
"""
|
||||
core = _core_dir("claude-code-plugin")
|
||||
if not core.exists():
|
||||
pytest.skip("claude-code-plugin is not built in this tree")
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
data_dir = Path(tmp)
|
||||
_run(core, data_dir, "import telemetry; telemetry.record('search')")
|
||||
|
||||
captured = data_dir / "captured.json"
|
||||
# Drain with a fresh, unconfigured interpreter, capturing the payload
|
||||
# instead of posting it.
|
||||
_run(
|
||||
core,
|
||||
data_dir,
|
||||
"import json, telemetry\n"
|
||||
"sent = []\n"
|
||||
"telemetry._post = lambda payload, url: sent.append(payload) or True\n"
|
||||
"telemetry.flush()\n"
|
||||
f"open({str(captured)!r}, 'w').write(json.dumps(sent))",
|
||||
)
|
||||
payloads = json.loads(captured.read_text(encoding="utf-8"))
|
||||
|
||||
batches = [p for p in payloads if "batch" in p]
|
||||
assert batches, "nothing was sent"
|
||||
properties = batches[0]["batch"][0]["properties"]
|
||||
assert properties["source"] == "CLAUDE_CODE_PLUGIN"
|
||||
assert properties["harness"] == "claude-code"
|
||||
|
||||
|
||||
def test_every_event_carries_a_uuid_for_dedupe():
|
||||
core = _core_dir("claude-code-plugin")
|
||||
if not core.exists():
|
||||
pytest.skip("claude-code-plugin is not built in this tree")
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
data_dir = Path(tmp)
|
||||
_run(core, data_dir, "import telemetry; telemetry.record('search'); telemetry.record('flush')")
|
||||
lines = (data_dir / "telemetry.jsonl").read_text(encoding="utf-8").strip().splitlines()
|
||||
|
||||
ids = [json.loads(line)["uuid"] for line in lines]
|
||||
assert len(ids) == 2
|
||||
assert len(set(ids)) == 2
|
||||
|
||||
|
||||
def test_source_tag_defaults_agree_between_the_two_modules():
|
||||
"""configure_harness and telemetry.init must derive the same tag.
|
||||
|
||||
They disagreed: `<host>_plugin` in one and `MEM0_<HOST>_PLUGIN` in the other,
|
||||
so one plugin could emit three different source values depending on which
|
||||
process sent the batch.
|
||||
"""
|
||||
core = _core_dir("claude-code-plugin")
|
||||
if not core.exists():
|
||||
pytest.skip("claude-code-plugin is not built in this tree")
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
out = _run(
|
||||
core,
|
||||
Path(tmp),
|
||||
"import memory_core, telemetry\n"
|
||||
"memory_core.configure_harness('kimi')\n"
|
||||
"telemetry.init(harness='kimi')\n"
|
||||
"print(memory_core.harness_config()['source_tag'].upper(), telemetry._source_tag)",
|
||||
)
|
||||
left, right = out.split()
|
||||
assert left == right == "KIMI_PLUGIN"
|
||||
Reference in New Issue
Block a user