The integration suite claimed exclusive ownership of a shared project. Its
globalSetup and globalTeardown each ran a project-wide wipe:
deleteAll({userId: '*', agentId: '*', appId: '*', runId: '*'})
deleteUsers()
CI runs this suite concurrently across PRs against one MEM0_API_KEY, so each
job's wipe deleted the memories its neighbours had just seeded. maxWorkers: 1
and max-parallel: 1 only serialize within a run, not across them.
Evidence from three jobs that overlapped on 2026-08-12:
6903 crud seeded 13:53:24-13:54:15, hit by 6928's wipe (13:53:20-30) FAIL
6928 crud seeded 13:53:30-13:54:19, hit by 6902's wipe (13:53:31-43) FAIL
6902 crud seeded 13:53:43-13:54:21, no overlapping wipe PASS
6902 batch ran 13:55:31-13:56:20, hit by 6928's teardown (13:55:59) FAIL
Every failure surfaced as waitForMemories timing out with zero memories, which
reads as pipeline latency but was deletion: probing the live API shows a seeded
user settling in 17s, well inside the 45s budget.
- drop globalSetup/globalTeardown; every suite already cleans up its own
UUID-scoped user in afterAll
- remove the fullProjectCleanup helper so the wipe cannot come back
- give updateWebhook a per-run URL; the project enforces uniqueness on
(project, url), so concurrent runs collided on the hardcoded constant
Verified: two suites run concurrently against the same key go from 3 failures
to 1. The remainder is updateProject() mutating project-level
custom_instructions, a singleton two runs cannot both own.
The live-API integration suites failed intermittently on a different
test each run: CRUD delete 404ing on a known ID, batch update/delete
failing, search returning nothing.
All three share one cause. waitForMemories returned as soon as one
memory existed, but the v3 pipeline was still consolidating, and
consolidation replaces a memory (delete + add) rather than editing it
in place. Tests captured memoryIds at that moment and later acted on
IDs the platform had already invalidated.
Wait for quiescence instead: two consecutive polls returning the same
sorted ID set. Compare IDs rather than length, since a replacement
leaves the count unchanged. Fixed in the one shared helper every
integration suite seeds through.
kartik-mem0