Mapping the AI-memory landscape

In Part 1 I described a gap: the record of AI-assisted work forms inside agent sessions, and almost nothing captures it faithfully. The obvious next question is whether anyone else had noticed. They had — loudly. “Everyone is doing LLM memory and self-learning agents” is barely an exaggeration. So before building anything, I spent a week reading the field.

”Almost every memory tool in this survey stores distilled facts and throws the transcript away. That absence is the whole opportunity.”

I ran the survey as four parallel research lanes — agent-memory frameworks, personal “second-brain” tools, LLM observability platforms, and coding-agent memory — each verified against the actual GitHub repos in June 2026, then deduplicated down to twelve distinct projects. What follows is the honest map: not who’s biggest, but what each one actually does with the record of a session, and where that leaves room for something different.

§1 · Four lanes, one question

Everything in this space is answering the same question — how does an agent remember? — but from four different starting points:

• Coding-agent memory — capture what a coding assistant did and feed it back to the next session. This is my lane; the direct competitors live here.
• Agent-memory frameworks — general-purpose libraries that give any agent a long-term memory layer.
• Personal “second brains” — self-hosted knowledge bases you and an assistant both read and write.
• LLM observability — dashboards that trace and log every model call, built for debugging rather than recall.

The interesting finding is how much these lanes have converged on the same handful of mechanisms — and the one design choice that still separates them.

§2 · The direct competitors

claude-mem (Apache-2.0, ~83k★) is the breakout. It hooks into an agent’s session lifecycle, AI-compresses everything into semantic summaries, and injects relevant context back into future sessions — with a clever “progressive disclosure” MCP retrieval flow that filters an index before fetching detail, claiming ~10× token savings. It’s multi-agent, local-first, and it ships a viewer. It’s also the closest thing to a large competitor I have. What it doesn’t do: it stores compressed summaries, not the byte-faithful transcript, and it leans on a heavy local stack (Bun + uv + a Chroma vector DB + a long-running worker on a fixed port).

claude-self-reflect (MIT, ~216★) is the architecturally adjacent one. Recent versions ship a single ~44 MB Rust binary with embedded SQLite — “no databases, no containers, no API keys” — that imports Claude Code’s .jsonl history and exposes semantic recall over MCP. Its best idea is a thoughtful relevance model: memory decay with a biomimetic 90-day half-life, plus recency and per-file search. It’s Claude-Code-only, single-machine, and churning fast (already at v9), but the decay-and-recency model is directly worth borrowing.

§3 · The memory frameworks — steal the ideas, not the dependencies

These four are where the reusable recall concepts live.

Mem0 (Apache-2.0, ~59k★) is the mindshare leader. Its contribution is a clean two-phase loop: extract atomic facts from a conversation, then reconcile each candidate against existing memory with an explicit ADD / UPDATE / DELETE / NOOP decision, so stale or contradictory facts get rewritten rather than endlessly appended. If I ever distil my transcripts into facts, that reconciliation step is the single most reusable idea in this whole survey. The catch: extraction is lossy and flattens conversations into short facts, discarding the raw record.

Letta (the former MemGPT, Apache-2.0, ~23k★) treats the agent itself as a stateful, server-resident object that manages its own memory across RAM-like core, recall, and archival tiers. Its standout is “sleep-time compute” — background self-reflection that reorganises memory between sessions. That’s the closest thing in the field to genuine self-learning, and it maps directly onto “future sessions learn from history.” But you adopt Letta’s whole runtime to get it; it doesn’t bolt onto an existing agent.

Zep / Graphiti (Apache-2.0, ~28k★) is the strongest conceptual benchmark. It builds a bi-temporal knowledge graph where every fact carries two timelines — when it was true in the world, and when it was ingested — so contradicted facts are invalidated, never deleted, preserving full history and provenance back to the source episode. Retrieval uses no LLM call at all (hybrid embedding + BM25 + graph traversal), making it fast and reproducible. The honest gap: it needs a graph database (Neo4j/FalkorDB), which fights a lightweight self-hosting stance. Borrow the temporal-provenance model, not the Neo4j.

Cognee (Apache-2.0, ~12k★) rounds out the lane with an “Extract → Cognify → Load” pipeline over 38+ source types into a hybrid vector+graph store. It mostly confirms that the field is converging on hybrid vector+graph memory — a useful signal for a distillation layer, if I build one.

§4 · The second brains

Basic Memory (AGPL-3.0, ~3.3k★) is my nearest neighbour, and I studied it hardest. It’s MCP-native, and its source of truth is plain Markdown files on disk — human-editable, git-friendly, with a throwaway SQLite index on top. Knowledge is captured as Observations (categorised facts like [decision] or [tip]) and Relations (wiki [[links]]) that compound into a graph the agent writes itself during a conversation. A companion skills repo even adds reflection and defragmentation passes where the agent consolidates and dedupes its own memory. The one thing it doesn’t do is keep the full raw transcript — it stores distilled notes.

Khoj (AGPL-3.0, ~35k★) is the flagship self-hosted “AI second brain”: semantic search and chat over your own documents across a dozen surfaces (web, Obsidian, Emacs, even WhatsApp). Its useful lesson is architectural — it proves pgvector-in-Postgres is a credible single store for both relational data and embeddings. But its memory model is “RAG over your files,” not learning from agent sessions.

Reor (AGPL-3.0, ~8.6k★, now archived) is included as a clean reference architecture: a fully-offline desktop notes app using LanceDB + Transformers.js + Ollama with zero backing services, surfacing related notes live as you write. It’s the opposite end of the spectrum from a server-backed system — and its limitations (no API, no programmatic surface, no transcript fidelity) are exactly what an agent-facing memory needs.

§5 · The observability platforms

Langfuse (MIT core, ~29k★) is the most popular OSS LLM observability platform, and its gift to me is vocabulary. Its data model — observations → traces → sessions — maps almost exactly onto a session→event hierarchy, and its multi-store split (Postgres for config, ClickHouse for high-volume traces, S3 for raw events) is a textbook reference for when telemetry outgrows a single database. It’s also a cautionary tale on weight: ClickHouse + Postgres + Redis + S3 is a lot to self-host.

Arize Phoenix (~10k★) is the anti-Langfuse — proof that a single relational store, SQLite by default, is a viable low-friction backend. It’s OpenTelemetry-native and ships Claude Agent SDK instrumentation. Its cautionary note is the license: Elastic License 2.0 is source-available, not OSI-open — a model to learn architecture from, not licensing.

Laminar (Apache-2.0, ~3k★) is the youngest but most aspirationally aligned with a recall thesis. Two features stand out: a built-in SQL editor over all your session data, and natural-language-defined events — you describe a behaviour in prose and it tags matching traces. “Query your history in SQL” is exactly the power-user recall surface worth aiming at. Its four-service backend (Postgres + ClickHouse + RabbitMQ + Qdrant) is the cautionary side.

§6 · What the whole field agrees on — and where I diverge

Reading twelve projects back to back, the consensus is striking:

The mechanisms have converged. MCP is the dominant way to hand memory to a live agent. Vectors — Qdrant, LanceDB, pgvector, FastEmbed — are how everyone searches. Fact reconciliation (Mem0), temporal provenance (Graphiti), sleep-time reflection (Letta) and defragmentation (Basic Memory) are the recall ideas worth stealing, and none of them require adopting the store they ship with.

But look down the right-hand column. Almost every one of these tools stores a distilled version — facts, summaries, notes, spans — and throws the original transcript away. That’s not an oversight; distillation is the point of a memory layer. It’s also, precisely, the gap I noticed in Part 1: the honest, byte-faithful record of what actually happened in a session is exactly what nobody keeps as ground truth.

So that became the design thesis. Keep the lossless, vendor-neutral transcript — every session, byte-for-byte, in a plain document store plus object storage — as the source of truth. Treat distillation, reconciliation, temporal graphs, and semantic search as optional derived layers on top, so an agent can have both recall and provenance. Borrow the best ideas from all twelve; inherit the dependencies of none. Stay deliberately lighter than Langfuse and Laminar, because Phoenix proves you can.

That’s the thing I decided to build. Part 3 is the build log.

This is Part 2 of The Session Record. Previously: where AI-assisted work actually leaves its record. Next: building the session record.