Skip to content

PATTERN Cited by 1 source

Tree-structured conversation memory

Pattern

Persist an agent's conversation history not as a flat list of messages but as a tree, where every message has a parent_id. The shape supports three operations that a flat list cannot:

  1. Forking — branch from any message to explore an alternative direction without losing the original path.
  2. Non-destructive compaction — summarise older messages to shrink the active prompt while retaining full history in storage.
  3. Full-text search across history — query the entire accumulated conversation server-side rather than shuffling it into the prompt.

Storage is per-agent (one tree per session, many sessions per agent), persistent across hibernation, and reachable from the agent itself as a first-class tool surface.

Canonical instance: Project Think Sessions (2026-04-15)

From Cloudflare's 2026-04-15 Project Think launch (Source: sources/2026-04-15-cloudflare-project-think-building-the-next-generation-of-ai-agents):

import { Agent } from "agents";
import { Session, SessionManager } from "agents/experimental/memory/session";

export class MyAgent extends Agent {
  sessions = SessionManager.create(this);

  async onStart() {
    const session = this.sessions.create("main");
    const history = session.getHistory();
    const forked = this.sessions.fork(session.id, messageId, "alternative-approach");
  }
}

Mechanics per the post:

"Conversations are stored as trees, where each message has a parent_id. This enables forking (explore an alternative without losing the original path), non-destructive compaction (summarize older messages rather than deleting them), and full-text search across conversation history via FTS5."

The storage substrate is the agent's own Durable Object SQLite; FTS5 is SQLite's built-in full-text search module.

Three operations the tree enables

1. Forking

At any message, create a child branch that shares ancestors with the original but diverges going forward. Cheap — only the forward-new messages are new storage; ancestor messages are shared via parent references. Lets the agent try "what if I had answered this differently" without destroying the original path.

API shape: sessions.fork(sessionId, messageId, "branch-name").

2. Non-destructive compaction

As a conversation grows, its active prompt hits the model's context window limit. Flat-list handling has two options: drop old messages (lossy) or hit the limit and fail. Tree storage adds a third: summarise older messages into a compact summary node that replaces them in the active prompt, while the original messages remain in storage.

If a later turn needs an old message verbatim, the agent looks it up through a search tool — the summary was a prompt-space optimisation, not a storage-space deletion. This is the "non-destructive" property: compaction is a function over presentation, not an eviction from memory.

3. Full-text search over history

With FTS5 as a storage backend, the agent has a search_context tool that runs SQL full-text queries over the entire session — or across all sessions for the same agent. For a long-lived agent (days, weeks) this turns history into a queryable store rather than a prompt-buffer.

Why each property is load-bearing

  • Forking — exploration-heavy agent use cases (coding agents evaluating design alternatives, research agents comparing hypotheses) benefit from reversible branching. Without forking the agent either commits every intermediate step or reconstructs from a snapshot.
  • Non-destructive compaction — long sessions that repeatedly hit the context limit would otherwise be forced to discard memory; compaction gives prompt-efficiency without memory loss.
  • Search — the alternative (include every potentially-relevant past message in context) defeats the purpose of compaction; the right shape is a small active prompt + a queryable archive.

Operational framing

  • Storage cost — tree nodes + FTS5 indexes grow with session volume. For per-user agents this is per-user cost, not fleet cost. Lifecycle policy (TTL, archival) decision for the product.
  • Compaction as tool — the agent itself (or the runtime) can invoke compaction; the trigger is usually "active prompt > some threshold of the context window."
  • Token accounting in the prompt — Project Think surfaces context-block occupancy in the model's prompt ("MEMORY [42%, 462/1100 tokens]"); the same shape applies to tree-compacted older conversation.
  • Retention policy — some use cases (privacy, regulation) require explicit forgetting; the tree model makes delete-by-branch straightforward and compaction reversible, which is a tension with some delete-for-compliance models.

Prerequisites

  • Persistent per-agent storage with support for:
  • Tree-structured data (parent_id references; any relational DB).
  • Full-text search (FTS5 in SQLite; Postgres tsvector; etc).
  • Summarisation — the compaction step needs an LLM call to compress old messages. The post doesn't detail the summarisation quality / consistency discipline; it's a live design question.
  • Tool surface exposing history to the agent — compaction + forking are only useful if the agent (or the runtime on its behalf) can invoke them; Project Think exposes them as SDK methods + search_context tool.

When the pattern fits

  • Long-running agents (days, weeks, indefinite) with accumulated state.
  • Tasks where branching / alternative-exploration is natural (coding, design, research).
  • Single-user sessions where privacy of one user's history is a product concern — the per-agent DO storage matches this.

When it doesn't

  • Short-lived agents (single-turn, minutes) — tree overhead isn't justified.
  • Multi-agent swarms coordinating via a shared transcript — the tree is per-agent; cross-agent coordination wants a shared topic / message bus instead.
  • Regulated workloads with strict delete-on-request semantics — the non-destructive property makes deletion explicit work rather than default.

Seen in

Last updated · 200 distilled / 1,178 read