‹ back to writing
Oct 12, 2026·8 min read
essaysagentsmemory

The agent needs a memory.

A bigger context window will not save a production agent. The context window is RAM. Memory is what you write on purpose — with a receipt.

Last Tuesday my operator agent contradicted a constraint I had given it at the start of the session.

Not a soft preference. A hard rule. It had followed the rule for six tool calls, then quietly stopped. The model was the same. The system prompt was the same. The only thing that changed was how much junk we had stuffed into the window while the task ran.

I did what everyone does the first time. I blamed the model. Then I opened the transcript and counted tokens.

memory.policy · live
[09:14:02] constraint.pin → set · turn 1
[09:14:18] tool.result → appended · raw
[09:15:01] tool.result → appended · raw
[09:16:44] context.window → swollen · mid-burial
[09:17:12] constraint.pin → diluted · attention
// The rule did not leave. The window buried it. That is not a smarter-model problem.

That is the whole essay. Production agents do not fail because the model "forgot." They fail because builders treat the context window like a database, and the context window is not a database. It is RAM.

The wrong conversation

The public argument about agent memory has two positions.

The first says buy a bigger window. A million tokens. Two million. Surely the agent will hold everything if you just give it more room. This is usually said by someone whose agent has not yet buried a hard constraint under forty tool results.

The second says "just summarise the history." Compaction as garbage collection — keep the important bits, drop the rest, hope the summary still contains the one sentence that made the agent safe. This is usually said by someone who has not yet watched a wrong conclusion get compacted into a permanent "fact."

Both treat memory as a storage problem for the prompt. That is the mistake. Memory is a write path — what gets extracted, where it lives, who can change it, and how you prove what the agent was allowed to remember.

The rails already agree on the shape

What surprised me this autumn is how quickly the research and the operators converged on the same split.

Agent memory · the rails catch up · 2023–20260 / 5
20232024APRSEPNOW
  1. 2023
    Lost in the middleLiu et al. · attention fails in the middle of long context
  2. 2024
    MemGPT patternActive window + external memory · OS analogy lands
  3. APR
    Compaction = write pathOperators treat memory like production data
  4. SEP
    PaMERPreaction signals · compress + retrieve · ~70% fewer tokens
  5. NOW
    The memory is the APIpin · retrieve · receipt · forget on purpose
// Different papers, different products. Same conclusion: an agent doesn't get a bigger window. It gets a memory.

Liu et al.'s Lost in the Middle result already told us models attend to the start and end of long context more reliably than the middle. A bigger window delays burial. It does not cancel it.

Then the long-horizon agent papers stopped arguing about window size and started arguing about when to compress and what to restore. PaMER (Wang et al., Sep 2026) showed that compression and recall needs are already readable from the model's preaction hidden state — AUROC 0.831 for compression and 0.765 for recall — well above length-and-turn metadata. Keeping only the task prefix plus the two most recent interaction blocks used about 26% of full-context tokens and still preserved most of that signal. On WorkBuddyBench, their controller cut token use by roughly 70% while holding task performance.

Operators are saying the same thing without the AUROC. Clawpilot's line in April was blunt: compaction is now a write path. Not cleanup. A write into production state — with provenance, isolation, retention, and rollback.

An agent doesn't get a bigger window. It gets a memory — bounded retrieval, explicit writes, and a way to forget on purpose.

What a memory actually requires

In my own stack, the gap between "agent with a long context" and "agent with a memory" came down to four things:

  1. Working memory stays small on purpose. The window holds the current task, the last few turns, and a short retrieved set. Raw tool dumps get summarised before they enter. If every API response lands in full, you are filling RAM with files you already read.

  2. Persistent memory lives outside the model. Preferences, hard constraints, and durable facts get written to a store you control — a table, a vector index, a memory service — and retrieved on the next turn. They do not survive as "somewhere in the transcript." Transcripts compress. Exact values die in summaries.

  3. Hard constraints are pinned, not hoped. Anything the agent must never violate is re-injected at the top of the system prompt every call. If the only copy of "never send to a lead without my go" is buried at turn three under twelve tool results, you do not have a policy. You have a coin flip.

  4. Every write has a receipt. Who wrote it, from which session, when, and under what trigger. When the agent does something weird on Thursday, you need to answer "which memory made that legal?" without guessing. Forgetting is a workflow — retention, incident purge, scoped delete — not a vague hope that compaction will drop the bad bits.

The layer that moved

In The agent needs a wallet I wrote that spending is a bag of decisions, and each one wants a different owner. Memory is the same bag one layer up.

The window can hold the active scratchpad. It cannot be your source of truth across sessions, across teammates, or across the moment compaction fires. The protocols and papers can tell you when to compress and what to retrieve. They cannot decide which facts are policy, which are preference, and which were a bad Tuesday that should never have been saved.

If you're building agents this year, stop asking whether yours needs a bigger context window. Ask the better question: what's its memory — and which writes does it never get without you?

Write that schema first. The window takes care of itself.

Bi-weekly essays

Owned, undistributed, and published only here — no newsletter, no funnel, no upsell.

All essays ›
‹ all essays

Let's build
something joyful.

Agents · Engineering · Talks · Podcasts · Interviews · Automations.