Large language models do not have memory. Nothing persists between calls unless something outside the model explicitly writes it down and hands it back later. Everything that feels like an AI tool remembering you is engineered, not learned by the model itself.
That fact surprised me more than it should have, given how much time I spend building on top of these models. Claude feels like it remembers our last conversation because a system around the model is retrieving and re-injecting the right pieces, not because anything changed inside the model, which is the same mechanism behind the second brain I rely on to keep a one person business from losing context between sessions.
The four kinds of memory a real system needs
Production AI agents need four distinct kinds of memory, and conflating them is where most homemade memory systems go wrong. Working memory is the current context, whatever is in the conversation right now. Episodic memory is specific past events, what happened, in what order, with what outcome.
Semantic memory is extracted facts and preferences, the durable stuff that survives long after the original conversation is gone. Procedural memory is the agent’s own instructions and learned behavior, how it should act in a given situation without being told again.
The distinction between episodic and semantic trips people up the most. “The user mentioned their daughter’s birthday last week” is episodic, a specific event tied to a specific moment in time. “The user is a parent” is semantic, a stable fact distilled out of that event.
Semantic memory is denser and easier to retrieve relevantly, which is exactly why systems compress episodic history into semantic facts over time rather than keeping everything as raw events forever. Procedural memory gets left out of most memory discussions entirely, but it matters just as much.
It is the how-to knowledge, the operational playbooks and tool-use routines that make an agent behave consistently instead of reinventing its approach every session. In practice this often lives as plain instruction files rather than anything resembling a database or vector store.
How the storage actually works
The dominant production pattern is tiered, as a 2026 engineering guide covering Letta, LangMem, Mem0, and Zep lays out in detail. A small always-in-context core holds whatever must never get missed. A vector-store-backed retrieval layer holds everything else, searchable by meaning rather than exact keyword match against stored text.
An explicit forgetting policy decides what ages out, because a memory system that never forgets anything becomes slower and less relevant over time, not more useful the way it might sound like it should.
This tiered structure exists because of a real cost problem, not elegance for its own sake. Feeding an entire history back into a model every time is expensive and eventually exceeds what any context window can hold no matter how large it grows.
The system has to decide, every single call, which slice of memory actually matters for this specific question being asked right now.
That decision is retrieval, and it works by turning both the stored memories and the current question into vectors, lists of numbers representing meaning, then finding the closest matches between them.
Ask about a past decision and the system searches for memories that are semantically close to the question, not memories that happen to share the same literal words as the question you typed.
Why this matters more than most explanations let on
Every serious memory architecture write-up mentions the four memory types and the tiered storage pattern. Fewer mention what happens when the forgetting policy is wrong, which is where most systems that feel unreliable actually break down in practice.
Retrieve too narrowly and the agent seems to have amnesia about things it should obviously know already.
Retrieve too broadly and irrelevant history crowds out what actually matters for the current question, which shows up as an agent that seems distracted or confused even though technically nothing was forgotten at all.
Getting the retrieval width right matters more than which specific vector database product you pick.
How this shows up in my own system
My own vault functions as exactly this kind of tiered memory system, even though I did not design it by reading an academic memory architecture paper first.
The always-in-context core is a handful of files Claude reads at the start of every session, who I am, the hard rules, what is currently in flight.
Everything else lives in a searchable vault that gets pulled in only when a specific task needs it, not loaded wholesale into every conversation regardless of relevance.
The forgetting policy is the part most personal systems skip entirely, and it is also the part that matters most in practice. A decision made weeks ago stays settled unless something explicitly reopens it, rather than getting silently re-derived from scratch every session.
That is functionally the semantic-memory compression step happening through deliberate documentation instead of an automated summarizer agent doing it on a schedule.
The gap between my system and a proper agent memory architecture is real and worth naming honestly rather than glossing over. Mine relies on me writing things down clearly and Claude reading the right files, not automatic vector retrieval choosing what matters on its own.
It works because the scope is one person’s business, not because the underlying mechanism matches what a production multi-user agent actually needs at scale.
Where the two versions genuinely differ
A production memory system for something like a customer support agent needs automatic consolidation, because no human is going to manually summarize ten thousand conversations into semantic facts by hand. My system gets to skip that automation entirely because the volume is human-scale.
I can write a settled-decision note myself in the time it would take to configure a summarizer agent correctly, which changes the calculus completely for a one person operation like mine.
The tradeoff runs the other direction too. Automatic retrieval scales to volumes no human curation could ever match, but it inherits every failure mode of retrieval width being wrong.
Manual curation scales worse but fails in ways a human notices and fixes immediately, since I am the one reading my own notes back later.
LLM memory, quick answers
Do large language models actually remember anything between conversations? No. Nothing persists inside the model itself. Anything that feels like memory is a system outside the model retrieving and re-injecting relevant information before each call it makes.
What is the difference between episodic and semantic memory? Episodic memory is specific past events tied to a moment. Semantic memory is stable facts distilled out of many events. Systems compress episodic history into semantic facts because facts are denser and easier to retrieve accurately.
Why do AI agents need a forgetting policy? Without one, retrieval gets slower and less relevant as history grows, since irrelevant old context competes with what actually matters for the current question. Forgetting is a feature here, not a limitation to work around.
Can a manual, written system work as agent memory? Yes, at human scale. It trades the ability to handle massive volume for retrieval failures a human notices and corrects immediately, a reasonable tradeoff for a one person operation.
Where this runs
I did not design my vault by implementing an academic memory architecture on purpose.
I built it to stop losing track of my own decisions, and it turned out to match the real structure production AI memory systems use, tiered storage, explicit forgetting, and a compressed core that survives every session.
If you want to see what that looks like running day to day, join the AIOS waitlist.
