Post cover image

July 27, 2026

Your Agent Forgets Everything: Watch What It Costs to Remember

My briefing agent had a beautiful digest and the memory of a goldfish.

By Sourav Bhattacharjee

10 min read

I would ask it "tell me more about the second story," and it would cheerfully reply, "Which story? I don't have any stories." We had literally just spoken. One question earlier it handed me twelve of them. By the next line it had forgotten the entire conversation.

This is not a bug. This is the default. And once you understand why, you understand the single most misunderstood thing about LLM agents: they don't remember anything. Not one word. Every turn is the first turn unless you build the memory yourself.

This is Part 2 of building a morning-briefing agent in LangChain 1.x. In Part 1 we built the thing that produces the digest and squashed three bugs that made it lie to us. Here we make it hold a conversation, and then we add the one piece of measurement every agent needs and almost none of them ship with: a way to see what each turn actually costs.

You can read this on its own. All you need to know from Part 1 is that we have a working agent, built with create_agent, that produces a daily news digest.

None

Table of contents

The goldfish problem

Let's isolate the problem with the smallest possible experiment. Forget the news agent for a second. Two messages, one after the other:

USER:  My favourite fruit is mango.
USER:  What is my favourite fruit?

A human answers the second question without thinking. An agent, by default, cannot. Here is a tiny script, memory_demo.py, that runs exactly those two turns through a real create_agent:

def turn(agent, user_msg: str, config: dict | None) -> None:
    print(f"\nUSER:  {user_msg}")
    result = agent.invoke(
        {"messages": [{"role": "user", "content": user_msg}]},
        config=config or {},
    )
    print(f"AGENT: {result['messages'][-1].content}")
# ...build a normal agent, no special arguments...
turn(agent, "My favourite fruit is mango.", None)
turn(agent, "What is my favourite fruit?", None)

Run it with no special flags:

python memory_demo.py
WITHOUT MEMORY (each invoke starts from an empty message list)
USER:  My favourite fruit is mango.
AGENT: That's great! Mango is delicious.
USER:  What is my favourite fruit?
AGENT: I don't know - you haven't told me that.

Read that last line again. We told it. It even responded warmly. And it still has no idea, because the second agent.invoke(...) started from a blank message list. The first exchange never happened, as far as turn two is concerned.

Here's the mental model that fixes this for good: an LLM call is a pure function. You give it a list of messages, it gives you back the next message. It keeps nothing. There is no hidden "conversation" living inside the model. If turn two is going to know about turn one, you have to hand turn one's messages back to the model. Memory is not something the model has. It is something you carry.

Memory is one keyword argument

The good news, and the mildly dangerous news, is that LangGraph makes carrying that history absurdly easy. You add a checkpointer.

None

A checkpointer is a place to save the conversation state after every step. Attach one and LangGraph will, on each invoke, load the saved messages, append the new ones, run the graph, and save the result back. The whole "hand turn one's messages back to the model" chore is done for you.

For local development the simplest one is InMemorySaver, which keeps everything in a Python dict:

from langgraph.checkpoint.memory import InMemorySaver

checkpointer = InMemorySaver()
agent = create_agent(
    model=llm,
    tools=[echo],
    system_prompt=PROMPT,
    checkpointer=checkpointer,   # <- the entire difference
)
config = {"configurable": {"thread_id": "demo-user-1"}}

Same two turns, this time adding the --memory flag so the agent is built with the checkpointer:

python memory_demo.py --memory
WITH MEMORY (InMemorySaver + thread_id='demo-user-1')
USER:  My favourite fruit is mango.
AGENT: Mango, nice choice!
USER:  What is my favourite fruit?
AGENT: Your favourite fruit is mango.

One argument. That's the whole gap between amnesia and recall.

And here's my mildly dangerous opinion: because memory is this easy to switch on, most people switch it on everywhere and never think about it again. That's a mistake, and the second half of this article is about why. Memory is not free. Every remembered turn is context you pay to re-send on the next turn. Easy to add is not the same as free to run.

thread_id is the whole trick

Look again at that config. The interesting part isn't InMemorySaver, it's thread_id.

config = {"configurable": {"thread_id": "demo-user-1"}}

The thread_id is the name of the conversation. The checkpointer stores history keyed by it. Use the same thread_id across invokes and you continue the same conversation. Use a different one and you get a clean, independent thread with zero shared memory.

That single string is how one process serves many separate conversations without them bleeding into each other. In a real app it's your user ID, or a session ID, or chat-2026-07-12. Get it wrong in the fun way and user A starts seeing user B's history. Get it wrong in the boring way and every request spins up a fresh, amnesiac thread and you're back to the goldfish, wondering why memory "isn't working" when it's working perfectly on a thread you never reuse.

Two practical rules I now follow without exception:

  • The thread_id is the conversation. Decide what a conversation is in your app, and derive the id from that. Don't let it be an afterthought.
  • Different users must never share a thread_id. It's a memory key, so it's also a privacy boundary.

Wiring memory into the briefing agent

Back to the real agent. I wanted a --chat mode: produce the daily digest, then drop into a simple prompt loop where I can ask follow-up questions about today's news. Memory is the thing that makes "the second story" a sentence the agent can actually resolve.

The setup is exactly what the demo taught, with a thread_id scoped to the day:

checkpointer = InMemorySaver()
chat_agent = create_agent(
    model=llm,
    tools=[search_todays_stories],
    system_prompt=CHAT_PROMPT,
    checkpointer=checkpointer,
)
thread_config = {"configurable": {"thread_id": f"chat-{today}"}}

Then a plain input loop, reusing thread_config on every turn so the whole chat is one continuous thread:

while True:
    user_input = input("YOU: ").strip()
    if user_input.lower() in {"quit", "exit"}:
        break
    result = chat_agent.invoke(
        {"messages": [{"role": "user", "content": user_input}]},
        config=thread_config,
    )
    print(f"AGENT: {result['messages'][-1].content}")

Now "tell me more about the second story" works, because turn two can see turn one. Ask a follow-up to the follow-up and it holds. The goldfish finally remembers.

I also added a --chat-only [DATE] fast path that loads a previously saved digest from disk and jumps straight into chat, skipping the whole fetch-and-write pipeline. When you're iterating on conversation behavior, you do not want to re-run the news gathering every single time. Save the expensive output once, replay it into chat as often as you like.

Try it against the digest you already built:

python briefing.py --chat-only
============================================================
CHAT MODE - ask follow-up questions about today's briefing.
Type 'quit', 'exit', or press Ctrl+C to leave.
============================================================
[chat] Briefing loaded. Ask away.

YOU: tell me more about the second story
AGENT: ...

Type quit or exit when you're done.

Seed it with the summary, not everything

Here's the design decision I want you to notice, because it's the hinge the whole chat design turns on.

When chat mode starts, I have twelve full article bodies sitting in memory. The obvious move is to stuff all twelve into the agent's context so it can answer anything. I deliberately did not do that.

Instead, the very first message on the thread seeds the conversation with the digest only:

seed_lines = [
    f"Here is today's briefing for {today}. Use this - and only this - to",
    "answer any follow-up questions I ask.",
    "",
    "=== FINAL DIGEST (Markdown) ===",
    digest,
]
chat_agent.invoke(
    {"messages": [{"role": "user", "content": "\n".join(seed_lines)}]},
    config=thread_config,
)

The digest is a few thousand characters. The twelve full bodies are tens of thousands. If I seed the bodies, every single chat turn re-sends all of them to the model, forever, whether the question needs them or not. That is the memory cost I mentioned earlier, paid in full on every turn.

So the deal I struck: seed the cheap summary, and make the expensive bodies available on demand through a tool called search_todays_stories. When a question actually needs a specific detail, the agent goes and fetches only the relevant slice. When it doesn't, we never pay for the bodies. The principle is the part worth keeping: put the small thing in context, keep the big thing one tool call away.

For this article, treat search_todays_stories as a black box: hand it a query, get back the most relevant chunks of today's articles. How it actually finds those chunks — embeddings, a vector index, similarity search — is a whole topic of its own, and it's the subject of the next part. Here, all that matters is where the bulk lives: behind a tool call, not in memory.

You can't manage what you can't see

I made a claim twice now: memory costs money, every turn re-sends the history. Claims are cheap. I wanted the number on my screen after every turn. So before adding anything clever, I added a way to measure it.

Here is the thing nobody hands you: the agent already records exactly what each call cost. Every AIMessage that comes back from create_agent carries a usage_metadata dict with input_tokens and output_tokens. And the checkpointer already holds the entire message history for the thread. Put those two facts together and you can compute the running bill without any external service.

The read is agent.get_state(config), which returns a snapshot whose .values["messages"] is the full history:

def get_chat_stats(agent, config: dict) -> dict:
    snap = agent.get_state(config)
    msgs = snap.values.get("messages", [])
    ai_msgs = [m for m in msgs if type(m).__name__ == "AIMessage"]
    last = (getattr(ai_msgs[-1], "usage_metadata", None) or {}) if ai_msgs else {}
    cum_in = sum((getattr(m, "usage_metadata", None) or {}).get("input_tokens", 0)
                 for m in ai_msgs)
    cum_out = sum((getattr(m, "usage_metadata", None) or {}).get("output_tokens", 0)
                  for m in ai_msgs)
    return {
        "n_messages": len(msgs),
        "last_input_tokens": last.get("input_tokens"),
        "last_output_tokens": last.get("output_tokens"),
        "cumulative_input_tokens": cum_in,
        "cumulative_output_tokens": cum_out,
    }

Then one line printed after every chat turn:

[stats] history: 8 msgs, 25,241 chars | this turn: in=9103, out=592 | cumulative: in=26,574, out=647

That single line changed how I think about the agent. Three things jump out of it:

  • history: N msgs tells you the conversation shape. It grows by 2 on a plain turn (your question, the answer) and by 4 when a tool fires (question, the tool-call message, the tool result, the final answer). You can see whether the agent reached for a tool without any tracing.
  • this turn: in= is the number I care about most. That's the tokens the model was billed to read this turn, and it includes the entire history re-sent. Watch it, and the cost of memory stops being a claim and becomes a graph.
  • cumulative: in= is what the thread has cost you so far. This is the real spend.

Strong opinion, and I'll die on this hill: token counting should be on from the first line of code, not bolted on after the bill scares you. It's about fifteen lines. It reads data the framework already collected. There is no reason to fly blind, and every agent I've seen ship without it eventually surprises its owner with an invoice.

Watching the bill climb

Now the fun part. Start a session, then keep asking specific questions, each one triggering the search tool, and just watch this turn: in= climb:

python briefing.py --chat-only
turn 1  ->  in=3,101
turn 3  ->  in=5,940
turn 5  ->  in=8,700
turn 7  ->  in=11,520

Nobody changed the question difficulty. The questions are all roughly the same size. The input cost climbs anyway, turn after turn, because every turn re-sends everything that came before it. Memory only adds up. It never shrinks on its own.

This is the moment the whole design clicked for me. Memory is what makes an agent feel like it's talking with you instead of at you. It's also a slow, growing cost on every future turn. The more it remembers, the more each next answer costs. And if I had seeded those twelve full article bodies into the thread the way my first instinct wanted, that curve wouldn't start at 3,000 tokens. It would start at 20,000 and climb from there.

So the seeding choice from earlier pays off right here. Because only the small digest lives in memory and the full bodies sit behind the search tool, the cost line starts low and stays manageable. The agent can still reach any detail it needs, but it only pays for what a given question actually touches. Memory is what gives you the conversation. Keeping the bulk out of memory is what keeps that conversation affordable.

What to remember

ConceptWhy it mattersLLM calls are statelessThe model keeps nothing between calls. Memory is history you re-send, not something the model has.checkpointer=One argument on create_agent turns amnesia into recall. InMemorySaver is the local default.thread_idNames the conversation. Same id continues it, different id forks a clean one. It's also a privacy boundary between users.--chat-only replaySave expensive pipeline output to disk; replay it into chat so you can iterate on conversation without re-running everything.Seed small, fetch big on demandPut the summary in context, keep bulk documents one tool call away. Don't pay for context you might not use.usage_metadata on AIMessageThe framework already records input_tokens/output_tokens per call. Free measurement, waiting to be read.agent.get_state(config)Reads the full saved history for a thread. Combine with usage_metadata for a running cost line.Memory only growsInput tokens climb every turn because history is re-sent. Watch the number, or the invoice will.

The code

Everything is at github.com/souravb65/morning-briefing-agent.

It's the same repo from Part 1, plus memory_demo.py:

LangChainAgent/
├── briefing.py         # now with --chat / --chat-only and stats output
├── memory_demo.py      # the amnesia-vs-memory experiment
└── articles/           # these write-ups

To feel the whole thing yourself:

.\.venv\Scripts\Activate.ps1
python memory_demo.py            # forgets everything
python memory_demo.py --memory   # remembers
python briefing.py --chat        # build today's digest, then chat about it
python briefing.py --chat-only   # skip the build, chat about the last digest

Ask a few follow-ups in chat mode and watch the [stats] line after each answer. Watching this turn: in= climb is the fastest way to feel, in your gut, what conversational memory actually costs.

Try the two memory_demo.py runs back to back and tell me your agent's answer to "what is my favourite fruit?" in each. If the no-memory run ever guesses "mango" correctly, I want to see it, because that would mean something genuinely weird is happening and I'd love to dig into it.

And once your agent remembers, keep an eye on that this turn: in= number after every answer. The day it climbs faster than you expected is the day you'll really feel what memory costs.

Next up in Part 3: what's actually inside search_todays_stories — turning those article bodies into a searchable index so the agent fetches only the slice a question needs. That's retrieval, and it's where this series goes next.

Happy building.