Going deeper

The context is a budget

Everything competes for the same window: instructions, tool definitions, history, tool results, and the room left for the answer. Tool results are usually what overflows it. One database query can return more text than the whole conversation so far.

Three techniques, from cheapest to most costly:

• Trim tool output at the source: return the fields the task needs, plus a way to fetch more.
• Remove old messages: simple, but it must keep each tool_use with its tool_result, as the lab shows.
• Compact: ask a model to summarize the older part. It keeps more meaning but costs a call and can lose an exact figure, so keep identifiers and decisions outside the summary.

What to recall

Searching memories by similarity alone brings back old, trivial matches. The paper on Generative Agents (Park et al., 2023) combines three signals:

score = α × recency + β × importance + γ × relevance
recency = decay ^ hours_since_last_access

Recency decays exponentially. Importance is rated when the memory is written, often by the model itself. Relevance is the similarity to the current situation. The scales differ, so normalize them before adding them up. That paper’s weights are a starting point, not a result that transfers to your data.

Writing is the hard part

Reading memories is a search problem. Writing them is a trust problem. An agent that saves whatever it reads can be taught something false by a single document, and it will keep believing it in later sessions. Record the source of each memory, set an expiry date for facts that age, and let contradictions replace old entries rather than pile up.

Personal data adds a legal limit. Under the GDPR, a person can ask for their data to be deleted, so you must be able to find every memory about them. That’s much harder once their details have been dissolved into summaries.