When the chat fills up

Session: twenty minutes. You need: a long conversation about a piece of work that's still in progress. If you don't have one, keep the same chat open for a few days on real work and come back here.

With every message the model rereads the whole conversation from the start, because it keeps nothing between one reply and the next. The space for this rereading has a limit, and even before the limit its attention thins out: the things said in the middle of a long chat are the ones the model uses worst. Today you learn to recognise the moment and to switch chats without losing the work.

1. The three signals

The assistant asks you for information you'd already given it. It does something that contradicts a decision taken shortly before. It gives a generic answer to a question that needs the context of the conversation. Any one of the three is enough. Explaining again every time doesn't fix it: the space stays full, and the new explanation fills it even more.

2. The measure

In the long chat, write: "Summarise in five lines the goal of this work, where we've got to, the decisions taken and the next steps." Compare the summary with what you remember yourself. Whatever is missing, or comes back in a different version, has dropped out of the model's attention or, if the chat is very long, out of its context window.

3. The handover note

Ask for the document you need to start again: "Prepare a handover note for picking up this work in a new chat: goal, current status, decisions taken with their reasons, documents and data used, next steps, things already tried that shouldn't be repeated. Write it for someone who hasn't read any of this conversation." Reread it and correct it yourself: from here on it's your document, and an error left in it passes into the new chat as a fact.

4. The new chat

Open a new chat, paste the handover note and, as your first message, ask: "What do you need to know that isn't here?" The questions it asks you point to the gaps in the document. Fill them, and carry on with the work.

5. Outside the chat

Save the handover note in a dated file. On long work the best remedy is not to reach saturation at all, and the rule that counts most is one: one chat per topic. Mixing the bid, the note for your boss and the question about the contract in a single conversation fills the window with things the question of the moment doesn't need, and the model rereads them all with every message: the answers get worse sooner, and when it makes a mistake you no longer know which part of the conversation the mistake comes from. One chat per topic keeps conversations short, makes the history readable when you have to find something again, and lets you throw away a chat that went badly without losing the rest. Every so often ask for the same summary halfway through, and keep documents, decisions and numbers outside the chat. The chat is for reasoning together, not for archiving.

What you just saw

The context window has two different measures: how much text fits in it, which has grown enormously in three years, and how well the model uses what's in it, which has grown much less. Chapter 4, "What sits around the model", explains this in the sections "The context window" and "Capacity and attention"; chapter 5 has the section "When the chat fills up", where the handover note comes from.

In the book. Chapter 5, section 5.2 "When the chat fills up", page 98; chapter 4, sections 4.2 "The context window" and 4.3 "Capacity and attention", pages 88 and 89. The book on Amazon.