Model Memory Wall 3/4: What to Keep and What to Throw Away
The five stages of working with models, and the wall everyone eventually hits (Part 3)
---
In the previous part, we found the answer. What the model lacks is a drawer, in other words, memory. But is attaching a drawer enough? No. A badly built drawer can be worse than having none.
The most common misunderstanding is this: "Then we should just save everything." Every conversation, every sheet of paper, stuffed into the drawer without omission. The result is miserable. If you have to search the entire drawer to find the one page you actually need, that is no different from filling the desk with paper again. You have returned to the collapsing desk you were trying to escape in the first place. Storage is not memory. A good drawer has discipline.
First, it chooses what to keep. Not every sentence in a conversation deserves to be remembered. Small talk should pass through. Decisions, principles, and turning points in context should remain. And they should not remain as whole transcripts, but as the smallest meaningful units. Several small cards that can be retrieved precisely are better than one large sheet of paper.
Second, it does not damage what it keeps. This is the hardest and most important part. Once a memory is written, it should not be casually deleted, rewritten, or reordered. If yesterday's decision quietly turns into something else today, you will never trust that drawer again. The value of memory comes not from being perfect, but from not changing behind your back. Untrustworthy memory is not memory. It is rumor.
Third, it retrieves only the one page needed at the right moment. A good drawer identifies what is relevant to the work at hand and brings that back. It does not mix in unrelated papers or let irrelevant signals distort the ranking. It quietly places only the memories that truly resemble the current task back onto the desk.
What happens when a drawer with these three disciplines is attached beside the desk?
The desk no longer collapses when it fills up. Old papers do not fall to the floor. They enter the drawer in an orderly way and return to the right place when needed. The principles agreed on yesterday remain alive today. Last week's design becomes the premise for this week's work. The circling stops. Instead of turning around the same point, the project finally starts moving straight forward.
The genius with amnesia becomes a genius who remembers yesterday. The difference is not intelligence. It is one drawer.
Once you get this far, a natural question follows. What if everyone builds this drawer in their own way? What if the memory I built with one model cannot be carried to another model? What if the drawer disappears the moment I switch tools? In the final part, we will talk about the last wall memory hits. And perhaps the largest one.