What fits on the desk at once

You hand a very capable person your company's entire file and ask them to find one detail in the middle. They read all of it, and remember the beginning and the end best. Language models do the same, and this has been measured properly.

What a context window is

Picture a desk. The top is exactly this wide, and every file you need to hold side by side has to sit on it. Put one more folder down and another has to leave the desk.

The context window is that desktop for a model: the most text it can hold at one time, counted in tokens, covering both what you put in and what it writes back. Go past that and the older part is pushed off, and the model no longer remembers it.

This desk has grown very fast. GPT-3, released in June 2020, held 2,048 tokens. By February 2024 Gemini 1.5 Pro was taking a million tokens, and by May of the same year, two million.

But wider does not mean better used. Liu and colleagues, in work published in the journal TACL in 2024, moved the position of the needed information around inside a long context and found a U-shaped curve: the model answers best when that information sits at the very beginning or the very end, and drops off clearly when it sits somewhere in the middle. The longer the context, the weaker the result, even for information that is plainly relevant.

How this applies to running a business

I find this mirrors exactly how an organisation takes information in. Every person, every meeting has a desk of a finite size. Piling more on does not widen the desk, it pushes something else out of view, usually the thing nobody notices has dropped.

The U-shaped curve is familiar too. In a long presentation or a two-hour meeting, what gets remembered is the opening and the close, while the middle thins out. The most important thing, placed in the middle, will almost certainly drift away.

Is the most important thing in your week sitting at the beginning, at the end, or somewhere in the middle?

To your desk

A model does not remember more simply because it was given more. Past a threshold the older part is pushed out, with no notice at all.

Something to try this week: open the minutes of your longest recent meeting and mark the minute at which the most important thing was said. If it falls in the middle, move it next time, to the opening or to a close that settles it. The same content, only moved, and people keep more of it.