Attention is the whole trick

Nearly every AI tool we use daily, from ChatGPT to Gemini, stands on the same idea, published in 2017 by a group at Google under a very modest title: “Attention Is All You Need”. It is now among the most cited papers in the field's history, with more than 100,000 citations.

The core idea is the attention mechanism. Before it, machines read a sentence sequentially, one word at a time, and so tended to lose what was at the beginning by the time they reached the end. Attention changed that: when processing each word, the model looks at all the other words at once and asks itself “which word here matters most to the one I am working on”, then concentrates on the place that carries the meaning. In the sentence “the cat did not chase the mouse because it was full”, to work out who “it” refers to, the model learns to send its attention straight to “the cat”. That ability to attend to the right place, rather than simply to read more, is what gives the “T” in GPT, for Transformer.

I find the principle very close to the work of running things. An organisation's attention and resources are always finite; value does not come from keeping an eye on everything, but from putting the weight in the right place.

This week, where is your attention actually going?

To your desk

A language model is strong not because it reads more, but because it knows to concentrate on the few words that decide the meaning of the whole sentence.

At the end of the day, try looking at your own calendar the same way. Of everything that took time today, which two or three actually decided the outcome of the week? And which merely produced the feeling of being busy?

Focus, in the end, is a way of allocating resources. Attending to the right place usually makes a far bigger difference than trying to do a great deal.