Passing on the core

This is the concept that convinced me "small enough to do the job" is often wiser than "large and cumbersome".

What knowledge distillation is

Picture a master of several decades, whose head is full of judgements hard to put into words, standing beside a young worker and pointing. The young one does not need to learn again everything the master once learned, they only need to catch the core well enough to do the work. Knowledge distillation in AI is just that: a small model, called the student, is trained to imitate how a large model, called the teacher, answers, and so keeps most of the ability in a far lighter form.

The subtle part is that the student learns not only the right answer but also how the teacher hesitates. The idea was set out by Hinton and colleagues in 2015. A well-known example is DistilBERT from Sanh and colleagues in 2019: the student is 40% smaller and runs 60% faster while keeping around 97% of the language understanding of its teacher, BERT.

How this applies to running a business

I find many businesses assume that a good result requires the largest, most expensive tool. But for most everyday work a small, cheap, fast model is both sufficient and far kinder to the budget. Choosing the right size of tool for the job is a management decision, not only a technical one.

The wider lesson is in how an organisation passes on a craft. The most capable person does not have to do everything by hand. The greater value is distilling how they work into a tidy process others can reuse. That too is a kind of distillation, and it stops capability from ending with one individual.

In your team, whose knowledge is sitting in one or two people's heads without yet being distilled into something the whole team can use?