Garbage in, garbage out

The field has a stock phrase so blunt it sounds at first like a joke: garbage in, garbage out. The point is that however sophisticated a model is, it cannot be better than the data it was fed. However gifted the cook, if the ingredients have turned, the dish will struggle.

The more I study, the more I see that most of the effort in this field does not sit in designing something impressive, but in the much quieter work: cleaning the data, removing duplicate records, patching the gaps, settling on one way of naming things. A frequently cited estimate holds that data workers spend around 80% of their time on preparation and cleaning alone, before a model learns anything worth having. The reason is practical: a small distortion at the input is magnified into a large one at the output.

This is as true of organisations as of machines. According to the research firm Gartner, poor quality data costs each business an average of around $12.9 million a year, mostly not through any loud incident but through decisions made on wrong figures that nobody caught in time. An outdated customer list, a report that merged the wrong sources, a metric measured the wrong way: each quietly nudges the person in charge slightly off course.

So before hoping AI will produce better answers, perhaps the question to ask first is: how trustworthy is the data we are already basing decisions on?

To your desk

A model is only as good as the data it learned from. Our work is the same: a decision is only as good as the information it rests on.

This week, try picking one report you use regularly to decide things, sales, stock, or customer feedback, and trace exactly one number in it back: where it came from, when it was last updated, whether it merged the wrong source anywhere.

Usually the trouble is not a shortage of data. It is that we trust a number we have never once checked the origin of.