Answering from your own documents

This is the most fundamental cure for the "chatbot that keeps making things up" that many people hit when bringing AI into their work.

What retrieval-augmented generation is

Picture a very quick-witted sales adviser who has not read the company's internal documents. Ask anything and they answer fluently, but sometimes they state the policy wrongly because they are guessing from general knowledge. Now, before each customer question, we require them to open the right page of the company's documents and read it before answering. The quality changes immediately.

That is retrieval-augmented generation, usually shortened to RAG. The technique was introduced by the Facebook AI Research group in 2020, in a paper by Lewis and colleagues at the NeurIPS conference, and that paper gave it the name still used today. Rather than letting the model answer from a hazy memory, the system looks up the relevant passage first, and only then has the model compose an answer based on it.

How this applies to running a business

I find retrieval-augmented generation is the most practical bridge between AI and a business's own data. An AI assistant tied to the company's documents, policies, price list and frequently asked questions will answer from material you have approved, rather than inventing.

What matters is that it is only as good as the library behind it. Documents that are outdated, contradictory or missing will skew the answers accordingly. So before dreaming of an AI assistant that knows everything, the work worth doing is tidying and updating your own body of knowledge.

If an AI assistant were only allowed to answer from the documents your company has today, how much of it would it get right?

To your desk

Before blaming an AI assistant for a wrong answer, there is a question worth asking first: have I given it the right documents to read? Most invented answers come not from a weak model but from having no source to stand on.

A small thing to try this week: pick a question customers keep repeating, gather the few documents that actually answer it, the price list, the policy, the guide, and then require the AI to answer only from those. You will see a clear difference between an answer with a basis and an answer that merely sounds good.