How to check AI for errors: spotting what it made up

A model does not know when it is guessing. That is the whole problem, and it is why knowing how to check AI for errors matters more than any prompting technique. The wrong sentence arrives in the same register as the right one. No wobble in the prose, no hedge, no smaller font on the invented number.

The short version: you do not verify everything. You verify the four things it invents, and you do it in the two minutes before the work leaves your hands.

Stop reading for the tell, there isn't one

People try to develop a feel for it. They read the output looking for a seam, some flatness that gives the fabrication away. The seam is not there. The fluency comes off the same machinery whether the underlying claim is a well-documented fact or a plausible-shaped hole, so an invented regulation reads exactly like a real one: same rhythm, same air of citation, same calm.

That is also why the error gets past your own review. You read it back in your own voice, it sounds like something you would have written, and you send it.

The dangerous output is never the one that sounds wrong. It is the one that sounds exactly like you.

How to check AI for errors in under two minutes

Four things, in this order:

• Numbers. Every figure, percentage, date and currency amount, and especially any arithmetic done across a table you pasted in.
• Sources. Names of regulations, section numbers, report titles, page references. If it cites something, the citation is the most likely part to be invented.
• People and firms. Job titles, company names, who said what, which bank did which deal.
• Quotes. Anything inside quotation marks is unverified until you have seen it in the original.

The rest (structure, phrasing, the argument, the summary of a document you supplied) is far more reliable. Spread your two minutes evenly and you will spend most of them on the safe part.

Where it invents most

The pattern is consistent: fabrication rises with how specific your question is and falls with how much of the answer exists in public. Ask for a general explanation of hedge accounting and it is solid. Ask for the paragraph number in the standard and you are exposed. A listed company's last results are usually fine. A mid-sized private Hong Kong firm gets an answer anyway, because producing something is what it does.

Local regulation is the sharpest edge here. HKMA circulars, SFC codes, PCPD guidance, section numbers in the PDPO. These have precisely the shape a model can imitate and not enough weight in the training data to be dependable. When I wrote about whether ChatGPT is safe for work data, every framework name, date and section number in it was pulled up and read by hand. Ask a model for that same list and you get one that looks identical and cannot be relied on.

The habit that does most of the work

Ask for the claim and its source in the same request. "For each figure, give me the source and the date, and flag anything you are not confident about." The flagging is imperfect, but it is not random, and it puts the weakest claims where your eye lands first.

The second habit matters more. Give it the document rather than asking it to remember. A model summarising a filing you pasted in behaves very differently from one recalling that filing from training, and most of the invention I see comes from the second mode. The prompts that force sourcing and summarise from a supplied file are in the free prompt library.

What this does not fix

Verification catches invention. It does not catch a confident, checkable, wrongly framed answer, and that failure costs more. Every number in a summary can be correct while the one qualification that changed the meaning has quietly gone, which is the same trap as AI note-taking for meetings: accurate line by line, wrong as a record.

No checklist rescues you from that. Reading the output as an analyst rather than a proofreader does.

Why this is the first habit worth building

Thirteen years at a bank taught me that the review step is where credibility is won and lost, and nobody remembers that the draft came out of a machine. The people who get the most out of AI are not the ones with the cleverest prompts. They built a ten-second reflex in week one, and now trust their own output enough to move fast on it.

That reflex is specific to the work you actually do and the errors your desk cannot afford. If you want to build it inside your real workload, start here.