Lesson 8 of 12 · 7 min

Reading an answer with a critical eye

You know where to look first to spot errors in a result produced by an agent.

ObjectiveBy the end of this lesson, you will be able to review an agent’s work by checking figures, dates, proper names, quotes and calculations first, with an effort that matches the stakes.

A language model writes with the same confidence whether it is right or wrong. How fluent a text is says nothing about its accuracy. Verifying means reading the result as you would read the work of a new colleague: with goodwill, and without taking anything for granted.

What to look at first

Figures, dates and proper names: these are the most frequent and most costly errors.
Quotes and references: do they exist, and do they really say that?
Sweeping statements: “all companies must…”, “the law requires…”.
What is missing: a point from your request that was forgotten, an option dismissed without reason.
Calculations: a total, a percentage, a conversion.

Match the effort to the stakes. A draft internal email can be skimmed. A note for a client, a lesson for two hundred learners or a figure that goes into a report must be checked point by point.

Less effective

Is this OK?

More effective

Reread your answer. List every figure and every factual claim, with the precise source that supports it. Mark [TO CHECK] anything that has no source.

You turn a vague review into an inventory you can check quickly.

Reading the step trail

Also look at how the result was obtained. If the agent states a conclusion but no search appears in the trail, the conclusion comes from its general knowledge, which may be out of date. If a calculation was done with a tool, such as a spreadsheet, the result is more reliable than if it was written from memory.

An error you spot is not a failure of the tool. It is verification doing its job. Point it out to the agent, it will correct it and you will know where to look next time.

Finally, be wary of your own agreement. An answer that confirms what you already thought gets read less carefully. That is precisely the one that deserves a second reading.