Outdated knowledge and overconfidence
You spot an answer based on old knowledge and you neutralise the model’s tendency to agree with you.
ObjectiveBy the end of this lesson, you will be able to spot an answer based on old knowledge and phrase your questions to get criticism rather than confirmation.
A model is trained on texts up to a certain date. After that, it learns nothing more on its own. Laws change, prices move, companies merge, recommendations are updated. Without a search tool, the model answers with the state of the world at its training date, without necessarily saying so.
Spotting outdated knowledge
- The answer cites no source although it concerns a topic that changes.
- The steps feed shows no search.
- The examples seem old or the figures too round.
- The model describes in the present tense a situation that may have changed.
The date of a source matters as much as its content. A five-year-old official page can describe a rule that no longer exists. Always ask for the date, especially for law, prices and recommendations.
Overconfidence
The second flaw is subtler. A model writes in the same confident tone whether it is right or wrong. It rarely qualifies its statements on its own and it tends to go along with the question. If you ask why a method is the best, it will explain why, even if it is not.
Why is our training plan good?
Here is our training plan. Identify its three weakest points and what a demanding outside reviewer would criticise. Be direct.
You ask for criticism instead of confirmation, which neutralises the model’s tendency to agree with you.
Learnya counters this with challenge. For important tasks, a dedicated agent looks for what contradicts the conclusion. Its objections give you what the model does not offer spontaneously: the limits of the answer and the reasons to doubt it.