Reasoning step by step
Why asking for the intermediate steps improves answers to multi-step problems.
ObjectiveExplain the effect of step-by-step reasoning and know when to ask for it.
A model produces its answer token by token. If it writes the result of a multi-step problem straight away, it has to do everything at once. If it writes the steps first, each one serves as support for the next.
In 2022, researchers showed that giving a few examples of detailed reasoning in the request clearly improves results in arithmetic, logic and common sense, especially for large models. Recent so-called reasoning models do this work on their own, before answering.
What is the annual cost if 12 people use the plan at CHF 24 per month with a 15% discount?
Calculate step by step: the monthly cost per person, then for 12 people, then over twelve months, then apply the discount. Show the details.
Each step becomes verifiable, and an error can be spotted where it occurs.
When it does not help
- For a simple question or a rewording, the steps make the answer longer without making it better.
- Well-written reasoning can still be wrong. The steps make checking easier, they do not replace it.
References
- Wei, Wang, Schuurmans et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems 35, p. 24824–24837. arxiv.org/abs/2201.11903