Predicting what comes next in a text

A language model is a program that has learnt, from a very large amount of text, to predict the most likely continuation of a text. It breaks words into small pieces called tokens, chooses the next one, then the next, until it forms an answer. Everything it writes comes from this mechanism.

It sounds modest. Yet this is what allows it to summarise, translate, rephrase, reason step by step or write code. To predict well what comes next in a text about accounting, it must have picked up a great many regularities about accounting. The model has not learnt index cards. It has learnt ways of saying things and of linking ideas.

Three consequences to keep in mind

• The model produces what is plausible before it produces what is true. When it knows, the plausible and the true coincide. When it does not know, it still produces something plausible.
• Its knowledge stops at the date of its training. It is unaware of what happened afterwards, unless it searches for it.
• It does not remember on its own from one conversation to the next. The memory of an assistant like Learnya is a feature added around the model.

Two models, two uses

Learnya offers two models. Learnya Flash is fast and suits most everyday tasks: rephrasing, summarising, writing an email. Learnya Max takes more time to reason at length. It suits difficult analyses, multi-step problems and texts where every nuance matters.

Keep this picture in mind: the model is an excellent writer who has read an enormous amount, but who has access neither to your files nor to the news, unless you or its tools bring them to it. Almost every good practice in this course follows from this picture.