The environmental footprint

In 2019, a study estimated the energy use and emissions of training natural language processing models. The heaviest experiment, an automated neural architecture search, was estimated at around 284 tonnes of CO₂, roughly five times the lifetime emissions of an American car, fuel included. The study started a lasting debate about measuring and publishing these costs.

Since then, most of the consumption has been shifting towards use. Each answer requires computation, and millions of users multiply that cost. The size of the model, the length of the exchanges and the data centre’s source of electricity make the difference.

Three leaner choices

• Use a fast model for simple tasks and keep the largest one for tasks that justify it.
• Attach the relevant passages rather than entire folders, which reduces the number of tokens processed.
• Prefer hosting powered by low-carbon electricity, which is the case for most Swiss electricity generation.