The machines behind the models

Computing power

Training Apertus took more than 10 million GPU hours on Alps, the supercomputer of the Swiss National Supercomputing Centre (CSCS). A GPU hour is one graphics processor working for one hour.

The scale is new, but the problem is old: in the 1950s ETH Zurich built ERMETH because it needed more computing power than it could get any other way.

Data

Apertus learned from 15 trillion tokens, the small pieces of words that language models work with, in more than 1,000 languages. Because the training data were published, researchers can study what went into the model, which matters for questions of quality, bias and copyright.