What is generative AI?

Discover what an LLM is, how it is trained, and why it seems to know everything.

6 min · 3 questions

You have probably already heard of ChatGPT, Claude, or Gemini. These are generative AIs: computer programs capable of producing brand-new text (an explanation, a summary, a poem, some code) from a written request. The engine hiding behind them is called an LLM, short for *Large Language Model*. In computing, a "model" is a program that has learned patterns from examples, instead of applying rules written out one by one by a human.

The basic principle is surprisingly simple: an LLM is a machine that predicts how a text continues. When you read "The cat is sleeping on the…", your brain spontaneously fills in "couch" or "bed". An LLM does exactly that, but on a massive scale: at every step, it calculates which word (or piece of a word) is most likely to come next, adds it, then starts again. An entire answer is therefore nothing more than a long series of predictions, word after word. It's the same mechanism as the word suggestions on your phone keyboard, only incomparably more powerful.

You are a patient teacher who explains computing to a complete beginner.

Explain to me what a large language model (LLM) is as if I were 12 years old, without any undefined technical terms.

Constraints:
- Use an everyday-life analogy.
- Maximum 10 sentences.
- End with a question to check that I understood.

How does this machine learn to predict so well? Through training: a phase (before your very first conversation) during which the model "reads" enormous quantities of text - books, articles, websites, computer code. For each sentence, the continuation is hidden and the model is asked to guess it. When it gets it wrong, its parameters are adjusted slightly - that is, its billions of tiny internal settings. Repeat this billions of times, and the model ends up capturing the grammar, the style, and a tremendous amount of the knowledge found in its training texts.

This is why a generative AI seems to know everything: it has "read" a library larger than what a human could go through in a thousand lifetimes. But be careful with that word: *seems*. The model does not consult an encyclopedia when it answers, and it does not "understand" the way you do. It calculates the most plausible sequence of words based on its training. Most of the time, plausible = true. But not always, and that is precisely the subject of the next lesson.

I've just learned that LLMs work by predicting text, that they are trained on huge corpora, and that they only produce the most plausible continuation.

Ask me 3 questions, one at a time, to check that I've really understood these three ideas.
Wait for my answer after each question, then correct me kindly before asking the next one.
Never give me the answer before I've tried.

Knowledge check

Make sure you remember the key points of this lesson.

1. Fundamentally, what does a large language model (LLM) do when it writes an answer?
   • It copies passages found live on the Internet
   • It predicts, word after word, the most plausible continuation of the text
   • It applies grammar rules written out one by one by engineers
2. What is meant by the training of an LLM?
   • The phase where humans write out all of its possible answers
   • The moment when the user teaches it things during the conversation
   • The phase where the model adjusts its parameters by guessing how enormous quantities of text continue
   • Installing the software on the servers
3. Why does a generative AI give the impression of knowing everything?
   • Because it checks every statement in an encyclopedia before answering
   • Because it was trained on enormous volumes of text covering almost every topic
   • Because it is permanently connected to human experts