🤯 How LLMs Work Under the Hood #MLSTART

When you first encounter large language models, it seems like there's some magic inside. 🪄

But in reality, everything is much simpler (and more interesting). If we simplify greatly, we can say that an LLM simply predicts the next character or word.

It “looks” at the context, evaluates the probabilities of all possible continuations, and selects the most appropriate one. That is, at its core, it's ordinary multiclass classification, just on a gigantic scale.

🎞 To see this visually, there are two excellent resources:

- 3Blue1Brown - visually explains how neural networks work. After his videos, many things become literally obvious.

- Andrej Karpathy - former OpenAI developer, breaks down how to build a GPT from scratch. No fluff, very systematic. He shows step by step how a language model is built and why it does what it does.

💬 These two playlists really help put the puzzle together in your head: from neural networks to transformers, and from there to understanding LLMs.

Back to table of contents

👩‍💻 Data Flow