Be the Model
Learn how LLMs work by being each piece of one.
The short version. AI is the hottest topic in technology today, and technical people want to understand how LLMs actually work more than almost anything else. But it's a tough nut to crack: people drift off as soon as they hit diagrams that only make sense if you already understand transformers, attention heads, and embeddings.
Be the Model takes the opposite path. You work with a language model small enough to hold in your head, and you do its job yourself, one piece at a time: picking its next words, steering where it looks, making it learn. There's no way to be the model without understanding what the model does. Once you can do a piece's job by hand, you guide an AI to build the real version without hand-writing any code, so even novice developers can understand how an LLM works and build one themselves (but advanced developers can take it even further). By the end you've assembled a complete working LLM, and you understand every piece of it. The interactive toys linked below are working prototypes of the hands-on step; the rest of the page explains the course they fit into.
The idea
For years my working rule was "if you can build it, you understand it." It was a big part of why Head First C# worked: building something real forced you to understand it.
Generative AI broke that rule. You can now build things you don't understand at all, because the AI fills in everything you don't know. There are plenty of courses out there where you build an LLM, and people finish them with a model that works, then come away unsatisfied, because they built the whole thing without ever really understanding it.
There are also a lot of great books and explainers that are technically accurate: the diagrams are right, the math checks out. They're just hard to learn from. If you already mostly understand transformers, they're great resources for filling in the gaps. But if you're a working developer trying to get off the ground, a picture of a transformer shows you everything at once and nothing in particular. And that's the frustrating part of the broken rule: you can finish the book, have the AI build you the LLM, even see where every part of what it built maps onto the diagrams you read, and it still doesn't click.
Be the Model takes advantage of the broken rule instead of fighting it. The AI builds the scaffolding, which is the part it's good at, and the understanding lives somewhere the AI can't do it for you: you personally do the model's job, piece by piece. You choose next words from real corpus statistics, you set the weights on a neuron, you aim the model's attention at the words that matter.
The model you work with is tiny on purpose: a 62-word vocabulary, two attention heads, 2,160 parameters. It's small enough to hold in your head, and the math is the actual math. By the end it runs in the same tools as the real thing.
Who it's for: developers. If you can write code and remember high-school math, you already know enough to understand how LLMs work. It's just been really hard to find a way in.
The journey
The course is one continuous build, and each stage below is one chapter. A chapter teaches one piece of what's inside an LLM, you "become" that piece in an interactive toy that you build with AI, and then you build the real version, which the next chapter builds on. Here's the whole arc, with what each piece actually does inside a language model. Each card links to a prototype of the toy the learner builds themselves.
- Chapter 1 – Introduction: Language is magic
- Chapter 2 – Markov Chains: What word comes next?
- Chapter 3 – Tokenizer: Turning words into numbers
- Chapter 4 – Embeddings: Giving words meaning
Markov chain
You are the sampler
The simplest thing that can generate text: count which words follow which, then pick next words from the counts. The page counts four facts about the first computers; you pick a starting word, read the counts and percentages under it, and click your way to a sentence. Some words give you several ways to go and some give you only one. Then feed it your own text and watch the counts change.
Play the toy →Tokenizer
You are the tokenizer
The model's front door. Neural networks can't read words, so the tokenizer turns text into numbered tokens and back. The page writes a sentence and keeps the list; you turn every word into its number by hand, then rebuild the sentence from your numbers alone, with the original hidden until you're done.
Play the toy →Embeddings
You are the embedder
Where meaning enters the picture. Each word gets a short list of numbers, and the only thing that matters is that similar words get similar numbers. You set each word's numbers with sliders, pick a focus word, and watch the others rank themselves by distance while you drag.
Play the toy →- Chapter 5 – Neuron: Making one number from many
- Chapter 6 – Layers: Building a neural network
- Chapter 7 – Training: Feeding data into the network
Neuron
You count the cards
The single working unit of every neural network: multiply the inputs by weights, add them up, clamp the result. Chapter 5 has no separate toy; you build its neurons in four sketches. What's here is the chapter's interlude, the Hi-Lo card counter: a neuron stripped down to its multiply-and-add, with one input for each of the fifty-two cards and the running count as what comes out.
Open the counter →Layers
You are the runner
Neurons working side by side, and then a second layer reading the first. This is the whole untrained machine: a word goes in, two layers of random numbers run, nineteen scores come out, and you pick the next word from the scores and the token IDs alone.
Play the toy →Be the model
You run the trainer
The pond machine with all 619 parameters and all 45 training examples. Measure, get the gradient, take one training step, then press Run and watch the loss curve fall off a cliff and crawl while the guesses come right.
Play the toy →- Chapter 8 – Attention: Creating the context window
- Chapter 9 – The transformer block: Reading its own reading
Attention
You are the attention head
The transformer's big idea: before predicting the next word, the model looks back at the earlier words that matter, weighted by how much they matter. Aim the attention yourself and watch the answer change. Point it at the right word and the correct answer appears. Point it at punctuation and the answer falls apart.
Play the toy →The transformer block
You watch its two jobs
Attention plus a feedforward layer: gather, then think. This pair is the repeating unit, and real LLMs are this block stacked dozens of times. The chapter ends with the finished machine writing, and a dial for how far it strays from its top guess.
Play the toy →Your model in the real world
There's no toy for this chapter, on purpose: instead of a prototype, you run your model in the live tools people use for actual AI work. You rebuild your tiny model in PyTorch and export it to Hugging Face, where it loads and answers questions through the exact same API as production models. Then you go the other way: take a small, real GPT-2, convert it into the same JSON files you've been writing all along, and run it in your own runner. You've built a true model, and you're running it the way you'd run any other model. The difference is that this time you understand everything you did: you built every layer of it, and you understand every layer of it. That's the destination the whole journey points at, and it already works end to end.
How every chapter teaches
Every stage above has the same three-step shape, so you always know where you are, and you always know whether you've actually got it.
About the prompts: we never hand you the prompts. The course guides you to create every prompt yourself, because the prompt is the test: if you can create the prompt that builds a piece of the model, you understand that piece. (We may provide reference prompts you can peek at, the way a textbook keeps answers in the back, but the work is writing your own.) That guidance is the bulk of the course. You can work anywhere on the spectrum from pure vibe coding to writing every line by hand. The entire course can be completed without hand-writing a single line of code, and it still leaves you with a working model and a real understanding of every layer, because the understanding never came from typing the code. It came from being the model. The method also does double duty: every chapter is practice directing an AI to build real software, so you come away better at AI-driven development too.
What this looks like: embeddings
Learn. An embedding is a short list of numbers assigned to a word. In our tiny model it's eight numbers per word, and the toy starts you with three so the whole table fits on one screen. Nobody decides what they mean. It doesn't matter what they mean. The only job is that similar words end up with similar numbers: "charles" and "babbage" should be near-twins, "ada" and "lovelace" should be close to each other but clearly different from the Babbage pair, and "analytical" and "engine" should be similar to each other and far from all of the people. Once the words have numbers, you can measure how alike two words are instead of guessing about it. That's the same move every real LLM makes. Three numbers are enough to sort out a handful of words by hand, and a bigger vocabulary needs more distinctions, so real models use much longer lists. The other difference is that nobody sets a real model's numbers by hand: training sets them, which is what chapter 7 is about.
Be the embedder. In the full chapter you create the prompt that builds this toy first; here it's already built. Open the toy and set the three numbers for each word yourself. A sample exercise from the chapter: make "lovelace" rank as ada's nearest neighbor and "engine" as her farthest, using any numbers you like. When the ranking shows it, you're done, and you know what an embedding is because you just made one by hand and watched similarity fall out of nothing but numbers.
Build the real piece. Prompt the AI to build the real embedding component: given the course's vocabulary file as input, it has to produce an embeddings file that passes the provided similarity checks. When your files pass, the next chapter picks up your embeddings and feeds them into the network.
Technical notes
- Everything on this site runs as static JavaScript in the browser, with no server and nothing to install. That's just what these prototypes use; learners build their own versions with whatever language and tools they want. The model math in these toys (the forward pass, single-head causal attention) is real, checked against the same arithmetic run in PyTorch.
- The pages for chapters 2 through 7 work everything out from the book's own texts as you use them, so nothing is baked in: the counts, the token numbers, the distances, the scores and the training run are all computed in the browser. The two pages for chapters 8 and 9 run one validated tiny transformer whose weights ship with the page. Every page checks itself, recomputing a set of known values on load and printing them to the browser console.
- The toys were themselves built with AI-driven development, from prompts, and teaching learners to write those prompts is part of the course's method.
The name is a hat tip to the Be the Compiler exercises in Head First Java by Kathy Sierra and Bert Bates, where you played the role of the machine by hand. The funny thing is that in that exercise, all you actually did was decide whether two snippets of Java would compile. You weren't really being the compiler. Here, you really are being the model.