Think Together · Start with a Short Story
The girl watches an answer appear bit by bit and thinks the model chooses one whole word at a time. In fact, a generation unit is not always a word.
Lay Out the Process
- Existing textThe current context
- Calculate probabilitiesPossible next tokens
- ChooseGet one token
- RepeatAdd it to the context and continue
Common autoregressive language models generate tokens step by step, using what they have already generated for further calculations.
A token may be only part of a word. The screen may also buffer its display, so the visible rhythm is not necessarily the rhythm of every internal step.
If an error appears early, later text may build on it, so long answers also need checking.
Now You Try
If a whole sentence appears on the screen at once, does that prove the model made only one calculation?
Ready? Open this to see the explanation
No. The display may combine several generated pieces before showing them.
One Idea to Take Away
Common autoregressive language models generate tokens step by step, using what they have already generated for further calculations.
Try covering the diagram above and explaining each step to someone in your family. If one step is hard to explain, return to that card and look again.