Think Together · Start with a Short Story
A robot cannot directly use letters printed on paper for matrix calculations. First, pieces of text get IDs and are turned into numerical representations.
Lay Out the Process
- TextThe original sentence
- TokensSmall pieces after splitting
- IDsMatching entries in a vocabulary
- VectorsA set of numbers for calculation
Text is usually split into tokens and token IDs, then mapped to numerical representations such as vectors.
An ID works like an index in a vocabulary to find the matching representation; vectors are used in later calculations. Nearby IDs do not automatically have similar meanings.
As the model processes the text through its layers, the representations change in context.
Now You Try
If two words have neighboring IDs in the vocabulary, does that mean their meanings are similar?
Ready? Open this to see the explanation
No. IDs are identifiers. Semantic relationships come from training and representations, not the numerical order.
One Idea to Take Away
Text is usually split into tokens and token IDs, then mapped to numerical representations such as vectors.
Try covering the diagram above and explaining each step to someone in your family. If one step is hard to explain, return to that card and look again.