The AI That Writes One Character at a Time
Contents › Chapter 5 A look inside
A number for every character
Computers do not know characters. They only work with numbers
View in 3DA number for every character
So how do the dials work out the percents? We will look inside the AI in four steps. First, a computer does not know characters. It can only calculate with numbers. So every character gets a number tag. The small number under each tile is its number.
Why?Numbers go in order 0, 1, 2... starting with the most common character. A space (␣) is a character too. This AI knows 60 characters. A character it has never seen has no number, so the AI cannot write it.
Learn moreChatbots often use a group of characters that tend to appear together as one piece, instead of a single character. These pieces are called tokens.
View this scene in 3D →More in the series
A small AI (a character-level language model with 62,652 dials) was really trained on 40 fairy tales and fables retold for this project (about 376,021 characters), and every number here was calculated by that AI. It is a tiny model for learning; real chatbots are far bigger.