The AI That Writes One Character at a Time
Contents › Chapter 5 A look inside
A face board for each character
Each character has a board of 48 dials: how the AI sees that character
View in 3D48 dials for every character
Second, a number is only a name tag. It says nothing about which characters are alike. So each character gets a board of 48 dials. The colorful board under each tile is that board. Each small square is one dial.
Why?A square's color shows which way its dial is turned and how far. Blue is the + side, red is the − side, and darker means turned further. These are some of the 63,000 dials from Chapter 2. They start out turned every which way and get fixed during learning, until characters that are used in similar ways have similar board patterns. This board is the AI's "face" for each character.
Learn moreThe group of numbers attached to each character is called an embedding. Big AIs attach a board with thousands of dials to each piece of text.
View this scene in 3D →More in the series
A small AI (a character-level language model with 62,652 dials) was really trained on 40 fairy tales and fables retold for this project (about 376,021 characters), and every number here was calculated by that AI. It is a tiny model for learning; real chatbots are far bigger.