The AI That Writes One Character at a Time
How does an AI write?
How text-writing AI works, seen through a small AI that read 40 fairy tales. It gives every possible next character a percent, learns by comparing its guesses with the right answers and turning its dials, and writes by pulling a lever to pick one character at a time — in 22 ideas, in order.
Watch from the start in 3DChapter 1 · Guessing the next character
A small AI that read fairy tales. It does just one thing: guess the next character.
- 1 Meet this AI
A small AI that practiced writing by reading 40 fairy tales
View in 3D → - 2 Guess the next character
This AI does just one thing: guess the next character
View in 3D → - 3 It answers in percents
Instead of one answer, it gives every character a percent
View in 3D →
Chapter 2 · How it learned
Give it a quiz, compare its answer with the correct one, and fix it a little. Then do it thousands of times.
- 4 At first, any character
Before learning, all the slots are about the same width
View in 3D → - 5 What if there are few dials?
With few dials, the reel cannot change with the characters before it
View in 3D → - 6 Human writing is the textbook
One sentence from a fairy tale becomes many practice quizzes
View in 3D → - 7 A quiz and a grade
Hide the next character of a story, then compare the AI's answer with the correct one
View in 3D → - 8 Nudging all 63,000 dials
Turn the dials a little, so the correct slot gets wider
View in 3D → - 9 Practice with thousands of quizzes
Repeat the dial-turning with many quizzes
View in 3D →
Chapter 3 · Pulling the lever
Spin the reel and pick one character by its percent. How you pick changes the text.
- 10 Pull the lever
The character where the reel stops becomes the next character
View in 3D → - 11 Always picking the top one
Picking only the top character tends to repeat the same words
View in 3D → - 12 Temperature
A setting a person chooses: flatter slots or sharper slots
View in 3D →
Chapter 4 · One character at a time
Add the picked character, then guess again. Repeat, and you get a story.
- 13 Add it, and again
Add the picked character, then guess again
View in 3D → - 14 A whole story
Writing on, up to 60 characters
View in 3D → - 15 Same start, different text
If the reel stops somewhere else, you get a different story
View in 3D →
Chapter 5 · A look inside
Where do the percents come from? We look inside the AI in four steps.
- 16 A number for every character
Computers do not know characters. They only work with numbers
View in 3D → - 17 A face board for each character
Each character has a board of 48 dials: how the AI sees that character
View in 3D → - 18 Clues from earlier characters
The blank tile looks at the boards of the characters before it
View in 3D → - 19 Summary: four steps
Number tag, board, clues, then scores and percents
View in 3D →
Chapter 6 · Compared with big AIs
Chatbots write the same way. They are just much bigger.
- 20 If you say "Hello"
Before and after chat practice: the same "Hello", a different reply
View in 3D → - 21 Sounding right vs. being true
It only picks a likely next character. It never checks whether it is true
View in 3D → - 22 Only the size is different
The same idea, at a much bigger size
View in 3D →
More in the series
A small AI (a character-level language model with 62,652 dials) was really trained on 40 fairy tales and fables retold for this project (about 376,021 characters), and every number here was calculated by that AI. It is a tiny model for learning; real chatbots are far bigger.