The AI That Writes One Character at a Time
Contents › Chapter 6 Compared with big AIs
If you say "Hello"
Before and after chat practice: the same "Hello", a different reply
View in 3DSaying "Hello" to an AI that only read fairy tales
What happens if you say "Hello" to this AI? It has only read fairy tales and has never seen a conversation. So it does not reply. It just keeps writing likely next characters.
Text written by the AI: Human: Hello ⏎ AI: "I Las thump it word to more o
Why?The only thing this AI can do is guess the next character. Even "when someone asks you something, you answer" is something it has to learn.
View this scene in 3D →The AI after chat practice
So we gave it more guessing practice with 471 short example conversations. It learned that "AI:" comes after "Human:", and that a reply comes after that. Now it answers "Hello". The reply is still just the characters after "AI:", picked one at a time.
Text written by the AI: Human: Hello ⏎ AI: Hello! I am a small I am still heart and story.
Why?Chatbots are made in the same two steps. First they learn to guess the next character from a huge amount of text. Then they practice again with good example conversations written by people. But this AI is very small, so apart from greetings it often gives odd replies.
Learn moreThe AI also decides when the reply ends. When a line break (⏎) is picked as the next character, the reply is over. The second step is called fine-tuning. The example conversations were written just for this project.
View this scene in 3D →More in the series
A small AI (a character-level language model with 62,652 dials) was really trained on 40 fairy tales and fables retold for this project (about 376,021 characters), and every number here was calculated by that AI. It is a tiny model for learning; real chatbots are far bigger.