The AI That Writes One Character at a Time
Contents › Chapter 6 Compared with big AIs
Only the size is different
The same idea, at a much bigger size
View in 3DOnly the size is different
This AI read about 380,000 characters and has about 63,000 dials. GPT-3, released in 2020, read about 300 billion pieces of text (a piece is a few characters grouped together) and had about 175 billion dials. Even so, it does the same job. Look at the text so far, give percents, pick one and add it.
Why?A big AI has read far more text, has far more dials on its boards, and has many more clue-finding steps. That is why it can copy not just grammar, but lots of facts and ways of speaking.
Learn moreThe GPT-3 numbers come from a 2020 paper (Language Models are Few-Shot Learners). Most AIs since then do not say how much text they read or how many dials they have.
View this scene in 3D →Read more
Wikipedia: GPT-3 · Brown et al. (2020), Language Models are Few-Shot Learners
More in the series
A small AI (a character-level language model with 62,652 dials) was really trained on 40 fairy tales and fables retold for this project (about 376,021 characters), and every number here was calculated by that AI. It is a tiny model for learning; real chatbots are far bigger.