Nudging all 63,000 dials

All 63,000 dials at once, a little each

It was wrong, so we fix it by turning the dials. We turn all 63,000 or so dials at once, each just a little, in the direction that makes the correct slot wider. Press "Turn once". The needles on the board on the left turn a little (the gray needle is where it was before), and along the wires, the correct slot on the reel gets wider.

Percent given to the correct next character ‘c’ after ‘…est wood’: before 0.3% → after 8 turns 79%

Why?Each turn, 62,556 dials move at once (31,362 turned up, 31,194 turned down). Every one of them turns the way that widens the correct slot. We turn them a little at a time, like adjusting the water temperature in a shower. Turn too much at once and things get worse.

Learn moreEven with 63,000 dials, one calculation tells us which way to turn each dial to widen the correct slot. That calculation is called backpropagation, and turning the dials a little in that direction is called gradient descent. You can see more in AI Through Signals, from the same series.