Clues from earlier characters

Finding clues in earlier characters

Third, the blank tile looks at the boards of the characters before it and gathers clues. It is like looking for clues in earlier sentences when you answer a test question. The brighter a look-back line from the blank tile to an earlier character, the more that character was used as a clue.

Why?It does not only look at the character right before. It can look at characters far back too. This AI looks back up to 64 characters. Which characters to use as clues is also set by dials, and gets fixed during learning.

Learn moreThe part that lets characters look at each other is called attention. The lines on screen are the average of the four attention heads in the last layer.