cover

The Pattern Parrot

How do chatbots learn patterns?
You type a question into a chatbot and โ€” ~~boom~~ โ€” it answers like it knows everything. But how does it learn all those

You type a question into a chatbot and โ€” boom โ€” it answers like it knows everything. But how does it learn all those patterns, all those ways words fit together? The secret isn't magic. It's more likeโ€ฆ training a very determined parrot who's read the entire internet.

First, the chatbot reads. **Millions and millions of sentences** โ€” books, websites, conversations, recipes, poems, every

First, the chatbot reads. Millions and millions of sentences โ€” books, websites, conversations, recipes, poems, everything humans have written down. It doesn't understand the words yet. It just sees patterns, like noticing that "The cat sat on the" is almost always followed by a thing you can sit on.

While it reads, the chatbot plays a ~~guessing game~~ with itself. It **hides the last word** of a sentence and tries to

While it reads, the chatbot plays a guessing game with itself. It hides the last word of a sentence and tries to predict it. "The sky is __." Blue? Clear? Falling? It guesses, checks the answer, and adjusts tiny dials inside its brain โ€” millions of them โ€” to get better next time.

Those dials are called ++weights++, and they're how the chatbot remembers patterns. When it guesses wrong, it tweaks the

Those dials are called weights, and they're how the chatbot remembers patterns. When it guesses wrong, it tweaks the weights a tiny bit. When it guesses right, it nudges them to remember what worked. After billions of guesses, the weights settle into a map of how language actually flows.

~~Here's the weird part:~~ the chatbot **never "knows" what a cat is**. It just knows that "cat" appears near "meow" and

Here's the weird part: the chatbot never "knows" what a cat is. It just knows that "cat" appears near "meow" and "purr" and "litter box" more often than near "rocket ship." It's learned the shape of cat-ness from the company words keep.

~~The real magic trick~~ is called ++attention++. When you ask, "Why do cats purr?" the chatbot doesn't treat every word

The real magic trick is called attention. When you ask, "Why do cats purr?" the chatbot doesn't treat every word equally. It focuses hard on "cats" and "purr," connects them to all the related patterns it's seen, and ignores the boring words like "do."

All this happens in layers, ~~like a relay race~~. The first layer notices simple patterns โ€” *common word pairs*. The ne

All this happens in layers, like a relay race. The first layer notices simple patterns โ€” common word pairs. The next layer notices slightly bigger ideas โ€” questions vs. statements. The deepest layers recognize huge patterns, like "this sounds like a request for scientific explanation."

So when you type your question, the chatbot doesn't look up an answer in a book. It surfs the patterns it learned โ€” **bi

So when you type your question, the chatbot doesn't look up an answer in a book. It surfs the patterns it learned โ€” billions of tiny weight adjustments, all firing at once โ€” to predict what helpful words should come next. It's guessing, extraordinarily well, one word at a time.

~~And here's the funny thing:~~ sometimes it gets stuff wrong, because *it's only ever learned patterns, not truth*. If

And here's the funny thing: sometimes it gets stuff wrong, because it's only ever learned patterns, not truth. If the internet said "unicorns love Tuesday" enough times, the chatbot might believe it. It's brilliantly good at language, and hilariously bad at telling real from nonsense.

That parrot we mentioned at the start? ~~Turns out~~ it's a parrot that's **read everything**, notices *every tiny patte

That parrot we mentioned at the start? Turns out it's a parrot that's read everything, notices every tiny pattern, and can remix all of it into something new. Not bad for a bunch of math and guessing games. Justโ€ฆ maybe don't ask it to count how many Rs are in "strawberry."

How was this book?

A Wonderleaf Book

The Pattern Parrot

โ€” How do chatbots learn patterns? โ€”

Wonderleaf Editions
โ€” ex libris โ€”
A Wonderleaf Book

The Pattern Parrot

How do chatbots learn patterns?

Wonderleaf Editions ยท MMXXVI
Scene 1
You type a question into a chatbot and โ€” ~~boom~~ โ€” it answers like it knows everything. But how does it learn all those
The Pattern Parrot2
Scene 1

You type a question into a chatbot and โ€” boom โ€” it answers like it knows everything. But how does it learn all those patterns, all those ways words fit together? The secret isn't magic. It's more likeโ€ฆ training a very determined parrot who's read the entire internet.

3The Pattern Parrot
Scene 2
First, the chatbot reads. **Millions and millions of sentences** โ€” books, websites, conversations, recipes, poems, every
The Pattern Parrot4
Scene 2

First, the chatbot reads. Millions and millions of sentences โ€” books, websites, conversations, recipes, poems, everything humans have written down. It doesn't understand the words yet. It just sees patterns, like noticing that "The cat sat on the" is almost always followed by a thing you can sit on.

5The Pattern Parrot
Scene 3
While it reads, the chatbot plays a ~~guessing game~~ with itself. It **hides the last word** of a sentence and tries to
The Pattern Parrot6
Scene 3

While it reads, the chatbot plays a guessing game with itself. It hides the last word of a sentence and tries to predict it. "The sky is __." Blue? Clear? Falling? It guesses, checks the answer, and adjusts tiny dials inside its brain โ€” millions of them โ€” to get better next time.

7The Pattern Parrot
Scene 4
Those dials are called ++weights++, and they're how the chatbot remembers patterns. When it guesses wrong, it tweaks the
The Pattern Parrot8
Scene 4

Those dials are called weights, and they're how the chatbot remembers patterns. When it guesses wrong, it tweaks the weights a tiny bit. When it guesses right, it nudges them to remember what worked. After billions of guesses, the weights settle into a map of how language actually flows.

9The Pattern Parrot
Scene 5
~~Here's the weird part:~~ the chatbot **never "knows" what a cat is**. It just knows that "cat" appears near "meow" and
The Pattern Parrot10
Scene 5

Here's the weird part: the chatbot never "knows" what a cat is. It just knows that "cat" appears near "meow" and "purr" and "litter box" more often than near "rocket ship." It's learned the shape of cat-ness from the company words keep.

11The Pattern Parrot
Scene 6
~~The real magic trick~~ is called ++attention++. When you ask, "Why do cats purr?" the chatbot doesn't treat every word
The Pattern Parrot12
Scene 6

The real magic trick is called attention. When you ask, "Why do cats purr?" the chatbot doesn't treat every word equally. It focuses hard on "cats" and "purr," connects them to all the related patterns it's seen, and ignores the boring words like "do."

13The Pattern Parrot
Scene 7
All this happens in layers, ~~like a relay race~~. The first layer notices simple patterns โ€” *common word pairs*. The ne
The Pattern Parrot14
Scene 7

All this happens in layers, like a relay race. The first layer notices simple patterns โ€” common word pairs. The next layer notices slightly bigger ideas โ€” questions vs. statements. The deepest layers recognize huge patterns, like "this sounds like a request for scientific explanation."

15The Pattern Parrot
Scene 8
So when you type your question, the chatbot doesn't look up an answer in a book. It surfs the patterns it learned โ€” **bi
The Pattern Parrot16
Scene 8

So when you type your question, the chatbot doesn't look up an answer in a book. It surfs the patterns it learned โ€” billions of tiny weight adjustments, all firing at once โ€” to predict what helpful words should come next. It's guessing, extraordinarily well, one word at a time.

17The Pattern Parrot
Scene 9
~~And here's the funny thing:~~ sometimes it gets stuff wrong, because *it's only ever learned patterns, not truth*. If
The Pattern Parrot18
Scene 9

And here's the funny thing: sometimes it gets stuff wrong, because it's only ever learned patterns, not truth. If the internet said "unicorns love Tuesday" enough times, the chatbot might believe it. It's brilliantly good at language, and hilariously bad at telling real from nonsense.

19The Pattern Parrot
Scene 10
That parrot we mentioned at the start? ~~Turns out~~ it's a parrot that's **read everything**, notices *every tiny patte
The Pattern Parrot20
Scene 10

That parrot we mentioned at the start? Turns out it's a parrot that's read everything, notices every tiny pattern, and can remix all of it into something new. Not bad for a bunch of math and guessing games. Justโ€ฆ maybe don't ask it to count how many Rs are in "strawberry."

21The Pattern Parrot

~ finis ~

Tiny picture books for big little questions.

โ€” a small constellation of questions โ€”
โœฆWonderleaf
Editions