How Machines Actually Learn

The three ingredients every AI system needs

Strip away the hype and every AI system is three things: training data, a model, and a training loop. That's it. Understand these three and you understand 80% of how AI works.

1. Training data

Imagine teaching a 4-year-old to recognise a dog. You don't say "a dog is a mammal with four legs, a tail, a wet nose, and barks." You point at 100 dogs and say "dog" and at 100 cats and say "not a dog". After a while, the child can identify a dog they've never seen before.

Machine learning is exactly that — but with a million examples instead of a hundred. The examples are called training data. Bad training data makes bad AI. If you only ever showed the child golden retrievers, they might struggle with a pug.

2. The model

The model is the thing doing the learning. For modern AI (like ChatGPT), the model is a neural network — a huge grid of tiny math equations, loosely inspired by how brain neurons fire.

Each connection in the network has a number called a "weight." Training adjusts those weights until the network can look at an input (e.g. a photo) and produce the right output (e.g. "dog"). ChatGPT has about 1.7 trillion of these weights. They were set by training on an estimated 13 trillion words of text.

3. The training loop

Training means running the model on an example, checking if it got the right answer, and tweaking the weights slightly when it got it wrong. Repeat a billion times. At the end, the model is "trained" — the weights are frozen and it can answer new questions.

That tiny loop, run billions of times, is almost everything.

Training vs inference — a distinction that matters

• Training is teaching the model. It's done ONCE by a large team with huge computers and millions of dollars of electricity. Training GPT-4 is estimated to have cost over $100M.
• Inference is USING the trained model. Every time you type into ChatGPT, you're doing inference. It's cheap and fast — a few cents at most per query.

Why it matters: once a model is trained, it cannot learn new information without being re-trained. ChatGPT doesn't "remember" your conversation with it yesterday (unless you explicitly use a memory feature). Each conversation starts fresh. The model itself is frozen.

Why AI "hallucinates" — and why it's not a bug

Large Language Models like ChatGPT work by predicting the most likely next word given everything before. They are, literally, very fancy auto-complete.

Here's what happens when you ask one a factual question:

1. The model looks at your question.
2. It predicts the statistically most likely answer, word by word.
3. If the correct answer appeared enough times in its training data, the prediction is likely right.
4. If the correct answer didn't appear in training data — or appeared rarely — the model still predicts something, because that's the only thing it knows how to do. It generates a plausible-sounding answer that might be completely wrong.

This is called hallucination. The model isn't lying — it has no concept of truth. It's pattern-completing. This is a feature of how the technology works, not a bug that will be fully fixed soon.

Practical rule: Never trust an AI-generated fact (names, dates, statistics, quotes) without verifying from another source. Always.

Hands-on: train your own AI in 5 minutes

Before you close this module, try this: open Google's Teachable Machine in a new tab. Click "Image Project" → "Standard Model."

1. Create a class called "Me with hand up" — click "Webcam" and capture 30 samples of yourself with your hand up.
2. Create a second class called "Me normal" — capture 30 samples of yourself sitting normally.
3. Click "Train Model". Wait ~15 seconds.
4. Point the webcam at yourself. The model tells you in real-time which class you're in.

You just trained an AI image classifier. That's the same technique used in M-Pesa fraud detection, Instagram filters, and self-driving cars — just with bigger training data and more classes.