Training: learning patterns from examples
Distinguish fitting a pattern from testing whether it works on new examples.
Training is the process of adjusting a model using examples. In one common setup, the model makes a prediction, compares it with a known answer, and changes its internal numbers to reduce the error. A message sorter might learn from messages labeled “personal” or “promotion.” Repeating this across many examples can produce a useful pattern, but the examples influence what it learns.
Imagine that every personal message in a tiny practice set is short and every promotion is long. A rule based only on length would look successful there. A long letter from a friend would expose its weakness. Testing on examples kept separate from training helps reveal problems like this. Our paper exercise uses a simple rule so you can see the issue directly; a modern language model is much more complex. The central question remains: does the learned pattern work beyond the examples used to build it?
Will it work on new messages?
Choose a model, explain your choice, then catch a mistake in how it was tested.
Worked example
Two models sort messages into “spam” or “ordinary.” Both train on the same 100 labeled messages. You compare them on a separate set of 100 messages that neither trained on.
| Model | Training 100 messages | Validation 100 different messages |
|---|---|---|
| A | 100 correct | 62 correct |
| B | 91 correct | 88 correct |
Take Model B to the final test. A is perfect on the examples it learned from, but misses 38 of the validation messages. B misses 12. Fitting familiar examples closely can hide a weaker pattern.
- Training: fit the model.
- Validation: compare choices while developing it.
- Test: estimate the final model’s performance, using examples kept out of those decisions.
These are made-up, balanced datasets with equal costs for the two kinds of error. Real comparisons also need representative data, error analysis, and uncertainty. A small validation score is evidence, not a guarantee.
Another check: training or inference?
Training or inference?
1 of 3 · A system adjusts its internal weights after comparing predictions with examples.
Try it yourself
- Write four pretend messages: two short personal reminders and two long advertisements. Label each one.
- Try the rule “short means personal; long means promotion” on those four messages.
- Add a long personal letter as a new test case and record why the rule gets it wrong.
You’re ready to move on when…
- You can distinguish the examples used to choose a rule from the new example used to test it.
- You can explain why success on the original four examples did not prove the rule was reliable.