Build & fine-tune

AI Builder: Small-Model Fine-Tuning

Fine-tune a small open model with LoRA, evaluate it against a frozen baseline, and package the adapter with reproducible documentation.

14-20 hours · estimatedSelf-guided · learn at your pace
Before you start

Python developers who have trained at least one supervised model and want to adapt a language model locally.

  • Comfort with Python and PyTorch tensors
  • Basic understanding of train/validation/test splits
  • A computer with 16 GB RAM recommended; CPU or Apple Silicon is sufficient for a tiny model and small dataset
LESSON 1 · 3-4 hours ESTIMATED

Write the task and data contract

Create a clean prompt-completion dataset with explicit licensing, provenance, splits, and acceptance tests.

Fine-tuning changes behavior by continuing training on examples, so the examples are the specification. Define one narrow task, such as rewriting technical error messages into concise user-facing explanations. Create 120 to 300 original prompt-completion pairs yourself; synthetic examples may be used only if you can inspect and license them. Store fields for prompt, completion, source, license, and split. Remove secrets, personal data, copied proprietary text, duplicates, and near-duplicates across splits. Hold out adversarial cases before training: empty inputs, ambiguous messages, long strings, and requests outside scope. TRL accepts standard or conversational language-modeling and prompt-completion formats, but format consistency matters because tokenization turns the text into the actual training sequence. Inspect ten rendered examples after the chat template is applied. Calculate input and output token-length distributions and set a truncation limit that preserves nearly all examples. Finally, write five behavioral acceptance tests in plain language. These tests should express what success means before optimization makes a score tempting.

Try it yourself

  1. Author a small, original dataset and record provenance plus license for every row.
  2. Deduplicate before splitting and reserve adversarial test cases.
  3. Render tokenized examples, inspect boundaries, and chart token lengths.

You’re ready to move on when…

  • No duplicate or near-duplicate pair crosses splits.
  • Every row has provenance and license metadata.
  • Five acceptance tests exist before training and ten rendered examples are manually checked.

Learn from the original sources

TRL: Dataset formats Hugging Face Datasets: Process Hugging Face: Dataset cards
Enroll above to keep your progress across visits.
PUT IT INTO PRACTICE

Local LoRA adapter for error-message rewriting

Author a licensed dataset, establish a frozen base-model benchmark, train one small LoRA adapter locally, and publish a paired evaluation plus model card.

What you’ll make

  • Dataset card and JSONL splits
  • Training script and logs
  • LoRA adapter checkpoint
  • Paired evaluation report and model card

Check your work

  • Adapter beats the frozen baseline on at least one preregistered target without hiding regressions.
  • All test data remains untouched until final evaluation.
  • A clean local process loads the adapter and recreates representative outputs.
Project estimate: 5-8 hours beyond lessons. Completion is self-reported; this is not an accredited qualification.