BIG IDEA Two completely different phases
People blur "training" and "running" a model, but they're worlds apart. Training is the giant, one-time process that creates the model by learning its weights from data. Inference is what happens every time you send a prompt — the finished model, weights frozen, produces an answer. As an app builder you live almost entirely in inference; training is done by the labs.
🏗️ Training (make the model)
- Done once by the lab (Anthropic, OpenAI…)
- Huge cost: thousands of GPUs, weeks–months
- Sets the weights
- Result: a reusable model
⚡ Inference (use the model)
- Happens on every request
- Cost = tokens × price, milliseconds–seconds
- Weights are frozen — read-only
- What your app and this study material are about
DIAGRAM The model lifecycle
next-token prediction"] PT --> BASE["Base model
(knows language, raw)"] BASE --> POST["Post-training
SFT + RLHF / alignment"] POST --> ALIGN["Aligned model
(helpful, safe, follows instructions)"] ALIGN --> DEP["Deploy behind an API"] DEP --> INF["⚡ Inference
your prompt → answer"] U["👤 You"] --> INF --> U classDef train fill:#f3f0ff,stroke:#764ba2,color:#4a3b7a; classDef run fill:#eef6ff,stroke:#3b7bd0,color:#1f5fa8; class PT,POST train; class DEP,INF run;
- Pre-training — the model reads a vast corpus and learns to predict the next token. This is where most compute goes and where raw knowledge/skills form. Output: a base (foundation) model.
- Post-training — the base model is refined to be useful and safe: SFT (supervised fine-tuning on good examples) and RLHF (learning from human preference feedback / alignment). This is what turns a raw text predictor into a helpful assistant.
- Deploy & inference — the finished, frozen model is served behind an API; your prompts run through it.
KEY POINT At inference, the model does not learn
This trips up almost everyone: when you chat with a model, it is not learning from you. The weights are fixed. Any "memory" of your earlier messages exists only because the conversation is re-sent with each call (the API is stateless). To actually change what a model knows or how it behaves, you must go back to a training step — or, far more commonly, work around it with prompting and RAG.
THE LADDER Four ways to adapt a model — cheapest to heaviest
When you need a model to know or do something new, climb only as high as you must:
| Method | What it changes | Effort / cost | When to use |
|---|---|---|---|
| Prompting | Just the input — no model change | 🟢 Instant | Steering behavior, small per-request context |
| RAG | Adds retrieved data at query time | 🟡 Moderate (build a pipeline) | Large or changing private knowledge |
| Fine-tuning | Adjusts the weights on your examples | 🟠 Heavier, slower to update | A fixed style/format the base won't hold |
| Pre-training | Builds a model from scratch | 🔴 Enormous (labs only) | Almost never — you buy this, not build it |
COST Why inference cost is the one you manage
Training cost is the lab's problem, amortized into the per-token price you pay. Inference is your recurring bill, and it scales with usage — so the whole of CCA Domain 5 (caching, batching, model tiers, token efficiency) is really about optimizing inference. Training a model is like building a factory; inference is paying for each item it makes.
TAKEAWAYS
GLOSSARY
| Term | In one sentence |
|---|---|
| Training | The process that sets a model's weights by learning from data. |
| Inference | Running the finished model to answer a prompt. |
| Pre-training | The large first phase: learn language/knowledge via next-token prediction. |
| Fine-tuning | A lighter follow-up that adapts a base model to a task or style. |
| SFT | Supervised fine-tuning — training on curated good input→output examples. |
| RLHF | Reinforcement Learning from Human Feedback — aligning a model to human preferences. |
| Base vs aligned model | Raw text predictor vs the helpful, instruction-following assistant. |
| Foundation model | A large general model you adapt, rather than train yourself. |