Domain 1 · Plan and manageChoose the right model
Choose the right model
The exam skill: choose an appropriate model for each task, including LLMs, small models, multimodal models and Foundry Tools. Exam questions test the category; model names are examples.
From CCA-F: the same idea as choosing Haiku, Sonnet or Opus: the cheapest model that passes your evaluation wins. Azure adds three things: a catalogue with many vendors, Foundry Tools that aren't LLMs at all, and a model router that chooses for you.
1 · The mind map: eight categories
Each branch is a category, with example models and the words in a question stem that point to it.
2 · The decision tree
Ask the questions in order. The first Yes wins. The order matters: a Foundry Tool beats any model, and a cheaper model beats a bigger one once it passes evaluation.
- 1
Does a Foundry Tool already do this job (PII redaction, translating a document, reading an invoice, transcribing audio, offline extraction)?
Yes →Use the Foundry ToolLanguage PII · Translator · Speech · Document IntelligenceDeterministic, cheaper, and the exam's 'least effort' answer. An LLM is the runner-up. - 2
Is the output something other than text: an image, a video, a voice, or vectors?
Yes →Use a generation or embedding modelgpt-image-2 · sora-2 · gpt-realtime / Voice Live · text-embedding-3-largeMatch the model to the output type; chat models don't make images or vectors. - 3
Does the input include images, screenshots or charts that the model must read?
Yes →Use a multimodal modelgpt-5 · gpt-4.1 · Phi-4-multimodalFor repeatable fields with confidence, Content Understanding beats a chat model. - 4
Does the task need multi-step reasoning: planning, maths, tricky code, or rules with many conditions?
Yes →Use a reasoning modelo4-mini · o3 · gpt-5 (reasoning_effort)Slower, and reasoning tokens bill as output. No temperature; use reasoning_effort and max_output_tokens. - 5
Must it run on a device or offline?
Yes →Use a small model on the edgePhi-4-mini (Foundry Local)Small open-weight models fit edge hardware; cloud-only models don't. - 6
Is the task simple, high-volume, and has a small model been proven accurate enough?
Yes →Use a small modelgpt-5-mini · gpt-5-nano · gpt-4.1-miniThe cheapest model that passes your evaluation wins. - 7
Do prompts vary from trivial to hard, and you'd rather not choose per request?
Yes →Use model routermodel-router: Balanced (default), Quality, or Cost modePicks a model per prompt. Smallest context window of its models; Global or Data Zone Standard only. - 8
Has prompting failed to get a consistent style or format, and you have hundreds of good examples?
Yes →Fine-tune a modelgpt-4.1-mini (SFT, DPO) · o4-mini (RFT)Try prompts and RAG first. Fine-tuning teaches style and format, not fresh facts. - ✓
No to everything?
Then →Otherwise: a general-purpose LLMgpt-5 · gpt-4.1 · Claude SonnetStart here, evaluate, then move down to a cheaper model if quality holds.
3 · The criteria to weigh
| Criterion | Ask | Exam angle |
|---|---|---|
| Task and modality | What goes in and comes out: text, image, audio, video, vectors. | Rules out whole categories at once. |
| Quality needed | How hard is the task? Does it need reasoning, or is it classification? | Proven by evaluation, not guessed. |
| Latency | Is a person waiting? Reasoning and large models are slower. | Real-time → small or router; batch → anything. |
| Cost and volume | Price per token × requests per day. | Reasoning tokens bill as output. |
| Context length | How much text per request? | Model router uses its smallest model's window. |
| Availability | GA, preview, or limited access? Available in your region and deployment type? | "GA only" in a stem rules out preview models. |
| Data residency | Must data stay in a geography? | That's the deployment type (Data Zone), not the model, but it limits which models you can pick. |
| Customisation | Does it need your style or format, consistently? | Prompt → RAG → fine-tune, in that order. |
| Where it runs | Cloud, or a device, or offline? | Offline → small open-weight model or a container. |
4 · In this scenario, choose this
| Scenario | Choose | Example | Why | Runner-up trap |
|---|---|---|---|---|
| Classify 5,000 claim emails an hour into six fixed categories | Small model | gpt-5-mini | Simple and high volume; a small model was proven accurate. | Reasoning model: slower and dearer for no gain. |
| Hide names and card numbers in transcripts | Foundry Tool | Language PII detection | Built for it, returns redacted text. | An LLM prompted to redact: less reliable, costs tokens. |
| Plan a multi-step claims investigation with many rules | Reasoning model | o4-mini | Multi-step logic. | General LLM: fine for writing, weaker at long chains of rules. |
| Describe a damage photo for an adjuster | Multimodal model | gpt-4.1 | Reads images and answers in text. | Image Analysis 4.0: deprecated. |
| Extract fields with confidence from varied repair estimates | Foundry Tool | Content Understanding analyzer | Schema, confidence, no training. | A vision chat model: no confidence scores. |
| Retrieve policy clauses by meaning | Embedding model | text-embedding-3-large | Vectors for search. | A chat model can't produce index vectors. |
| Generate a product image with a transparent background | Image generation | gpt-image-2 | GA image model. | DALL-E 3: retired. |
| A live phone agent that can be interrupted | Realtime audio | gpt-realtime via Voice Live | Speech to speech in one service. | STT + LLM + TTS chained: more latency. |
| A support chat with mixed easy and hard questions, cut cost | Model router | model-router (Cost mode) | Picks per prompt. | Model subset limited to small models: blocks hard questions. |
| Run on a technician's laptop with no internet | Small model on the edge | Phi-4-mini on Foundry Local | Runs locally. | Any cloud deployment. |
| Replies must match the house style; prompting is inconsistent | Fine-tuned model | gpt-4.1-mini (SFT) | Teaches style from examples. | Bigger model: costs more and still drifts. |
5 · Categories compared
| Category | Best at | Weak at | Cost | Speed | Examples |
|---|---|---|---|---|---|
| General-purpose LLM | Writing, broad knowledge, tool use, agents | Cost at volume; slower than small models | $$$ | Medium | gpt-5, gpt-4.1, Claude Sonnet, Llama, Mistral Large |
| Small model (SLM) | Classification, extraction, routing, high volume | Hard reasoning, long nuanced writing | $ | Fast | gpt-5-mini/nano, gpt-4.1-mini/nano, Phi-4-mini |
| Reasoning model | Multi-step logic, maths, code, planning | Latency; reasoning tokens bill as output; no temperature | $$–$$$$ | Slow | o3, o4-mini, gpt-5 (reasoning_effort), DeepSeek-R1 |
| Multimodal model | Images in, text out: captions, visual Q&A, charts | Repeatable fields with confidence (use Content Understanding) | $$ | Medium | gpt-5, gpt-4.1, Phi-4-multimodal |
| Embedding / rerank | Vectors for search, similarity, clustering; reranking | Doesn't generate text | ¢ | Fast | text-embedding-3-large/small, Cohere Embed, Cohere Rerank |
| Image / video generation | Create and edit images; generate and remix video | Faces in video inputs; real people | $$ per image / per second | Seconds to minutes | gpt-image-2, sora-2 (preview) |
| Realtime audio / speech | Live voice conversations, transcription, voices | Content filters don't apply to audio | $$ | Real-time | gpt-realtime, gpt-4o-transcribe, gpt-4o-mini-tts |
| Model router | Mixed difficulty without choosing | Smallest context window of its models; limited deployment types | Varies | Varies | model-router |
| Foundry Tools | PII, translation, OCR, forms, speech, offline containers | Open-ended reasoning | $ | Fast | Language, Translator, Speech, Document Intelligence, Content Understanding |
Costs are relative per request, not prices. Model names change fast; check the Foundry Models catalogue (open during the exam) for current names, regions and GA status.
6 · Which model for developing agents
Recommendation: start a new prompt agent on a mid-size general-purpose model with strong tool calling (for example gpt-5-mini). Run the agent evaluators: task adherence, tool call accuracy, intent resolution. Move up (gpt-5, or a reasoning model) only if those fail, and move narrow subagents down to small models once they pass.
From CCA-F: the same split as Opus or Sonnet orchestrating and Haiku subagents doing narrow work. The model is the agent's brain; the tools, instructions and guardrails make it an agent, and they don't change with the model.
From CCA-F: the same split as Opus or Sonnet orchestrating and Haiku subagents doing narrow work. The model is the agent's brain; the tools, instructions and guardrails make it an agent, and they don't change with the model.
| Agent role | Choose | Example | Why | Watch out |
|---|---|---|---|---|
| Default prompt agent (tools, RAG, a few steps) | General-purpose model with strong tool calling | gpt-5-mini → move to gpt-5 or gpt-4.1 if evaluations fail | Agents live or die on tool-call accuracy and following instructions; start mid-size and let the evaluations decide. | Check the Agent Service supported-models list: not every catalogue model supports tools. |
| Orchestrator in a multi-agent system | Larger general-purpose or reasoning model | gpt-5 · o4-mini | It plans, delegates and merges results: the hardest job in the system. | Reasoning models are slower; use them where planning, not chatting, is the work. |
| Worker / subagent with one narrow job | Small model | gpt-5-mini · gpt-5-nano · gpt-4.1-mini | Narrow, repeated tasks with clear tools: cheap and fast. | Prove it with task adherence and tool-call accuracy evaluations first. |
| Planning-heavy agent (many rules, multi-step decisions) | Reasoning model | o4-mini · o3 · gpt-5 with reasoning_effort | Thinks before choosing tools. | No temperature; set reasoning_effort and max_output_tokens. No parallel tool calls at minimal effort. |
| Triage or routing step in front of agents | Small model, or rules first | gpt-5-nano · a rules engine | Classifying a request is simple; fixed answers belong in rules. | Don't put a large model on a classification job. |
| Voice agent | Realtime audio model through Voice Live | gpt-realtime | Speech in, speech out, with interruptions. | Content filters don't apply to audio models. |
| Agent that reads screenshots, photos or forms | Multimodal model | gpt-5 · gpt-4.1 | Images go straight into the conversation. | Repeatable fields with confidence: give the agent a Content Understanding tool instead. |
| Your own agent code (LangGraph, Agent Framework) | Any deployed model; run it as a hosted agent | the deployment your code calls | Hosted agents run your code; the model is your code's choice. | The model choice rules above still apply inside your code. |
What makes a model good for agents, in order: reliable tool calling (right tool, valid arguments), instruction following over many turns, structured outputs, enough context for tool results, and latency per turn, since an agent makes several model calls per answer. Exam cue: a stem about an agent picking the wrong tool points to tool_choice, tool descriptions or a stronger model, not to temperature.
7 · The biggest cost levers
Is prompt caching the biggest lever? Usually not: not calling a model and choosing a smaller one come first, because they cut every token. Caching only discounts repeated input. For agents it's often the biggest single lever, because every turn resends the instructions, tools and history, so input dwarfs output.
Order of attack: avoid the call, then pick the smaller model, then trim output, then cache, then batch.
Order of attack: avoid the call, then pick the smaller model, then trim output, then cache, then batch.
| Lever | How | Typical effect | Azure mechanism |
|---|---|---|---|
| 1 · Don't call a model | Answer fixed questions with rules; use a Foundry Tool for PII, OCR, translation; cache whole responses for repeat questions. | Up to 100% for that traffic | Rules step, Language, Translator, Document Intelligence |
| 2 · Right-size the model | Use the smallest model that passes your evaluation; flagships only where they earn it. | Often 5–25× cheaper per token | gpt-5-nano / mini, Phi; model router in Cost mode |
| 3 · Cut output and reasoning tokens | Output costs several times input, and reasoning tokens bill as output. Lower reasoning_effort, cap max_output_tokens, ask for short structured answers. | Large on reasoning-heavy apps | reasoning_effort, max_output_tokens, structured outputs |
| 4 · Prompt caching | Stable content first (instructions, tools, documents), changing content last; append-only history. | Cached input at about 10% of the input price on GPT-5-family models; doesn't touch output | Automatic from 1,024 tokens; prompt_cache_key; 24h retention on supported models |
| 5 · Batch what can wait | Overnight scoring, bulk extraction, evaluations. | About 50% off | Global Batch / Data Zone Batch deployments |
| 6 · Send less input | Fewer RAG chunks (top_k), summarise old turns, trim large tool results. | Proportional to tokens removed | Search top_k, conversation compaction |
| 7 · Fewer agent turns | Every turn resends the conversation and tool results. Cap turns, give tools that return exactly what's needed. | Multiplies with every lever above | Clear tool schemas, tool_choice, max turns in your loop |
| 8 · Commit capacity at steady volume | Provisioned throughput or reservations when utilisation stays high. | Cheaper per token when busy; waste when idle | Provisioned deployments (cached input up to 100% off) |
Hidden costs to watch: playground evaluations (on by default, billed), idle Provisioned capacity, AI Search tier, and trace and log storage in Application Insights.
Prompt caching across providers
| Provider | Turn it on | Saving | Lifetime | Writing the cache |
|---|---|---|---|---|
| Azure OpenAI (Foundry) | Automatic, on by default. GPT-5.6+: optional explicit breakpoints and prompt_cache_key | Discount on Standard; up to 100% on Provisioned | In-memory 5–10 min idle (max 1 h); 24 h extended on GPT-4.1 / GPT-5.x; GPT-5.6+: 30 min minimum | Free before GPT-5.6; can be charged from GPT-5.6 |
| OpenAI | Automatic from 1,024 tokens; prompt_cache_key | Cached input at 0.1× on GPT-5 family | 5–10 min idle; 24 h extended option | Free on most models |
| Anthropic (Claude) | Top-level cache_control (automatic) or up to 4 explicit breakpoints | Reads 0.1× (lower on newest models) | 5 min default; 1 h option | 1.25× (5 min) or 2× (1 h) |
| Google Gemini | Implicit on 2.5 and newer (automatic); explicit cache objects too | Discount on hits (see pricing) | Explicit: you set the lifetime | Explicit caches charge storage |
Checked on 8 October 2026 against Azure prompt caching, Claude prompt caching and Gemini caching. Same rule everywhere: the cache matches the start of the prompt, so one changed character early on (a timestamp, a user ID) is a miss. Check hits in usage.prompt_tokens_details.cached_tokens.
Exam traps: there's no "enable caching" switch on an Azure deployment: it's on by default for GPT-4o and newer. A prompt under 1,024 tokens never caches. Caching makes repeated input cheaper and faster; it never changes the answer.
8 · Model choice isn't deployment choice
Picking the model answers "what can do this job?". Picking the deployment type answers "where is it processed, and how do I pay?" (Global, Data Zone, Standard, Provisioned, Batch). A stem about residency or throughput is asking about the deployment, not the model. See D1.