Azure AI Engineer · AI-103
Domain 2 · 30–35%Implement generative AI and agentic solutions

Generative AI and agents

The biggest domain, and the one closest to CCA-F. Each card says what you already know from CCA-F, then the Azure detail the exam tests.

How the exam asks: about 80% of questions are Azure-specific (which service, setting, role, SDK call, or GA vs preview) and about 20% are concept-led. CCA-F gives you the why; these cards give you Azure's names, defaults, limits and traps.
PracticeD2 practice bank →

22 items, all formats, practice or timed exam mode.

ABuild generative applications with Foundry

2.1

Connect an app to a Foundry project

One project endpoint, one keyless credential, and you get an OpenAI-compatible client for models and agents.

From CCA-F: Like creating an Anthropic() client with an API key, except Azure prefers your Entra identity over a key, and the client is bound to a project.

How it works

  1. AIProjectClient(endpoint, DefaultAzureCredential())
  2. project.get_openai_client()
  3. client.responses.create(model=..., input=...)

Must know

  • Endpoint: https://<resource>.services.ai.azure.com/api/projects/<project>.
  • Package azure-ai-projects 2.x; 1.x code doesn't run on it.
  • Direct model calls can also use OpenAI(base_url=".../openai/v1/") with an Entra token: no api-version.

Traps

  • AzureOpenAI() with an api-version is the older pattern.
  • Agents and evaluations need Entra ID; keys fail there.

Check yourself: d2-c1 · d2-q18 · Microsoft Learn

2.2

Deploy and consume models

Pick a model from the catalogue, deploy it with a deployment type, then call it through the Responses API.

From CCA-F: Same as choosing Haiku, Sonnet or Opus for a job; here the catalogue also has GPT, Llama, Phi and Claude, and each needs a deployment first (except preview instant models).

How it works

  1. Choose a model (small, large, reasoning, multimodal, code)
  2. Deploy it (Global Standard to start)
  3. Call it by deployment name

Must know

  • Reasoning models reject temperature, top_p, max_tokens; use max_output_tokens and reasoning_effort.
  • Reasoning tokens are billed as output tokens.
  • A developer message equals a system message; don't send both.

Traps

  • max_tokens on a reasoning model is an error.
  • Bigger isn't the default answer: the stem's constraint (cost, latency, modality) decides.

Check yourself: d2-q12 · Microsoft Learn

2.3

Implement RAG

Retrieve the right passages first, then ask the model to answer only from them.

From CCA-F: Exactly the RAG pattern from CCA-F context management; Azure supplies the retriever (Azure AI Search, Foundry IQ, file search).

How it works

  1. Index content in Azure AI Search
  2. Attach the Search tool (or a knowledge base) to the agent
  3. The agent retrieves, then answers with citations

Must know

  • Search tool query_type defaults to vector_semantic_hybrid; top_k defaults to 5.
  • The project's managed identity needs Search Index Data Contributor and Search Service Contributor.
  • File search = documents the user uploads; Foundry IQ = curated organisational knowledge.

Traps

  • Pure vector search misses exact codes and IDs; hybrid catches them.
  • File search isn't for shared company knowledge.

Check yourself: d2-n1 · d2-m2 · Microsoft Learn

2.4

Design workflows and multi-step pipelines

Chain model calls, tools and rules into a defined flow; build new ones on Microsoft Agent Framework.

From CCA-F: The workflow-vs-agent choice from CCA-F: a fixed sequence (workflow) vs the model choosing steps (agent). Same decision here.

How it works

  1. Fixed steps → a workflow (graph)
  2. Model chooses steps → an agent
  3. Use Agent Framework for both in new code

Must know

  • Agent Framework 1.0 (Python, .NET) succeeds Semantic Kernel and AutoGen.
  • Prompt flow retires 20 April 2027.
  • Foundry Workflows (preview) retire 1 December 2026.

Traps

  • Prompt flow or Workflows for a new long-lived build = the retiring answer.

Check yourself: d2-q13 · Microsoft Learn

2.5

Evaluate models and apps

Score quality, groundedness, relevance and safety on a dataset before release, with built-in evaluators.

From CCA-F: LLM-as-judge and golden sets from CCA-F; Foundry ships the judges and runs them in the cloud.

How it works

  1. Upload a JSONL dataset
  2. Pick evaluators (builtin.<name>)
  3. evals.create, then evals.runs.create

Must know

  • RAG evaluators: retrieval, groundedness (1–5), relevance; Groundedness Pro is pass/fail and needs no judge deployment.
  • Safety evaluators (violence, protected material, indirect attack…) need no model parameter.
  • Default pass threshold is 3 on the 1–5 scale.

Traps

  • Fabrication = groundedness, not relevance or coherence.

Check yourself: d2-n2 · Microsoft Learn

BBuild agents with Foundry

2.6

Prompt agents and hosted agents

A prompt agent is declared (model + instructions + tools) and Foundry runs it; a hosted agent is your own code in a container.

From CCA-F: A prompt agent is like a Claude agent defined only by system prompt and tools; a hosted agent is like running your own Agent SDK loop, but hosted by Foundry.

How it works

  1. project.agents.create_version(agent_name, PromptAgentDefinition(...))
  2. Each change creates a new agent version
  3. Hosted agents: container with Agent Framework, LangGraph or others

Must know

  • Prompt agents are GA.
  • Agent versions are snapshotted automatically, handy for CI/CD.
  • A Responses call without a stored agent is an 'ephemeral agent'.

Traps

  • create_agent and assistants are the classic API; v2 is create_version.

Check yourself: d2-q9 · d2-o1 · Microsoft Learn

2.7

Conversations: tracking state

Agents v2 keep history in conversations; each model turn is a response.

From CCA-F: In the Claude Messages API you resend the whole history yourself; here a conversation object can hold it server-side for you.

How it works

  1. oa = project.get_openai_client(agent_name=...)
  2. conv = oa.conversations.create()
  3. oa.responses.create(conversation=conv.id, input=...)

Must know

  • Threads → conversations; runs → responses; assistants → agent versions.
  • Agents in one project share conversation storage; other projects can't see it.
  • Standard setup puts conversations in your own Cosmos DB.

Traps

  • Thread and run vocabulary in an answer option signals the classic API.

Check yourself: d2-q9 · Microsoft Learn

2.8

Tools and tool schemas

Tools give an agent actions; each has a name, a description and a JSON schema for its arguments.

From CCA-F: Same as CCA-F tool design: name + description + input schema. Function calling runs in your code, like tool_use handled by your loop.

How it works

  1. Define the function schema
  2. Model returns a function-call item
  3. Your app runs it and sends the output back

Must know

  • Built-in tools: code interpreter, file search, Azure AI Search, OpenAPI, MCP, Azure Functions, Bing grounding, web search, A2A.
  • Computer use and memory are preview.
  • tool_choice: auto, required or none.

Traps

  • Bing grounding and web search send queries outside the Azure compliance boundary.
  • Azure Functions tool needs the Standard agent setup.

Check yourself: d2-q14 · d2-m1 · Microsoft Learn

2.9

MCP tools and the toolbox

Connect MCP servers as tools; a toolbox shares curated tools with many agents through one MCP endpoint.

From CCA-F: The same Model Context Protocol from CCA-F, unchanged. New: approval is built into the tool definition.

How it works

  1. MCPTool(server_label, server_url, require_approval=...)
  2. Model returns an approval request
  3. You approve or reject; the call runs

Must know

  • require_approval defaults to always.
  • Non-streaming MCP calls time out at 100 seconds; use background mode for longer.
  • Private MCP servers need the Standard setup with your VNet.

Traps

  • require_approval="never" on a write tool removes the human.

Check yourself: d2-q11 · d2-q16 · d2-y2 · Microsoft Learn

2.10

Memory and knowledge for agents

Pick by source: memory for what the agent learns about a user, Foundry IQ for organisational knowledge, file search for uploaded files.

From CCA-F: CCA-F's context-management choices (what to keep, what to retrieve), turned into three Azure services.

How it works

  1. About the user, across chats → memory (preview)
  2. Curated company content → Foundry IQ
  3. Files brought into this chat → file search

Must know

  • Memory types: user profile, chat summary, procedural.
  • Memory limits: 100 scopes per store, 10,000 memories per scope; no VNet support.
  • Foundry IQ connects agents to knowledge bases over MCP.

Traps

  • Memory is preview; it won't be the answer when the stem demands GA.

Check yourself: d2-m2 · Microsoft Learn

2.11

Multi-agent solutions

When one agent hands work to another, use the A2A tool; for fixed multi-step flows, use Agent Framework workflows.

From CCA-F: CCA-F's orchestrator–subagent pattern. Here the 'call another agent' primitive is the A2A tool.

How it works

  1. Agent A gets an a2a tool pointing at agent B
  2. A delegates; B answers
  3. Both traces join in App Insights

Must know

  • A2A tool (protocol v1.0) is GA.
  • Connected Agents isn't available in the new Agent Service.
  • Workflows are preview and retire 1 Dec 2026.

Traps

  • 'Connected Agents' is the classic answer.

Check yourself: d2-q10 · Microsoft Learn

2.12

Safeguards and approvals

Keep a human in the loop for risky actions: MCP approvals, gates around function calls, and guardrails on tool calls.

From CCA-F: Claude Code's permission modes and hooks, in agent form: the agent proposes, a person or rule approves.

How it works

  1. Classify actions: read vs write vs irreversible
  2. Require approval on writes
  3. Add guardrail checks at tool call and tool response

Must know

  • Guardrail intervention points for agents: user input, output, tool call, tool response (tool points are preview).
  • Guardrails for agents are preview; annotate-only isn't available for agents.
  • An agent's own guardrail fully overrides its model's.

Traps

  • A guardrail detects risk; it doesn't replace a human approval.

Check yourself: d2-q11 · d2-s1 · d2-s2 · d2-s3 · Microsoft Learn

2.13

Monitor and evaluate agents

Agent evaluators score whether it understood the user, followed instructions and used tools correctly.

From CCA-F: CCA-F trajectory evaluation (was the right tool called with the right arguments?), as built-in evaluators.

How it works

  1. Intent resolution: did it understand?
  2. Task adherence: did it follow instructions?
  3. Tool call accuracy: right tool, right arguments?

Must know

  • Agent evaluators include task adherence, intent resolution, tool call accuracy, tool selection, tool input accuracy, task completion.
  • Evaluation level: turn (default) or conversation.
  • Continuous evaluation samples live traffic; alerts go through Azure Monitor.

Traps

  • Coherence and fluency don't tell you whether tools were used correctly.

Check yourself: d2-y1 · Microsoft Learn

COptimise and operationalise

2.14

Tune generation: prompts and parameters

Improve answers with prompt changes first; then adjust parameters; reasoning models use effort instead of temperature.

From CCA-F: Prompt engineering from CCA-F applies as-is; extended thinking's budget maps to reasoning_effort.

How it works

  1. Clear instructions and examples
  2. Classic models: temperature / top_p
  3. Reasoning models: reasoning_effort, max_output_tokens

Must know

  • reasoning_effort ranges from none to max, depending on the model.
  • max_output_tokens includes reasoning tokens.
  • Parallel tool calls aren't supported at minimal effort.

Traps

  • Raising effort before measuring is the costly answer.

Check yourself: d2-q12 · d2-q17 · Microsoft Learn

2.15

Reflection and self-critique

Add a critic step: an evaluator or a second pass checks the answer before it's used.

From CCA-F: CCA-F's evaluator–optimizer pattern.

How it works

  1. Generate
  2. Critique with an evaluator (e.g. groundedness, task adherence)
  3. Revise or hand off if it fails

Must know

  • Docs favour evaluating before raising effort.
  • A judge needs its own model deployment, except Groundedness Pro and safety evaluators.

Traps

  • Calling the model twice on every request doubles cost before you know what's wrong.

Check yourself: d2-q17 · Microsoft Learn

2.16

Observability and orchestration

Connect Application Insights for tracing with no code change; orchestrate several models with model router or rules-first logic.

From CCA-F: CCA-F's tracing and cost-awareness; here OpenTelemetry GenAI traces land in App Insights automatically.

How it works

  1. Connect App Insights to the project
  2. Server-side traces appear (90 days in the portal)
  3. Add client-side spans for extra detail

Must know

  • Tracing is GA for prompt and hosted agents; preview for workflow and external agents.
  • Querying traces needs Log Analytics Reader.
  • Model router: Balanced/Quality/Cost modes; smallest model's context window.

Traps

  • 'Add OpenTelemetry spans in code' is wrong when the stem says no code changes.

Check yourself: d2-q15 · Microsoft Learn