WHY THIS VERSION Not every Claude build is a loop
The Text-to-SQL companion taught the agent loop (stop_reason
β tool_use β tool_result). G1 deliberately teaches the other half of the exam: a
single, grounded, structured, cached Messages API call β no tools, no loop. Detection stays
deterministic; Claude only narrates. Knowing when not to build a loop is itself a D1
judgement the exam probes.
output_config.format), grounding / shape-vs-truth, prompt caching a system prefix, and the trap
that you must still check stop_reason even on a one-shot call.LOOP vs SINGLE-CALL Which shape, and why G1 is the right call
| Question | π Agent loop (Text-to-SQL / G4) | π£ Single structured call (G1) |
|---|---|---|
| Who decides the next action? | The model, from observed tool results | Nobody β there's one step; the model only narrates |
| Does correctness need running something? | Yes β you can't know the SQL works without executing it | No β "is it broken?" is deterministic math, done before the call |
| Tools? | Yes (run_sql, drill-downs) | None. Evidence is passed in the prompt |
| Output | Free-text answer at end_turn | Schema-constrained JSON (an incident list) |
| Main exam surface | D1 loop Β· D4 tool contract | D3/D4 structured output Β· D5 caching & grounding |
DIAGRAM Deterministic detect β one grounded call
(system tables)"] --> D{"detect()
DETERMINISTIC"} D -->|"no issues"| Q["β quiet β no LLM call"]:::ok D -->|"issues + evidence"| C["ONE Messages API call
system(cached) Β· adaptive thinking
output_config.format(schema)"]:::core C --> S{"check stop_reason"} S -->|"refusal / max_tokens"| H["handle explicitly"]:::warn S -->|"end_turn"| P["parse JSON"] P --> G["ground(): drop any run_id
not in the evidence"]:::core G --> OUT["digest"]:::ok classDef core fill:#f3f0ff,stroke:#764ba2,color:#4a3b7a; classDef ok fill:#dcfce7,stroke:#15803d,color:#14532d; classDef warn fill:#fef3c7,stroke:#b45309,color:#7c2d12;
CODE The whole sentinel, on the Claude API
Deterministic detection, then one structured call with the instructions cached and the output constrained to a schema. Every line is annotated with the CCA concept it embodies.
# pip install anthropic Β· export ANTHROPIC_API_KEY=sk-ant-... (or: ant auth login) import anthropic, json client = anthropic.Anthropic() # --- 1. DETERMINISTIC detection (NO Claude). "Is it broken?" NEVER touches the model. --- def detect(runs): issues = [] for r in runs: if r["state"] == "FAILED": issues.append({"kind":"failure", "job":r["job_name"], "run_id":r["run_id"], "evidence":r["error"]}) elif r["retries"] >= 3: issues.append({"kind":"retry_spike", "job":r["job_name"], "run_id":r["run_id"], "evidence":f"{r['retries']} retries"}) elif r["duration_min"] > 3 * r["baseline_min"]: issues.append({"kind":"slow", "job":r["job_name"], "run_id":r["run_id"], "evidence":f"{r['duration_min']}m vs {r['baseline_min']}m"}) return issues # --- D3/D4: STRUCTURED OUTPUT β the model must return THIS shape, nothing else. --- TRIAGE_FORMAT = {"format": { "type": "json_schema", "schema": { "type": "object", "properties": {"incidents": {"type": "array", "items": { "type": "object", "properties": { "run_id": {"type": "string"}, "job": {"type": "string"}, "severity": {"type": "string", "enum": ["low", "med", "high"]}, "likely_cause":{"type": "string"}, "cited_run_id":{"type": "string"}, # it MUST cite the evidence it used }, "required": ["run_id","job","severity","likely_cause","cited_run_id"], "additionalProperties": False, }}}, "required": ["incidents"], "additionalProperties": False, }}} # --- D5: stable instructions marked for PROMPT CACHING; the grounding rule lives here. --- SYSTEM = [{ "type": "text", "text": ("You are an Ops triage assistant. Cluster and rank ONLY the issues you are given. " "For each, set a severity and a plain-English likely cause, and cite the run_id of the " "evidence you used. NEVER invent a cause, a job, or a run_id that is not in the input."), "cache_control": {"type": "ephemeral"}, # D5: cache the stable prefix (mind the min-token floor) }] def triage(issues): if not issues: # D5/cost: never call the model with nothing to say return {"incidents": []} resp = client.messages.create( model="claude-opus-4-8", # D5: model tier by task β Haiku 4.5 is plenty for narration max_tokens=4000, system=SYSTEM, thinking={"type": "adaptive"}, # D3/D1: adaptive thinking for clustering (NOT budget_tokens β 400) output_config=TRIAGE_FORMAT, # D3/D4: constrain the response to the schema (deprecated: output_format) messages=[{"role": "user", "content": json.dumps(issues)}], # D1: stateless β evidence goes in every call ) sr = resp.stop_reason # D1: CHECK stop_reason even with NO loop β a single call can still fail if sr == "refusal": return {"error": "declined for safety"} # HTTP 200, but not an answer if sr == "max_tokens": return {"error": "truncated β raise max_tokens"} # cut off β done if sr != "end_turn": return {"error": f"unexpected stop_reason: {sr}"} data = json.loads(next(b.text for b in resp.content if b.type == "text")) return ground(data, issues) # D5: schema guarantees SHAPE, not TRUTH β validate the VALUES # --- D5: structured output can't break the SHAPE, but it CAN invent VALUES. Enforce grounding in code. --- def ground(data, issues): real = {i["run_id"] for i in issues} data["incidents"] = [c for c in data["incidents"] if c["cited_run_id"] in real] # drop invented run_ids return data runs = json.load(open("mock/runs.json")) # PROD: SELECT system.lakeflow.job_run_timeline via databricks-sdk digest = triage(detect(runs)) # detect (deterministic) β triage (one grounded, cached call) print(json.dumps(digest, indent=2))
ground() step proves the exam's favourite lesson that schema-valid β
correct.DEPLOY ON AZURE DATABRICKS Same code, routed through Model Serving
In her environment the LLM call goes through Mosaic AI Model Serving, not straight to Anthropic. The
good news: Databricks serves Claude via the native Anthropic Messages API, so only the client wiring
changes β the entire messages.create(...) call (adaptive thinking, output_config.format,
stop_reason, grounding) is byte-for-byte identical. Three lines move:
# pip install anthropic (on the Databricks compute) import anthropic, os # DIRECT (what the exam code shows): # client = anthropic.Anthropic() # β api.anthropic.com, ANTHROPIC_API_KEY # THROUGH AZURE DATABRICKS (Model Serving, native Anthropic Messages API): client = anthropic.Anthropic( base_url = f"https://{os.environ['DATABRICKS_HOST']}/serving-endpoints", # workspace serving URL api_key = os.environ["DATABRICKS_TOKEN"], # Databricks token, not an Anthropic key ) # ...and `model` becomes the SERVING-ENDPOINT name you configured, e.g. "claude-opus-4-8-endpoint". # Everything else β system(cached), thinking:adaptive, output_config.format, stop_reason β is UNCHANGED.
databricks-openai / the OpenAI client against the
same endpoint). It works, but it reshapes the request/response into the OpenAI schema β so native bits like
output_config.format, adaptive thinking, and typed tool_use blocks look different.
For both cert prep and feature parity, prefer the Anthropic Messages API route shown above.STOP REASONS Yes β even a single call
The loop version routes on stop_reason to keep going. Here there's no loop β but you still
check it, because a one-shot call can come back refusal (HTTP 200, no answer) or max_tokens
(truncated JSON that will fail to parse). Reading content before checking stop_reason is the trap.
| stop_reason | On this single call it means⦠| This code does |
|---|---|---|
end_turn | Finished β the JSON is complete | Parse it, then ground() the values |
max_tokens | The JSON was cut off β parsing will fail | Return an error β raise max_tokens / stream, don't parse |
refusal | Safety classifier declined (200, empty) | Return a graceful "declined" |
pause_turn / stop_sequence / other | Not expected here (no server tools / stop strings) | Catch-all error β never assume it's a valid answer |
refusal and
max_tokens are both HTTP 200 responses you must branch on before touching content.CERT MAP Every API line β an exam concept
output_config.format β json_schema + additionalProperties:false) β D3/D4. Trap: the old output_format is deprecated; combining it with citations is a 400.ground() step validates the values the schema can't β D5 Β· Silent Failures. Trap: trusting a parsed object because it validated.cache_control, order toolsβsystemβmessages, byte-stable) β D5 Β· Prompt Caching. Trap: the min cacheable prefix (~1024 Haiku / 2048 / 4096 Opus tokens) β a short instruction silently won't cache.{"type":"adaptive"}, not budget_tokens β D3/D1. Trap: budget_tokens is a 400 on Opus 4.8/4.7/Sonnet 5.stop_reason even without a loop (refusal, max_tokens) β D1 Β· Stop Reasons. Trap: reading content before branching.TRADE-OFFS Single structured call vs an agent loop
β Single structured call (G1)
- Cheap, fast, deterministic-first β one call, capped
- No tool-contract surface, no runaway loop
- Machine-parseable output by construction
- Correct when the model has nothing to decide
π Agent loop (G4 / Text-to-SQL)
- Needed when the next action depends on an observed result
- More power, more cost, more failure modes
- Requires
stop_reasonrouting + the tool contract - Use it for RCA drill-down, not for detection
SWAP-INS The same code, bigger
- Go live β replace the
runsload with adatabricks-sdkcall / aSELECToversystem.lakeflow.job_run_timelineusing a read-only service principal. Nothing else changes. - Auto-validated parsing β use the SDK helper
client.messages.parse(...)with your schema instead of hand-parsing the JSON. - Nightly digest at scale β run the triage over many workspaces via the Batch API (~50% cheaper, async) β D5 economics.
- Cheaper β
model="claude-haiku-4-5"for narration; the detection is free either way. - Grow it into G4 β add read-only drill-down tools and a
stop_reasonloop and this becomes the RCA Copilot.