Gen AI Fundamentals › Stack Decisions

🧭 Stack Decisions

Seven layers · what actually constrains what · and why the permutation count is a red herring
The confusion this page resolves. "Claude" names two different things at two different layers — a model, and an agent framework. So does "OpenAI". Once the layers are separated, the question "does choosing X restrict me to Y?" has a short, mechanical answer.

1 · The seven layers concept

Read from the bottom up. Each layer is chosen separately — but only two edges between them are real constraints.

1LanguagePython · TypeScript · Java · .NET · Go  — decides which SDKs exist for you, and nothing else
2Runtimelaptop · container · serverless function · Databricks job  — orthogonal to every other layer
3Orchestrationnone (plain code) · agent framework · graph framework · durable execution  — the only genuinely free choice, and the one that decides lock-in
4Client SDKAnthropic SDK · OpenAI SDK · boto3/Bedrock · Vertex SDK · LiteLLM · raw HTTP  — follows from the wire format
5GatewayKong · Azure APIM · LiteLLM Proxy · Portkey · none  — pins the wire format
6Provider / hostingAnthropic API · Azure OpenAI · Bedrock · Vertex · Databricks Serving · self-hosted vLLM
7ModelClaude Sonnet · GPT-4-class · Llama · Gemini  — a config value, if layer 3 was chosen well
free — choose independently constrained — usually decided for you the decision that matters
The same word at two layers. Claude at layer 7 is a model. Claude Agent SDK at layer 3 is an agent framework. You can use the model without the framework — most systems should. You cannot use the framework with a non-Claude model. The same trap exists for OpenAI (a company, a wire format, an SDK, and an agent framework).

2 · Only two edges are real constraints key insight

Constraint A — the wire format pins the client

There are effectively two formats in the world:

  • OpenAI Chat Completions — the de-facto standard. Azure OpenAI, most gateways, most open models, most self-hosted servers.
  • Anthropic Messages — Anthropic API; also available on Bedrock and Vertex.

Your gateway speaks one of them, and that decides your client SDK. It is not a judgement call.

Constraint B — orchestration decides lock-in

The rule that resolves almost every "am I locked in?" question:

Frameworks that own the loop are tied to a model family.
Frameworks that own the graph are model-agnostic.
Plain code is always agnostic.

An agent framework decides what to call next, so it is built around one model's tool-calling and stop semantics. A graph framework only decides what runs after what and takes whatever client you hand it.

3 · Layer 3 in depth — who owns the loop? the confusing one

First, the distinction people skip. A client SDK talks to one model, once — send text, get text. Orchestration sequences many steps — model calls, tool calls, branches, retries, state. The client is the HTTP library; orchestration is the application framework. You always need the first. You often need nothing for the second.

The four options differ on exactly one question: who owns the while loop?

1 · Plain code — you own the loop

def heal(failure):
    cause = analyse(failure, graph)          # your function
    text  = llm.complete(system, prompt)     # ← client SDK, one call
    fix   = generate(cause)                  # your function
    if gate(fix).approved:
        deploy(fix)

Nothing hidden, every branch written by you. Debuggable with a breakpoint.

2 · Agent framework — the model owns the loop

agent = Agent(model="claude-sonnet-5",
              tools=[read_logs, query_graph, run_sandbox])
result = agent.run("Diagnose why curate_orders failed")

# the model decides the sequence at runtime:
#   read_logs → query_graph → read_logs again → conclude

Inversion of control. You supply tools; it supplies the plan. You cannot know the sequence in advance — which is the point, and the cost.

3 · Graph framework — you declare the loop, it runs it

g = StateGraph(Case)
g.add_node("rca", rca_fn)
g.add_node("deploy", deploy_fn)
g.add_conditional_edges("gate", route, {"auto": "deploy", "human": END})
app = g.compile(checkpointer=SqliteSaver("cases.db"))

You still decide the sequence — but declared rather than written, and you get checkpointing, resume and a picture of the topology in return.

4 · Durable execution — looks like plain code, survives death

@workflow.defn
class HealWorkflow:
    @workflow.run
    async def run(self, failure):
        cause = await workflow.execute_activity(analyse, failure)
        await workflow.wait_condition(lambda: self.approved)   # can wait DAYS
        await workflow.execute_activity(deploy, fix)

That wait_condition spans a process restart, a deployment, a weekend. The runtime persists every step.

OptionOwns the loop You getYou pay Model-agnostic?
Plain codeYou Full control, trivial debugging, no dependency You build resume and retries yourself✅ always
Agent frameworkThe model Handles paths you cannot enumerate Non-determinism, cost, harder debugging, lock-in❌ per model family
Graph frameworkThe framework Checkpointing, resume, branch/parallel, visualisation A DSL to learn; indirection✅ you supply the client
Durable executionThe runtime Survives crashes and long waits; retries and timers A server to operate; a programming model✅ you supply the client
Why this is the lock-in layer. Only the agent framework has to speak the model's tool-calling and stop-reason protocol — a per-model-family contract. The other three never touch it: they call your function, which calls your client. Model-agnostic by construction.

Choosing, in order

  1. Can you write the steps down? → plain code. Stop here far more often than feels natural.
  2. Must a run survive a crash or a long human wait? → durable execution.
  3. Is the topology genuinely complex — branches, parallel, joins? → graph framework.
  4. Does the model need to choose the sequence because you cannot know it? → agent framework.
They compose. Temporal can wrap plain code. A graph can have an agent as a single node. The common mature shape is durable execution for the spine, plain code for the steps, and one agentic node where the path is genuinely unknowable.
The expensive mistake is reaching for an agent framework because the work is "AI work". If you can enumerate the steps, an agent buys you non-determinism, latency and a debugging problem in exchange for nothing. Agents earn their place only where the path depends on what is discovered mid-run.

4 · Compatibility — orchestration × provider matrix

Layer-3 choice Owns Works with Through Kong → Azure OpenAI?
Plain code + a model clientnothing Every provider, every model✅ yes
LangGraph, LlamaIndex Workflows, Burrthe graph Every provider — you supply the client✅ yes
Temporal, Restate, Step Functionsdurability Everything; wraps any of the above✅ yes
OpenAI Agents SDKthe loop OpenAI, Azure OpenAI, OpenAI-compatible endpoints✅ yes
MS Agent Framework / Semantic Kernelthe loop Azure OpenAI first; others via connectors✅ yes
Claude Agent SDKthe loop Claude models only — Anthropic API, Bedrock, Vertex❌ no
Strands (AWS)the loopBedrock first⚠️ not the usual path
Google ADKthe loopVertex first⚠️ not the usual path
So: does the Claude Agent SDK restrict you? Yes — but only if you adopt the framework. Using Claude the model restricts nothing, and using the Anthropic client for a plain "system prompt + messages → text" call is the most portable shape a model call has. Keep the model at the edges and the provider stays a configuration choice rather than an architectural one.

5 · Why the permutation count is a red herring

Multiplied out, the stack looks terrifying:

5 languages× 4 runtimes× 6 orchestration× 6 clients× 5 gateways× 6 providers× 10 models =~216,000

But the layers cascade rather than combine. In practice you take four decisions, in this order, and each one collapses the space:

#DecisionWho really makes itWhat it collapses
1Provider + gatewayYour organisation, usually before you arriveLayers 5 and 6 — down to one
2Wire formatNobody — it falls out of #1Layer 4 — down to one
3OrchestrationYou. The only free architectural choiceDecides whether layer 7 stays free
4ModelYou — measured, not assertedA config value if #3 was chosen well

Language and runtime are orthogonal to all four. So the real decision tree is four nodes deep, not seven — and only one node is genuinely yours.

6 · A worked example applied

The same system, running in a corporate estate and on a personal free-tier account. One code path.

LayerEnterpriseDemo / laptopChanged?
LanguagePythonPython—
RuntimeContainer (control plane)Laptopdeployment only
OrchestrationPlain codePlain code—
Client SDKOpenAI SDKOpenAI SDK—
GatewayKongnonebase URL
ProviderAzure OpenAIDatabricks Servingbase URL
ModelGPT-4-classllama-4-maverickconfig value
# The entire provider surface. Three implementations, no call-site changes.
class LLMClient(Protocol):
    def complete(self, system: str, user: str, max_tokens: int = 800) -> str: ...

class OpenAICompatible:          # Kong → Azure OpenAI, or any OpenAI-shaped endpoint
    def __init__(self, base_url, api_key, model):
        self._c = OpenAI(base_url=base_url, api_key=api_key)   # base_url = the Kong route

class AnthropicDirect: ...       # Anthropic Messages API
class DatabricksServing: ...     # in-workspace models, no external key

Selected by one environment variable. Two config values separate a corporate deployment from a laptop demo.

7 · Building the mind map practice

Draw it as a ladder, not a web — seven rungs bottom to top — and annotate each rung with three things:

  1. Who decides it — you, your organisation, or physics (the wire format).
  2. What it constrains — draw an arrow only where a real constraint exists. There are exactly two: gateway → client, and orchestration → model family.
  3. Reversibility — how much work to change it later. Model: a config value. Gateway: a base URL. Orchestration: a rewrite. Language: a rebuild.
The one thing to internalise: the layers people argue about — model, provider, gateway — are the cheap, reversible ones. The layer nobody argues about, orchestration, is the expensive irreversible one. Spend your design time there and treat the rest as configuration.
Related: AI Engineering Blueprint — what each layer must provide · AI gateways — layer 5 in depth · Orchestration frameworks — layer 3 in depth.