Gen AI Fundamentals β€Ί Tooling Landscape

🧰 AI Tooling Landscape

Observability Β· AI gateways Β· Orchestration Β· Graph, memory & code graphs β€” open source and commercial
How to use this. Four categories where the choice actually matters and the market is crowded. Each table ends with a default recommendation β€” not because it is always right, but because having a default means you only argue about the cases that deserve an argument. Companion to the AI Engineering Blueprint, which says what you need; this says what to use.

Going deeper on two of them: Observability & Eval β€” What Each Tool Offers has the feature-by-feature matrices and the cloud-provider equivalents, and AI Gateway β€” The Control Point covers that layer on its own page.
OSS open source, self-host OSS + SaaS open core with a hosted option cloud-native tied to one platform

1 Β· Observability & tracing landscape

LLM observability is ordinary distributed tracing plus three things ordinary APM does not capture: the prompt and response, token usage and cost, and a quality score. That last one is why generic APM alone is rarely enough β€” it can tell you the call took 900 ms, not that the answer was wrong.

Tool What it is best at Watch out for Reach for it when
LLM-native, open source
LangfuseOSS + SaaS Tracing, prompt management, evals, cost β€” the most complete open-source single answer Self-hosting means running Postgres + ClickHouse You want one OSS tool and are willing to run it
Arize PhoenixOSS + SaaS Trace inspection and evaluation notebooks; strong on retrieval debugging Leans experimentation more than long-run production You are debugging RAG relevance
MLflow TracingOSS Traces, prompt registry and evaluation alongside model lifecycle Best experience inside Databricks You already run MLflow β€” do not add a second tool
Opik (Comet)OSS + SaaS Tracing plus an opinionated evaluation library Younger ecosystem Eval-first teams
TruLensOSS Feedback functions β€” programmatic quality scoring Not a full observability backend You want quality scoring, not infrastructure
HeliconeOSS + SaaS Proxy-based capture β€” one line to adopt Proxy sits on the critical path You want logging without touching app code
Laminar, Literal AIOSS + SaaS Traces plus dataset curation from production Smaller communities You want production traces to become eval sets
LLM-native, commercial
LangSmith Deep LangChain/LangGraph integration, datasets, annotation queues Pulls you toward one framework's worldview You are already on LangGraph
Braintrust Eval-centric workflow, side-by-side experiment comparison Eval-first framing suits some teams and not others Prompt iteration is your bottleneck
W&B Weave Traces plus the ML experiment lineage teams already know Heavier than needed for pure GenAI apps You already run Weights & Biases
Galileo, Humanloop, PromptLayer Guardrail metrics, prompt collaboration for non-engineers Overlaps tools you may already own Product/PM colleagues need to edit prompts
General APM that now speaks LLM
Datadog LLM Observability One pane with the rest of your estate; mature alerting Cost at volume; prompt payloads inflate ingest Datadog is already the company standard
New Relic, Dynatrace, Elastic, Splunk, Coralogix AI monitoring bolted onto existing APM Quality evaluation is usually thinner than LLM-native tools Consolidation matters more than depth
SigNoz, HyperDX, Uptrace, Grafana + Tempo/LokiOSS + SaaS OTel-native backends you can self-host You assemble the LLM semantics yourself You want OTel end to end with no vendor
Cloud-native
Databrickscloud-native MLflow Tracing, Agent Evaluation, Lakehouse Monitoring, system tables for cost Only for workloads on Databricks The workload already lives there
Azurecloud-native Application Insights + AI Foundry tracing and evaluation Split across several services Azure-standardised estates
AWScloud-native CloudWatch + X-Ray, Bedrock invocation logging and model evaluation Assemble-it-yourself for LLM specifics Bedrock-centric builds
GCPcloud-native Cloud Trace/Logging plus Vertex AI evaluation and monitoring As above Vertex-centric builds
Default: instrument with OpenTelemetry, and send it to Langfuse (or your existing APM if one is mandated). That keeps the instrumentation portable and makes the backend a reversible decision β€” which is the only property that reliably survives a two-year tool cycle.

2 Β· OpenTelemetry and its alternatives trade-off

Worth being precise, because the question is usually asked wrongly: OpenTelemetry is a standard, not a product. Its true alternatives are proprietary agents. The LLM-specific libraries people name as alternatives are mostly conventions built on top of it.

Option Relationship to OTel Trade-off Choose when
OpenTelemetry + GenAI semantic conventionsOSS The standard itself Vendor-neutral, huge ecosystem; GenAI conventions are still settling, so expect some churn Default. Especially if the backend may change.
OpenLLMetry (Traceloop)OSS On top of OTel Auto-instruments LLM SDKs and emits OTel spans β€” not a replacement You want LLM spans without writing them
OpenInference (Arize)OSS On top of OTel Alternative span conventions for LLM/RAG; pairs with Phoenix You use Phoenix or want richer RAG spans
Vendor SDKs β€” Datadog, New Relic, Dynatrace OneAgent, Elastic APM True alternatives Deeper out-of-the-box features and support; re-instrumenting later is real work The vendor is mandated and unlikely to change
SentryOSS + SaaS Alternative, with OTel support Error-first rather than trace-first Exceptions matter more than latency
Prometheus + StatsDOSS Metrics only No traces β€” cannot answer "why did this request behave that way" You need aggregates, not per-request forensics
Zipkin / Jaeger clients, OpenTracing / OpenCensusOSS Predecessors, merged into OTel Legacy β€” OpenTracing and OpenCensus are archived Only existing systems; do not start here
SDK-native tracing β€” Langfuse SDK, LangSmith SDK, MLflowmixed Alternative, most now OTel-compatible Fastest to adopt; check there is an OTel export path before committing Speed matters and the tool is already chosen
The question to ask a vendor: "can I export raw OTel spans to a backend of my choosing?" If yes, adopting them is reversible. If no, you are choosing a backend and an instrumentation layer at the same time β€” which is how observability becomes the thing you cannot migrate off.

3 Β· AI gateways landscape

This section is the short version. AI Gateway β€” The Control Point is the full page: the 14-capability checklist, an expanded OSS matrix, a proper side-by-side of the five cloud gateways, routing strategies, semantic caching, MCP gateways and failure modes.

An AI gateway is a reverse proxy that understands model traffic. It exists so that key management, routing, fallback, rate limiting, caching, cost attribution, guardrails and logging live in one place instead of being reimplemented in every application. If you have more than two applications calling models, you already need one.

Gateway Strengths Trade-off Choose when
Open source
LiteLLM ProxyOSS + SaaS 100+ providers behind one OpenAI-shaped API; virtual keys, budgets, fallbacks, cost tracking Python service to operate; a hop on the critical path Default for most teams β€” quickest route to multi-provider
Portkey GatewayOSS + SaaS Very fast edge gateway, routing, retries, caching, guardrails Richer features sit in the paid tier Latency-sensitive, want OSS with a hosted upgrade
Kong AI GatewayOSS + SaaS AI plugins on a mature API gateway β€” auth, quotas, transformations Kong concepts to learn if you are not already on it Kong is already your API layer
Envoy AI GatewayOSS Kubernetes-native, built on Envoy Gateway; strong traffic control Operationally heavier; k8s assumed Platform teams standardised on Envoy
Apache APISIX, Higress, Traefik AI pluginsOSS AI routing on an existing ingress AI features are newer than the core proxy You want one proxy for all traffic
Gloo AI Gateway (Solo.io)OSS + SaaS Envoy-based, service-mesh adjacent, policy-rich Enterprise-shaped You run a mesh already
Bifrost, GPT Router, llm-gatewayOSS Lightweight, low-overhead routers Smaller feature sets and communities You want the thinnest possible hop
Managed & cloud-native
Databricks Mosaic AI Gatewaycloud-native Rate limits, payload logging, guardrails, PII detection, usage tracking, all governed by Unity Catalog Databricks-resident workloads You are on Databricks β€” this is the obvious answer
Azure API Management (AI gateway features)cloud-native Token-based rate limiting, semantic caching, load balancing across Azure OpenAI deployments APIM policy language is its own skill Azure estates with PTU + pay-go mixes
Apigee (GCP)cloud-native Mature API management with LLM policy templates Enterprise pricing Apigee already in place
AWS β€” Bedrock + API Gateway / generative-AI gateway patternscloud-native IAM-native, no new vendor More reference architecture than product β€” you assemble it AWS-only estates that want IAM everywhere
Cloudflare AI Gateway Edge caching, analytics, rate limiting; trivial to switch on Traffic leaves your perimeter Public-facing apps already on Cloudflare
OpenRouter, Unify, Martian, Not Diamond Aggregation and quality/price-aware routing across providers A third party in the data path β€” a real review item in regulated settings Experimentation and model comparison
Default: if you are on a platform, use its gateway β€” Mosaic AI Gateway on Databricks, APIM on Azure. Otherwise LiteLLM Proxy to start, and move to Portkey or Envoy AI Gateway when latency or scale makes the hop expensive.

What the gateway takes off every application

The reason a gateway is the highest-leverage shared component: it applies to every application, in any language, with zero application code. Nine of the thirty-seven requirements in the blueprint can be satisfied here and nowhere else as cheaply.

Concern What you stop writing in every app
Model access & routingProvider SDKs, endpoint config, one API shape for every model
Fallback & retries"Primary down β†’ secondary" logic, backoff, load balancing across deployments
Rate limits & quotaPer-key, per-user, per-team limits β€” including token-based, not only request-based
Budgets & cost capsSpend ceilings per team or project, alerts, hard stops
Cost attributionSpend tagged to team, project and use case without touching app code
Payload loggingRequest and response capture into your observability backend
CachingExact and semantic caching across all callers, not once per app
GuardrailsPII redaction and injection screening applied uniformly
Key managementVirtual keys per team β€” rotate centrally, never distribute provider keys

Open-source gateways β€” capability matrix

Gateway Lang Routing & fallback Rate limit / budget Caching Guardrails Logging Notes
LiteLLM ProxyPython βœ… 100+ providersβœ… per key / user / team, token-basedβœ… Redis + semantic βœ… Presidio, Lakera, Bedrock Guardrailsβœ… Langfuse, OTel, S3, Datadog Richest OSS feature set. Admin UI, virtual keys, spend tracking
Portkey GatewayTypeScript βœ…βœ…βœ… simple + semantic⚠️ mostly paid platformβœ… hooks Very low latency, edge-deployable
BifrostGo βœ…βœ…βœ…βš οΈ limitedβœ… Lowest overhead; MCP support
Envoy AI GatewayC++/Go βœ…βœ… token-based⚠️ via Envoy⚠️ externalβœ… OTel Kubernetes/CRD-native; scales hardest
Kong AI GatewayLua βœ… ai-proxy⚠️ advanced limiting is enterprise⚠️ semantic cache enterprise βœ… ai-prompt-guardβœ…Right answer if Kong is already your ingress
Apache APISIXLua βœ…βœ…βš οΈβœ… ai-prompt-guardβœ… Same logic if APISIX is your ingress
HigressGo βœ…βœ… token limitβœ…βœ… security guardβœ… Strong AI plugin set
Several of these projects move features between the open-source and paid tiers. Verify the current tier of anything you are depending on before it becomes load-bearing.

What "gateway config" actually looks like

LiteLLM, because it is the most likely default. Note that the applications never learn any of this exists β€” they keep calling claude-sonnet at the gateway URL.

# config.yaml
model_list:
  - model_name: claude-sonnet                 # the alias apps ask for
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: claude-sonnet                 # same alias again = load balanced
    litellm_params:
      model: bedrock/anthropic.claude-sonnet-5
      aws_region_name: eu-west-1

router_settings:
  routing_strategy: latency-based-routing
  fallbacks: [{ claude-sonnet: [claude-haiku] }]
  num_retries: 2
  allowed_fails: 3                            # circuit breaker
  cooldown_time: 30

litellm_settings:
  cache: true
  cache_params: { type: redis, ttl: 3600, supported_call_types: [acompletion] }
  callbacks: [langfuse, otel]                 # tracing + payload logging
  guardrails:
    - guardrail_name: pii
      litellm_params: { guardrail: presidio, mode: pre_call }
    - guardrail_name: injection
      litellm_params: { guardrail: lakera, mode: pre_call }

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  max_budget: 5000                            # org ceiling, USD per period
  budget_duration: 30d

Per-team virtual keys then carry their own limits and cost tags:

curl -X POST $GATEWAY/key/generate -H "Authorization: Bearer $MASTER_KEY" \
  -d '{"team_id":"ops-eai","models":["claude-sonnet"],
       "max_budget":400,"budget_duration":"30d",
       "tpm_limit":200000,"rpm_limit":500,
       "metadata":{"cost_centre":"EAI-1042","use_case":"incident-search"}}'
That single call is quota, budget and cost attribution for an entire team β€” and the application never knows it happened. This is the concrete reason the gateway belongs outside the application rather than inside a shared library: it governs code you do not control, in languages you did not choose.

Which one to pick

SituationChooseWhy
On DatabricksMosaic AI GatewayGoverned by Unity Catalog; no extra system to run
On AzureAPI ManagementToken-based limiting and balancing across PTU and pay-go deployments
Anywhere else, starting outLiteLLM ProxyBest OSS coverage, fastest to value
Kubernetes, high scaleEnvoy AI GatewayCRD-native, built on proven traffic infrastructure
Latency-critical / edgePortkey or BifrostLowest added hop
Kong or APISIX already in placeTheir AI pluginsOne proxy, not two
Decide this before you adopt one: the gateway is on the critical path of every request. What happens when the gateway is down? Either a direct-to-provider fallback in the client, or an accepted outage with a stated SLO. "We will add HA later" quietly means the gateway is a single point of failure for every AI application at once β€” and you find out during the first incident.

4 Β· Orchestration & agent frameworks landscape

The alternatives to LangGraph fall into three families, and the third is the one most teams overlook.

Framework Family Strengths Choose when
Graph & state-machine frameworks
LangGraphOSSGraph Explicit nodes and edges, checkpointing, human-in-the-loop, mature ecosystem Complex branching you want to see as a graph
LlamaIndex WorkflowsOSSEvent-driven Event/step model rather than a graph DSL; strong retrieval heritage RAG-heavy systems
Burr (DAGWorks)OSSState machine Explicit state machine with good introspection You think in states, not graphs
Pydantic AIOSSTyped agents Type-safe, dependency-injected, familiar to FastAPI teams Python-first teams who value types
Agent-first frameworks
Claude Agent SDKOSSAgent The agent loop, tools, skills, subagents and permissions as first-class concepts Building on Claude β€” the loop is handled for you
OpenAI Agents SDKOSSAgent Lightweight handoffs and guardrails OpenAI-centric builds
Microsoft Agent Framework / Semantic Kernel / AutoGenOSSAgent Enterprise .NET and Python story; multi-agent conversation patterns Microsoft-standardised organisations
Google ADKOSSAgent Pairs with Vertex Agent Engine for managed deployment GCP-centric builds
Strands Agents (AWS)OSSAgent Model-driven loop, deploys neatly to AWS Bedrock-centric builds
CrewAI, Agno, smolagents, Griptape, Atomic AgentsOSSAgent Role-based crews and fast prototyping Getting to a demo quickly
DSPyOSSCompiler Optimises prompts against a metric instead of hand-tuning them You have an eval set and want it to do the tuning
Durable execution β€” the overlooked family
TemporalOSS + SaaSDurable Battle-tested durability, retries, timers, versioning, resumption β€” in plain code, no DSL Long-running or must-not-lose-progress work
Restate, Inngest, DBOSOSS + SaaSDurable Lighter durable execution with modern developer experience You want durability without operating Temporal
Airflow, Prefect, Dagster, Databricks WorkflowsmixedData Scheduling, backfills, lineage, data-aware retries The work is a pipeline that happens to call a model
Step Functions, Durable Functions, Cloud Workflowscloud-nativeDurable Managed, IAM-integrated, nothing to run Cloud-standardised estates
The unfashionable answer is often right. The genuinely hard parts of orchestration are durability, resumption and versioning β€” not the graph DSL. Temporal or Step Functions plus plain Python solves those properly and has done for a decade. Reach for a graph framework when the topology itself is the complexity; reach for durable execution when surviving failure is the complexity. Most production systems have the second problem.

5 Β· Graph & memory frameworks landscape

Two layers people conflate: the framework that builds and queries a knowledge graph for an agent, and the database underneath it. Choose them separately.

Frameworks β€” graph-shaped memory and retrieval

Framework What it does Trade-off Choose when
Graphiti (Zep)OSS Temporal knowledge graph β€” facts carry validity intervals, so it models what was true when Ingest cost; needs a graph database Agent memory where facts change over time
Microsoft GraphRAGOSS Builds entity/community graphs over a corpus for global "what are the themes" questions Expensive indexing pass; batch-oriented Corpus-wide sense-making that vector RAG cannot do
LightRAG, nano-graphragOSS Cheaper, simpler GraphRAG implementations Fewer features than the original You want GraphRAG without the indexing bill
CogneeOSS Pipelines that build graph + vector memory together Younger project You want both substrates from one pipeline
Mem0OSS + SaaS Memory layer with extraction and consolidation; hybrid vector + graph Opinionated about what is worth remembering Per-user long-term memory in an assistant
ZepOSS + SaaS Managed memory service built on Graphiti A service to depend on You want memory without building it
Letta (MemGPT)OSS Agents with self-managed tiered memory A whole agent runtime, not a library Memory management is the core problem
LlamaIndex Property Graph IndexOSS Graph construction and querying inside a familiar RAG framework Tied to LlamaIndex abstractions Already on LlamaIndex
neo4j-graphragOSS First-party GraphRAG patterns for Neo4j Neo4j-specific Neo4j is already chosen

Databases β€” where the graph actually lives

Database Strengths Trade-off Choose when
Neo4jOSS + SaaS Most mature; Cypher, tooling, vector index built in Enterprise features are licensed Default for a real knowledge graph
FalkorDBOSS Very low latency, Redis-based, built for agent memory access patterns Smaller ecosystem Graph reads on the request path
KΓΉzuOSS Embedded β€” no server, like SQLite for graphs Not for large multi-writer workloads Per-agent or per-tenant local graphs
Apache AGE (Postgres)OSS Graph queries inside the Postgres you already run Slower than native graph engines at depth Modest graphs; one fewer system
Memgraph, ArangoDB, NebulaGraph, Dgraph, TigerGraphmixed In-memory speed, multi-model, or very large scale Each has a distinct query language to learn Specific scale or latency demands
Neptune (AWS), Spanner Graph (GCP), Cosmos DB Gremlin (Azure)cloud-native Managed, IAM-integrated, no operations Less portable; ecosystem tooling thinner Cloud-standardised estates
Before adopting any of this, ask: is the question actually multi-hop? GraphRAG and temporal graphs earn their cost on questions like "what depends on this service, transitively" or "what were the themes across ten thousand tickets". If your real question is "find me something similar", a hybrid vector + keyword search is cheaper, faster and easier to explain β€” and it is what most teams needed.

6 Β· Code structure graphs landscape

A knowledge graph of a codebase β€” files, symbols, calls, imports, inheritance, data flow. Distinct from document RAG, and increasingly the substrate that makes coding agents useful: it answers "what calls this?" and "what breaks if I change it?", which similarity search cannot.

Name collision worth knowing. Glean is Meta's open-source source-code indexing system. It has nothing to do with the enterprise search product of the same name. Say "Meta's Glean" or "Glean code indexing" in design discussions.
Tool What it produces Coverage Best for
Built for agents & LLM consumption
GraphifyApache 2.0 On-device graph via tree-sitter; ships as an MCP server, slash command, pre-tool hook and memory-file directive 36 languages, plus SQL, Terraform, docs Dropping a code graph straight into an AI coding assistant with no glue code
BlarifyOSS Parses a repo into a Neo4j/FalkorDB graph of files, classes, functions and their relationships Multi-language via tree-sitter + LSP Giving an agent multi-hop code context β€” the closest thing to "graphify my codebase"
PotpieOSS Repo knowledge graph plus prebuilt agents over it Multi-language Q&A and onboarding on an unfamiliar codebase
tree-sitter + tree-sitter-graphOSS Parse trees, and a DSL for turning them into graphs on your own schema 40+ grammars Rolling your own β€” most custom tools start here
LlamaIndex CodeHierarchyNodeParserOSS Hierarchical code chunks preserving structure for retrieval Multi-language Code RAG without standing up a graph database
Repo maps β€” Aider, ContinueOSS Ranked symbol map of a repo, sized to a context window Multi-language Cheap context for a coding agent, no graph store at all
Precise code intelligence
SCIP / LSIF (Sourcegraph)OSS Definitions, references and implementations as an index Per-language indexers Accurate cross-repo navigation at scale
Kythe (Google)OSS Language-agnostic graph schema for cross-references Several, via extractors Very large monorepos
Stack Graphs (GitHub)OSS Incremental name-binding graphs β€” resolves symbols without a full build Growing set Fast, incremental indexing
Glean (Meta)OSS A queryable database of facts about source code Multi-language via indexers Building your own code-intelligence backend
Semantic & data-flow analysis
JoernOSS Code Property Graph β€” AST + control flow + data flow in one queryable graph C/C++, Java, Python, JS, Go and more The richest structural graph available; security and impact analysis
CodeQL (GitHub)free for OSS Code as a relational database you query Major languages Vulnerability hunting and deep semantic queries
jQAssistantOSS Scans Java/Maven into Neo4j β€” architecture rules as Cypher Java/JVM Java estates; enforcing architecture constraints in CI
Moose / PharoOSS Meta-model of a system for custom analyses Many, via importers Research-grade or bespoke analysis
Dependency & architecture views
dependency-cruiser, madge, pydeps, jdeps, go-callvisOSS Module and call dependency graphs One language each Quick answers with no infrastructure
CodeScene, NDepend, Structure101, Understand, CAST Imaging Architecture maps, coupling metrics, hotspots; CAST targets whole application portfolios Broad Legacy modernisation programmes and estate-wide assessment
Picking one. Feeding an agent β†’ Blarify, or tree-sitter into KΓΉzu/FalkorDB if you want your own schema. Impact and data-flow analysis β†’ Joern. A Java estate β†’ jQAssistant into Neo4j. Accurate navigation across many repos β†’ SCIP. Just enough context for a coding agent and nothing more β†’ a repo map, which needs no database at all.
Start with the repo map. A ranked tree-sitter symbol map solves most coding-agent context problems for a fraction of the effort. Build the graph when you actually need multi-hop answers β€” blast radius, transitive callers, dead code across module boundaries β€” because that is when the traversal earns the pipeline you now have to maintain.

7 Β· Token efficiency on large codebases practice

Working an AI coding assistant against a large repository is mostly a context problem, not a model problem. The levers below are ordered by impact, and the first four cost nothing.

# Lever Why it works Cost
1Delegate exploration to subagents A subagent burns its own context reading twenty files and returns a paragraph. Your main window receives the conclusion, not the dumps. "Find where retry logic lives across the services" should almost never run in the main session. Free
2Clear between unrelated tasks Compaction is lossy and costs tokens to perform; clearing is clean. Check where the window is actually going before optimising anything else β€” the answer is usually surprising. Free
3Keep the project memory file lean CLAUDE.md is prepended to every request. Point at files rather than inlining them. A 500-line memory file is a tax on every turn of every session. Free
4Prune MCP servers Every connected server's tool schemas sit in context on every request. Three chatty servers can cost more than the code you are reading. The most common silent bloat. Free
5Reference files explicitly @src/auth/session.py skips an entire search-and-read cycle versus "find the session handling code". Free
6Skills instead of a large memory file Skills load on demand; the memory file is always resident. Free
7Plan before executing Design first, then act β€” avoids exploratory thrash filling the window with dead ends. Free
8Structural search over text search ast-grep matches code shapes; far fewer false positives than grep, so fewer files opened. Install
9Symbol-level retrieval A language server returns one function, not one file. Setup
10A code knowledge graph Structural questions answered as nodes and edges instead of file text. Worth it once the repo is large, unfamiliar and the questions are genuinely structural. Setup + upkeep
Do 1–4 first. They are free, take an afternoon, need no maintenance, and in most codebases they deliver the bulk of the available saving. Tooling is the fourth step, not the first.

Free and open-source options

Tool Licence What it gives you Best for
Built in β€” nothing to install
Subagentsincluded Isolated context for exploration; only the conclusion returns Start here. Biggest single win
SkillsincludedProcedures loaded on demand rather than always residentConventions, checklists, workflows
HooksincludedIntercept tool calls β€” e.g. consult an index before a broad searchEnforcing retrieval discipline
Search & retrieval
ripgrepMITMIT Fast literal and regex search, already used under the hoodEverything. Narrow before you read
ast-grepMITMIT Structural search on the syntax tree β€” match code shapes, not stringsRefactors and pattern audits with far fewer false hits
SerenaMITMIT MCP server over language servers β€” symbol-level reads and editsLarge repos where whole-file reads are the cost
Language-server MCPsOSSVaries Go-to-definition, find-references, hover as callable toolsTyped languages with good LSP support
Zoekt / SourcebotOSSApache 2.0 Indexed cross-repo code searchMany repositories rather than one large one
Packing & context shaping
RepomixMITMIT Packs a repo into one structured, compressible file with token countsHanding a whole small repo over once
Aider repo mapApache 2.0Apache 2.0 Ranked tree-sitter symbol map sized to a context windowCheapest useful whole-repo overview β€” no database
Context7MITMIT Current library documentation on demandStopping guessed API signatures and the retries they cause
Code knowledge graphs
GraphifyApache 2.0Apache 2.0 On-device graph over 36 languages via tree-sitter, no embeddings. Ships as an MCP server, a slash command, a pre-tool hook and a memory-file directive Structural questions on a large, unfamiliar codebase
BlarifyMITMIT Repo into Neo4j or FalkorDB; you own the schemaBuilding your own graph-backed tooling
JoernApache 2.0Apache 2.0 Code Property Graph β€” AST plus control and data flowImpact analysis and security, not day-to-day coding
PotpieApache 2.0Apache 2.0 Repo knowledge graph with agents over itOnboarding to an unfamiliar codebase
On the token-reduction numbers. Figures quoted around code-graph tooling β€” "49Γ—", "71.5Γ—" β€” are community-reported on particular workloads, not independent benchmarks, and they measure structural queries where a graph is inherently favourable. Expect a real but far smaller average across a normal working session, and measure it on your own repo before quoting it to anyone.
The staleness trap. A code graph must be rebuilt after significant changes. A stale graph is worse than none, because it is confidently wrong β€” and if a pre-tool hook makes the assistant consult it before searching, wrong answers arrive first. Wire the rebuild into a post-merge git hook or CI. Never rely on remembering to re-run it.
An ontology is not the answer to this problem. An ontology encodes business meaning β€” Study, Compound, Incident. A code graph encodes structural facts β€” this function calls that one. Tools like Graphify derive the second automatically from source. Nothing about making a coding assistant cheaper on your repository requires the first. See Ontology Builder for where ontologies genuinely earn their cost.
Related: the AI Engineering Blueprint maps every conceptual requirement to platform options and covers memory selection; Implementing Guardrails shows how the safety controls are actually built.