THE IDEA A concern is cross-cutting when it touches every call but isn't the logic
Logging, tracing, metrics, cost, redaction, caching, rate limits, audit β none of these are "call the model," yet
every request needs them. If you scatter them through the core you get an unmaintainable, untestable mess. The fix
is the same principle you liked: code to interface. The core depends only on a port (e.g. LogSink);
where it physically logs is a swappable adapter (stdout / CloudWatch / Azure Monitor). That's
ports-and-adapters (hexagonal) architecture + dependency injection, applied to
GenericAnthropicMessageClient.
THE SHAPE core depends on ports; adapters plug in
pure logic Β· depends only on PORTS"]:::core CORE --> L["port: LogSink"]:::p --> LA["stdout Β· JSON file Β· CloudWatch Β· Azure Monitor Β· Loki"] CORE --> T["port: Tracer"]:::p --> TA["Phoenix Β· Langfuse Β· MLflow Β· App Insights Β· X-Ray"] CORE --> M["port: MetricSink"]:::p --> MA["Prometheus Β· StatsD Β· CloudWatch Β· Cloud Monitoring"] CORE --> B["port: Budget"]:::p --> BA["in-mem counter Β· Redis Β· Delta ledger"] CORE --> S["port: SecretProvider"]:::p --> SA["dotenv Β· Key Vault Β· Secrets Manager Β· secret scopes"] CORE --> AU["port: AuditSink"]:::p --> AUA["JSONL Β· Log Analytics Β· CloudTrail Β· Delta"] classDef core fill:#e7ecff,stroke:#3b5bdb,color:#1e3a8a; classDef p fill:#f3f0ff,stroke:#764ba2,color:#4a3b7a;
The core never names a concrete sink. Point it at a different adapter and behaviour changes with zero core edits β the Dependency-Inversion Principle in practice.
THE FULL SET every cross-cutting concern, as a port
| Concern | What it does on every call | Port interface | Pattern |
|---|---|---|---|
| Observability | |||
| π Logging | Structured record of request/response/outcome | LogSink.emit() | Port/Adapter |
| π Tracing | A span per request & per loop step (parent/child) | Tracer.span() | Port + Observer |
| π Metrics | tokens, cost, latency, retries, cache-hit, success | MetricSink.incr/observe() | Port/Adapter |
| π§Ύ Audit (GxP) | Immutable, provable record (who/what/when + versions) | AuditSink.record() | Port/Adapter |
| Cost & governance | |||
| π° Budget / cost accounting | Estimate + enforce per-request/session/tenant spend | Budget.check/record() | Strategy + Port |
| π¦ Rate limiting / concurrency | Client-side throttle so you don't self-inflict 429s | RateLimiter.acquire() | Port/Adapter |
| ποΈ Feature flags / kill switch | Enable/disable a model/tool/agent instantly | FlagSource.enabled() | Port/Adapter |
| π·οΈ Prompt / model versioning | Pin & log the exact prompt+model for reproducibility | PromptRegistry.get() | Port/Adapter |
| Security & safety | |||
| π Secrets | Fetch keys/tokens from a vault, never hard-code | SecretProvider.get() | Port/Adapter |
| π©Ή Redaction (PII/secrets) | Scrub sensitive data before log/trace/model | Redactor.scrub() | Port + Decorator |
| π‘οΈ Guardrails | Input/output moderation & injection filtering | GuardrailPolicy.check_in/out() | Strategy + Port |
| Resilience & performance | |||
| β»οΈ Retry / circuit-breaker / fallback | Survive transient errors; degrade gracefully | RetryPolicy, Fallback | Strategy + Chain of Resp. |
| π Idempotency | Keys so a retry can't double-execute a side effect | IdempotencyStore | Port/Adapter |
| ποΈ Caching | Client-side response/semantic cache (beyond prompt caching) | CacheStore.get/put() | Port + Decorator |
| Configuration | |||
| βοΈ Config source | Timeouts, model, budgets, TTLs β from env/file/vault/dynamic | ConfigSource.get() | Port/Adapter |
| π Backend / identity | Which endpoint + how to auth (key / OAuth / cloud IAM) | Backend (adapter) | Adapter |
THE PORTS interfaces the core depends on (Python Protocols)
from typing import Protocol, Any class LogSink(Protocol): def emit(self, record: dict) -> None: ... # adapter decides WHERE (stdout/CloudWatch/...) class Tracer(Protocol): def span(self, name: str, **attrs): ... # context manager; nests per loop step class MetricSink(Protocol): def incr(self, name: str, value: float = 1, **tags): ... def observe(self, name: str, value: float, **tags): ... class Budget(Protocol): def check(self, est_tokens: int) -> bool: ... # deny before the call if over def record(self, usage) -> None: ... # debit after class Redactor(Protocol): def scrub(self, obj: Any) -> Any: ... # PII/secrets out BEFORE anything logs it class SecretProvider(Protocol): def get(self, name: str) -> str: ... class CacheStore(Protocol): def get(self, key: str): ... def put(self, key: str, value, ttl: int) -> None: ... class AuditSink(Protocol): def record(self, event: dict) -> None: ... # append-only, immutable class RateLimiter(Protocol): def acquire(self, cost: int) -> None: ... # block/throttle client-side class GuardrailPolicy(Protocol): def check_input(self, req) -> None: ... def check_output(self, resp) -> None: ...
HOW THEY WEAVE IN the middleware order matters
Cross-cutting behaviour applies as middleware around the core call (Decorator/Chain-of-Responsibility). Order is a design decision, not an accident:
THE COMPOSITION ROOT wire concrete adapters in ONE place
# composition_root.py β the ONLY file that names concrete adapters (dependency injection) client = MessageClient( backend = DatabricksBackend(host=cfg.host, token=secrets.get("DBX_TOKEN")), # Adapter ports = Ports( log = CloudWatchLog() if prod else StdoutLog(), # swap WHERE we log, 1 line tracer = LangfuseTracer() if prod else PhoenixTracer(), metric = PrometheusMetrics(), budget = RedisBudget(cap_usd=cfg.daily_cap), redact = PIIRedactor(), cache = RedisCache() if prod else InMemoryCache(), audit = DeltaAudit() if prod else JsonlAudit(), limiter= TokenBucket(rpm=cfg.rpm), guard = CompositeGuardrail([InjectionFilter(), Moderation()]), ), ) # Tests: pass FakeLog(), FakeBudget(), InMemoryCache() β no network, deterministic.
PHYSICAL ADAPTERS the same port, per environment
| Port | π» Local | π· Azure | π§ AWS | π΄ GCP | π§± Databricks |
|---|---|---|---|---|---|
| LogSink | stdout / JSON file | Azure Monitor / Log Analytics | CloudWatch Logs | Cloud Logging | driver logs / system tables |
| Tracer | Phoenix / Langfuse | App Insights | X-Ray + CloudWatch | Cloud Trace | MLflow Tracing |
| MetricSink | Prometheus + Grafana | Azure Monitor | CloudWatch | Cloud Monitoring | MLflow / Lakehouse |
| SecretProvider | dotenv | Key Vault | Secrets Manager | Secret Manager | Databricks secret scopes |
| CacheStore | in-memory / local Redis | Azure Cache for Redis | ElastiCache | Memorystore | Delta / Redis |
| AuditSink | JSONL file | Log Analytics | CloudTrail + S3 | Cloud Audit Logs | Delta + system.access.audit |
| ConfigSource | env / file | App Configuration | AppConfig / SSM | Runtime Config | job params / widgets |
CONCEPTS β CCA MAP
| The cross-cutting port⦠| CCA domain |
|---|---|
| Logging, tracing, metrics, request-id | D5 Β· observability |
| Budget, rate limiting, caching, prompt/model versioning | D5 Β· cost |
| Secrets, redaction, guardrails (injection/moderation) | D5 Β· guardrails Β· D4 |
| Retry / circuit-breaker / fallback / idempotency | D5 Β· reliability |
| Audit trail (GxP), kill switch | D5 Β· governance |
LEVEL IT UP
- Build two adapters per port β e.g.
StdoutLog+JsonlAuditlocally, then a cloud one; prove the swap. - Order the middleware deliberately β put
Redactfirst and verify no PII reaches the log sink. - Inject fakes in tests β assert the client logs/meters/audits correctly with zero network.
- Add a kill switch β a
FlagSourcethat disables a model/tool instantly (a guardrail in practice). - Fold into GenericMessageClient β these ports are its constructor dependencies.