BP-02 · Technical deep dive
A token factory you own, end to end.
Four owned layers. One swappable model bus. The token factory is a thin, model-agnostic architecture that sits between your application and whichever frontier model wins this quarter — so swapping providers becomes a router-config change, not a rewrite.
The blueprint
Four owned layers. One swappable model bus.
A request enters at the top and traces a single path: context is assembled, the prompt is compiled, the router selects a provider, and the eval harness gates the response.
The boundary on the right is the only contract that touches a vendor. Everything left of it — context, prompts, routing policy, evals — is yours.
The four layers
Each layer is a contract, not a framework.
Every layer publishes a typed input and output. Replace the implementation; the contract — and the application above it — stays put.
01
Layer
Context engine
ingest(source) → embed → retrieve(query)
Your proprietary context graph. Owned schema, portable embeddings, no vendor-specific retrieval API. Built so the same retrieval call works whether the downstream model is Claude, GPT, or a quantised open-weight running on your own GPUs.
- In
- source documents, telemetry, structured records
- Out
- ranked context chunks + provenance
You own
- schema
- embeddings
- retrieval policy
- provenance
02
Layer
Prompt compiler
compile(intent, context) → ProviderPayload
Declarative prompt definitions compiled to provider-native payloads. One source of truth, many targets — Anthropic message blocks, OpenAI chat completions, raw chat-template tokens for self-hosted models. Versioned, diff-able, code-reviewed.
- In
- intent + retrieved context
- Out
- provider-native request payload
You own
- prompt source
- templates
- tool schemas
- version history
03
Layer
Model router
route(task, cost, SLA) → Provider
Per-task routing across providers. Policy lives in code: route long-context summarisation to one provider, structured extraction to another, latency-critical traffic to a self-hosted model. Swap providers by changing a routing rule, not the application.
- In
- compiled payload + task metadata
- Out
- provider response (raw)
You own
- routing policy
- fallback chains
- rate-limit budget
- cost ledger
04
Layer
Eval harness
gate(response, golden) → {pass, score, drift}
Your golden evals gate every model swap. Replays a frozen set of representative requests against any candidate provider, scores them on your rubric, blocks deploys that regress. Independence you can measure, not just promise.
- In
- candidate response + golden set
- Out
- pass/fail + drift report
You own
- golden set
- scoring rubric
- drift baseline
- release gate
Request lifecycle
One trip through the factory.
A trace of a single typed intent — from application call to gated response — at sub-second wall-clock. Every step is observable; every decision is loggable.
T+0ms
App emits intent
The application calls the factory with a typed intent — `summarise(contract)` or `extract(invoice)`. No model name, no provider SDK.
T+12ms
Context engine retrieves
The retrieval call hits your owned vector index plus structured stores. Returns ranked chunks with provenance, never the raw documents.
T+18ms
Prompt compiler binds
Intent + retrieved context are bound to the active prompt template, then compiled to the payload shape the chosen provider expects.
T+22ms
Router selects provider
Routing policy reads the task class, the cost ceiling, and the SLA. Picks one provider and a fallback chain. Logs the decision.
T+820ms
Provider responds
Raw provider response returns. The factory normalises tool calls, citations, and stop reasons into a provider-agnostic shape.
T+830ms
Eval harness gates
Inline evals run on the response — schema validation, refusal detection, drift checks. Fail-closed by default; fall through to the next provider in the chain on regression.
What you keep
Six dimensions of independence — six places vendor SDKs leak.
Independence is not a posture; it is a list of artifacts. If you can’t name what you own at each row, you don’t own the stack.
Dimension
Vendor SDK
Token factory
Prompt source of truth
Lives in vendor playground or notebook
Versioned in your repo, code-reviewed
Context store
Vendor-managed embeddings + retrieval API
Owned schema, portable embeddings
Routing policy
Hard-coded to one provider's model family
Per-task routing across providers
Eval set
Vendor benchmarks; no replay on swap
Golden set replayed on every candidate
Cost ledger
Monthly invoice; no per-task attribution
Per-route, per-tenant attribution
Exit cost
Re-implementation across the application
Routing-rule change + eval replay
Where regulation meets architecture
The four layers map onto what EU regulators already ask for.
DORA, NIS2, GDPR, and the EU AI Act are substantive in this work — not decoration. The token factory is not a compliance service. It is an architecture that happens to produce the artefacts a compliance conversation needs: data residency, auditability, vendor independence.
Framework
In scope
What the factory produces
DORA
ICT risk + third-party dependency, financial entities
Router-layer visibility into every provider call, per-route cost ledger, documented fallback chain — the artefacts an ICT-risk review asks for.
NIS2
Supply chain security, essential & important entities
Owned context store and owned prompts mean no hidden third-party processor inside the critical path. Retrieval and evals run on infrastructure you can account for.
GDPR
Personal data processing + cross-border transfer
Context engine enforces residency and provenance per chunk. The prompt compiler stops PII from drifting into vendor-hosted prompt history by making the payload code-reviewed, not free-text.
EU AI Act
High-risk AI systems, transparency, post-market monitoring
The eval harness is the post-market monitoring loop: golden replays, drift baselines, release gates — the evidence a conformity assessment is looking for.
Blue Abraham does not sell a compliance product. It builds architecture that makes the conversation with your compliance and legal stakeholders shorter.
Next step
Start with an architecture review.
A bounded engagement. I walk your current stack against the four-layer reference, name what you already own, and write up where the architecture is sound and where it is at risk. One to three weeks, fixed price, delivered in writing.
Get in touch