What a token factory is and isn't

A token factory is the shape of system I build for companies that have proprietary documents worth treating as the source of truth. It is four owned layers — a context engine, a prompt compiler, a model router, and an eval harness — wrapped around one deliberately swappable component, the language model itself. The factory metaphor is not ornamental. It carries a claim: the valuable work of an AI system is not the generation step that happens inside a model. It is the upstream preparation of the right context and the downstream assessment of the result. Those are the parts a company can own, and the parts a company has to own if the output is going to hold up to a regulator, a board, or a future auditor.

That much fits on the back of a napkin. The more useful question, and the one this essay is about, is what the term rules out. Token factory is not a framework, not a product, not a managed service, and not the name of a vendor offering. It is a methodology. The difference matters, and the line is easy to blur in a market where every assistant is now "agentic" and every RAG pipeline is marketed as a platform.

The four layers, in enough detail to be useful

The context engine is the part of the system that turns a proprietary corpus into retrievable, ranked, citation-bearing material. It is where the chunking policy lives, where the embedding model is chosen, where hybrid retrieval is tuned, where the rerankers sit. In practice it is also where most of the domain-specific work happens: how to handle a technical manual that references another manual three sections away, how to weight a table against the paragraphs around it, how to surface the canonical document when two near-duplicates disagree. The context engine is the factory's intake. If it is wrong, nothing downstream recovers.

The prompt compiler is the layer that takes a task specification and produces the concrete prompt sent to the model. In a mature token factory this is not a templating engine. It is a small library of composable units — instructions, exemplars, schemas, guard clauses — that get assembled for a given task, a given tenant, a given model. The compiler version is part of the artefact set. When a prompt changes, the change is traceable, diff-reviewable, and reversible, in exactly the way a migration is traceable in a reasonable database system.

The model router is the thinnest of the four layers, and deliberately so. It is the component that sends a compiled prompt to a specific model — frontier, open-weights, self-hosted — and returns a typed result. Its only job is to make the model decision a routing decision rather than an architectural one. In practice the router also encodes cost ceilings, latency budgets, and fallback policy. It is the place where "use GPT-5 for the hard cases and a 7B self-hosted model for the easy cases" becomes a two-line policy rather than a three-month rewrite.

The eval harness is the layer most organisations skip in their first AI product and regret in their second. It is the offline and online evaluation surface that turns every change to any other layer into a measurable event. Canonical eval sets, scoring functions, regression gates, a way to replay production traffic against a candidate configuration without touching production users. The eval harness is what makes the difference between "the AI feature is working" as a vibe and "the AI feature is working" as a number the company can put in a board deck.

Around those four owned layers, the model is the one component the factory does not pretend to own. That is the point. A frontier model is a rented input — priced, supplied, versioned, and eventually deprecated by someone else. The entire economic and regulatory argument for the token factory shape depends on keeping the expensive, proprietary work outside the rented component.

What the term deliberately does not mean

Token factory is not a framework. There is no NPM package to install. I have implementations in TypeScript, Python, and earlier in a couple of JVM languages; the code looks different in each case, tuned to the team and the surrounding stack. The methodology is what travels. If someone offers you a "token factory framework" in a box, they are selling you a framework and calling it a methodology. Those are different things, and the framework almost always ends up in the way.

Token factory is not a product. Blue Abraham does not sell a SaaS that you integrate. The factory is a shape a company builds, on its own infrastructure, with the architecture the engagement documents. A delivered engagement produces: a production system, a runbook, a set of evals, and enough written architecture for another team to pick the work up. The thing a company pays for is the architecture decision record and the expert build. The thing they keep is a proprietary asset.

Token factory is not a chatbot. A chatbot is a surface on top of a factory — one of several. I have built plenty of non-chat surfaces on top of the same underlying factory: structured extraction endpoints, classification pipelines, decision-support scorers, internal APIs that return typed results to another service. The chat surface is overrepresented in public discourse because it is what users see in consumer products. It is underrepresented in high-stakes enterprise work, where the ask is almost never "a chatbot for our engineers" and almost always "a pipeline that returns a typed, citation-bearing answer my reviewer can audit."

Token factory is not a vendor layer. The model router in particular is sometimes confused with OpenRouter, the various LLM gateway products, or the "bring your own key" features bolted onto every new wrapper. Those products solve a real but narrow problem: routing an outgoing request to the cheapest acceptable model. A token factory uses that kind of routing internally, but the routing is the easy part. The hard part is the four owned layers around it — the ones a gateway cannot provide because they require domain knowledge the gateway does not have.

Why the factory framing matters

I use the word factory deliberately, and in full awareness that it sounds industrial in a field that prefers the vocabulary of magic. The reason is that a factory is the right mental model for a system whose job is to produce tokens — very specific tokens, for very specific customers, against very specific quality standards — at scale, reliably, and with enough process discipline that a defective batch can be traced and corrected.

A factory does not promise the best possible output for any given input. It promises a consistent output within stated tolerances, and it promises that when the output drifts, someone can find out why. That is exactly the promise a company needs from its AI system if the output feeds a regulated process, an external customer, or a decision that touches an individual. The generative magic of the underlying model is an ingredient. The factory is what makes the ingredient usable at industrial cadence.

The other reason the framing matters is that it restores the work of engineering to visibility. In the current market, it is common to hear that "prompt engineering" is the whole of the discipline, and that once you know how to write a good prompt the rest is plumbing. This is not true of the systems I have built. The prompt is one artefact inside the prompt compiler layer. The quality of the system is determined far more by the context engine — whether the right material reaches the model at all — and by the eval harness — whether anyone can tell if the output is good. Calling the whole thing a factory makes it harder to pretend the interesting work happens in one text box.

What ownership actually changes

The last thing worth saying about the term is that the ownership claim is not ideological. It is operational, and it shows up in places most introductions to the topic skip.

It shows up in the model migration. Companies that have built their AI on a rented wrapper discover, around the first model deprecation, that the cost of migrating is not the model change itself but the prompt, eval, and context assumptions baked into the wrapper. In a token factory those assumptions live in layers the company controls, so the migration is a scoped engineering task rather than a vendor negotiation.

It shows up in the regulatory conversation. When a regulator asks what evidence a company has that a given decision was supported by the correct source material, the token factory produces that evidence as a matter of course — the context engine emits citations, the prompt compiler logs the exact prompt, the eval harness carries the regression history. A rented-first architecture usually cannot produce those artefacts without a vendor support ticket, and the ticket almost always returns a partial answer.

It shows up in the cost curve. At low volume the token factory looks expensive relative to a wrapper: there is more to build. Somewhere past the first few hundred thousand euros in annual AI spend, the curves cross. The rent is linear in usage; the factory is roughly fixed, with the expensive parts now sitting on your balance sheet rather than the vendor's invoice. I will write the economics essay separately.

None of this is a reason to build a token factory on day one of every AI initiative. There are plenty of systems that should be a thin call to a frontier API, a light prompt, and a sensible UI. The token factory becomes the right shape when the documents are proprietary, the outputs are high-stakes, and the time horizon is long enough that owning the architecture pays for itself.

A working definition to end on

A token factory is an AI system whose value sits in four owned layers — context engine, prompt compiler, model router, eval harness — with the language model itself treated as a deliberately swappable input. It is a methodology, not a framework; a shape, not a product. It is what I build, in cases where building is the right answer. When the right answer is to rent, I say so, and the engagement ends with a shorter document and a clearer vendor choice.

The rest of the writing on this site is in service of that distinction: what the layers look like in specific domains, when to build, when not to, and where the regulatory and economic surface pushes the decision one way or the other. Reading on from here, the two aviation essays and the economics essay are the places to go next.