Why the Model Should Be the Smallest Part of Your Stack.
The dominant narrative around agentic AI in 2026 is that the models are not yet good enough. The standard reading is wrong. The technology has not been over-sold; it has been under-engineered. The models are not the problem. The architectures wrapped around them are.
Thesis
The dominant narrative around agentic AI in 2026 is that the models are not yet good enough. The MIT NANDA report, published in July 2025, found that 95% of enterprise generative AI pilots delivered no measurable impact on profit and loss, despite $30–40 billion of cumulative spend. Forrester’s analysis later that year predicted that half of all AI-attributed layoffs would be quietly reversed within a year. The obvious reading is that the technology has been over-sold. I read those numbers differently. The technology has been under-engineered. The companies failing at scale are not failing because the models cannot do the work; they are failing because of the architectures wrapped around the models, and a newer model will not fix that on its own.
This paper makes a single load-bearing claim: agentic AI in production succeeds or fails on the strength of the infrastructure surrounding the model, not the capability of the model itself. Designed properly, that infrastructure does four things. It gives the agent a curated local data layer to read from, rather than the system of record. It bounds the action surface with well-typed tools. Output is validated deterministically before anything is persisted. Write-back to source systems goes through a separate code path that requires explicit human approval. When those four conditions hold, the model becomes the smallest and most replaceable component in the stack, and the conclusions of Belcak et al. (2025) at NVIDIA Research follow naturally: small, fine-tuned language models are sufficient for the great majority of agentic invocations, and large frontier models stop being a technical necessity and become an expensive habit.
None of this came from theory. Between September 2025 and April 2026 I built and shipped a production AI platform, alone, that synchronises with an enterprise field-service management system and produces scheduling recommendations for engineers across thousands of jobs per month. The development budget was fixed at six hundred pounds per month, which bought roughly two days of paid time per week for the majority of the build, rising to five later in the project with no corresponding budget increase. No team, no evaluation harness, no frontier-model wrapper waiting to solve the problem. I iterated against live production data, kept what survived load, and folded each failure back in as design input for the next cycle. The shorthand for that was Build-Test-Measure. The architecture that emerged was forced by constraint into the only shape that could survive at enterprise scale: a tightly bounded agent loop running against deterministic tools, reading from a synchronised local store, gated by human approval, audited end to end. What remains of the lesson is the version history of the scheduling agent: four iterations of the core decision logic, a complete data-stack rebuild, and a documented migration away from an early agent-framework dependency.
What that build taught me, and what this paper sets out to formalise, is that the failure mode named by MIT — “brittle workflows, weak contextual learning, misalignment with day-to-day operations” — is not a description of AI itself. It is what happens when AI is bolted onto systems that were never designed to host it. The companies whose layoffs are now being reversed wrapped a frontier model around an enterprise stack that could not give the model what it needed: clean data, bounded tools, deterministic validation, and a path back to the system of record that did not depend on the model’s discretion. The 5% that MIT identifies as having succeeded are domain-specific, workflow-integrated, vendor-led; independently, they have arrived at the same architecture I had to build under constraint. That convergence is the strongest evidence I can offer that the architecture is not idiosyncratic. Different teams, different industries, different paths in, and the same shape keeps emerging.
The remainder of this paper formalises that architecture. Section 2 describes the bolted-on failure mode in detail, using the early phases of my own build as evidence. Section 3 names the five architectural primitives that, taken together, constitute infrastructure-first AI. Section 4 connects those primitives to the small-language-model thesis advanced by Belcak et al., explaining why the architecture is the precondition for that shift, not its consequence. The strategic case follows in Section 5: why infrastructure independence matters at a moment when enterprise AI is failing in public and provider concentration risk is rising. Section 6 presents the case study, redacted for client confidentiality. In Section 7 I propose the remaining work, a controlled experiment in which a fine-tuned small model replaces the current frontier model on this exact production task, and set out what would constitute its falsification. Section 8 closes with what this thesis means for the next generation of AI startups.
The Bolted-On Failure Mode
The bolted-on failure mode has a recognisable shape. A frontier model is wrapped around a stack that was not designed to host it, and the model is asked to compensate, inside its own context window, for everything the surrounding infrastructure does not provide: clean data, bounded actions, deterministic validation, controlled write-back. For a while, on small examples, it appears to work. At production scale, on a real budget, it does not.
I learned this shape twice in close succession.
The first lesson came from version three of the scheduling agent. The architecture, at that point, was conventional. Assemble the entire decision surface inside an edge function, serialise it into a single user prompt, and hand the whole thing to the model. The prompt carried the job, the engineers, the matching candidates, the schedules, the travel times, the cost-centre rules, and the constraints. Each call cost between ten and twenty-five pence. That number sounds modest until it is placed against the budget the build was actually running under. The six hundred pounds per month I had to work with was not a model budget; it was the entire envelope for the platform. It covered the AI-assisted development tools I was writing the code in (Lovable consumed up to three hundred pounds of it on a heavy month), the production API tokens, every external service the platform depended on, and the inference cost of the agents themselves. Inside that envelope, ten to twenty-five pence per scheduling call was not a number I could plan around. It was a number that would close the project.
The architecture also dictated which models could be used. I was developing, by necessity, on the cheapest viable inference. GPT-4.1-mini and nano carried most of the build, and I migrated to Mistral in late January and early February for the same reason. The v3 prompt-stuffing approach did not work on any of them. The payload was too large; the smaller models lost coherence inside it. Only the larger frontier models could carry the context, and those models cost an order of magnitude more per call than I could afford at production volume. The architecture was demanding a model class the budget could not support. That is the failure mode in its purest form: a design decision in the prompt layer with no available exit at the cost layer.
The second lesson came from the agent framework I was using to orchestrate the early agents. Over a single weekend, the agent began hallucinating tool calls. It would claim, inside its reasoning, that it had checked an engineer’s schedule and report a confident answer. It had not checked anything. The schedule it described did not exist. There was no single runaway loop to point at; the framework imposed a coded iteration cap of ten to twenty steps per execution, and the cap held. The damage came from below the cap. Each execution ran its full iteration budget reasoning over fabricated state, produced an unusable answer, and exited. The next execution did the same. Over a weekend, the cumulative cost of many failed executions reached approximately sixty pounds of API tokens, roughly ten per cent of the monthly budget, all of it spent inside the iteration limits that were supposed to be the safety mechanism.
The cause, once I worked it out, was simple and architectural. The framework’s memory layer stored input and output messages but did not store the tool messages between them. The model could see that it had asked a question and that it had eventually produced an answer, but the intermediate evidence on which the answer depended had been silently dropped from the history. The model filled the gap with a plausible reconstruction. This was not a model failure. The model was doing what any language model does when it is shown a partial conversation. It was a memory-layer failure presenting as a hallucination, and it is now documented as GitHub Issue #14361 in the framework’s own tracker.
Those two episodes name three failure modes that recur wherever AI is bolted on rather than built in.
The first is context poisoning. When the prompt is asked to carry every piece of information the model might need, signal and noise arrive together, and the model has no principled way to separate them. Capability does not compound with context size beyond a certain point; it degrades. The fix is not a longer context window. The fix is to take the logic out of the prompt and put it in deterministic code, leaving the model only the decision the model is actually qualified to make.
The second is tool-call drift. When the conversation history visible to the model is not a faithful record of what the model has actually done, the model invents the missing steps. Iteration caps will stop a single execution from running forever; they will not stop the underlying behaviour from repeating across executions, and they will not stop the bill arriving on Monday morning. The fix is not a tighter cap. The fix is an orchestrator that owns the conversation, persists every tool exchange, and never lets the model see a history it cannot trust.
The third is cost-context coupling. When the model is the load-bearing component of the system, the system inherits the model’s pricing curve. The only path to cheaper operation is a smaller model, and a smaller model cannot carry the context, so the path is closed. On a fixed budget that closure is not an inconvenience; it is the end of the project. The fix is to make the model the smallest, not the largest, component in the stack: bound its action surface, hand it less context, and let deterministic tools and validation absorb the work that prompt-stuffing was trying to do.
Three named failure modes; one common cause. The model was asked to be the infrastructure. Section 3 names the five primitives that, taken together, give the model the infrastructure it needs and let it return to being the model.
The Five Primitives of Infrastructure-First AI
The bolted-on failure mode emerges from a single architectural error: giving the model responsibility for decisions it should not have to make and denying it access to infrastructure it needs to make the decisions it should. The fix is not a better model. The fix is better infrastructure.
Five architectural primitives, taken together, constitute that infrastructure. Figure 1 shows them in composition. None of them are individually novel; the contribution of this section is naming the set, defending each against the failure modes Section 2 described, and showing why they only do their work as a set.
Air gap as a hard rule. The agent has zero direct connection to the system of record. It reads only from a synchronised local store. This is not a guideline to be relaxed when schedule pressure builds. It is a permanent design boundary. The dashed line across Figure 1 names the air gap; nothing crosses it except the synchronisation layer on the read side and the write-back function on the write side, and the agent itself is on neither path. The air gap solves two problems at once. First, it protects the production system: if the agent hallucinates, the hallucination stays inside the sandbox. No corrupted record lands in the operational system; no cascading failures propagate outward. Second, it protects the agent: the model receives clean, curated data rather than the system of record’s full complexity, noise, and edge cases. The agent-framework memory bug described in Section 2 cost me sixty pounds in API tokens over a weekend; without the air gap, the same failure mode running against the live operational system would have written hallucinated schedules directly into the field, and the cost would have been measured in operational damage rather than tokens. The air gap is what made it safe to develop on live production data at all. It is now a formalised design principle of the build, not a convention I observe when convenient. When the agent receives only what it needs and writes nowhere it should not, the context poisoning failure mode becomes impossible by design.
Curated local store as the agent’s only data surface. Between the system of record and the agent sits a deliberately AI-shaped consumption layer. Not a generic cache of every field the operational system contains. Not raw vector embeddings of raw documents. A constructed knowledge base, built across the seven months of the project: relational tables for structured facts, vector embeddings for semantic search over engineer capabilities and job descriptions, and a graph layer for relationship traversal between engineers, sites, customers, and jobs. The three sub-boxes inside the local store in Figure 1 name each of these by function. This is not RAG-as-an-afterthought. This is RAG-as-the-substrate. The model does not decide what data is relevant; the data layer decides, ahead of time, what the model is allowed to see. That decision is formalised in a schema, refined across each iteration of the agent, and enforced by the database itself. The outcome is a model that receives smaller context windows, cleaner signal, and fewer degrees of freedom to go wrong.
Action-bounded by tool surface. The agent can only do what its tools permit. The tools are defined in advance; their signatures are strict; their return types are known. The agent cannot invent new tools, cannot call tools with arbitrary arguments, cannot step outside its action space. The production scheduler exposes seven tools to the model and nothing else. Even the “submit final answer” function is itself a tool, intercepted by the orchestrator in application code before it can reach the persistence layer. The agent loop sits nested inside the tool surface in Figure 1, with the surface itself owning every action the agent can take. The agent literally cannot escape. This is what Anthropic’s Building Effective Agents (2024) describes as the hybrid pattern: agent discretion inside a workflow, with the workflow owning the action surface. The boundary eliminates the cost-context coupling failure mode by making the model’s computational budget independent of its action budget. A smaller model can drive the same tool surface as a larger one because the tools, not the model, enforce correctness.
Server-side validation of agent output before persistence. The agent’s proposal is treated as a hypothesis, not a result. Before any recommendation lands in the database, deterministic code in the orchestrator validates it: checking each proposed time block against the engineer’s existing schedule, enforcing fifteen-minute boundaries on all start and end times, requiring the travel block to end exactly where the work block begins, capping the recommendation at five engineers, normalising field names regardless of which naming convention the model has chosen this run, and silently dropping any engineer whose blocks fail the rules. The tool layer does not trust the model; the persistence layer does not trust the tool layer. Over ninety per cent of jobs in production arrive with incomplete cost-centre metadata, which is enough to break a less defensive architecture, but the validation layer absorbs the noise before it becomes a fault. This layering is what stops tool-call drift from becoming an operational disaster. A hallucinated tool call still produces an invalid result, and the validation layer catches it. The model does not pay for its mistakes; the system does, silently, by discarding them.
Human-gated write-back through a separate code path. Even after the validation layer passes a recommendation, even after a human has reviewed it and chosen to approve it, the agent does not act. A deterministic system function performs the write to the system of record. Figure 1 makes this separation visible: the entire right-hand column, from the validation block up through the approval diamond to the write-back function, is code the agent has no part in, running on its own runtime, sharing no state with the agent loop on the left. The read path (agent queries local store, calls tools, produces recommendation) and the write path (human approves, system executes the approved action) are separate code entirely, in separate edge functions, sharing no common runtime. This separation is the final firewall. Approval is not a token the model sees; it is a gate the model cannot cross. The human’s decision becomes a parameter to the write function, not a prompt to the model. The result is accountability without ambiguity: when something goes wrong, it is not because the model misinterpreted the human’s intent. It is because the system executed exactly what was asked.
These five primitives are not independent. Figure 1 shows the dependency: the air gap enables the curated store by separating it from the source of truth; the curated store enables action-bounding by giving the tool surface something safe to read from; action-bounding enables server-side validation by making the model’s output legible to deterministic code; and server-side validation enables human-gated write-back by making the proposal trustworthy enough to present to a human in the first place. Taken as a set, they make the model the smallest component in the stack, the most replaceable piece, and the least load-bearing part of the system.
Why This Architecture Enables Small Language Models
The five primitives prevent the failure modes named in Section 2. They do something else as well. They change the shape of the task the model is being asked to perform, and once the shape of the task changes, the class of model that suffices for it changes too.
When the tool surface bounds the action space, the model does not have to reason about whether an action is safe; the tool layer’s own code enforces that. When the curated local store filters operational data down to what the agent actually needs, the model does not have to separate signal from noise; the data layer has already done it. Server-side validation turns the model’s output into a hypothesis the system can check, normalise, or quietly discard. The model is not less important under this arrangement. Its job is different. The work that used to live inside the model’s reasoning now lives in code I can read, test, and fix.
What the model is left with is narrow. It is not building a decision from first principles inside a large context window. It is classifying a well-shaped problem and proposing a bounded answer. That is a different task. And once the task is different, the model that can do it is different too.
I went looking for evidence of this when I started writing the paper. I needed to know whether the substitution my architecture was implying was something other people had already demonstrated, or whether I was reasoning from one production system to a claim the literature would not support. The answer was the first. Jhandi et al. (AWS, 2025) fine-tuned a 350-million-parameter model on tool-calling tasks and achieved a 77.55 per cent pass rate on ToolBench, the standard public benchmark for agent tool calling. On the same benchmark, ChatGPT-CoT, a much larger general-purpose model, scored 26 per cent. ToolLLaMA-DFS, a specialised agent model, scored 30.18 per cent. A sub-one-billion-parameter model, fine-tuned for a bounded task inside bounded tooling, outperformed much larger general models by roughly three to one. The smaller model is not more capable in the abstract. The task, once it has been bounded, no longer demands general capability. It demands specificity.
Belcak et al. (2025) at NVIDIA Research go further. Their position paper defines small language models as sub-ten-billion-parameter models and argues they are “sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems.” Section six of their paper sets out a conversion algorithm for migrating LLM-based agentic applications to SLM-based ones. The algorithm does not promise any model can do any task. It formalises a process: breaking the decision surface into smaller pieces, introducing deterministic validation at each step, reducing the model’s context and responsibility until the model is doing only the pattern-matching and bounded classification it is genuinely capable of. When I read it, I recognised what I had been forced to build.
The five primitives, taken together, are a worked instance of that conversion algorithm. The architecture I built under budget pressure, without the literature in front of me, turned out to be a specific realisation of the design Belcak and his co-authors describe in the abstract.
What this makes available is a configuration in which a fine-tuned small model becomes viable. The model has not grown more capable. The task has shrunk to fit the capability the model already has. The production scheduling agent described in Section 6 runs on Mistral Large today, not on a fine-tuned seven-billion-parameter model. The reason is straightforward. I did not have the budget to run the substitution experiment during the build. The reason that is recoverable rather than fatal is that the architecture treats the model as a configuration choice. The data layer is mine. The tool layer is mine. The validation is mine. The write path is mine. The model is a parameter I can change. Section 7 takes up that substitution and explains why the operators with the most at stake will be the first to commit to it.
The Strategic Case for Infrastructure Independence
5.1 Data Sovereignty and the Compliance Ceiling
The question of who owns data, and who can see it, sits at the centre of every enterprise’s decision to adopt AI. GDPR, the EU AI Act, sector-specific regulations. These are not obstacles to AI adoption. They are the boundary conditions within which AI systems must operate.
A frontier model provider offers a data processing agreement, claims GDPR readiness, promises residency options in EU zones. The promise is real but structural. It solves the immediate compliance concern: your data is processed by someone who has committed to following the rules. What it does not solve is the fundamental question of leverage. When your AI system depends on a frontier model accessed through a provider’s API, four distinct compliance corners collapse into one: data sovereignty, training data usage, breach liability, and audit and forensics. All four become the provider’s responsibility. All four become your exposure. When the provider’s capacity fills, you cannot access the model. When the provider deprecates a model, you must rebuild around the replacement. When geopolitical events disrupt the provider’s infrastructure, your systems go offline with it. The provider’s terms can shift. Your forensic access to what actually happened to your data inside their systems is limited by their willingness to share logs. You are compliant, on paper, because they say so. You are compliant in practice because you have structured your dependency that way.
The architecture in Section 3 inverts that relationship. When the model is the smallest and most replaceable component in the stack, the data layer becomes yours to design. The curated local store is your data. The sync layer is your responsibility. The validation before persistence is your code. The write-back gate is your call. The five primitives do not require you to choose between compliance and capability. They require you to choose compliance first, and then decide which model to plug in.
That choice is the difference between compliance-as-assurance and compliance-as-design. A frontier model with a DPA is assurance: you are relying on someone else’s commitment to follow the rules. An infrastructure-first architecture is design: you are building the rules into the stack from the first decision. You control whether data leaves your infrastructure at all. You control whether training happens. You control the audit trail end to end. The model becomes a choice you can make and remake without rebuilding the compliance structure around it.
For organisations operating under GDPR, under sector-specific regulations, under the EU AI Act, this is not theoretical. It is structural reality. The regulations assume data residency, data ownership, and audit trails. They assume the ability to delete data on request, to prove what happened, to demonstrate safeguards. A provider-dependent architecture leaves you explaining constraints that the regulation did not anticipate. An infrastructure-first architecture leaves you compliant by design. The model layer can change. The compliance layer is yours, and it stays.
5.2 The Hidden Cost of Bolted-On AI
The MIT NANDA report identifies 95% of enterprise generative AI pilots as delivering no measurable profit and loss impact. The standard explanation is that the models are not yet mature enough. The more honest explanation is structural. The companies failing at scale are not failing because their models lack capability. They are failing because they bolted AI onto systems that were never designed to host it, and when the model cannot deliver the promised return on investment, the human workers who were laid off to fund the experiment are still gone.
Klarna, the Swedish fintech, laid off 700 customer service workers in 2023 with the explicit intention to replace them with AI agents. By early 2025, CEO Sebastian Siemiatkowski had concluded that “the use of AI agents without human support is no longer the right fit.” IBM replaced several hundred HR roles with AskHR, its internal AI system; CEO Arvind Krishna later told the Wall Street Journal the headcount impact was a fraction of the figure originally circulated. Salesforce cut 4,000 customer support roles, with CEO Marc Benioff explicitly framing the move as AI substitution. Amazon cut 14,000 corporate roles, then announced 14,000 more, describing both rounds as investments in AI. Microsoft replaced an entire team at Candy Crush with the AI tools they had built. In October 2025, Forrester found that 55 percent of employers surveyed regretted laying off staff because of AI. The regret is not philosophical. It is operational. The models could not do what the organisation expected them to do. The failure is not the model’s fault. The failure is the architecture’s.
When a frontier model is asked to operate inside infrastructure that was not designed for it, the model inherits every constraint the infrastructure imposes. It must reason inside a bloated prompt. It must work with data it cannot trust. It must make decisions without deterministic validation. It must write back to systems it cannot safely touch. The only way to make the model work under those constraints is to make the model larger, more capable, more expensive. The path to cost control is a smaller model. The path to a smaller model is better infrastructure. But the infrastructure is already built, locked into an enterprise stack, and expensive to change. The only path that remains is the expensive one. That is cost-context coupling: the model is the load-bearing component, and the system inherits the model’s pricing curve.
Provider dependence amplifies the risk. When your AI system depends on a frontier model accessed through an API, your costs depend on the provider’s pricing. Inference prices have fallen since 2023, but capability drift, deprecation cycles, geopolitical exposure, and concentration risk remain. A provider can deprecate a model. A provider can shift their pricing without notice. A provider can face regulatory pressure, acquisition, or capacity constraints that affect your access. A provider can be subject to geopolitical events outside your control. On 1 March 2026, Iranian drone strikes hit three AWS data centres in the UAE and Bahrain, taking two of three availability zones offline in AWS ME-CENTRAL-1 and one in ME-SOUTH-1. Anthropic, which routes a meaningful share of Middle East and South Asia inference through those regions, experienced degraded availability and elevated error rates for Claude in the affected geographies for several days, even as the rest of the world saw the service stay up. Companies that had concentrated their AI operations in those regions, on a single provider, on a single model, absorbed an outage they had no fallback for.
The companies regretting their AI layoffs are not regretting the decision to use AI. They are regretting the decision to make AI their only path to the outcome they wanted, and to lock themselves into a provider relationship in order to make that path work. The architecture in Section 3 makes the model the smallest, not the largest, component in the stack. Swap one model for another. The infrastructure does not move. The data layer does not move. The validation does not move. The compliance posture does not move. The model becomes a choice you can make and remake without rebuilding everything around it. That is not a theoretical advantage when your workforce decisions depend on it. That is survival.
Case Study: A Production Scheduling Agent
I went into this build with one principle settled in advance: a language model evaluated against a fixed test set learns the answers, not the reasoning behind them. The empirical literature on this is unambiguous. Recent work at ICLR 2025 on memorisation and generalisation in language models demonstrates that performance on knowledge-intensive tasks correlates strongly with the frequency of relevant patterns in the training distribution, which is to say the model is recognising what it has seen rather than reasoning about what it has not. Parallel research on data composition shows that training on real data continues to improve performance, while training on synthetic or model-generated data plateaus quickly or degrades. The practical implication for a production system is direct: if I built a synthetic scenario suite and iterated the agent against it, the agent would learn to pass my scenarios. It would not learn to handle the real ones. Every edge case my suite failed to imagine would be an edge case the agent had never seen.
That principle determined the development methodology from day one. Every iteration of the scheduling agent, from the first prototype to the most recent stabilisation, ran against live production data. Engineer availability changed daily. Schedule blocks moved hour to hour. Jobs arrived with the full operational mess: missing cost-centre assignments, incomplete geocoding, conflicting constraints. The agent had to react to whatever the day brought, not to whatever I had imagined the night before. This was not a luxury. It was the only way to know whether the architecture was actually working.
The first weeks of the build were not about AI at all. They were about understanding the operational system the AI would eventually have to serve. I sat with users of the existing dashboard and watched what they did with it. I logged into the operational system and explored the data. The early value came from places I had not expected. A combination of geocoded site coordinates, engineer mobile-log GPS pings, and timesheet records turned out to be enough to reconstruct a rough picture of who had actually been where, when, against what they had claimed in their hours. It was not perfectly accurate. It did not need to be. It demonstrated, to the client and to me, that the operational data contained more signal than the existing system was extracting. That was the first quick win, and it bought me the credibility to keep building.
The first agent I built was a job-drafting system. It read inbound emails, identified the fields a job record needed, and produced a draft for human approval. Ten emails in, ten drafts out, iterate. It worked, on a small sample. I did not ship it. The reason I did not ship it was that the work had taught me the underlying database structure was wrong. The schema had been designed for what I had thought the system would need at the start, not for what the parallel work on the scheduling agent was now telling me it actually needed. I stopped, rebuilt the foundations, and started again.
Scheduling version one had read directly from the live operational system. It was the version-one mistake everyone makes. The agent had no air gap, no curated data layer, and no protection against the noise of operational metadata bundled in with the facts it actually needed. Version two introduced the local synchronised store and pulled entire rows from it. The air gap began to take shape, but the rows still carried every operational field, useful or not. The data synchronisation layer itself was developed as a separate workstream, running at intervals between ten minutes and four hours depending on the volatility of the source table. Schedules synced more often. Static reference data synced less. By the time the scheduling agent reached maturity, the sync layer was operating reliably enough that the agent could trust its local view of the world.
Version three was where the curated local store became deliberate. Instead of pulling entire rows, I wrote bespoke queries that returned only the columns the agent needed, structured for reasoning rather than for transactional display. The same iteration introduced the vector database. The earlier versions had been matching jobs to engineers by cost centre alone, which assumed the cost-centre assignment captured what an engineer could actually do. It did not. An engineer assigned to one cost centre might have skills that crossed several. A job description might call for capabilities the engineer record never explicitly listed. Keyword matching produced false negatives constantly. The fix was semantic: embed engineer capabilities and job descriptions in the same vector space, and let the agent retrieve by meaning rather than by exact string match. This work landed at the end of November and into early December. The curated local store now had two distinct layers, relational tables for structured facts and vector embeddings for semantic retrieval, with a graph layer added later for relationship traversal.
Version three improved the agent’s reasoning significantly, but it never reached real testing in production. The architecture still asked Claude Sonnet to carry the decision logic inside the prompt, which meant each recommendation took roughly three minutes to produce. In a field-service operation, three minutes is not waiting time, it is abandonment time. Users would initiate a recommendation, watch the loading indicator, and revert to manual scheduling before the agent finished. The system existed but did not enter their workflow. There was no usage data to learn from because there was no usage. The architecture had to change before testing was even possible.
By January the client had been watching me build for three months with nothing running in their day-to-day operations. The pressure to ship something was significant. I shipped the job-drafting agent to UAT on the agent framework I had been using for orchestration. It was the fast deliverable. The client had something to test. I continued working on the scheduling agent in parallel, with the job-drafting agent running against live email and surfacing edge cases I had not anticipated.
February was when the architecture failed in a way I could not ignore. I was testing the scheduling agent against production data. On easy scenarios, where availability was generous and time windows were wide, it produced recommendations. On harder scenarios, where engineers were constrained or no one was in the right area, it began to hallucinate. It would claim to have checked an engineer’s schedule and report a confident answer. The schedule had not been checked. The agent had reached the framework’s iteration cap of twenty steps having reasoned over fabricated state and produced an unusable result. Over a single weekend, the cumulative cost of these failures reached approximately sixty pounds, roughly ten per cent of the monthly development budget. The root cause was architectural. The agent framework’s memory layer stored input and output messages but not the tool messages between them. The model could see what it had asked and what it had answered, but not the intermediate evidence the answer depended on. It filled the gap with plausible fabrication. This is documented as GitHub Issue #14361 in the framework’s tracker.
Version four was the answer. I migrated the orchestration off the framework and into native edge functions running inside the database itself, with custom deterministic tooling sitting alongside the native agent. Memory became something I controlled in application code, not something a framework managed. The deterministic tooling absorbed the decision drag that had previously sat inside the prompt: gathering context, validating output, normalising values, applying hard constraints, and persisting results. The model’s job narrowed to bounded classification: given this job and this candidate engineer, is this a feasible recommendation, and if so why? With the decision logic running in deterministic code rather than inside the model’s reasoning loop, the inference workload dropped substantially. Mistral Large and Mistral Medium could both run the workflow cost-effectively, producing recommendations in roughly thirty to sixty seconds depending on the complexity of the scheduling scenario. That change in latency was the change in adoption. Users who had previously abandoned the system before it could finish now waited the half-minute and used the result.
The most important property of version four was not the speed. It was the failure behaviour. Where the earlier architecture would hallucinate its way to the iteration cap and produce a confident-sounding wrong answer, version four would return “no engineers available” when no engineers were available. The deterministic validation layer caught invalid output silently. The bounded tool surface meant the agent could not invent actions outside its action space. The twenty-iteration cap remained, but it was now a final backstop, not the primary safety mechanism. The primary safety mechanism was the surrounding architecture. The system became operationally trustworthy, in the specific sense that a wrong answer would now look like a refusal rather than a confident fabrication.
All of this was only safe because of the air gap. Testing an agent against live production data is not a methodology to attempt without isolation. Every hallucination during development happened inside the sandbox. The system of record was never touched by the agent. The synchronisation layer kept the local store fresh. The validation layer discarded invalid output before persistence. The human approval gate stood between any recommendation and any actual schedule write-back. The agent could fail, and it did fail, hundreds of times during development, without a single bad write reaching the production system. That property is what made it possible to develop on live data at all. The architecture was not just the goal of the build. It was the precondition for the development methodology that produced it.
The scheduling agent in its current state is the outcome of this version history. Hundreds of working recommendations in production. Users actually using the system in daily workflows, though formal training has not yet been delivered. Dozens of edge cases captured and handled because they arrived from the wild rather than from a scenario I had to imagine. The architecture did not emerge from theory. It emerged from the constraint of building something real, on a real budget, against real data, while client pressure pushed deliveries out of order, while the framework I started on broke under load, while every problem I solved exposed two more behind it. The five primitives are visible in every step of the version history. They were not designed in advance. They were what survived the constraint.
Future Work: From Architecture to Ownership
The architecture described in Section 3 makes the model the smallest and most replaceable component in the stack. The substitution that follows, replacing a frontier model accessed through a cloud API with a fine-tuned smaller model under the operator’s own control, is not a hypothesis the manifesto is testing. It is the direction the architecture commits to. This section names the work that completes the commitment and the reason that work matters.
The reason is compliance, and the reason compliance matters is that there are sectors in which the cost of getting it wrong is not measured in fines.
Consider the operators for whom this is most acute. A healthcare provider processing patient records, clinical notes, prescription histories, diagnostic outputs. A law firm holding privileged communications, client matter files, case materials that sit under attorney-client privilege. A hospital network running clinical decision support against imaging data that identifies the patient as clearly as their name on the file. These operators sit under GDPR, sit under the EU AI Act, and sit under sector-specific rules that treat their data as the most sensitive category of personal information. Medical records carry layered protection under Article 9 of the GDPR; legal communications carry protections that predate the regulation by centuries. Sending any of this through a cloud API hosted outside the operator’s jurisdiction, or processed by a company headquartered in a third country with extraterritorial data laws, creates legal exposure that no commercial advantage can offset. The EU AI Act’s full enforcement provisions arrive on 2 August 2026 with penalties reaching seven per cent of global annual turnover, higher than GDPR. [Note, 2026-10-22: in July 2026 the EU moved the high-risk obligations to December 2027.] The arithmetic does not favour the operator who chose convenience.
For these operators, the choice is not whether to fine-tune. It is where to host. A fine-tuned small model running through a cloud provider’s hosted service inherits every compliance question the provider answers: its jurisdiction, its data handling, its training-use clauses, its CLOUD Act exposure, its outage history. A fine-tuned small model running on infrastructure the operator controls, in the jurisdiction the operator operates from, answers the same questions through its own architecture. The model’s weights are the operator’s. The inference path is the operator’s. The audit trail is the operator’s. The five primitives in Section 3 already separate the model from the system of record; the substitution extends that separation to the inference layer itself.
The methodology for fine-tuning, in this architecture, is settled rather than speculative. Belcak et al. set out a conversion algorithm in 2025; Jhandi et al. demonstrated a sub-one-billion-parameter model outperforming much larger general models on a standard tool-calling benchmark in the same year. The architecture in Section 3 is a worked instance of the conditions both papers identify as necessary. What remains is execution, and execution turns on training data.
Training data discipline matters more than training data volume. A small model fine-tuned on a thousand unfiltered production examples produces a model that is more confidently wrong, not less. The methodology I would apply to any workflow built on this architecture begins by collecting roughly a thousand production executions, screening them against human approval outcomes and downstream operational signals, and taking the top hundred as the initial training set. Fine-tune on those. Measure acceptance rate, end-to-end latency, and consistency against the cloud-API baseline. If the resulting model falls short on any axis, widen to two and a half thousand executions filtered to the top five hundred and retrain. Phase two runs only if phase one is insufficient. The discipline is the same regardless of where the model is eventually hosted.
The hosting decision is the operator’s. The two viable postures are fine-tuning through a provider’s hosted service, which trades some of the compliance benefit for ease of operation, and fine-tuning into self-hosted weights on infrastructure the operator owns or rents in the appropriate jurisdiction. The self-hosted posture can be realised on prosumer-grade hardware, the kind a smaller operator buys outright and runs on electricity, or on a single rented enterprise-grade GPU for operators who prefer a fixed monthly cost without owning hardware. For a regional clinic or a mid-sized law firm, the prosumer floor is the realistic entry. For a hospital trust or a national legal practice, rented enterprise infrastructure is the more natural posture. Both deliver the same compliance position: data residency, audit independence, and freedom from provider concentration risk. Section 5’s argument about the AWS strikes and the Claude outage was about exactly this. When the inference layer is yours, third-party failures cannot take you offline with them.
The economics align past a threshold. Published numbers on on-premise inference put per-token costs orders of magnitude below cloud API pricing once utilisation crosses a moderate floor; the precise crossover for a given workflow is one of the empirical contributions the next phase of work will produce. The point is not that self-hosting is always cheaper. The point is that for the operators who must self-host on compliance grounds, the financial penalty of doing so collapses past a volume that most production workflows will eventually reach. The right thing and the affordable thing converge.
What the substitution makes available, finally, is a configuration in which the model is no longer the load-bearing dependency the architecture has consistently said it should not be. The work to demonstrate this on the scheduling agent, and on the wider class of workflows built on the same primitives, is the next phase. Section 8 takes up what that configuration means for the operators choosing it, and for the next generation of AI startups choosing the architecture that produces it.
What Comes Next
The dominant narrative around agentic AI in 2026 has now shifted. The models are not the moat. The infrastructure is. The most influential capital in the venture market is no longer asking founders to build the next chatbot or the next copilot. It is asking them to build companies in which AI is the foundation, not a feature. The phrase that has emerged for what that means is the queryable company: every meeting recorded, every ticket tracked, every customer interaction captured and made legible to an intelligence layer that learns from it. The vision is a closed loop, not an open one. The work is to make the loop close.
That work is exactly the work this paper has described.
A queryable company is a curated local store. A closed loop is human-gated write-back through a separate code path. The integration burden investors now name as the barrier to entry, the brutal stitching together of Slack, Linear, GitHub, call recordings, ticketing systems, and the dozen other tools every operational system depends on, is the construction of the AI-shaped consumption layer between source of truth and agent. The “company brain” is not a model. It is the infrastructure that lets a model behave intelligently against an organisation’s actual data. The five primitives are what that infrastructure looks like when it is built right.
This is why the next generation of AI startups will look different from the last one. The previous wave bolted frontier models onto existing software and called the combination an AI product. It is now clear what that strategy produces. The 95% MIT pilot failure rate. The Klarna reversal. The Forrester data showing that more than half of employers regret the layoffs they made in pursuit of AI returns. The companies winning the long game are not the ones with the best wrapper around someone else’s model. They are the ones that built the infrastructure first, then plugged a model into it last. When the model is the smallest and most replaceable component in the stack, the company that owns the infrastructure is the company that owns the customer relationship, the compliance posture, the audit trail, and the operational reality the AI actually serves.
I did not set out to write a manifesto. I set out to ship a system. The system imposed the architecture on me, because no other architecture could survive at production scale on the budget I had. What I learned, developing and delivering alone for seven months against live enterprise data, is that the architecture the constraint forced is the architecture the market is now describing in its own language. Infrastructure-first AI is what survives. It survives the pricing volatility of frontier models. It survives the deprecation cycles. It survives the geopolitical exposure that disrupted Claude in the Middle East regions in March 2026. It survives the compliance ceiling that GDPR, the EU AI Act, and sector-specific regulations impose on every organisation that holds data worth protecting. It survives because the model is no longer the load-bearing part.
The companies being built right now on this architecture will not look like AI companies, in the way the previous wave looked. They will look like the operational systems they replace, with AI as the substrate rather than the surface. The model will be a configuration choice. The infrastructure will be the product. That is the bet the smartest capital is now placing. The next generation of founders will be the ones who place it with them.
References
-
Amazon (Galetti, B.) (2025, 28 October). Amazon to lay off 14,000 corporate employees. NBC News. nbcnews.com
-
Anthropic (2024, 19 December). Building effective agents. Anthropic Engineering. anthropic.com/engineering/building-effective-agents
-
Belcak, P., Heinrich, G., Diao, S., Fu, Y., Dong, X., Muralidharan, S., Lin, Y. C., & Molchanov, P. (2025). Small Language Models are the Future of Agentic AI. arXiv:2506.02153 [cs.AI]. arxiv.org/abs/2506.02153
-
Benioff, M., via Singh, M. (2025, 2 September). Salesforce CEO confirms 4,000 layoffs ‘because I need less heads’ with AI. CNBC. cnbc.com
-
Carpenter, N. (2025, 15 July). Report: Laid-off staff at Candy Crush maker say they’ve been training their AI replacements. Kotaku. kotaku.com
-
Challapally, A., Pease, C., Raskar, R., & Chari, P. (2025, July). The GenAI Divide: State of AI in Business 2025. MIT NANDA Initiative, MIT Media Lab. nanda.media.mit.edu
-
European Union (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 (General Data Protection Regulation), Article 9 — Processing of special categories of personal data. Official Journal of the European Union, L 119/1. eur-lex.europa.eu/eli/reg/2016/679/oj
-
European Union (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), Article 99. Official Journal of the European Union, 12 July 2024. eur-lex.europa.eu/eli/reg/2024/1689/oj
-
Forrester Research / Le Clair, C., et al. (2025, 28 October). Predictions 2026: The Future of Work. Forrester Research. forrester.com
-
Jhandi, P., Kazi, O., Subramanian, S., & Sendas, N. (2025, 17 December; rev. 9 March 2026). Small Language Models for Efficient Agentic Tool Calling: Outperforming Large Models with Targeted Fine-tuning. arXiv:2512.15943 [cs.AI]. arxiv.org/abs/2512.15943
-
Levin, M. & Roe, D. (2026, 9 March). Iranian attacks on Amazon data centers in UAE, Bahrain signal a new kind of war. Fortune. fortune.com
-
n8n / fjrdomingues (2025, 2 April). AI Agent doesn’t store the Tool usages in memory (n8n-io/n8n issue #14361). GitHub. github.com/n8n-io/n8n/issues/14361
-
Shibu, S. (2025, 5 May). IBM replaced hundreds of HR workers with AI, according to its CEO. Entrepreneur. entrepreneur.com
-
Shibu, S. (2025, 8 May). Klarna is hiring customer service agents after AI couldn’t cut it on calls, according to the company’s CEO. Entrepreneur. entrepreneur.com