The 3 Layers of Modern AI: Systems → Infrastructure → Products

You type a question into an AI app and the answer appears.

Behind it: a building chosen years ago for its access to electricity. A supply contract signed with a power generator before the app existed. A shipment of chips that needed a government’s permission to cross a border. And a company you’ve never heard of whose job is keeping that answer coming to you fast.

None of it is visible to you. A finished product hides its supply chain - that’s what finished means. But the hidden part is where the cost sits, where the risk sits, and where it gets decided what that app is allowed to do at all.

The model everyone talks and argues about is one step in that chain.

Three layers. The order matters, because most times each only becomes possible once the one above it is settled.

01 - Systems

Who decides whether any of this is allowed to happen.

Nothing is built at this layer. No chip, no model, no product. What gets set here is the outer boundary of what everyone underneath is permitted and able to do.

Rules: The EU AI Act is the one with a name. Most of the rules that actually bite don’t have names, they’re the existing regulations governing banks, hospitals, insurers and public institutions, and they attach to a model the moment it touches that work. A bank cannot deploy a system it can’t explain to its supervisor. A hospital cannot use a device that hasn’t been cleared. Neither is an AI rule; both were already there, and a model that can’t clear them may as well not exist for that sector.

Capital: Frontier training of models is a capital expense, not an operating one. The money is committed years before anyone knows what it bought, and if the run fails it isn’t recoverable. Very few pools of capital tolerate that shape - venture funds, sovereign wealth, public markets, and the balance sheets of the handful of firms large enough to self-finance. That constraint, more than any technical one, is why there are five or six frontier labs rather than five hundred.

Chips, and where they’re allowed to go: Advanced fabrication is concentrated in very few places, and the lithography equipment behind it in fewer still - effectively one company. Export controls now determine which countries can buy what, and that single set of decisions draws the map for everything below: where training capacity can be sited, which labs get the current generation of hardware, and who spends the next two years working a generation behind.

Standards and security: Risk management frameworks, management system certifications, evaluation requirements, procurement checklists. Unglamorous, and decisive - they determine what a regulated buyer is permitted to sign for, which quietly determines which products at the bottom of this stack have a market at all.

When something moves at this layer, everything below it re-prices. An export rule changes on a Tuesday and a two-year build plan is worth less by Wednesday.

02 - Infrastructure

Where intelligence gets built, then made to run somewhere.

Two stages sit inside this one layer. They get collapsed constantly, and that collapse is where most of the confusion about who does what in building AI comes from.

a) Build / Train

This part is physical. A data centre is a building full of computers, and almost none of it comes from one supplier. Six separate supply chains have to line up, each with its own constraint, and any one of them can be the reason a site opens two years late.

Someone has to own the building - land, permits, shell, and the fibre running into it. Sites are chosen years ahead on the basis of power availability and network routing rather than anything resembling ordinary property logic. Equinix is one of the companies in that business.

Someone has to supply the power. A single large site can draw what a mid-sized city draws, which puts it in a different relationship with the grid than any ordinary customer - you negotiate directly with generators, sometimes for the output of a named plant, sometimes by building your own. This is the binding constraint on new capacity right now. Chips can be bought faster than power can be connected, and the queue for a grid interconnection runs in years. Generators like Vistra sit on the other side of those contracts.

Someone has to sell the compute. Nvidia is the obvious name; the largest buyers increasingly design their own silicon alongside it, which is itself a signal about where the margin sits.

Someone has to wire the chips to each other. A training run isn’t thousands of separate jobs - it’s one job split across thousands of chips that have to stay in step, passing results back and forth the entire time. If the link between them is slow, adding more chips stops helping. There is an entire business in selling that link, and companies like Arista live in it.

Someone has to keep the accelerators fed. Compute is the expensive line item, so the storage tier exists to make sure none of it is sitting idle waiting on a read. Everpure and the high-throughput vendors compete on exactly that.

Someone has to take the heat away. Racks now concentrate enough of it that moving air across them no longer works, so water gets plumbed through the building, bringing in liquid cooling specialists like Motivair. And in dry regions, how much water a site draws has become a permitting question. Which means the cooling decision travels back up to Layer 01 and lands in front of a regulator.

Inside that building, the labs run the training - OpenAI, Anthropic, Google, Meta, Mistral, DeepSeek. Weeks of it, sometimes months. What comes out is a set of weights in a file.

The file does nothing on its own. It can’t answer a question, read a camera feed, or be opened by a user. Which is precisely why the next stage exists.

b) Deploy / Runtime

Taking that trained model and preparing it to actually run somewhere.

The runtime is what turns weights into something callable: loading the model into memory, batching incoming requests so expensive hardware isn’t idle, caching recent work so it isn’t recomputed, quantizing the model to run on cheaper silicon, scheduling across available capacity, and scaling machines up and down with traffic. Or, on the other path, compressing and compiling the whole thing until it fits on a single chip inside a device.

Every model call, without exception, passes through a runtime. The only question is who owns it and whether it’s visible as its own product.

Bundled runtime - here, the lab that trained the model also owns and operates the serving layer, hidden behind its own API or device. You never encounter “runtime” as a separate thing.

• Cloud: OpenAI, Anthropic and Google serve their own models through their own APIs. A developer calling one of those is using that lab’s in-house serving stack without ever being told about it.

• Edge: Tesla and Apple built their own on-device inference stack. The intelligence in the car or the phone runs on hardware they designed, with serving software they wrote, and nothing leaves the device to make it work.

Unbundled runtime - the company that trained the model and the company serving it are different businesses. This happens mainly with open-weight models: Llama, Mistral, DeepSeek. The creator publishes the weights and doesn’t operate them for anyone. Somebody still has to. That gap is a business.

• Cloud, specialists: Together AI, Fireworks, Baseten, Modal, Groq. Serving other people’s models is the entire company, and that focus is why they tend to win on price-performance for inference specifically, carry the broadest model selection, and have new open models running first.

• Cloud, hyperscalers: AWS Bedrock, Azure AI Foundry, Google Vertex AI. Here serving is one feature inside a much larger platform that also sells storage, databases and compute. The pitch isn’t raw performance - it’s that the customer is already inside. Billing, identity, security and compliance all carry over, so there’s no new vendor to approve and no new contract to negotiate. At enterprise scale that regularly outweighs a faster response time.

• Edge: General Instinct on the specialist side, Hailo on the chipmaker side. Both take a model you bring them and compile it down onto physical silicon.

You’d expect edge to come out the same shape as cloud, and it doesn’t. The companies attaching runtime software to something larger aren’t cloud giants - they’re chipmakers. Hailo and Intel’s OpenVINO give you the software because the software is what makes their silicon worth buying. AWS Greengrass is about the only case of a genuine cloud giant crossing into edge.

So the edge version isn’t “cloud giant adds edge as a feature.” It’s “chip company ships free software to sell more chips.” A small difference, and the accurate one rather than the tidy one.

Runtime creates nothing new. It makes the intelligence usable - as an API call, or as something flashable onto a device. Whether it’s visible as its own business depends entirely on whether the model-maker and the runtime-runner are the same company.

03 - Products

Where someone else - a separate builder - takes that usable intelligence and puts it inside a real thing a person buys or opens.

Software products. A company builds an application and reaches a model through a runtime, either by:

• calling a lab’s own API directly, using that lab’s in-house runtime invisibly, or

• calling a third-party runtime serving an open model.

Either way the end user sees only the finished application. Never the model, never the runtime beneath it, never the supply chains beneath that. The work at this layer isn’t intelligence - it’s interface, workflow, trust, distribution, and knowing a customer’s job well enough to know which part of it is worth automating. That is a different discipline from training a model, which is why the companies doing it are usually different companies.

Examples: Cursor, Perplexity, Harvey. All three started here in the ordinary way - calling someone else’s API and building a product around it. All three now post-train their own models on open weights, because paying a per-call toll to a company that is also moving into your market stops making sense at scale. Harvey’s runs on Moonshot’s Kimi; Perplexity’s on Llama. Note what that requires: open weights from someone who trained them, and someone else’s infrastructure to run the training on. Harvey’s post-training went through Fireworks, which you met two sections ago. ChatGPT and Claude also sit at this layer, but they’re the collapsed case - the lab that built the model shipping the product itself.

Hardware products. A company builds a physical device - camera, robot, drone - and works with an edge runtime provider to compress a model onto that device’s chip, or builds that capability in-house.

The constraints here are harsher. There’s no round trip to a data centre, so latency is whatever that one chip can manage. The model has to fit inside a fixed power and thermal budget, because the device has neither active cooling nor a wall socket. And once it ships, updating it stops being a deployment problem and becomes a logistics one.

Examples: security cameras with on-device recognition, autonomous drones, industrial robots.

Genta Sentinel sits here, as an intelligence layer deployed onto camera infrastructure that is already installed. That’s the more interesting version of the hardware case. The cameras exist. The footage exists. Most of it is watched by nobody and reviewed only after something has already gone wrong. What’s missing isn’t hardware, it’s the intelligence to make the footage useful while the event is still happening. Closing that gap doesn’t require shipping new cameras. It requires getting a model to run on what’s already installed.

Systems decide who’s allowed to build → Infrastructure builds the intelligence and runs it through a runtime, owned in-house or rented from a third party → Products are separate builders who put that intelligence into an app or a device and hand the finished thing to a user.

You may have spotted that a company like OpenAI trains models, runs the servers that answer the requests, and sells ChatGPT directly to the public - all three layers, one company. Nvidia sells the silicon and the software that runs models on it. Google does the whole thing end to end. If the same companies keep turning up at every layer, what is the point of separating them?

Because the layers were never groups of companies. They’re jobs. Each has to be done before the next becomes possible, and every AI product that reaches a person has been through all three, whether one company did all the work or different companies divided it between them.

The question worth asking - which job is this company actually doing, and does it control the job above it?

Take a company selling an AI product built on somebody else’s API. It owns Layer 03 (products) and nothing above it. If the lab it depends on raises prices, it absorbs that. If the lab ships a competing product, it absorbs that too. Both its largest cost and its most serious competitor sit inside a business it has no say in. That is the position companies like Cursor and Perplexity spent the past year climbing out of, and it’s why each of them now trains a model instead of only renting one.

The same test applies anywhere on the stack. It won’t tell you who’s ahead. It tells you who’s exposed, and to what.