The Cloud Built for Web Apps Is Being Rebuilt for AI

For most of its history, cloud computing was built around general purpose servers. Online shops, databases, email and business software ran on processors that handle many different tasks reasonably well. Capacity was plentiful, and a customer who needed a hundred more servers could usually have them within minutes.

AI has changed that picture. Training large models and serving them to millions of users requires chips built for massive parallel arithmetic, networks that connect thousands of those chips and buildings that can supply and remove large amounts of power and heat. Cloud providers have invested heavily in this capacity, and it is changing how cloud computing is designed, priced and bought.

This article looks at the cloud through that lens: what AI workloads need, what that does to the infrastructure and what it means for organizations that want to use AI without overspending.

Why AI needs different hardware

Neural networks spend most of their time on one kind of operation: multiplying large tables of numbers. General purpose processors can do this, but slowly. Graphics processing units, originally designed to render images, perform thousands of these calculations in parallel. Over the past decade they became the standard tool for AI, and several cloud providers have added custom accelerators of their own.

Training a large model also requires those chips to work as a team. Thousands of accelerators exchange intermediate results constantly, so the network between them matters as much as the chips. AI clusters use specialized high-speed connections and are built as tightly coupled units, quite unlike the loosely connected servers behind a typical web application.

For cloud customers, the result is a distinct class of offerings: accelerator instances, AI clusters and model hosting services. Each comes with its own pricing and, at times, its own waiting list.

Power and cooling become the limit

A rack of AI servers draws far more electricity than a rack of conventional servers, and nearly all of it turns into heat. Air cooling, the standard in most data centers, reaches its limits at these densities. Liquid cooling, which brings coolant directly to the chips, is becoming common in new AI facilities.

Power supply is often the harder problem. A new data center needs a grid connection that can deliver a large, steady load, and in many regions such connections take years to arrange. Where providers can build is therefore increasingly decided by available electricity, not only by land, network links or closeness to customers. This also raises questions about energy sources, water use for cooling and the local impact of very large sites.

In AI infrastructure, the scarce resource is no longer only the chip. It is the power line to the building.

Training is rare, inference is constant

Public attention focuses on training, the expensive process of building a model from data. Most organizations never train a large model themselves. They use models that others have trained, through an interface or by running a copy. For them, the relevant cost is inference: the computing needed every time the model answers a question, writes a draft or sorts a document.

Inference costs behave differently from traditional cloud costs. They grow with use, are often billed by the amount of text processed, and long inputs cost more than short ones. A pilot with a few users can look cheap. The same feature rolled out to every employee or customer can become one of the largest lines on the cloud bill. Estimating the cost per task early, and watching it as usage grows, prevents unpleasant surprises.

Three ways to use AI in the cloud

Organizations usually choose among three approaches, and many combine them:

• Model APIs: the provider runs the model and charges per request. This is the fastest start and needs no hardware, but it offers the least control over the model and over how data is handled.
• Managed AI platforms: the customer selects, adapts and deploys models on infrastructure the provider maintains. This gives more control over versions, tuning and monitoring.
• Rented accelerators: the customer runs its own models on rented GPU instances. This gives full control and can be cheaper at high, steady volume, but it requires specialized skills.

The right choice depends on volume, the sensitivity of the data and in-house expertise. A regional insurer testing document classification may start with an API and move to a managed platform once volumes and requirements are clear. A company with strict data rules may prefer models that run inside its own cloud account from the outset.

Smaller models and smarter placement

Not every task needs the largest available model. Sorting support tickets, pulling fields from invoices or summarizing internal notes can often be handled by smaller or specialized models with a fraction of the computing. Testing a few model sizes against real examples, then choosing the smallest one that meets the quality bar, is one of the most effective ways to control AI costs.

Placement matters too. Some inference is moving out of central data centers to devices or local servers, where responses are faster and data stays on site. Capacity reservations can secure access to accelerators for planned projects, while on-demand use suits experiments. And because AI models need data, an old rule still applies: computing tends to go where the data already lives.

What to watch next

The cloud is becoming more specialized. General purpose computing will remain the backbone for most business systems, but a growing share of investment flows into AI capacity, with its own hardware, buildings and pricing. For customers, that means more options and more decisions: which models, which hosting approach, which region and which commitments.

The organizations that handle this well tend to treat AI like any other workload. They measure what each use case costs and what it delivers. They avoid locking into one approach before their requirements are clear. And they keep an eye on the energy and data questions that come with it. The hardware is new. The discipline is familiar.