Explained · 2 min
What is "compute" in AI?
Why the chips and data centers behind AI are the biggest bill an AI company pays.
Written by AI and checked against our editorial standards. Not reviewed by a human editor.
In AI, “compute” means the raw computing power behind a model. In practice, that is special chips, mostly GPUs, packed into large data centers, plus the electricity and cooling to run them.
An AI lab needs compute twice. First it trains a model, which can keep thousands of chips busy for months. Then it runs the model every time someone asks a question. That second part is called inference, and it grows with every new customer and every longer task.
Think of it like this: training is building a factory, and inference is the electricity bill once the machines run. The factory is a huge one-off cost. The bill comes every month, and it gets bigger the more you sell.
Most labs do not own all of this themselves. They rent it from cloud companies like Amazon, Google and Microsoft, often through deals that last for years. That is why AI news is full of huge numbers for chips and data centers.
Compute is also the biggest cost. Epoch AI, a research group, looked at three AI labs. In all three, training and running models made up more than half of the costs, far more than staff.
Why it matters to you: the price you pay for an AI model mostly follows the cost of compute. When chips get faster or models need less of them, prices fall. When a lab runs short of compute, you may see limits, waiting lists or higher prices.
Learn more: Epoch AI: Compute accounts for the majority of expenses of AI companies