The New Cost Stack of AI: Research, Training, Inference, Agents, and Distribution
AI companies carry a layered cost structure that starts with research and training but continues through inference, agent orchestration, data centers, customer acquisition, and distribution.
AI economics begin long before the first customer sends a prompt
A traditional software company can spend heavily on engineering and still deliver each additional copy of its product at very low marginal cost. Advanced AI companies begin with a much heavier research stack. Researchers run experiments, curate data, evaluate models, build training systems, and compete for scarce technical talent before a production model exists. Epoch AI’s cost work estimates that hardware and research labor dominate the development cost of leading frontier models, showing that the economic burden begins during experimentation rather than only when a finished model is served.[1]
The final training run is only one line in the research bill
Failed experiments, ablations, data work, evaluations, safety testing, and engineering all consume resources before a model becomes a sellable service.
Training converts research ambition into a concentrated infrastructure expense
Frontier training requires accelerators, servers, high-speed networking, storage, power, cooling, and engineering systems capable of keeping enormous clusters productive. Epoch AI estimates that training costs for frontier systems have grown rapidly over time and projects that billion-dollar final training runs become plausible as scaling continues.[1] The important accounting insight is that a training run may be episodic while the competitive need to train new generations is recurring.
Inference turns product usage into an ongoing cost of revenue
After deployment, every user request consumes computation. Stanford’s AI Index records a dramatic decline in the price of capable model inference, which is one of the most encouraging trends for AI profitability.[2] Yet declining unit cost does not eliminate the expense when usage rises just as quickly. A company serving billions of tokens or generating video at scale can experience enormous aggregate inference bills even while each unit becomes cheaper.
Usage growth can hide efficiency gains
If the cost per task falls by half while customers perform ten times as many tasks, the absolute infrastructure bill can still rise rapidly.
Agentic systems multiply both value and computation
Agents can plan, browse, write code, call APIs, invoke several models, critique their own work, and retry failed steps. A single customer request may therefore trigger an entire computational workflow rather than one inference. The economic case is that the agent may complete work worth far more than a chat response. The cost risk is that poorly controlled loops consume tokens and tool calls without producing proportionate value. Agent profitability depends on orchestration efficiency as much as model price.
Data centers move AI spending from software budgets toward industrial capital
Epoch AI’s finance research tracks the extraordinary growth in physical assets associated with AI, including accelerators and data-center capacity.[3] Owning infrastructure introduces depreciation, maintenance, financing, and utilization risk. Leasing infrastructure converts some of that burden into contractual operating commitments. Either way, AI companies increasingly resemble capital-intensive operators beneath software-like interfaces.
Utilization determines whether expensive capacity becomes an asset or a drag
A fully used cluster can support revenue and research. An underused cluster still consumes capital, depreciation, and often energy while producing little economic return.
Distribution takes a share of economics that model benchmarks do not reveal
AI companies reach customers through cloud marketplaces, app stores, enterprise resellers, browser integrations, device platforms, and strategic partners. Those channels can accelerate growth, but they may require revenue sharing or discounts. The result is that headline revenue and gross-basis ARR can overstate the amount of economic value the model provider ultimately retains. Klover’s profitability research on Anthropic specifically highlights how reseller structures and compute arrangements matter when interpreting reported economics.[4]
Sales, support, compliance, and trust add ordinary business costs back into the stack
Once AI moves from demonstration to enterprise infrastructure, customers expect security reviews, support, compliance, reliability, procurement integration, and account management. These costs resemble SaaS, but they sit on top of the research and compute stack rather than replacing it. McKinsey’s software research shows why mature technology businesses must balance growth with margins and cash generation.[5] AI companies face the same requirement after adding several unusually expensive technical layers.
Enterprise revenue can be high quality and expensive to acquire
Large contracts improve predictability but often require long sales cycles, dedicated deployment teams, customized security work, and ongoing service commitments.
Architecture determines how much of the cost stack a company must own
Not every AI company bears every layer equally. A foundation-model lab may own research, training, serving, and platform distribution. An application company may rent models and concentrate spending on product and customer acquisition. An agentic research company may innovate through orchestration while using multiple external models. Klover’s 2026 financial analysis argues that these architectural differences help explain why AI companies can produce radically different profitability profiles even when all are described as frontier or advanced AI businesses.[4]
Why the AI cost stack will determine who becomes sustainably profitable
The AI profitability race will not be won simply by the company with the most revenue or the cheapest model. It will be won by organizations that understand which layers create strategic differentiation and which should be bought from others. Training efficiency, inference cost, infrastructure utilization, agent design, distribution economics, and enterprise delivery all ultimately meet in the financial statements.[1][3]
This is the central economic difference between the AI era and the software eras that came before it. Intelligence is becoming cheaper to access, but producing and operating it at the frontier still requires a complex physical and organizational system. Sustainable AI companies will be those that convert that entire stack—not merely the model—into more durable value than it consumes.
Works Cited
- 01
- 02Stanford HAI — 2025 AI Index: Research and Development hai.stanford.edu
- 03Epoch AI — The Finances of AI epoch.ai
- 04
- 05
CodeHistory is a living archive. Citations document the evidence used for this edition; later evidence may refine the account.
Submit a research lead