Imagine you are about to sign a six-figure contract for GPU capacity you cannot explain to your CFO. That is the situation a surprising number of enterprise teams now find themselves in. A new survey of 107 enterprises finds that AI infrastructure spending is accelerating far ahead of anyone's ability to measure, attribute, or control its costs. Most organizations still run their AI workloads on a familiar base of hyperscaler cloud and model-provider APIs, yet the next dollar is pointed at specialized compute that almost none of them use today. A majority intend to switch or add providers within the year, and many within a single quarter. The AI compute gap is widening: the money is moving, the metering is not, and the people writing the checks do not have the tooling to know what they are buying. If you build or ship with AI, this is your budget on the line, whether you report to engineering or finance.
What does the survey actually find about AI compute spending?
The core finding from the VentureBeat survey is that 45 percent of enterprises plan to add or switch AI compute providers within the next year, with a significant chunk aiming to do so within a quarter. The spending trajectory is rising across the board: enterprises are pouring budget into inference and training capacity faster than they can build the dashboards, chargeback models, and unit-economics frameworks to track it. The same piece reports that approximately 8 percent of surveyed organizations have what the authors consider a mature approach to measuring AI compute costs, meaning over nine in ten are operating with partial or no visibility.
The chart above shows where the next dollar is aimed. 58 percent of the next wave of spending is directed at specialized AI compute providers, the kind of bare-metal GPU clouds and inference platforms that sit outside the hyperscaler tent. Only 25 percent stays with the hyperscalers, and 12 percent goes to model-provider APIs, the layer most builders interact with daily. The remaining 5 percent is on-prem GPUs. That split matters because specialized compute is the category where attribution tooling is weakest, contracts are newest, and pricing models vary the most.
Why is this a problem if everyone is already spending on AI anyway?
The problem is not that spending exists. It is that spending is outrunning the plumbing around it. If your team launches an agent that calls a frontier model API, you can usually see the token bill. If you move that workload to a specialized inference provider with reserved capacity, a custom container image, and a shared GPU pool, the cost surface gets murky fast. Who consumed what? Was that batch inference job for the product team or the research team? Did the fine-tuning run on the A100s justify the spend, or did it sit idle for six hours?
The survey signals that almost none of the organizations buying specialized compute today have the internal tooling to answer those questions. That means teams are making architecture decisions, provider commitments, and capacity reservations based on instinct and vendor demos rather than measured unit economics. If you have ever had to reverse-engineer a cloud bill at the end of the quarter to figure out which team owes what, you know how painful that conversation is. Now imagine it with a technology stack that changes every few months and a provider landscape where the key players did not exist a year ago.
This connects to a pattern we have tracked before. The enterprise AI agent evaluation gap showed that half of organizations are shipping agents they cannot properly test. The compute gap is the infrastructure layer of the same disease: teams are running ahead of their own ability to observe what they have built.
How does this change the decisions a builder or team lead should make?
If you are a developer, founder, or product lead running AI workloads, this survey should change how you think about your next three moves.
-
Build cost attribution before you migrate. If you are one of the 45 percent planning to switch or add a compute provider, instrument your current spend first. Tag every inference call, every batch job, every fine-tuning run with a project and team label. If you cannot answer "what did this feature cost last month" before you migrate, you will not be able to answer it after, when the bill is bigger and the provider is less familiar.
-
Treat specialized compute contracts as a build-vs-buy decision for observability. Specialized providers often hand you raw usage logs and expect you to build the chargeback layer. Factor that engineering cost into the total cost of ownership. A provider that is 20 percent cheaper per GPU-hour but requires your team to build a billing aggregator from scratch may cost more in engineer hours than it saves on silicon.
-
Watch the quarter-over-quarter provider churn. The survey says many enterprises plan to add or switch providers within a quarter. That is fast. If your team is mid-migration and the provider landscape shifts again, you need an abstraction layer, an inference router, or at minimum a clean interface that lets you swap backends without rewriting your application layer. Lock-in is not just a vendor problem, it is a velocity problem.
-
Push finance to set a compute budget with a unit-economics anchor. Do not accept a flat monthly budget. Insist on a cost-per-inference, cost-per-agent-action, or cost-per-fine-tuned-model metric that both engineering and finance agree on. The survey suggests most organizations lack this shared metric, which is exactly how budgets get blown and projects get cancelled mid-stream.
The deeper issue is that only 8 percent of enterprises have mature cost-measurement practices for AI compute. That means the other 92 percent are flying with instruments that would not pass muster in a traditional cloud-spend review, let alone a board meeting. The gap between spend velocity and measurement maturity is the story here, and it is widening.
What should you watch over the next two quarters?
The provider landscape is moving fast enough that the survey's snapshot may age quickly. Moonshot AI just released Kimi K3, a 2.8-trillion-parameter open-source model priced at $3 per million input tokens and $15 per million output tokens, with cached input at $0.30 per million, according to VentureBeat's reporting. That kind of pricing from a Chinese lab changes the economics of inference for any team that can run open-weight models on rented GPU capacity. Axios reports that the model benchmarks competitively with proprietary systems from Anthropic and OpenAI, meaning the gap between frontier proprietary and frontier open-weight is narrowing in a way that directly affects your build-vs-buy calculus. If specialized compute providers start offering Kimi K3 or comparable open-weight models as a managed service, the pricing pressure on the entire inference market will intensify, and the cost-attribution problem becomes even more urgent because the price floor is dropping.
There is also a regulatory and infrastructure dimension. New York's data center moratorium is already reshaping where AI compute gets sited in the United States, and Apple's AI price hike tied to memory costs shows that the hardware supply chain can move consumer prices, not just enterprise contracts. If your compute strategy assumes hyperscaler capacity will always be available at predictable prices, the next two quarters may disabuse you of that.
The open question is whether observability and FinOps tooling for AI compute catches up before the spending wave crests. The venture market is funding startups that want to be the Datadog or Vercel of GPU spend, but adoption lags. Most teams still rely on provider-native dashboards and hand-rolled spreadsheets. If you are evaluating tooling, prioritize platforms that can ingest data across multiple providers, not just the one you happen to be leaving.
The compute gap is a governance problem, not just a tech one
The hardest part of this story is not the technology. It is that AI compute spending has become a governance problem disguised as an infrastructure problem. Boards are approving budgets for AI initiatives without the dashboards to hold them accountable. Engineering teams are choosing providers without the data to prove the choice was right. Finance teams are getting bills they cannot allocate. The 107 enterprises in this survey are not outliers, they are the market. If your organization is further along than the survey median, you have a competitive advantage you may not even realize: you can see your own spend. That sounds basic, but in a market where 92 percent of enterprises cannot do it well, visibility is a moat.
Sources
- VentureBeat: The AI compute gap: Enterprises are buying infrastructure faster than they can measure what it costs
- VentureBeat: China's Moonshot AI releases Kimi K3, the largest open-source model ever, rivaling top U.S. systems
- Axios: China's open-weight Kimi model stuns AI world with frontier-level results
