Companies that raced to put AI tools in the hands of their workers are starting to rein in their use, as the cost of deploying the technology at scale begins to test corporate budgets.
Amazon, Walmart, Cisco, Uber and Meta are among early adopters that have introduced caps, discouraged wasteful use or pushed employees to cheaper models in a bid to keep AI spending under control.
A new phase in AI adoption
The shift marks a new phase in corporate AI adoption. As workers move beyond chatbots to AI agents, which can perform complex tasks autonomously but require far more computing power, companies are being forced to scrutinise whether each query and task is worth the cost.
This has intensified as groups including Anthropic and OpenAI have moved some services from flat subscriptions to token-based billing, which tracks the units of data processed by models. The change has exposed companies more directly to the cost of every prompt and automated workflow
“Compute costs are now beginning to enter the minds of both CFOs and boards. Consumers and businesses have been taught that AI is cheap or free and that is definitely not the case,” said Costi Perricos, global generative AI leader at Deloitte.
Sam Altman, OpenAI’s chief executive, said this month that cost had emerged as a “huge issue” for customers this year. “The issue never came up [last year] . . . People were totally happy with the amount they were spending.”
All about the infrastructure
“The amount of infrastructure needed for an agent is meaningfully higher than for a chatbot,” said Patel. “For every human you might have 10, 100 or on the aggressive side 1,000 agents . . . They just keep working and that consumes a chunk of [compute].”
Goldman Sachs analysts last month predicted that use of AI agents would result in a 24-fold increase in token consumption by 2030 and that the huge rise in demand would exacerbate a shortage of chips over the next 12 to 18 months.
Since the start of the year, Chinese AI models have overtaken their US counterparts in token consumption, according to data from OpenRouter, an aggregation platform that allows users to access multiple AI models.
China is taking the lead
China’s cheaper energy and more efficient models have allowed the country’s AI labs to charge less than leading US groups for tokens, giving China a new edge on the AI battleground.
Smaller companies are also feeling the cost pressure. Software group Workato said its AI use exploded after its 1,300 employees started using AI agents last summer. “It took off like wildfire, people started really transforming their jobs with agents,” said chief information officer Carter Busse.
But the company got a shock when Anthropic switched it over to token-based pricing in May. “Our spend went up 7x the first day and I’m like, oh shit, we created a monster,” said Busse. “
[Large language model] companies have been subsidising all of our usage and now no longer. User-based pricing shelters you.”
Source: FT
https://www.ft.com/content/1d37cc08-e0aa-45a4-a45d-4ad282529314?syn-25a6b1a6=1
