Everyone is jumping on the AI bandwagon, trying to see how it can help their business. But controlling costs is critical.
Chances are you are starting to use artificial intelligence platforms in your business and seeing some positive results from how the technology can make your company more efficient. But you may also have found that getting a handle on costs can be a real challenge.
A new study found that only about one in ten users of nearly 400 surveyed businesses were able to accurately forecast their spending on artificial intelligence. And in a third of those tested cases, which AI model you used had a big impact on the cost, with lower-priced platforms often outperforming higher-priced models.
The Wall Street Journal recently reported that the study—which was conducted by researchers from Stanford University, Carnegie Mellon University and the University of California, Berkeley, as well as Microsoft Research— concluded that after running models through more than 6,800 tasks spanning math, programming, science and other areas, lower-priced models actually cost more than higher-priced models in about a third of the cases.
“While more-advanced models tend to cost more per token, they can sometimes perform tasks more efficiently, leading to lower overall costs,” the WSJ wrote. “Likewise, asking a ‘cheap’ model to do something it isn’t suited to handle could cause a token run-up.”
“The practical takeaway is clear,” said Lingjiao Chen, one of the researchers. “Price alone should not be used to infer which model is actually cheaper.”
How to project costs is a daunting task, the report said. Unlike traditional software, AI more often functions like a human worker, making decisions, acting and sometimes making mistakes — on the clock. It’s causing more companies to focus on the use of tokens — the basic measurement for AI use. Tokenmaxxing, the original tool for AI use, is being rethought so companies can better budget their AI usage.
When asked to comment, Google responded to the WSJ, “Some prompt-level fluctuation is inherent to AI, and our testing shows this averages out across a high volume of real-world, diverse workloads. Total costs depend on many factors for a given task, which can make it hard to forecast new and evolving technology with precision.” Google said it offers customers more control via spending caps and flexible pricing.
The WSJ does offer some advice. “As more companies add AI to their daily workflows, they will also need strategies to manage their use, such as training employees on which model to use for a particular task. Otherwise, they could end up with IT bills that come out of a black box.”
