As artificial intelligence technology continues to infiltrate all aspects of business, many companies are having trouble understanding what it’s costing them.
A recent report in the Wall Street Journal cited a study that only 11 percent of nearly 400 businesses were able to accurately forecast what their AI spending would be. This comes as many companies encouraged their workers to use AI as much as possible — tokenmaxxing is what they called it — only to find out how expensive that was.
“Unlike traditional software,” the WSJ wrote, “AI behaves more like a human worker: It takes action, makes decisions, sometimes even makes mistakes—all on the clock.”
The newspaper said that researchers from Stanford University, Carnegie Mellon University and the University of California, Berkeley, as well as Microsoft Research, ran models through more than 6,800 tasks spanning math, programming, science and other areas. In 32 percent of those cases, lower-priced models actually cost more than higher-priced models. “The practical takeaway is clear,” Lingjiao Chen, one of the researchers, told the WSJ. “Price alone should not be used to infer which model is actually cheaper,” noted Chen in reference to the various AI platforms now available. In one test the study ran, the less expensive model failed to produce an answer after running up $14 worth of token use, while the pricier one succeeded for just $1. “While this result could have been an anomaly, it is something that can happen in regular model use,” the researchers wrote.
“Some prompt-level fluctuation is inherent to AI, and our testing shows this averages out across a high volume of real-world, diverse workloads,” said a Google spokeswoman in a statement to the WSJ on the study. “Total costs depend on many factors for a given task which can make it hard to forecast new and evolving technology with precision.”
The Journal’s advice to companies using AI? “They will need strategies to manage their use, such as training employees on which model to use for a particular task. Otherwise, they could end up with IT bills that come out of a black box.”
