What Comes After Tokenmaxxing?: How to avoid getting locked in to just one AI provider

Dear friends,

I’m glad the idea of “tokenmaxxing” — that individuals and companies should use as many tokens as possible to boost productivity — is finally dying out. As much as I encourage everyone to make ample use of AI, the practical reality is that increasing token usage beyond a certain point gives diminishing returns because there are still bottlenecks in organizations that burning more tokens alone cannot resolve.   

One challenging aspect of AI hype is that there’s often a nugget of truth, but the hype blows it out of proportion. Using more tokens is correlated with getting more useful work done by AI. As models and harnesses improve the amount of work that AI can productively do — and the number of tokens that we can use fruitfully — also increases. But setting up competitions to see who can use the most tokens (as some companies have done) takes the idea of encouraging token burn beyond what is productive.   

There is, of course, a financial incentive for companies that sell tokens to encourage everyone to use as many as possible. Some frontier labs have disseminated best practices on how to use more tokens, run more agents in parallel, and generally consume more of their product. This follows a long history of companies trying to get people to use more of whatever they sell: 

• Car repair shops routinely recommend people get an oil change every 3,000 miles, which is much more frequent than is needed for most cars.  
• Most dentists in the US will tell you that adults need only a pea-sized dollop of toothpaste; yet TV toothpaste ads routinely show people using a long strip to promote using more.   

Using tokens and multiple agents is valuable, just like oil changes and toothpaste are very useful products. But it is also important to have trustworthy sources of information that provide advice based on your interest. (This is why, as I wrote previously, DeepLearning.AI has never accepted payment for creating any course.)  

 To be clear, I do think society should find more ways to use tokens effectively. But to make sure our use is productive, consider these two practices:  

• Once an application scales beyond a basic level, instrument it to tell you how much it costs to run. For example, I know one of my applications costs about $0.50 per query and another about $3.00 per 10-minute conversation. Knowing these numbers helps me make quick back-of-the-envelope calculations to approach spending systematically.   
• When architecting your software, preserve optionality and avoid getting locked into any one model provider. For example, even when building an initial prototype, I often keep in mind the possibility of switching to an alternative model provider, including specifically open weight options — or perhaps even have my first prototype work with multiple LLM providers, so that I can switch models easily.   

This sort of advice is hard for any model provider to give. To be clear, I’m rooting for all of the frontier AI labs — they are building amazing technology that helps us all build better. At the same time, I will make sure DeepLearning.AI continues as a a neutral, trustworthy source of information that is committed, first and foremost, to the success of learners who want to build with AI.   

So, please productively use lots of tokens, but don’t tokenmaxx. And keep building!  

Andrew  

P.S. Something fun: If you want a brief diversion, check out this space shooter game that I built last weekend. (Gamepad support works better for me in Firefox than Chrome; but can also be played with keyboard and mouse.) There has been much debate about AI enabling new forms of creative expression through creating images, music and writing (or damaging them through slop). I think creative expression through games will be another exciting direction — and fun too!