Artificial Intelligence & The End Of Silicon Valley's Money Factory

This week, Sam Altman gave away the economics of the AI world. And they may not be all that attractive.

The fact that OpenAI is losing money isn’t surprising. WHY OpenAI is losing money on a $200/month subscription is what’s new to Silicon Valley.

To explain how the economics of AI have been upended in the last few months, we need to go through some history.

Most of the scaling that took place through the end of 2023 was related to pre-training.

Simply, developers were able to use more data and more compute to make AI models smarter. This is often called the foundation model and it’s the core of the models we see today.

This type of scaling is also why you heard talk about a model costing $1 billion and $100 billion models being just a few years away.

But the idea of an all-knowing $100 billion model has subsided as the gains from more data and more compute have leveled off. Now, almost every model is the same after pre-training because models have trained on all of the data the internet had available.

The second scaling step to scale models was reinforcement learning. This involves human feedback to tell the model if it got something right or not. Reinforcement training now also uses AI feedback and synthetic data generation, which makes post-training more scalable.

But like pre-training, there are limits to the scaling of post-training.

So, developers have found that test-time scaling, or “reasoning”, is the next scaling law for AI. Jensen Huang showed what this looks like in his CES keynote. (Notice, he doesn’t acknowledge the diminishing returns of scaling pre-training or scaling, the lines just go faint.)

“Reasoning” is somewhat akin to an AI system thinking the way humans do. It’s going down different paths and determining which one is the best. I think of it a little like this graphic where the "prompt” is on the far left and “Today” is the single output you see from the AI.

What you don’t see in this image is the compute going into creating these alternate branches that aren’t used. That compute is EXPENSIVE.

Think of it like this:

This means the model doesn’t need to be smarter, the prompt simply uses GPU power for longer than older techniques.

OpenAI doesn’t release the number of reasoning tokens (which we should use as a proxy for cost and GPU time) it uses for the o3 model yet, but data from Artificial Analysis shows reasoning tokens are a growing piece of the tokens used in AI based on the early reasoning model o1.

Prices for reasoning models have also reflected this higher compute cost associated with reasoning.

This will have a profound impact on the economics of artificial intelligence tools and businesses.

Being smarter will be more costly. And companies are already pricing services based on how much intelligence you want to pay for.

That may lead to pricing that looks more like electricity (pay for what you use) than today’s SaaS model (per seat). And the impact on Silicon Valley’s economy could be profound.

The internet of the past 20 years and the multi-trillion dollar tech companies we see today are built on two assumed truths.

This is why Facebook was trading for 100x earnings in 2014 and 2015. Investors knew it could eventually scale profits to the $55.5 billion in net income we see today.

It’s why Uber could raise tens of billions of dollars chasing ride-sharing.

It’s why winner-take-all markets are valuable. It’s why the Smiling Curve works.

Margins are 80% to 90% for most of these companies. The marginal cost isn’t important. What’s important is growing. Fast.

AI may not look like this. AI may look more like a utility or a manufacturing business.

The business model won’t be selling seats of software at an 80%-90% margin.

Pricing will be based on usage.

Usage will be based on the best model for the job and the ROI of the job being done.

There’s far less leverage on development and operating expenses than there was previously.

It looks like models are quickly commoditized and so are software developers building those models.

What happens if Silicon Valley’s margins go to 30%, or lower?

OpenAI’s gross margin on its most expensive product is negative, so this isn’t a wild projection.

The impact for investors may be massive improvements in efficiency and better products and features for business.

And lower profits that we need to judge based on earnings (the return of the P/E multiple) and not just growth (out with the P/S multiple).

Use Salesforce and Microsoft as examples.

Is Salesforce a better business charging $2 per conversation for Agentforce than it was charging $3,960 per year per employee for the Unlimited SaaS plan?

Maybe. But Salesforce (likely) won’t have an 80% margin on AI agents as it does on software seats.

Margins will compress and Salesforce will need to “make it up in volume”.

Hyperscalers like Microsoft are also spending tens of billions — $80 billion in 2025 — to build out infrastructure to serve AI models and run inference. But returns on capital are falling from tech-like levels to energy-like levels as the scale of the investment goes up.

This looks more like an oil & gas company or a utility, the opposite of what tech used to be with unlimited leverage and sky-high margins.

The new paradigm rarely looks like the old paradigm.

We don’t know exactly how AI businesses will develop, who will be disrupted, or what the economics look like. And I want to acknowledge that costs of inference are coming down rapidly and will continue to improve at an exponential pace.

My caution is not to assume the high profit margins and operating leverage Silicon Valley has benefitted from for 30 years will exist in the new paradigm.

AI may be more of a commodity product with lower gross margins and fewer massive winners. If that’s the case, we need to rethink how we value AI-focused companies across the value chain.

Disclaimer: Asymmetric Investing provides analysis and research but DOES NOT provide individual financial advice. Travis Hoium may have a position in some of the stocks mentioned. All content is for informational purposes only. Asymmetric Investing is not a registered investment, legal, or tax advisor or a broker/dealer. Trading any asset involves risk and could result in significant capital losses. Please, do your own research before acquiring stocks.