Yes, we're now at an inflection point where this is starting to become possible. gpt-4o mini costs $0.15 per 1 million input tokens, and $0.60 per 1 million output. This is cheap enough that it can, at least in some cases, be funded by ad impressions.
Of course, the implications here are mixed. If you want to build an ad-supported tool that actually helps people, that's great. But it also means it now makes clear financial sense to fill the web with AI-generated garbage with the assumption that that ad impressions will pay for it.
Another implication is that you're dependent on it remaining cheap enough. You risk VC money running out and them having to jack up prices, or them doing so because they managed to capture the whole market.
I don’t think it’s cheap because VC money is subsidizing losses on every token. It’s getting cheaper because models and infrastructure are becoming more efficient.
And I really don’t think any of the AI API providers can “capture the whole market”. There are at least 3 of ballpark equal capability, so I don’t see how dramatically raising prices is compatible with dominant market share.
Even if the inference is getting cheaper, all the frontier companies are running massive losses building and serving it. That has to come back eventually, that's just how capitalism works.
Just remember that Netflix didn't start really jacking up the price till after the other players entered the streaming war, when they were pioneers it was dirt cheap. The existence of Disney+ didn't stop them at all.
One major difference is that Netflix has a monopoly on much of its content, but LLMs are fungible.
Dell was never able to jack up its prices, even when it was dominant in the market, because people would just go to another vendor. I think OpenAI is closer to a Dell than a Netflix.
if new models won't require to have significantly better hardware then I think eventually many cases will be run on locally. Currently you have macbook pro with even 128GB RAM - of course almost nobody has it but those hardware will be mainstream common in ~5 years. You already can have ollama - you just need to have equivalent preinstalled by Microsoft/Apple/Google on their all new devices - again most likely this will happen in <5 years.
Haha, maybe Gen AI needs will finally drive up memory needs and the memory installed in most device, but i doubt it.
Apple still happily takes 1800$ for 8GB laptops. Wouldn't be surprised if that were their most sold SKU ans the laptops sold today will be around for quite some time.
Yes, but that's what's relevant, right? If I create a service, I don't really care if OpenAI makes a killing or is subsidizing my cost with venture capital, what matters to me is how much I pay to OpenAI, and how much revenue my service generates.
It's relevant as far as the providers of said LLM inference could always swoop in, undercut you and take your business if they feel like it. But they could do that anyways, no matter if they do it at a profit or not.
I'm kind of out of touch with current AI API pricing and ad revenues, so i'm curious how the economics work out.