Egregoros

@hfalcke

LLMs will stay and people will use it.

There's a big assumption here, that comes from the previous one:

It is not that AI does not work, as often said here, it does work for many applications.

That's not the question. The first-order question is: do they deliver more value than they cost. And the answers to that are skewed by the fact that the current services are heavily subsidised and customers are paying 10-50% of the minimum viable price. So, if people actually have to pay the real price, how many of those use cases still exist?

The follow-on to that is, given that increasing the price by a factor of 2-10x will significantly reduce the demand, what will the resulting price be? Inference costs won't change much, but the price to users is the inference costs plus the training costs amortised across users (plus profit). Fewer users means that the training costs are amortised across fewer users and so the per-user costs go up. So how many use cases are still economically feasible if the prices go up by a factor of 20x?

But that's just the first order. The second order is: how affordable is it when you factor in the cost of externalities. Current LLMs are trained by doing something that, in any other context, would be a clear-cut case of large-scale copyright infringement. If LLM vendors had to actually pay for licenses for the things that they are ingesting in training, the cost of training would go up by at least a factor of 10x.

So there's a huge assumption in 'LLMs will stay'. The companies that train them do so on the assumption that they'll eventually be able to recoup that (huge) investment. If you don't train new ones, they become stale quickly. To be useful, they need to include recent events. For software, an LLM that doesn't know about new APIs and language features and does know about deprecated / removed ones is useless. For other uses, the shifts may be slower or faster. But training isn't a one-shot cost as a result, you're constantly needing to do retraining. And that's what needs the massive datacentres full of GPUs.

So, if you want to claim LLMs are here to stay, you need to answer a simple question: Who is going to pay for the training? If no one is, then how will LLMs remain useful?

The entire industry is spending enormous amounts of money for tiny returns. How do you expect this to shift to actually making money? Note that the numbers in this page look much better than they are, because all of that $516 B in revenue for NVIDIA is spending from other companies (which must be passed on to customers for anyone else to become profitable).

@david_chisnall

Of course they're useful. You don't need to retrain LLMs so that they have internalized knowledge for them to be useful. The point of the LLMs is that they're able to go outside their training data and look up API descriptions etc and then perform work.

Even the existing open weight models we have today are enough for people to get good usage out of purely local AI on regular graphics cards for many years to come.

New foundation models can then be trained by companies who sell local datacenter deployment subscriptions. When you don't need to satisfy VCs and a quarterly market you can take much longer (=cheaper) to train them.

@hfalcke