He is pointing out that the current costs to create the data centres means you will never be able to make a profit to cover those costs. $800 Billion just to cover the interest.
OpenAI is already haemorrhaging money and the space data centres has already been debunked. There is even a recent paper that points out that LLMs will never become AGI.
The article also finishes out with some other experts giving the same results.
It's already quite useful. While not all AI service providers are profitable, I've worked on projects that saved a lot of money for the company - a lot more than it cost us running the servers.
took me a while but i read it. thought it was actually a pretty good and well researched paper that does a good job rationalizing its thesis. thanks for sharing
Is this AI paper written by a reputable subject matter expert? It seems to be written by a physicist and also be the only academic work by this author in English
How many articles on this topic do we imagine there are? Thousands? Hundreds of thousands? It is hopeless to read every one by any author, no matter how unrelated to the domain, and judge them individually on their merits. Being a subject domain expert is not a perfect measure of paper quality but it's the only feasible way to make a first pass at filtering.
Even if I did read it, I have no hope of understanding if it has made a fundamental mistake because I don't have the subject matter expertise either.
(I imagine it has made a fundamental mistake anyway: for LLMs to be useful progress toward AGI they don't have to be a feasible way to create AGI by themselves. Innovation very often involves stepping through technologies that end up only being a component of the final solution, or inspiration for the final solution. This was always going to be an issue with trying to prove a negative.)
The core piece as quoted from the abstract: "AGI predictions fail not from
insufficient compute, but from fundamental misunderstanding of what intelligence demands structurally."
Then goes in detail as to what that is and why LLMs don't fit that. There are plenty other similar papers out there.
Sry to say but the fact that you argue with LLMs never become AGI, you are not up-to-date.
People don't assume LLM will be AGI, people assume that World Models will lead us to AGI.
I personally never asumed LLM will become AGI, i always assumed that LLM broke the dam for investment and research into massivce scale compute ML learning and LLMs are very very good in showing were the future goes because they are already so crazy good that people can now imagine a future were AGI exists.
And that was very clear already when / as soon as GPT-3 came out.
The next big thing will probably be either a LOT more RL or self propelling ai architecture discovery. Both need massive compute to work well but then will potentially provide even faster progress as soon as humans are out of the loop.
AlphaGo Zero doesn't need much human intervention at all.
Regarding world models: All the big ones. LeCun, Demis Hassabis, Fei-Fei Li too. And they are all working on it.
LLMs will definitly play some type of role in AGI. After all you can ask an LLM already a lot of basic things like 'what are common tasks to make a tea'. A type of guide, long term fact memory or whatever this can be called.
> AlphaGo Zero doesn't need much human intervention at all
You should research it and not just read news articles. RL did not work and required human intervention numerous times before it got close to what it is now.
Are OpenAI or Anthropic et al seriously building towards “world models”? I haven’t seen any real evidence of that. It seems more like they are all in on milking LLMs for all they are worth.
He is pointing out that the current costs to create the data centres means you will never be able to make a profit to cover those costs. $800 Billion just to cover the interest.
OpenAI is already haemorrhaging money and the space data centres has already been debunked. There is even a recent paper that points out that LLMs will never become AGI.
The article also finishes out with some other experts giving the same results.
[edit] Fixed $80 to $800