Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Definitely engineering. It’s not entirely wrong to say that the two reasons it took us until 2022 to make a ChatGPT are 1) the computing power needed and 2) the size of the training corpus needed. The same goes for other generative AI – it took a corpus of a couple billion images to train a Stable Diffusion model.


You may argue that it took a leap of insight to get to transformer models, though.


That was not an innovation of ChatGPT.


It's pretty clear that ChatGPT is being used here as a synecdoche for recent LLMs, and transformer LLMs in particular.


"Transformer is All You Need" is from 6 years ago. It isn't an advancement that happened last year.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: