1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random.
2. LLMs dramatically increases productivity for a limited subset of tasks
3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background.
LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup. If I need to write a small bit of glue code in a language I do not know, LLMs not only save me time, but they make it so I don't have to learn something that I'll likely never use again.
Fixing up existing large code bases? Productivity is at best a wash.
Setting up a scaffolding for a new website? LLMs are amazing at it.
Writing mocks for classes? LLMs know the details of using mock libraries really well and can get it done far faster than I can, especially since writing complex mocks is something I do a couple times a year and completely forget how to do in-between the rare times I am doing it.
Navigating a new code base? LLMs are ~70% great at this. If you've ever opened up an over-engineered WTF project, just finding where HTTP routes are defined at can be a problem. "Yo, Claude, where are the route endpoints in this project defined at? Where do the dependency injected functions for auth live?"
Right tool, right job. Stop using a hammer on nails.
> LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup. If I need to write a small bit of glue code in a language I do not know, LLMs not only save me time, but they make it so I don't have to learn something that I'll likely never use again.
I wax and wane on this one.
I've had the same feelings, but too often I've peaked behind the curtain, read the docs and got familiar with external dependencies and then realize whatever the LLM responds with paradoxically either wasn't following convention or tried to shoehorn your problem to fit code examples found online, used features inappropriately, took a long roundabout path to do something that can be done simply, etc.
It can feel like magic until you look too closely at it, and I worry that it'll make me complacent with the feeling of understanding without actually taking away an understanding.
Yeah LLMs get me _an_ answer far faster than I could find it myself, but it's often not correct. And then I have to verify it myself which was exactly the work I was trying to skip by using the LLM to start with.
If I have to manually verify every answer, I may as well read the docs myself.
Is it really that different from scrolling through Stack Overflow answers and rejecting the ones that aren't suitable? A lot of times you can tell it what specifically you didn't like about the solution and get another crack anyway (e.g., "let's iterate over the characters to do this rather than using a regex")
I wasn't using stack overflow before LLMs. It had already declined too much in quality and discouraged me from posting legitimate questions on there. I was more focused on reading documentation to gain a solid understanding. So for me, it's a much different experience.
It's incredible how quickly an LLM can answer. I've also crossed checked its responses with documentation before and discovered that it suggested implementing a deprecated feature that had a massive warning banner in the documentation that the LLM failed to indicate. I'm still a fan of reading documentation.
I've asked AIs for help in doing things in Salesforce, and the answers tend to be a 50/50 mix of correct steps and garbage. It's not hard to see why, because there's a lot of garbage on support sites, much of it because it's outdated. Garbage in, garbage out.
The difference is that if I go directly to the support site, there's a decent chance I can quickly spot and reject the garbage based on the date, the votes it's gotten, even the quality of the writing. AI doesn't include any of those clues; it mixes good and bad together and offers it up for me to pick apart through trial and error.
Lot of support sites are AI generated themselves now. Lenovo's support site is absolute garbage, it has entire articles dedicated to explaining how to use your laptop's screen as a monitor by setting the HDMI to INPUT instead of OUTPUT, something flatly not possible on any laptop. Also mentions VGA and DVI in 2025, something no laptop they sell features.
It doesn't completely solve this problem but definitely helps to have something like context7 MCP server running that `Copilot et al.` can reach dramatically reduces hallucinations for most tools. And additionally I've used Continue.dev VSCode along with manually specified docs and guides that you can selectively inject into your context. Both of those tactics make a huge difference in answer quality.
Personally, I don't trust LLMs to write code for me, generally speaking. That said, as of late I've been very pleased with the whole "shoehorn your problem to fit code examples found online" thing these LLMs do, in the very special case of massaging unix scripts, and where the "code examples found online" part seems to mostly amount to making fairly canonical reference to features documented in man pages that are plastered all over the web and haven't changed much in decades.
For questions that I know should have a straightforward answer, I think it beats searching Stackoverflow. Sure, I'll typically end up having to rewrite most of the script from scratch; however, if I give it a crude starting point of a half-functional script I've already got going, pairing that with very clear instructions on how I'd like it extended is usually enough to get it to write a proof of concept demonstration that contains enough insightful suggestions for me to spend some time reading about features in man pages I hadn't yet thought to use.
The biggest problem maybe is a propensity for these models to stick in every last fancy feature under the sun. It's fun to read about a GNU extension to awk that makes my script a couple lines shorter, but at best I'll take this as an educational aside than something I'd accept at the expense of portability.
Before accepting an answer I’ve started asking “is there a simpler more straight forward way of achieving that?” Ans most of the time it changes the whole thing it just wrote lol
> LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background.
What is this supposed busy work that can be done in the background unsupervised?
I think it's about time for the AI pushers to be absolutely clear about the actual specific tasks they are having success with. We're all getting a bit tired of the vagueness and hand waving.
Not really. This is a thread that attracts a certain subset of people so it's expected. Compare the comments here to the ones in a thread with a more neutral premise.
They are. There's no vagueness. It's rare to see people that still don't believe LLMs can do anything at all nowadays. Most other naysayers have moved on to the much more relevant question on whether the perceived productivity gains are real or not.
"LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup"
Just a random personal anecdote I wanted to throw out. I recently had to build some custom UI with Qt. I hadn't worked with Qt in a decade and barely remembered it. Seems like a perfect use case for AI to get me "up to speed" on the library, right? It's an incredibly well documented library with lots written on it, perfect fodder for an AI to process.
So, I gave it a good description of the widget I was trying to make, what I needed it to look like and how it should be behave, and behold, it spit out the specific widget subclass I should use and how I should be overriding certain methods to customize behavior. Wow, it worked exactly like promised.
So I implemented it like it suggested and was seemingly happy with the results. Went on with working on other parts of the project, dealing with Qt more and more here and there, gaining more and more experience with Qt over time.
A month or two later, after gaining more experience, I looked back at what AI had told me was the right approach on that widget and realized it was completely messed up. It had me subclassing the completely wrong type of widget. I didn't need to override methods and write code to force it to behave the way I wanted. I could instead just make use of a completely different widget that literally supported everything I needed already. I could just call a couple methods on it to customize it. My new version removes 80% of the code that AI had me write, and is simpler, more idiomatic, and actually makes more sense now.
So yeah, now any time I see people write about how "well, it's good for learning new libraries or new languages", I'll have that in the back of my mind. If you don't already know the library/language, you have zero idea whether what the AI teaching you is horrible or not. Whether there's a "right/better" way or not. You think it's helping you out when really you're likely just writing horrible code.
Just recently I was having trouble getting something to work with a large well documented framework library. Turned out the solution I needed was to switch to a similar but different API. But that's not what Claude told me. Instead it wanted me to override and rewrite a bunch of core library code. Fortunately I was able to recognize that the suggested solution was almost certainly bad and did some more digging to find the right answer, but I could easily see nightmarish code that solves immediate problems in terrible ways piling up fast in a vibe coded project.
I do find LLMs useful at times when working in unfamiliar areas, but there are a lot of pitfalls and newly created risks that come with it. I mostly work on large existing code bases and LLMs have very much been a mildly useful tool, still nice to have, but hardly the 100x productivity booster a lot of people are claiming.
This keeps happening to me. I keep coming across big files written during my Cursor hype period from 3 months ago and finding huge non DRY chunks and genuinely useless nonsense. Yes, I should have reviewed better, but it's a lot to wade through and it ostensibly "worked," as in, the UI looked as it should.
Today I asked Claude how to ignore Typescript type checking in some vendored js files in my project. It churned on this and ended up turning off type checking on all js files in my project and proudly declaring it a great success because the errors were gone. Hurray. If I knew nothing about my project then I would be none the wiser.
> Setting up a scaffolding for a new website? LLMs are amazing at it.
So amazing that every single stat showed by the author in the article has been flat at best, despite all being based on new development rather than work on existing code-bases.
Maybe the world has run out of interesting websites to create. That they are created faster doesn't necessarily imply they'll be created more frequently.
AI still fails to extrapolate. It can interpolate between things it’s trained on but that’s not exactly a new interesting product. If it truly could extrapolate at human-ish levels we would actually maybe have 10x more games and websites and whatnot
Working with LLMs has fundamentally changed how I approach documentation and development.
Traditional documentation has always been a challenge for me - figuring out where to start, what syntax conventions are being used, how pieces connect together. Good docs are notoriously hard to write, and even harder to navigate. But now, being able to query an LLM about specific tasks and get direct references to the relevant documentation sections has been a game-changer.
This realization led me to flip my approach entirely. I’ve started heavily documenting my own development process in markdown files - not for humans, but specifically for LLMs to consume. The key insight is thinking of LLMs as amnesiac junior engineers: they’re capable, but they need to be taught what to do every single time. Success comes from getting the right context into them.
Learning how to craft that context is becoming the critical skill.
It’s not about prompting tricks - it’s about building systematic ways to feed LLMs the information they need.
I’ve built up a library of commands and agents for my Claude Code installation inspired by AgentOS (https://github.com/buildermethods/agent-os) to help engineer the required context.
The tool is a stochastic parrot, you need to feed it the right context to get the right answer. It is very good at what it does but you need to use it to its strengths in order to get value from it.
I find people complaining about LLMs often expect vibe coding to be this magic tool that will build the app for you without thinking, which it unfortunately has been sold as, but the reality is more of a fancy prompt based IDE.
Recently I tried to scaffold a website with a well known coding agent.
It didn’t work. I asked a colleague. He had the same problem. Turned out it was using out of date setup instructions for a major tool that has changed post training.
After spending time fixing the problem, I realised (1) it would have been faster to do it myself and (2) I can no longer trust that tool to set anything up - what if it’s doing something else wrong?
You'd run into the same issues if you followed a slightly outdated medium tutorial on the topic!
React had this issue when they changed away from create react app. Basically every tutorial for years was instantly made obsolete, despite occupying all the top search results slots.
Forums where filled with confusion for quite awhile.
Now days most libraries seem to rely on "content producers" to document how to actually do things and don't bother with great docs filled with examples. In decades gone past, companies would hire actual authors to write real physical books on how to use new APIs.
LLMs just have a higher latency on updating their reference material.
"My shitty company's shitty app has an API for that! It's so good sales has decided to present it to the industry as though it should be incorrect not to use it"
If it can figure out where dependencies come from I'm going to have to look more into this. I really hate the way injection makes other people's code bases impenetrable. "The framework scans billions of lines of code to find the implementation, and so can you!"
1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random.
2. LLMs dramatically increases productivity for a limited subset of tasks
3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background.
LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup. If I need to write a small bit of glue code in a language I do not know, LLMs not only save me time, but they make it so I don't have to learn something that I'll likely never use again.
Fixing up existing large code bases? Productivity is at best a wash.
Setting up a scaffolding for a new website? LLMs are amazing at it.
Writing mocks for classes? LLMs know the details of using mock libraries really well and can get it done far faster than I can, especially since writing complex mocks is something I do a couple times a year and completely forget how to do in-between the rare times I am doing it.
Navigating a new code base? LLMs are ~70% great at this. If you've ever opened up an over-engineered WTF project, just finding where HTTP routes are defined at can be a problem. "Yo, Claude, where are the route endpoints in this project defined at? Where do the dependency injected functions for auth live?"
Right tool, right job. Stop using a hammer on nails.