"Yeah, but LLMs can't handle millions of lines of crufty old Java" is a guaranteed reply any time this topic comes up.
(That's not to say it isn't a valid argument.)
Short answer: LLMs are amazingly useful on large codebases, but they are useful in different ways. They aren't going to bang out a new feature perfectly first time, but in the right hands they can dramatically accelerate all sorts of important activities, such as:
- Understanding code. If code has no documentation, dumping it into an LLM can help a lot.
- Writing individual functions, classes and modules. You have to be good at software architecture and good at prompting to use them in this way - you take on the role of picking out the tasks that can be done independently of the rest of the code.
- Writing tests - again, if you have the skill and experience to prompt them in the right way.
Yes, LLMs are very useful, when used properly. But the linked change request is not a good example of how they would be used by a typical software developer. The linked pull request is essentially output from a compiler that's been hardcoded.
> Writing individual functions, classes and modules. You have to be good at software architecture and good at prompting to use them in this way - you take on the role of picking out the tasks that can be done independently of the rest of the code.
If you have enough skill and understanding to do this, it means you already have enough general software development experience and domain-specific experience and experience with a specific, existing codebase to be in rarefied air. It's like saying, oh yeah a wrench makes plumbing easy. You just need to turn the wrench, and 25 years of plumbing knowledge to know where to turn it.
> Writing tests - again, if you have the skill and experience to prompt them in the right way.
This is very true and more accessible to most developers, though my big fear is it encourages people to crap out low-value unit tests. Not that they don't love to do that already.
> If you have enough skill and understanding to do this, it means you already have enough general software development experience and domain-specific experience and experience with a specific, existing codebase to be in rarefied air.
Yes, exactly. That's why I keep saying that software developers shouldn't be afraid that they'll be out of a job because of LLMs.
> "Yeah, but LLMs can't handle millions of lines of crufty old Java" is a guaranteed reply any time this topic comes up.
That's not at all what the GP was saying, though:
> There's no need to implement half the code, realize the requirements weren't specified properly, and go back and have a conversation with the PM about it. Which is, you know, what developers actually do.
> This is the kind of stuff LLMs are great at, but it's not representative of a typical change request by Java Developer #1753 at Fortune 500 Enterprise Company #271.
(That's not to say it isn't a valid argument.)
Short answer: LLMs are amazingly useful on large codebases, but they are useful in different ways. They aren't going to bang out a new feature perfectly first time, but in the right hands they can dramatically accelerate all sorts of important activities, such as:
- Understanding code. If code has no documentation, dumping it into an LLM can help a lot.
- Writing individual functions, classes and modules. You have to be good at software architecture and good at prompting to use them in this way - you take on the role of picking out the tasks that can be done independently of the rest of the code.
- Writing tests - again, if you have the skill and experience to prompt them in the right way.