AI slows down some experienced software developers, study finds
Technology
121
Beiträge
52
Kommentatoren
8
Aufrufe
-
Code reviews seem like a good opportunity for an LLM. It seems like they would be good at it. I’ve actually spent the last half hour googling for tools.
I’ve spent literally a month in reviews for this junior guy on one stupid feature, and so much of it has been so basic. It’s a combination of him committing ai slop without understanding or vetting it, and being too junior to consider maintainability or usability. It would have saved so much of my time if ai could have done some of those review cycles without me
This has been solved for over a decade. Include a linter and static analysis stage in the build pipeline. No code review until the checkbox goes green (or the developer has a specific argument for why a particular finding is a false positive)
-
I’m seeing exactly the opposite. It used to be the junior engineers understood they had a lot to learn. However with AI they confidently try entirely wrong changes. They don’t understand how to tell when the ai goes down the wrong path, don’t know how to fix it, and it takes me longer to fix.
So far ai overall creates more mess faster.
Don’t get me wrong, it can be a useful tool you have to think of it like autocomplete or internet search. Just like those tools it provides results but the human needs judgement and needs to figure out how to apply the appropriate results.
My company wants metrics on how much time we’re saving with ai, but
- I have to spend more time helping the junior guys out of the holes dug by ai, making it net negative
- it’s just another tool. There’s not really a defined task or set time. If you had to answer how much time autocomplete saved you, could you provide any sort of meaningful answer?
I've always had problems with junior engineers (self included) going down bad paths, since before there was Google search - let alone AI.
So far ai overall creates more mess faster.
Maybe it is moving faster, maybe they do bother the senior engineers less often than they used to, but for throw-away proof of concept and similar stuff, the juniors+AI are getting better than the juniors without senior support used to be... Is that a good direction? No. When the seniors are over-tasked with "Priority 1" deadlines (nothing new) does this mean the juniors can get a little further on their own and some of them learn from their own mistakes? I think so.
Where I started, it was actually the case that the PhD senior engineers needed help from me fresh out of school - maybe that was a rare circumstance, but the shop was trying to use cutting edge stuff that I knew more about than the seniors. Basically, everything in 1991 was cutting edge and it made the difference between getting something that worked or having nothing if you didn't use it. My mentor was expert in another field, so we were complimentary that way.
My company (now) wants metrics on a lot of things, but they also understand how meaningless those metrics can be.
I have to spend more time helping the junior guys out of the holes dug by ai, making it net negative
Shame. There was a time that people dug out of their own messes, I think you learn more, faster that way. Still, I agree - since 2005 I have spend a lot of time taking piles of Matlab, Fortran, Python that have been developed over years to reach critical mass - add anything else to them and they'll go BOOM - and translating those into commercially salable / maintainable / extensible Qt/C++ apps, and I don't think I ever had one "mentee" through that process who was learning how to follow in my footsteps, the organizations were always just interested in having one thing they could sell, not really a team that could build more like it in the future.
it’s just another tool.
Yep.
If you had to answer how much time autocomplete saved you, could you provide any sort of meaningful answer?
Speaking of meaningless metrics, how many people ask you for Lines Of Code counts, even today?___
-
Adding context is “knowing more” for a computer program.
Maybe it’s different in VS code vs regular VS, because I never get issues like what you’re describing in VS. Haven’t really used it in VS Code.
Are you using agent mode?
-
You shouldn't think of "AI" as intelligent and ask it to do something tricky. The boring stuff that's mostly just typing, that's what you get the LLMs to do. "Make a DTO for this table <paste>" "Interface for this JSON <paste>"
I just have a bunch of conversations going where I can paste stuff into and it will generate basic code. Then it's just connecting things up, but that's the fun part anyway.
Most ides do the boring stuff with templates and code generation for like a decade so that's not so helpful to me either but if it works for you.
-
I have asked questions, had conversations for company and generated images for role playing with AI.
I've been happy with it, so far.
That's kind of outside the software development discussion but glad you're enjoying it.
-
I've always had problems with junior engineers (self included) going down bad paths, since before there was Google search - let alone AI.
So far ai overall creates more mess faster.
Maybe it is moving faster, maybe they do bother the senior engineers less often than they used to, but for throw-away proof of concept and similar stuff, the juniors+AI are getting better than the juniors without senior support used to be... Is that a good direction? No. When the seniors are over-tasked with "Priority 1" deadlines (nothing new) does this mean the juniors can get a little further on their own and some of them learn from their own mistakes? I think so.
Where I started, it was actually the case that the PhD senior engineers needed help from me fresh out of school - maybe that was a rare circumstance, but the shop was trying to use cutting edge stuff that I knew more about than the seniors. Basically, everything in 1991 was cutting edge and it made the difference between getting something that worked or having nothing if you didn't use it. My mentor was expert in another field, so we were complimentary that way.
My company (now) wants metrics on a lot of things, but they also understand how meaningless those metrics can be.
I have to spend more time helping the junior guys out of the holes dug by ai, making it net negative
Shame. There was a time that people dug out of their own messes, I think you learn more, faster that way. Still, I agree - since 2005 I have spend a lot of time taking piles of Matlab, Fortran, Python that have been developed over years to reach critical mass - add anything else to them and they'll go BOOM - and translating those into commercially salable / maintainable / extensible Qt/C++ apps, and I don't think I ever had one "mentee" through that process who was learning how to follow in my footsteps, the organizations were always just interested in having one thing they could sell, not really a team that could build more like it in the future.
it’s just another tool.
Yep.
If you had to answer how much time autocomplete saved you, could you provide any sort of meaningful answer?
Speaking of meaningless metrics, how many people ask you for Lines Of Code counts, even today?___
Shame. There was a time that people dug out of their own messes, I think you learn more, faster
Yes, that’s how we became senior guys. But when you have deadlines that you’re both on the hook for and they’re just floundering, you can only give them so much opportunity. I’ve had too many arguments with management about letting them merge and I’m not letting that ruin my code base
Speaking of meaningless metrics, how many people ask you for Lines Of Code counts, even today?
We have a new VP collecting metrics on everyone, including lines of code, number of merge requests, times per day using ai, days per week in the office vs at home
-
This has been solved for over a decade. Include a linter and static analysis stage in the build pipeline. No code review until the checkbox goes green (or the developer has a specific argument for why a particular finding is a false positive)
Not really.
Linter in the build pipeline is generally not useful because most people won’t give results time or priority. You usually can’t fail the build for lint issues so all it does is fill logs. I usually configure a linter and prettifier in a precommit hook, to shift that left. People are more willing to fix their code in small pieces as they try to commit.
But this is also why SonarQube is a key tool. The scanners are lint-like, and you can even import some lint output. But the important part is it tries to prioritize them, score them, and enforce a quality gate based on them. I usually can’t fail a build for lint errors but SonarQube can if there are too many or too priority, or if they are security related.
But this is not the same as a code review. If an ai can use the code base as context, it should be able to add checks for consistency and maintainability similar to the rest of the code. For example I had a junior developer blindly follow the AI to use a different mocking framework than the rest of the code, for no reason other than it may have been more common in the training data. A code review ai should be able to notice that. Maybe this is too advanced for current ai, but the same guy blindly followed ai to add classes that already existed. They were just different enough that SonarQube didn’t flag is as duplicate code but ai ought to be able to summarize functionality and realize they were the same. Or I wonder if ai could do code organization? Junior guys spew classes and methods everywhere without any effort in organizing like with like, so someone can maintain it all. Or how about style? I hope yo never revisit style wars but when you’re modifying code you really need to follow style and naming of what’s already there. Maybe ai code review can pick up on that
-
That's kind of outside the software development discussion but glad you're enjoying it.
As a developer
- I can jot down a bunch of notes and have ai turn it into a reasonable presentation or documentation or proposal
- zoom has an ai agent which is pretty good about summarizing a meeting. It usually just needs minor corrections and you can send it out much faster than taking notes
- for coding I mostly use ai like autocomplete. Sometimes it’s able to autocomplete entire code blocks
- for something new I might have ai generate a class or something, and use it as a first draft where I then make it work
-
Experienced software developer, here. "AI" is useful to me in some contexts. Specifically when I want to scaffold out a completely new application (so I'm not worried about clobbering existing code) and I don't want to do it by hand, it saves me time.
And... that's about it. It sucks at code review, and will break shit in your repo if you let it.
On that last note, important thing they left out here being general news reporting tech stuff is that this was specifically bug fixing tasks. It can typically only provide the broadest of advice on that, and it’s largely incapable of tackling problems holistically when you often need to be thinking big picture while tackling a bug.
Interesting that the AI devs thought they were being quicker though.
-
Shame. There was a time that people dug out of their own messes, I think you learn more, faster
Yes, that’s how we became senior guys. But when you have deadlines that you’re both on the hook for and they’re just floundering, you can only give them so much opportunity. I’ve had too many arguments with management about letting them merge and I’m not letting that ruin my code base
Speaking of meaningless metrics, how many people ask you for Lines Of Code counts, even today?
We have a new VP collecting metrics on everyone, including lines of code, number of merge requests, times per day using ai, days per week in the office vs at home
I’ve had too many arguments with management about letting them merge and I’m not letting that ruin my code base
I guess I'm lucky, before here I always had 100% control of the code I was responsible for. Here (last 12 years) we have a big team, but nobody merges to master/main without a review and screwups in the section of the repository I am primarily responsible for have been rare.
We have a new VP collecting metrics on everyone, including lines of code, number of merge requests, times per day using ai, days per week in the office vs at home
I have been getting actively recruited - six figures+ - for multiple openings right here in town (not a huge market here, either...) this may be the time...
-
Experienced software developer, here. "AI" is useful to me in some contexts. Specifically when I want to scaffold out a completely new application (so I'm not worried about clobbering existing code) and I don't want to do it by hand, it saves me time.
And... that's about it. It sucks at code review, and will break shit in your repo if you let it.
Same. I also like it for basic research and helping with syntax for obscure SQL queries, but coding hasn't worked very well. One of my less technical coworkers tried to vibe code something and it didn't work well. Maybe it would do okay on something routine, but generally speaking it would probably be better to use a library for that anyway.
-
I’ve had too many arguments with management about letting them merge and I’m not letting that ruin my code base
I guess I'm lucky, before here I always had 100% control of the code I was responsible for. Here (last 12 years) we have a big team, but nobody merges to master/main without a review and screwups in the section of the repository I am primarily responsible for have been rare.
We have a new VP collecting metrics on everyone, including lines of code, number of merge requests, times per day using ai, days per week in the office vs at home
I have been getting actively recruited - six figures+ - for multiple openings right here in town (not a huge market here, either...) this may be the time...
Interesting idea… we actually have a plan to go public in a couple years and I’m holding a few options, but the economy is hitting us like everyone else. I’m no longer optimistic we can reach the numbers for those options to activate
-
"Using something that you're not experienced with and haven't yet worked out how to best integrate into your workflow slows some people down"
Wow, what an insight! More at 8!
As I said on this article when it was posted to another instance:
AI is a tool to use. Like with all tools, there are right ways and wrong ways and inefficient ways and all other ways to use them. You can’t say that they slow people down as a whole just because some people get slowed down.
I use github copilot as it does speed things up. but you have to keep tight reins on it, but it does work because it sees my code, sees what i'm trying to do, etc. So a good chunk of the time it helps.
Now something like Claude AI? yeah...no. Claude doesn't know how to say "I don't know." it simply doesn't. it NEEDS to provide you a solution even if the vast majority of time it's one it just creates off the top of it's head. it simply cannot say it doesn't know. and it'll get desperate. it'll provided you with libraries or repos that have been orphaned for years. It'll make stuff up saying that something can magically do what you're really looking for when in truth it can't do that thing at all and was never intended to. As long as it sounds good to Claude, then it must be true. It's a shit AI and absolutely worthless. I don't even trust it to simply build out a framework for something.
Chatgpt? it's good for providing me with place holder content or bouncing ideas off of. that's it. like you said they're simply tools not anything that should be replacements for...well...anything.
Could they slow people down? I think so but that person has to be an absolute moron or have absolutely zero experience with whatever their trying to get the AI to do.
-
Fun how the article concludes that AI tools are still good anyway, actually.
This AI hype is a sickness
Upper management said a while back we need to use copilot. So far just used Deepseek to fill out the stupid forms that management keep getting us to fill out
-
As a developer
- I can jot down a bunch of notes and have ai turn it into a reasonable presentation or documentation or proposal
- zoom has an ai agent which is pretty good about summarizing a meeting. It usually just needs minor corrections and you can send it out much faster than taking notes
- for coding I mostly use ai like autocomplete. Sometimes it’s able to autocomplete entire code blocks
- for something new I might have ai generate a class or something, and use it as a first draft where I then make it work
I’ve had success with:
- dumping email threads into it to generate user stories,
- generating requirements documentation templates so that everyone has to fill out the exact details needed to make the project a success
- generating quick one-off scripts
- suggesting a consistent way to refactor a block of code (I’m not crazy enough to let it actually do all the refactoring)
- summarize the work done for a slide deck and generate appropriate infographics
Essentially, all the stuff that I’d need to review anyway, but use of AI means that actually generating the content can be done in a consistent manner that I don’t have to think about. I don’t let it create anything, just transform things in blocks that I can quickly review for correctness and appropriateness. Kind of like getting a junior programmer to do something for me.
-
That's happening right now. I have a few friends who are looking for entry-level jobs and they find none.
It really sucks.
That said, the future lack of developers is a corporate problem, not a problem for developers. For us it just means that we'll earn a lot more in a few years.
Only if you are able to get that experience though.
-
They wanted someone with experience, who can hit the ground running, but didn't want to pay for it, either with cash or time.
- cheap
- quick
- experience
You can only pick two.
I know which 2 I am in my current role and I don't know if I like it or not. At least I don't have to try very hard.
-
Not really.
Linter in the build pipeline is generally not useful because most people won’t give results time or priority. You usually can’t fail the build for lint issues so all it does is fill logs. I usually configure a linter and prettifier in a precommit hook, to shift that left. People are more willing to fix their code in small pieces as they try to commit.
But this is also why SonarQube is a key tool. The scanners are lint-like, and you can even import some lint output. But the important part is it tries to prioritize them, score them, and enforce a quality gate based on them. I usually can’t fail a build for lint errors but SonarQube can if there are too many or too priority, or if they are security related.
But this is not the same as a code review. If an ai can use the code base as context, it should be able to add checks for consistency and maintainability similar to the rest of the code. For example I had a junior developer blindly follow the AI to use a different mocking framework than the rest of the code, for no reason other than it may have been more common in the training data. A code review ai should be able to notice that. Maybe this is too advanced for current ai, but the same guy blindly followed ai to add classes that already existed. They were just different enough that SonarQube didn’t flag is as duplicate code but ai ought to be able to summarize functionality and realize they were the same. Or I wonder if ai could do code organization? Junior guys spew classes and methods everywhere without any effort in organizing like with like, so someone can maintain it all. Or how about style? I hope yo never revisit style wars but when you’re modifying code you really need to follow style and naming of what’s already there. Maybe ai code review can pick up on that
Yeah, I’ve added AI to my review process. Sure, things take a bit longer, but the end result has been reviewed by me AND compared against a large body of code in the training data.
It regularly catches stuff I miss or ignore on a first review based on ignoring context that shouldn’t matter (eg, how reliable the person is who wrote the code).
-
Who says I made my webapp with ChatGPT in an afternoon?
I built it iteratively using ChatGPT, much like any other application. I started with the scaffolding and then slowly added more and more features over time, just like I would have done had I not used any AI at all.
Like everybody knows, Rome wasn't built in a day.
So you treated it like a junior developer and did a thorough review of its output.
I think the only disagreement here is on the semantics.
-
It may also be self fulfilling. Our new ceo said all upcoming projects must save 15% using ai, and while we’re still hiring it’s only in India.
So 6 months from now we will have status reports talking about how we saved 15% in every project
And those status reports will be generated by AI, because that’s where the real savings is.
