Skip to content

AI slows down some experienced software developers, study finds

Technology
127 54 31
  • They wanted someone with experience, who can hit the ground running, but didn't want to pay for it, either with cash or time.

    • cheap
    • quick
    • experience

    You can only pick two.

    I know which 2 I am in my current role and I don't know if I like it or not. At least I don't have to try very hard.

  • Not really.

    Linter in the build pipeline is generally not useful because most people won’t give results time or priority. You usually can’t fail the build for lint issues so all it does is fill logs. I usually configure a linter and prettifier in a precommit hook, to shift that left. People are more willing to fix their code in small pieces as they try to commit.

    But this is also why SonarQube is a key tool. The scanners are lint-like, and you can even import some lint output. But the important part is it tries to prioritize them, score them, and enforce a quality gate based on them. I usually can’t fail a build for lint errors but SonarQube can if there are too many or too priority, or if they are security related.

    But this is not the same as a code review. If an ai can use the code base as context, it should be able to add checks for consistency and maintainability similar to the rest of the code. For example I had a junior developer blindly follow the AI to use a different mocking framework than the rest of the code, for no reason other than it may have been more common in the training data. A code review ai should be able to notice that. Maybe this is too advanced for current ai, but the same guy blindly followed ai to add classes that already existed. They were just different enough that SonarQube didn’t flag is as duplicate code but ai ought to be able to summarize functionality and realize they were the same. Or I wonder if ai could do code organization? Junior guys spew classes and methods everywhere without any effort in organizing like with like, so someone can maintain it all. Or how about style? I hope yo never revisit style wars but when you’re modifying code you really need to follow style and naming of what’s already there. Maybe ai code review can pick up on that

    Yeah, I’ve added AI to my review process. Sure, things take a bit longer, but the end result has been reviewed by me AND compared against a large body of code in the training data.

    It regularly catches stuff I miss or ignore on a first review based on ignoring context that shouldn’t matter (eg, how reliable the person is who wrote the code).

  • Who says I made my webapp with ChatGPT in an afternoon?

    I built it iteratively using ChatGPT, much like any other application. I started with the scaffolding and then slowly added more and more features over time, just like I would have done had I not used any AI at all.

    Like everybody knows, Rome wasn't built in a day.

    So you treated it like a junior developer and did a thorough review of its output.

    I think the only disagreement here is on the semantics.

  • It may also be self fulfilling. Our new ceo said all upcoming projects must save 15% using ai, and while we’re still hiring it’s only in India.

    So 6 months from now we will have status reports talking about how we saved 15% in every project

    And those status reports will be generated by AI, because that’s where the real savings is.

  • Explain this too me AI. Reads back exactly what's on the screen including comments somehow with more words but less information
    Ok....

    Ok, this is tricky. AI, can you do this refactoring so I don't have to keep track of everything. No... Thats all wrong... Yeah I know it's complicated, that's why I wanted it refactored. No you can't do that... fuck now I can either toss all your changes and do it myself or spend the next 3 hours rewriting it.

    Yeah I struggle to find how anyone finds this garbage useful.

    If you give it the right task, it’s super helpful. But you can’t ask it to write anything with any real complexity.

    Where it thrives is being given pseudo code for something simple and asking for the specific language code for it. Or translate between two languages.

    That’s… about it. And even that it fucks up.

  • If you give it the right task, it’s super helpful. But you can’t ask it to write anything with any real complexity.

    Where it thrives is being given pseudo code for something simple and asking for the specific language code for it. Or translate between two languages.

    That’s… about it. And even that it fucks up.

    I bet it slows down the idiot software developers more than anything.

    Everything can be broken into smaller easily defined chunks and for that AI is amazing.

    Give me a function in Python that if I provide it a string of XYZ it will provide me an array of ABC.

    The trick is knowing how it fits in your larger codebase. That's where your developer skill is. It's no different now than it was when coding was offshored to India. We replaced Ravinder with ChatGPT.

    Edit - what I hate about AI is the blatant lying. I asked it for some ServiceNow code Friday and it told me to use the sys_audit_report table which doesn't exist. I told it so and then it gave me the sys_audit table.

    The future will be those who are smart enough to know when AI is lying and know how to fix it when it is. Ideally you are using AI for code you can do, you just don't want to. At least that's my experience. In that, it's invaluable.

  • This post did not contain any content.

    I work for an adtech company and im pretty much the only developer for the javascript library that runs on client sites and shows our ads. I dont use AI at all because it keeps generating crap

  • So you treated it like a junior developer and did a thorough review of its output.

    I think the only disagreement here is on the semantics.

    Sure, but it still built out a full-featured webapp, not just a bit of greenfielding here or there.

  • Explain this too me AI. Reads back exactly what's on the screen including comments somehow with more words but less information
    Ok....

    Ok, this is tricky. AI, can you do this refactoring so I don't have to keep track of everything. No... Thats all wrong... Yeah I know it's complicated, that's why I wanted it refactored. No you can't do that... fuck now I can either toss all your changes and do it myself or spend the next 3 hours rewriting it.

    Yeah I struggle to find how anyone finds this garbage useful.

    This was the case a year or two ago but now if you have an MCP server for docs and your project and goals outlined properly it's pretty good.

  • Same. I also like it for basic research and helping with syntax for obscure SQL queries, but coding hasn't worked very well. One of my less technical coworkers tried to vibe code something and it didn't work well. Maybe it would do okay on something routine, but generally speaking it would probably be better to use a library for that anyway.

    I actively hate the term "vibe coding." The fact is, while using an LLM for certain tasks is helpful, trying to build out an entire, production-ready application just by prompts is a huge waste of time and is guaranteed to produce garbage code.

    At some point, people like your coworker are going to have to look at the code and work on it, and if they don't know what they're doing, they'll fail.

    I commend them for giving it a shot, but I also commend them for recognizing it wasn't working.

  • I like the saying that LLMs are good at stuff you don’t know. That’s about it.

    FreedomAdvocate is right, IMO the best use case of ai is things you have an understanding of, but need some assistance. You need to understand enough to catch atleast impactful errors by the llm

  • Fun how the article concludes that AI tools are still good anyway, actually.

    This AI hype is a sickness

    LLMs are very good In the correct context, forcing people to use them for things they are already great at is not the correct context.

  • That's still not actually knowing anything. It's just temporarily adding more context to its model.

    And it's always very temporary. I have a yarn project I'm working on right now, and I used Copilot in VS Code in agent mode to scaffold it as an experiment. One of the refinements I included in the prompt file to build it is reminders throughout for things it wouldn't need reminding of if it actually "knew" the repo.

    • I had to constantly remind it that it's a yarn project, otherwise it would inevitably start trying to use NPM as it progressed through the prompt.
    • For some reason, when it's in agent mode and it makes a mistake, it wants to delete files it has fucked up, which always requires human intervention, so I peppered the prompt with reminders not to do that, but to blank the file out and start over in it.
    • The frontend of the project uses TailwindCSS. It could not remember not to keep trying to downgrade its configuration to an earlier version instead of using the current one, so I wrote the entire configuration for it by hand and inserted it into the prompt file. If I let it try to build the configuration itself, it would inevitably fuck it up and then say something completely false, like, "The version of TailwindCSS we're using is still in beta, let me try downgrading to the previous version."

    I'm not saying it wasn't helpful. It probably cut 20% off the time it would have taken me to scaffold out the app myself, which is significant. But it certainly couldn't keep track of the context provided by the repo, even though it was creating that context itself.

    Working with Copilot is like working with a very talented and fast junior developer whose methamphetamine addiction has been getting the better of it lately, and who has early onset dementia or a brain injury that destroyed their short-term memory.

    From the article: "Even after completing the tasks with AI, the developers believed that they had decreased task times by 20%. But the study found that using AI did the opposite: it increased task completion time by 19%."

    I'm not saying you didn't save time, but it's remarkable that the research shows that this perception can be false.

  • Most ides do the boring stuff with templates and code generation for like a decade so that's not so helpful to me either but if it works for you.

    Yeah but I find code generation stuff I've used in the past takes a significant amount of configuration, and will often generate a bunch of code I don't want it to, and not in the way I want it. Many times it's more trouble than it's worth. Having an LLM do it means I don't have to deal with configuring anything and it's generating code for the specific thing I want it to so I can quickly validate it did things right and make any additions I want because it's only generating the thing I'm working on that moment. Also it's the same tool for the various languages I'm using so that adds more convenience.

    Yeah if you have your IDE setup with tools to analyze the datasource and does what you want it to do, that may work better for you. But with the number of DBs I deal with, I'd be spending more time setting up code generation than actually writing code.

  • This was the case a year or two ago but now if you have an MCP server for docs and your project and goals outlined properly it's pretty good.

    Not to sound like one of the ads or articles but I vice coded an iOS app in like 6 hours, it's not so complex I don't understand it, it's multifeatured, I learned a LOT and got a useful thing instead of doing a tutorial with sample project. I don't regret having that tool. I do regret the lack of any control and oversight and public ownership of this technology but that's the timeline we're on, let's not pretend it's gay space communism (sigh) but, since AI is probably driving my medical care decisions at the insurance company level, might as well get something to play with.

  • They're also bad at that though, because if you don't know that stuff then you don't know if what it's telling you is right or wrong.

    I...think that's their point. The only reason it seems good is because you're bad and can't spot that is bad, too.

  • I have limited AI experience, but so far that's what it means to me as well: helpful in very limited circumstances.

    Mostly, I find it useful for "speaking new languages" - if I try to use AI to "help" with the stuff I have been doing daily for the past 20 years? Yeah, it's just slowing me down.

    and the only reason it's not slowing you down on other things is that you don't know enough about those other things to recognize all the stuff you need to fix

  • I actively hate the term "vibe coding." The fact is, while using an LLM for certain tasks is helpful, trying to build out an entire, production-ready application just by prompts is a huge waste of time and is guaranteed to produce garbage code.

    At some point, people like your coworker are going to have to look at the code and work on it, and if they don't know what they're doing, they'll fail.

    I commend them for giving it a shot, but I also commend them for recognizing it wasn't working.

    I think the term pretty accurately describes what is going on: they don't know how to code, but they do know what correct output for a given input looks like, so they iterate with the LLM until they get what they want. The coding here is based on vibes (does the output feel correct?) instead of logic.

    I don't think there's any problem with the term, the problem is with what's going on.

  • This post did not contain any content.

    Yeah... It's useful for summarizing searches but I'm tempted to disable it in VSCode because it's been getting in the way more than helping lately.

  • Interesting idea… we actually have a plan to go public in a couple years and I’m holding a few options, but the economy is hitting us like everyone else. I’m no longer optimistic we can reach the numbers for those options to activate

    Always keep an open mind. I stuck around in my first job until the sad and pathetic end for everyone, and when I finally did start looking the economy was worse than it had been when the writing was first on the wall.

  • 32 Stimmen
    6 Beiträge
    46 Aufrufe
    G
    Yes. I can't imagine that they will go after individuals. Businesses can't be so cavalier. But if creators don't pay the extra cost to make their models compliant with EU law, then they can't be used in the EU anyway. So it probably doesn't matter much. The Llama models with vision have the no-EU clause. It's because Meta wasn't allowed to train on European's data because of GDPR. The pure LLMs are fine. They might even be compliant, but we'll have to see what the courts think.
  • best porn sites

    Technology technology
    1
    0 Stimmen
    1 Beiträge
    12 Aufrufe
    Niemand hat geantwortet
  • 216 Stimmen
    13 Beiträge
    57 Aufrufe
    J
    It’s DEI’s fault!
  • 1k Stimmen
    95 Beiträge
    19 Aufrufe
    G
    Obviously the law must be simple enough to follow so that for Jim’s furniture shop is not a problem nor a too high cost to respect it, but it must be clear that if you break it you can cease to exist as company. I think this may be the root of our disagreement, I do not believe that there is any law making body today that is capable of an elegantly simple law. I could be too naive, but I think it is possible. We also definitely have a difference on opinion when it comes to the severity of the infraction, in my mind, while privacy is important, it should not have the same level of punishments associated with it when compared to something on the level of poisoning water ways; I think that a privacy law should hurt but be able to be learned from while in the poison case it should result in the bankruptcy of a company. The severity is directly proportional to the number of people affected. If you violate the privacy of 200 million people is the same that you poison the water of 10 people. And while with the poisoning scenario it could be better to jail the responsible people (for a very, very long time) and let the company survive to clean the water, once your privacy is violated there is no way back, a company could not fix it. The issue we find ourselves with today is that the aggregate of all privacy breaches makes it harmful to the people, but with a sizeable enough fine, I find it hard to believe that there would be major or lasting damage. So how much money your privacy it's worth ? 6 For this reason I don’t think it is wise to write laws that will bankrupt a company off of one infraction which was not directly or indirectly harmful to the physical well being of the people: and I am using indirectly a little bit more strict than I would like to since as I said before, the aggregate of all the information is harmful. The point is that the goal is not to bankrupt companies but to have them behave right. The penalty associated to every law IS the tool that make you respect the law. And it must be so high that you don't want to break the law. I would have to look into the laws in question, but on a surface level I think that any company should be subjected to the same baseline privacy laws, so if there isn’t anything screwy within the law that apple, Google, and Facebook are ignoring, I think it should apply to them. Trust me on this one, direct experience payment processors have a lot more rules to follow to be able to work. I do not want jail time for the CEO by default but he need to know that he will pay personally if the company break the law, it is the only way to make him run the company being sure that it follow the laws. For some reason I don’t have my usual cynicism when it comes to this issue. I think that the magnitude of loses that vested interests have in these companies would make it so that companies would police themselves for fear of losing profits. That being said I wouldn’t be opposed to some form of personal accountability on corporate leadership, but I fear that they will just end up finding a way to create a scapegoat everytime. It is not cynicism. I simply think that a huge fine to a single person (the CEO for example) is useless since it too easy to avoid and if it really huge realistically it would be never paid anyway so nothing usefull since the net worth of this kind of people is only on the paper. So if you slap a 100 billion file to Musk he will never pay because he has not the money to pay even if technically he is worth way more than that. Jail time instead is something that even Musk can experience. In general I like laws that are as objective as possible, I think that a privacy law should be written so that it is very objectively overbearing, but that has a smaller fine associated with it. This way the law is very clear on right and wrong, while also giving the businesses time and incentive to change their practices without having to sink large amount of expenses into lawyers to review every minute detail, which is the logical conclusion of the one infraction bankrupt system that you seem to be supporting. Then you write a law that explicitally state what you can do and what is not allowed is forbidden by default.
  • 19 Stimmen
    1 Beiträge
    14 Aufrufe
    Niemand hat geantwortet
  • 0 Stimmen
    1 Beiträge
    8 Aufrufe
    Niemand hat geantwortet
  • 248 Stimmen
    232 Beiträge
    600 Aufrufe
    U
    Repair technicians see by far the most of seagate drives
  • Lapsus$: GTA 6 hacker sentenced to life in hospital prison

    Technology technology
    2
    1
    0 Stimmen
    2 Beiträge
    23 Aufrufe
    A
    From Aurealisa perspective, the leak may have been a momentary distraction, but it had serious consequences for the person behind it. Hopefully, this serves as a lesson to other potential hackers.