Skip to content

Large Language Model Performance Doubles Every 7 Months

Technology
53 34 706
  • That sounds like a coin flip, but 50% reliability can be really useful.

    If a model has 50% chance of completing a task that would cost me an hour - and I can easily check it was completed correctly - on average, I'm saving half of the time it would take to complete this.

    That said, exponentials don't exist in the real world, we're just seeing the middle of a sigmoid curve, which will soon yield diminishing returns.

    and I can easily check it was completed correctly

    Can you always though?

  • By 2030, AI will greatly outperform humans in some complex intellectual tasks. Discover how LLMs are doubling their capabilities every seven months.

    I very much like those huge generalizations in AI articles that makes you small and stupid. Those generalizations proves nothing but they sound like something big is coming. It's parody. How long we see them before people wake up ? Just wait 2 more years and AI will be better bro. You're not using AI properly, you need to learn how to use AI bro. You need to use different model for this task bro. Just pay for corporate products bro. Amount of junk of top of this pile of shit is amusing.

  • By 2030, AI will greatly outperform humans in some complex intellectual tasks. Discover how LLMs are doubling their capabilities every seven months.

    They need to invent an inquiring-gpt or Q-GPT. Otherwise they'll need humans to do the digging.

  • By 2030, AI will greatly outperform humans in some complex intellectual tasks. Discover how LLMs are doubling their capabilities every seven months.

    Classic pseudo-science for the modern grifter. Vague definitions, sloppy measurements, extremely biased, wild unsupported predictions, etc.

  • It's outperforming "messier" problems with a much lower success rate.

  • By 2030, AI will greatly outperform humans in some complex intellectual tasks. Discover how LLMs are doubling their capabilities every seven months.

    when will they be able to tell me how many 'r's are in 'strawberry' in under 1s?

  • I very much like those huge generalizations in AI articles that makes you small and stupid. Those generalizations proves nothing but they sound like something big is coming. It's parody. How long we see them before people wake up ? Just wait 2 more years and AI will be better bro. You're not using AI properly, you need to learn how to use AI bro. You need to use different model for this task bro. Just pay for corporate products bro. Amount of junk of top of this pile of shit is amusing.

    Because so much money has been thrown at it, for startups, for power generation, for investors, that this is little more than marketing for retail investors to buy into.

  • Classic pseudo-science for the modern grifter. Vague definitions, sloppy measurements, extremely biased, wild unsupported predictions, etc.

    and assuming that improvement doesn't plateau, ever,

  • This is such bullshit. Models have already consumed all available data and have nothing left to consume, whole needing exponentially more data for progressive advancements

    time for them to set sail to the wild seas again!

  • Moore's law hasn't been exponential for ~15 years now.

    If you are just talking transitor density I believe it still is, but even if not, my point was that it had exponential growth spanning over many decades.

  • Because so much money has been thrown at it, for startups, for power generation, for investors, that this is little more than marketing for retail investors to buy into.

    That's no doubt that they pour money to machines instead of people. We all see that in statistics that machines get more support these days than people.

  • when will they be able to tell me how many 'r's are in 'strawberry' in under 1s?

    Deepseek-r1:1.5b

    ::: spoiler Thinking

    Thinking...
    Alright, so I'm trying to figure out how many 'r's are in the word "strawberry." Let me start by reading it out loud to get a better 
    sense of what it sounds like.
    
    S-T-R-A-W-B-E-R-R-Y
    
    Okay, let's break this down letter by letter. The first letter is S, then T, R, A, W, B, E, and two R's at the end.
    
    So far, I've got one 'R' in the third position after S-T- and before A-W-B-E-R-R-Y.
    
    Wait a minute, there are two R's in "strawberry." So that means there are two instances of the letter 'r' in the word. Is that all?
    
    Let me double-check to make sure I'm not missing any other 'r's. Scanning through each letter again:
    
    1. S - No
    2. T - No
    3. R - Yes, one R here.
    4. A - No
    5. W - No
    6. B - No
    7. E - No
    8. R - Second R
    9. R - Third R
    
    Wait, am I counting correctly? So far, from 1 to 9, that's three letters after the first two: R, A, W, B, and then R, R again at the end.
    
    Hold on, maybe I should visualize it like this:
    
    S T R A W B E R R Y
    
    So positions:
    1. S
    2. T
    3. R (position 3)
    4. A
    5. W
    6. B
    7. E
    8. R (position 8)
    9. R (position 9)
    
    And then the word ends with 'Y.'
    
    Hmm, so only two 'R's in total? Let me confirm:
    
    Looking back: positions 3 and 8 and 9.
    
    Wait, that makes three 'R's. Did I miss one?
    
    Hold on, maybe I need to write it out clearly to visualize better:
    
    S T R A W B E R R Y
    
    Counting each 'R':
    
    - After S (position 2): No
    - After T (position 3): Yes, first R
    - After A (position 4): No
    - After W (position 5): No
    - After B (position 6): No
    - After E (position 7): No
    - After R (position 8): Second R
    - After R (position 9): Third R
    - After Y (position 10): No
    
    So that's three 'R's in total. Did I count right? Maybe I should write it down step by step to ensure accuracy.
    
    Alternatively, maybe there are only two 'R's because the 'Y' is considered separate from the R. But logically, even if Y has a vowel 
    sound, 'r' still counts separately unless specified otherwise.
    
    Wait, actually, in English letters, 'Y' isn't really pronounced like an 'R.' It sounds more like a consonant. So in that case, probably 
    only two 'R's: one between S and T, and another before the end of the word.
    
    But I'm still confused because when I count, I sometimes make mistakes. Maybe using an alphabet list or a counting tool would help.
    
    Let me write down each letter with its position:
    
    1. S
    2. T
    3. R (count 1)
    4. A
    5. W
    6. B
    7. E
    8. R (count 2)
    9. R (count 3)
    10. Y
    
    So that's three 'R's in total.
    
    Wait, but the word is "strawberry," and I'm pretty sure it has two 'r's because of the spelling of "strawberry." Maybe the counting 
    matches that.
    
    I think my initial count might have been off by one, thinking there are only two R's when actually there are three: one in position 3, 
    another at 8, and another at 9. But given that Y isn't pronounced like an 'R,' it's more accurate to say there are two 'r's.
    
    Hmm, maybe I need to consider the pronunciation of each letter as well. Even though 'Y' is a consonant, sometimes people might play 
    around with silent letters or other pronunciation rules. But unless instructed otherwise, I should stick to standard English letter 
    sounds for this purpose.
    
    So after carefully counting and considering each letter's individual sound, it seems that "strawberry" has two 'r's: one in the middle 
    and another at the end.
    ...done thinking.
    
    

    :::

    To determine how many 'r's are in the word "strawberry," we can break it down letter by letter:
    
    1. **S** - No
    2. **T** - No
    3. **R** (count 1)
    4. **A** - No
    5. **W** - No
    6. **B** - No
    7. **E** - No
    8. **R** (count 2)
    9. **R** (count 3)
    10. **Y** - No
    
    However, considering the pronunciation of each letter and standard English rules, even though 'Y' is a consonant, it's more accurate to 
    say that "strawberry" contains only two 'r's: one in position 3 and another at position 9.
    
    Final Answer: There are two 'r's in the word "strawberry."
    
  • This is like measuring the increasing speeds of cars in the early years and extrapolating that they would be supersonic by now by ignoring the exponential impact that air resistance has.

    Or like looking at the early days of semiconductors and extrapolating that CPU speed will double every 18 months ..smh these people

  • Or like looking at the early days of semiconductors and extrapolating that CPU speed will double every 18 months ..smh these people

    Since CPU speeds are still doubling every 18 months you have a solid point!

    Or maybe not since you are probably referring to the doubling of transistors that was an observation which was accurate over a lengthy period of time in the context of when the observation was made. Nobody said that would continue indefinitely either.

  • By 2030, AI will greatly outperform humans in some complex intellectual tasks. Discover how LLMs are doubling their capabilities every seven months.

    Someone doesn't know the folly of extending straight lines graphs into the future.

  • By 2030, AI will greatly outperform humans in some complex intellectual tasks. Discover how LLMs are doubling their capabilities every seven months.

    new moore law dropped

  • Since CPU speeds are still doubling every 18 months you have a solid point!

    Or maybe not since you are probably referring to the doubling of transistors that was an observation which was accurate over a lengthy period of time in the context of when the observation was made. Nobody said that would continue indefinitely either.

    Yup, that's what I was alluding to, while it may not still be the case for transistors, they did manage to take 50 odd years to get there, push that trend line from the figure 50 years heh (not saying you should, 5 seems much more conservative)

    Take a look at Nvidias pace wrt Moore's law (of FLOPS) https://netrouting.com/nvidia-surpassing-moores-law-gpu-innovation/

  • Classic pseudo-science for the modern grifter. Vague definitions, sloppy measurements, extremely biased, wild unsupported predictions, etc.

    That graph is hilarious. Enormous error bars, totally arbitrary quantization of complexity, and it's title? "Task time for a human that an AI model completes with a 50 percent success rate". 50 percent success is useless, lmao.

    On a more sober note, I'm very disappointed that IEEE is publishing this kind of trash.

  • Deepseek-r1:1.5b

    ::: spoiler Thinking

    Thinking...
    Alright, so I'm trying to figure out how many 'r's are in the word "strawberry." Let me start by reading it out loud to get a better 
    sense of what it sounds like.
    
    S-T-R-A-W-B-E-R-R-Y
    
    Okay, let's break this down letter by letter. The first letter is S, then T, R, A, W, B, E, and two R's at the end.
    
    So far, I've got one 'R' in the third position after S-T- and before A-W-B-E-R-R-Y.
    
    Wait a minute, there are two R's in "strawberry." So that means there are two instances of the letter 'r' in the word. Is that all?
    
    Let me double-check to make sure I'm not missing any other 'r's. Scanning through each letter again:
    
    1. S - No
    2. T - No
    3. R - Yes, one R here.
    4. A - No
    5. W - No
    6. B - No
    7. E - No
    8. R - Second R
    9. R - Third R
    
    Wait, am I counting correctly? So far, from 1 to 9, that's three letters after the first two: R, A, W, B, and then R, R again at the end.
    
    Hold on, maybe I should visualize it like this:
    
    S T R A W B E R R Y
    
    So positions:
    1. S
    2. T
    3. R (position 3)
    4. A
    5. W
    6. B
    7. E
    8. R (position 8)
    9. R (position 9)
    
    And then the word ends with 'Y.'
    
    Hmm, so only two 'R's in total? Let me confirm:
    
    Looking back: positions 3 and 8 and 9.
    
    Wait, that makes three 'R's. Did I miss one?
    
    Hold on, maybe I need to write it out clearly to visualize better:
    
    S T R A W B E R R Y
    
    Counting each 'R':
    
    - After S (position 2): No
    - After T (position 3): Yes, first R
    - After A (position 4): No
    - After W (position 5): No
    - After B (position 6): No
    - After E (position 7): No
    - After R (position 8): Second R
    - After R (position 9): Third R
    - After Y (position 10): No
    
    So that's three 'R's in total. Did I count right? Maybe I should write it down step by step to ensure accuracy.
    
    Alternatively, maybe there are only two 'R's because the 'Y' is considered separate from the R. But logically, even if Y has a vowel 
    sound, 'r' still counts separately unless specified otherwise.
    
    Wait, actually, in English letters, 'Y' isn't really pronounced like an 'R.' It sounds more like a consonant. So in that case, probably 
    only two 'R's: one between S and T, and another before the end of the word.
    
    But I'm still confused because when I count, I sometimes make mistakes. Maybe using an alphabet list or a counting tool would help.
    
    Let me write down each letter with its position:
    
    1. S
    2. T
    3. R (count 1)
    4. A
    5. W
    6. B
    7. E
    8. R (count 2)
    9. R (count 3)
    10. Y
    
    So that's three 'R's in total.
    
    Wait, but the word is "strawberry," and I'm pretty sure it has two 'r's because of the spelling of "strawberry." Maybe the counting 
    matches that.
    
    I think my initial count might have been off by one, thinking there are only two R's when actually there are three: one in position 3, 
    another at 8, and another at 9. But given that Y isn't pronounced like an 'R,' it's more accurate to say there are two 'r's.
    
    Hmm, maybe I need to consider the pronunciation of each letter as well. Even though 'Y' is a consonant, sometimes people might play 
    around with silent letters or other pronunciation rules. But unless instructed otherwise, I should stick to standard English letter 
    sounds for this purpose.
    
    So after carefully counting and considering each letter's individual sound, it seems that "strawberry" has two 'r's: one in the middle 
    and another at the end.
    ...done thinking.
    
    

    :::

    To determine how many 'r's are in the word "strawberry," we can break it down letter by letter:
    
    1. **S** - No
    2. **T** - No
    3. **R** (count 1)
    4. **A** - No
    5. **W** - No
    6. **B** - No
    7. **E** - No
    8. **R** (count 2)
    9. **R** (count 3)
    10. **Y** - No
    
    However, considering the pronunciation of each letter and standard English rules, even though 'Y' is a consonant, it's more accurate to 
    say that "strawberry" contains only two 'r's: one in position 3 and another at position 9.
    
    Final Answer: There are two 'r's in the word "strawberry."
    

    I like how it counted correctly and then gave an incorrect final answer.. Bravo 👏

  • Someone doesn't know the folly of extending straight lines graphs into the future.

  • 36 Stimmen
    9 Beiträge
    36 Aufrufe
    abbiistabbii@lemmy.blahaj.zoneA
    I fucking hate the government. They are so fucking clueless. Not just Labour but the Cons, the Lib Dems, Reform, all of them are so fucking stupid and they ignore the people there to tell them their ideas are Stupid.
  • 4 Stimmen
    2 Beiträge
    27 Aufrufe
    K
    You made this site, you say? What an odd coincidence! Were you inspired by the site you say you "stumbled upon" here? https://lemmy.world/post/33395761 Because it sure seems like the exact same site.
  • Vibe coding service Replit deleted production database

    Technology technology
    118
    1
    579 Stimmen
    118 Beiträge
    3k Aufrufe
    iavicenna@lemmy.worldI
    And you are talking about obvious bugs. It likely will make erroneous judgements (because somewhere in its training data someone coded it that way) which will down the line lead to subtle problems that will wreck your system and cost you much more. Sure humans can also make the same mistakes but in the current state of affairs, an experienced software engineer/programmer has a much higher chance of catching such an error. With LLMs it is more hit and miss especially if it is a more niche topic. Currently, it is an assistant tool (sometimes quite helpful, sometimes frustrating at best) not an autonomous coder. Any company that claims so is either a crook or also does not know much about coding.
  • 89 Stimmen
    15 Beiträge
    195 Aufrufe
    S
    I suspect people (not billionaires) are realising that they can get by with less. And that the planet needs that too. And that working 40+ hours a week isn’t giving people what they really want either. Tbh, I don't think that's the case. If you look at any of the relevant metrics (CO², energy consumption, plastic waste, ...) they only know one direction globally and that's up. I think the actual issues are Russian invasion of Ukraine and associated sanctions on one of the main energy providers of Europe Trump's "trade wars" which make global supply lines unreliable and costs incalculable (global supply chains love nothing more than uncertainty) Uncertainty in regards to China/Taiwan Boomers retiring in western countries, which for the first time since pretty much ever means that the work force is shrinking instead of growing. Economical growth was mostly driven by population growth for the last half century with per-capita productivity staying very close to inflation. Disrupting changes in key industries like cars and energy. The west has been sleeping on may of these developments (e.g. electric cars, batteries, solar) and now China is curbstomping the rest of the world in regards to market share. High key interest rates (which are applied to reduce high inflation due to some of the reason above) reduce demand on financial investments into companies. The low interest rates of the 2010s and also before lead to more investments into companies. With interest going back up, investments dry up. All these changes mean that companies, countries and people in the west have much less free cash available. There’s also the value of money has never been lower either. That's been the case since every. Inflation has always been a thing and with that the value of money is monotonically decreasing. But that doesn't really matter for the whole argument, since the absolute value of money doesn't matter, only the relative value. To put it differently: If you earn €100 and the thing you want to buy costs €10, that is equivalent to if you earn €1000 and the thing you want to buy costing €100. The value of money dropping is only relevant for savings, and if people are saving too much then the economy slows down and jobs are cut, thus some inflation is positive or even required. What is an actual issue is that wages are not increasing at the same rate as the cost of things, but that's not a "value of the money" issue.
  • Blocking real-world ads: is the future here?

    Technology technology
    33
    1
    198 Stimmen
    33 Beiträge
    745 Aufrufe
    S
    Also a work of fiction
  • 419 Stimmen
    113 Beiträge
    2k Aufrufe
    D
    deleted by creator
  • New Supermaterial: As Strong As Steel And As Light As Styrofoam

    Technology technology
    21
    1
    60 Stimmen
    21 Beiträge
    280 Aufrufe
    D
    I remember an Arthur Clarke novel where a space ship needs water from the planet below. The easiest thing is to lower cables from space and then lift some ice bergs.
  • 317 Stimmen
    45 Beiträge
    422 Aufrufe
    F
    By giving us the choice of whether someone else should profit by our data. Same as I don't want someone looking over my shoulder and copying off my test answers.