"performance"

Then why share it?

Is the performance increase related to computing power? I suspect the undelying massive datacenters running the cloud based LLMs are expanding at a similar rate...

2 X 0 = 0

This is such bullshit. Models have already consumed all available data and have nothing left to consume, whole needing exponentially more data for progressive advancements

All that power used for a fucking coin flip.

So we can mock it!

How is completely fucking up literally 50% of the time outperforming exactly???

That sounds like a coin flip, but 50% reliability can be really useful.

If a model has 50% chance of completing a task that would cost me an hour - and I can easily check it was completed correctly - on average, I'm saving half of the time it would take to complete this.

That said, exponentials don't exist in the real world, we're just seeing the middle of a sigmoid curve, which will soon yield diminishing returns.

You see, in 7 months, they'll fuck up literally 100% of the time! Progress.

Apparently, throwing more data at it will not help much from now on... But anyway what they're saying, I can't trust the snake oil seller, he is suspicious...

Is it just me, or is this graph (first graph in the article) completely unintelligible?

The X-axis being time is self-explanatory, but the Y-axis is somehow exponential time but then also mapping random milestones of performance, meaning those milestones are hard-linked to that time-based Y-axis? What?

Very good analogy. They're also ignoring that getting faster and faster at reaching a 50% success rate (a totally unacceptable success rate for meaningful tasks) doesn't imply ever reaching acceptable success rates consistently.