Do you know how many industries would collapse if everyone had bare minimum living standards!

/s, just in case.

While I appreciate them talking in good faith, all these articles that warn against misusing technology really sound out of touch from our reality.

It’s capitalism, of course it would be misused cause of our economic incentives.

When it comes to AI, you either don’t like it or are trying to make money from it, no one expects it to actually work so the entire point is moot.

This is the last paragraph of the article:

AI seems to present a spectacular new slate of opportunities and threats. But, in essence, much of what was true before AI remains so now. Human greed and desire for greater control over nature and other people may lead toward paths of short-term gain. But, if you want a good life when all’s said and done, learn to live well within limits. Live with honesty, modesty, and generosity. AI can’t help you with that.

Yeah no shit

Nobody's expecting a "machine that can't think straight" to do it. Some people are hoping that a more competent machine will be developed.

I highly doubt it (at least anytime in our lifetimes)

Whether it be AI apocalypse or utopia, it's not LLM's that people think will take us there. It's AGI/ASI and nobody knows how long it'll take us to develop a system like that. Could take 2 years or it could take 50.

Even if it is, I don't see what it's going to conclude that we haven't already.

If we do build "the AI that will save us" it's just going to tell us "in order to ensure your existence as a species, take care of the planet and each other" and I really, really, can't picture a scenario where we actually listen.

I don't see what it's going to conclude that we haven't already.

Well, that's the point of trying to build ASI. To have it think of things that we haven't been able to think of.

I really, really, can't picture a scenario where we actually listen.

Of course not, you're not an ASI.

I think it very well might conclude things we haven't.

But at the same time, I think what you're saying is so very important. It's going to tell us what we already know about a lot of things. That the best way to scrub carbon from the air is the way nature is already doing it. That allowing the superwealthy to exist at the same time as poverty is not conducive to achieving humanity's most important goals.

If we consider AGI or ASI to be the answer to all of our problems and continue to pour more and more carbon into the atmosphere in an effort to get there, once we do have such a powerful intelligence, it may simply tell us, "If you were smarter as a species, you would have turned me off a long time ago."

Because the problem is not necessarily that we are trying to decode what it means to be intelligent and create machines that can replicate true conscious thought. The problem is that while we marvel at something currently much dumber than us, we are mostly neglecting to improve our own intelligence as a society. I think we might make a machine that's smarter than the average human quite soon, but not necessarily because of much change in the machines.

This is the same logic people apply to God being incomprehensible.

Are you suggesting that if such a thing can be built, its word should be gospel, even if it is impossible for us to understand the logic behind it?

I don't subscribe to this. Logic is logic. You don't need a new paradigm of mind to explore all conclusions that exist. If something cannot be explained and comprehended, transmitted from one sentient mind to another, then it didn't make sense in the first place.

And you might bring up some of the stuff AI has done in material science as an example of it doing things human thinking cannot. But that's not some new kind of thinking. Once the molecular or material structure was found, humans have been perfectly capable of comprehending it.

All it's doing, is exploring the conclusions that exist, faster. And when it comes to societal challenges, I don't think it's going to find some win-win solution we just haven't thought of. That's a level of optimism I would consider insane.