I agree. This is the exact problem I think people need to face with nural network AIs. They work the exact same way we do.

I don't think this is a fair way of summarizing it. You're making it sound like we have AGI, which we do not have AGI and we may never have AGI.

I agree, but I'm not sure it matters when it comes to the big questions, like "what separates us from the LLMs?" Answering that basically amounts to answering "what does it mean to be human?", which has been stumping philosophers for millennia.

It's true that artificial neurons are significant different than biological ones, but are biological neurons what make us human? I'd argue no. Animals have neurons, so are they human? Also, if we ever did create a brain simulation that perfectly replicated someone's brain down to the cellular level, and that simulation behaved exactly like the original, I would characterize that as a human.

It's also true LLMs can't learn, but there are plenty of people with anterograde amnesia that can't either.

This feels similar to the debates about what separates us from other animal species. It used to be thought that humans were qualitatively different than other species by virtue of our use of tools, language, and culture. Then it was discovered that plenty of other animals use tools, have language, and something resembling a culture. These discoveries were ridiculed by many throughout the 20th century, even by scientists, because they wanted to keep believing humans are special in some qualitative way. I see the same thing happening with LLMs.

Can humans think?

Maybe I should rephrase my question:

Outside of comment sections on the internet, who has claimed or is claiming that LLMs have the capacity to reason?

The environmental toll doesn’t have to be that bad. You can get decent results from single high-end gaming GPU.

Humans don't know how they work internally either. I mean what are you expecting?

Let's get something straight, no I'm not saying we have our modern definition of AGI but we've practically got the original definition coined before LLMs were a thing. Which was that the proposed AGI agent should maximise "the ability to satisfy goals in a wide range of environments". I personally think we've just moved the goal posts a bit.

Wether we'll ever have thinking, rationalised and possibly conscious AGI is beyond the question. But I do think current AI is similar to existing brains today.

Do you not agree that animal brains are just prediction machines?

That we have our own hallucinations all the time? Think visual tricks, lapses in memory, deja vu, or just the many mental disorders people can have.

Do you think our brain doesn't follow path of least resistance in processing? Or do you think our thoughts comes from elsewhere?

I seriously don't think animal brains or human to be specific are that special that nurural networks are beneath. Sure people didn't like being likened to animals but it was the truth, and I as do many AI researches, liken us to AI.

AI is primitive now, yet it can still pass the bar, doctors exams, compute complex physics problems and write a book (soulless as it may be like some authors) in less than a few seconds.

Whilst we may not have AGI the question was about math. The paper questioned how it did 36+59 and it did things in an interesting way where it half predicted what the tens column would be and 'knew' what the units column was, then put it together. Although thats not how I or even you may do it there are probably people who do it similar.

All I argue is that AI is closer to how our brains think, and with our brains being irrational quite often it shouldn't be surprising that AI nural networks are also irrational at times.

Maybe work is the wrong word, same output. Just as a belt and chain drive does the same thing, or how fluorescent, incandescent or LED lights produce light even though they're completely different mechanisms.

What I was saying is that one is based on the other, so similar problems like irrational thought even if the right answer is conjured shouldn't be surprising. Although an animal brain and nural network are not the same, the broad concept of how they work is.

“the ability to satisfy goals in a wide range of environments”

That was not the definition of AGI even back before LLMs were a thing.

Wether we’ll ever have thinking, rationalised and possibly conscious AGI is beyond the question. But I do think current AI is similar to existing brains today.

That's doing a disservice to AGI.

Do you not agree that animal brains are just prediction machines?

That's doing a disservice to human brains. Humans are sentient, LLMs are not sentient.

I don't really agree with you.

LLMs are damn impressive, but they are very clearly not AGI, and I think that's always worth pointing out.

The first person to be recorded talking about AGI was Mark Gubrud. He made that quote above, here's another:

The major theme of the book was to develop a mathematical foundation of artificial intelligence. This is not an easy task since intelligence has many (often ill-defined) faces. More specifically, our goal was to develop a theory for rational agents acting optimally in any environment. Thereby we touched various scientific areas, including reinforcement learning, algorithmic information theory, Kolmogorov complexity, computational complexity theory, information theory and statistics, Solomonoff induction, Levin search, sequential decision
theory, adaptive control theory, and many more.
Page 232 8.1.1 Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability

As UGI largely encompasses AGI we could easily argue that if modern LLMs are beginning to fit the description of UGI then it's fullfilling AGI too. Although AGI's definition in more recent times has become more nuanced to replicating a human brain instead, I'd argue that that would degrade the AI trying to replicate biology.

I don't beleive it's a disservice to AGI because AGI's goal is to create machines with human-level intelligence. But current AI is set to surpase collective human intelligence supposedly by the end of the decade.

And it's not a disservice to biological brains to summarise them to prediction machines. They work, very clearly. Sentience or not if you simulated every atom in the brain it will likely do the same job, soul or no soul. It just brings the philosophical question of "do we have free will or not?" And "is physics deterministic or not". So much text exists on the brain being prediction machines and the only time it has recently been debated is when someone tries differing us from AI.

I don't believe LLMs are AGI yet either, I think we're very far away from AGI. In a lot of ways I suspect we'll skip AGI and go for UGI instead. My firm opinion is that biological brains are just not effective enough. Our brains developed to survive the natural world and I don't think AI needs that to surpass us. I think UGI will be the equivalent of our intelligence with the fat cut off. I believe it only resembles our irrational thought patterns now because the fat hasn't been striped yet but if something truely intelligent emerges, we'll probably see these irrational patterns cease to exist.

They work the exact same way we do.

Citation needed.

What I was saying is that one is based on the other

Not in any direct way, no. At least not in any way more rigorous than handwavey analogies.

Because Blue Steel.

People that can not do Matrix multiplication do not possess the basic concepts of intelligence now?

As a mathematician (at least by education), I think that's a great definition, yes.

You're definitely overselling how AI works and underselling how human brains work here, but there is a kernel of truth to what you're saying.

Neural networks are a biomimicry technology. They explicitly work by mimicking how our own neurons work, and surprise surprise, they create eerily humanlike responses.

The thing is, LLMs don't have anything close to reasoning the way human brains reason. We are actually capable of understanding and creating meaning, LLMs are not.

So how are they human-like? Our brains are made up of many subsystems, each doing extremely focussed, specific tasks.

We have so many, including sound recognition, speech recognition, language recognition. Then on the flipside we have language planning, then speech planning and motor centres dedicated to creating the speech sounds we've planned to make. The first three get sound into your brain and turn it into ideas, the last three take ideas and turn them into speech.

We have made neural network versions of each of these systems, and even tied them together. An LLM is analogous to our brain's language planning centre. That's the part that decides how to put words in sequence.

That's why LLMs sound like us, they sequence words in a very similar way.

However, each of these subsystems in our brains can loop-back on themselves to check the output. I can get my language planner to say "mary sat on the hill", then loop that through my language recognition centre to see how my conscious brain likes it. My consciousness might notice that "the hill" is wrong, and request new words until it gets "a hill" which it believes is more fitting. It might even notice that "mary" is the wrong name, and look for others, it might cycle through martha, marge, maths, maple, may, yes, that one. Okay, "may sat on a hill", then send that to the speech planning centres to eventually come out of my mouth.

Your brain does this so much you generally don't notice it happening.

In the 80s there was a craze around so called "automatic writing", which was essentially zoning out and just writing whatever popped into your head without editing. You'd get fragments of ideas and really strange things, often very emotionally charged, they seemed like they were coming from some mysterious place, maybe ghosts, demons, past lives, who knows? It was just our internal LLM being given free rein, but people got spooked into believing it was a real person, just like people think LLMs are people today.

In reality we have no idea how to even start constructing a consciousness. It's such a complex task and requires so much more linking and understanding than just a probabilistic connection between words. I wouldn't be surprised if we were more than a century away from AGI.

Maybe I am over selling current AI and underselling our brains. But the way I see it is that the exact mechanism that allowed intelligence to flourish within ourselves exists with current nural networks. They are nowhere near being AGI or UGI yet but I think these tools alone are all that are required.

The way I see it is, if we rewound the clock far enough we would see primitive life with very basic nural networks beginning to develop in existing multicellular life (something like jellyfish possibly). These nural networks made from neurons neurotransmitters and synapses or possibly something more primitive would begin forming the most basic of logic over centuries of evolution. But it wouldn't reassemble anything close to reason or intelligence, it wouldn't have eyes, ears or any need for language. At first it would probably spend its first million years just trying to control movement.

We know that this process would have started from nothing, nural networks with no training data, just a free world to explore. And yet over 500 million years later here we are.

My argument is that modern nural networks work the same way that biological brains do, at least the mechanism does. The only technical difference is with neurotransmitters and the various dampening and signal boosting that can happen along with nuromodulation. Given enough time and enough training, I firmly believe nural networks could develop reason. And given external sensors it could develop thought from these input signals.

I don't think we would need to develop a consciousness for it but that it would develop one itself given enough time to train on its own.

A large hurdle that might arguably be a good thing, is that we are largely in control of the training. When AI is used it does not learn and alter itself, only memorising things currently. But I do remember a time when various AI researchers allowed earlier models to self learn, however the internet being the internet, it developed some wildly bad habits.

Formatting might be off on some of these, had to convert some papers to text as some were only scanned and I couldn't be bothered writing it all out by hand:

"The PDP models are inspired by the structure and function of the brain. In particular, they are based on networks of neuron-like units whose interactions resemble those among neurons in the cerebral cortex."
Parallel Distributed Processing: Explorations in the Microstructure of Cognition McClelland, Rumelhart, & the PDP Research Group

"The design of artificial neural networks was inspired by knowledge of the brain, in particular the way biological neurons are interconnected and the way they communicate through synapses."
Deep Learning LeCun, Bengio, Hinton

"The design of deep learning architectures owes much to our understanding of the hierarchical structure of the visual cortex, particularly as revealed by Hubel and Wiesel’s work on simple and complex cells."
Neuroscience-Inspired Artificial Intelligence Hassabis et al.

"The relationship between biological and artificial neural networks has now become a central issue in both neuroscience and AI. Deep networks trained with backpropagation may offer a plausible model of some aspects of human cognition."
Cognitive computational neuroscience Kriegeskorte & Douglas (2018)

"Goal-driven deep learning models, when trained to solve behavioral tasks, can develop internal representations that match those found in the brain."
Using goal-driven deep learning models to understand sensory cortex Yamins & DiCarlo

And longer excepts on the similarities of AI neural networks to biological brains, more specifically human children, in the pursuit of study with improving learning and education development. Super interesting papers that are easily accessible to anyone:

"Humans are imperfect reasoners. We reason most effectively about entities and situations that are consistent with our understanding of the world. Our experiments show that language models mirror these patterns of behavior. Language models perform imperfectly on logical reasoning tasks, but this performance depends on content and context. Most notably, such models often fail in situations where humans fail — when stimuli become too abstract or conflict with prior understanding of the world. Beyond these parallels, we also observed reasoning effects in language models that to our knowledge have not been previously investigated in the human literature. For example, the patterns of errors on the ‘violate realistic’ rules, or the relative ease of ‘shuffled realistic’ rules in the Wason tasks. Likewise, language model performance on the Wason tasks increases most when they are demonstrated with realistic examples; benefits of concrete examples have been found in cognitive and educational contexts (Sweller et al., 1998; Fyfe et al., 2014), but remain to be explored in the Wason problems. Investigating whether humans show similar effects is a promising direction for future research."
5.9-10 Language models show human-like content effects on reasoning
Ishita Dasgupta*,1, Andrew K. Lampinen*,1, Stephanie C. Y. Chan1, Antonia Creswell1, Dharshan Kumaran1, James L. McClelland1,2 and Felix Hill1 *Equal contributions, listed alphabetically, 1DeepMind, 2Stanford University

"In this article we will point out several characteristics of human cognitive processes that conventional computer architectures do not capture well. Then we will note that connectionist models {neural networks} are much better able to capture these aspects of human processing. After that we will mention three recent applications in connectionist artificial intelligence which exploit these characteristics. Thus, we shall see that connectionist models offer hope of overcoming the limitations of conventional AI. The paper ends with an example illustrating how connectionist models can change our basic conceptions of the nature of intelligent processing."

"The framework for building connectionist models is laid out in detail in Rumelhart, McClelland and the PDP Group (1986), and many examples of models constructed in that framework are described.
Two examples of connectionist models of human processing abilities that capture these characteristics are the interactive activation model of visual word recognition from McClelland and Rumelhart (1981), and the model of past tense learning from Rumelhart and McClelland (1986). These models were motivated by psychological experiments, and were constructed to capture the data found in these studies. We describe them here to illustrate some of the roots of the connectionist approach in an attempt to understand detailed aspects of human cognition."

"The models just reviewed capture important aspects of data from psychological experiments, and illustrate how the characteristics of human processing capabilities enumerated above can be captured in an explicit comptutational framework. Recently connectionist models that capture these same characteristics have begun to give rise to a new kind of Artificial Intelligence, which we will call connectionist AI. Connectionist AI is beginning to address several topics that have not been easily solved using other approaches. We will consider three cases of this. In each case we will describe recent progress that illustrates the ability of connectionist networks to capture the characteristics of human performance mentioned above."

"This paper began with the idea that humans exploit graded information, and that computational mechanisms that aim to emulate the natural processing capabilities of humans should exploit this kind of information as well. Connectionist models do exploit graded information, and this gives them many of their attractive characteristics."
Parallel Distributed Processing: Bridging the Gap Between Human and Machine Intelligence
James L. McClelland, Axel Cleeremans, and David Servan-Schreiber Carnegie Mellon University

"Artificial neural networks have come and gone and come again- and there are several good reasons to think that this time they will be around for quite a while. Cheng and Titterington have done an excellent job describing that nature of neural network models and their relations to statistical methods, and they have overviewed several applications. They have also suggested why neuroscientists interested in modeling the human brain are interested in such models. In this note, I will point out some additional motivations for the investigation of neural networks. These are motivations arising from the effort to capture key aspects of human cognition and learning that have thus far eluded cognitive science. A central goal of congnitive science is to understand the full range of human cognitive function"..."{there are} good reasons for thinking that artificial neural networks, or at least computationally explicit models that capture key properties of such networks, will play an important role in the effort to capture some of the aspects of human cognitive function that have eluded symbolic approaches."
Neural Networks: A Review from Statistical Perspective]: Comment: Neural Networks and Cognitive Science: Motivations and Applications
James L. McClelland Statistical Science, Vol. 9, No. 1. (Feb., 1994), pp. 42-45.

"The idea has arisen that as the scale of experience and computation begins to approach the scale of experience and computation available to a young child—who sees millions of images and hears millions of words per year, and whose brain contains 10–100 billion neuron-like processing units each updating their state on a time scale of milliseconds—the full power and utility of neural networks to capture natural computation is finally beginning to become a reality, allowing artificially intelligent systems to capture more fully the capabilities of the natural intelligence present in real biological networks in the brain."

"One major development in the last 25 years has been the explosive growth of computational cognitive neuroscience. The idea that computer simulations of neural mechanisms might yield insight into cognitive phenomena no longer requires, at least in most quarters, vigorous defense—there now exist whole fields, journals, and conferences dedicated to this pursuit. One consequence is the elaboration of a variety of different computationally rigorous approaches to neuroscience and cognition that capture neural information processing mechanisms at varying degrees of abstraction and complexity. These include the dynamic field theory, in which the core representational elements are fields of neurons whose activity and interactions can be expressed as a series of coupled equations (Johnson, Spencer, & Sch€oner, 2008); the neural engineering framework, which seeks to understand how spiking neurons might implement tensor-product approaches to symbolic representations (Eliasmith & Anderson, 2003; Rasmussen & Eliasmith, 2011); and approaches to neural representation based on ideal-observer models and probabilistic inference (Deneve, Latham, & Pouget, 1999; Knill & Pouget, 2004). Though these perspectives differ from PDP in many respects, all of these efforts share the idea that cognition emerges from interactions among populations of neurons whose function can be studied in simplified, abstract form."
Parallel Distributed Processing at 25: Further Explorations in the Microstructure of Cognition T. T. Rogers, J. L. McClelland / Cognitive Science 38 (2014) p1062-1063

If all you're saying is that neural networks could develop consciousness one day, sure, and nothing I said contradicts that. Our brains are neural networks, so it stands to reason they could do what our brains can do. But the technical hurdles are huge.

You need at least two things to get there:

1. Enough computing power to support it.
2. Insight into how consciousness is structured.

1 is hard because a single brain alone is about as powerful as a significant chunk of worldwide computing, the gulf between our current power and what we would need is about... 100% of what we would need. We are so woefully under resourced for that. You also need to solve how to power the computers without cooking the planet, which is not something we're even close to solving currently.

2 means that we can't just throw more power or training at the problem. Modern NN modules have an underlying theory that makes them work. They're essentially statistical curve-fitting machines. We don't currently have a good theoretical model that would allow us to structure the NN to create a consciousness. It's not even on the horizon yet.

Those are two enormous hurdles. I think saying modern NN design can create consciousness is like Jules Verne in 1867 saying we can get to the Moon with a cannon because of "what progress artillery science has made in the last few years".

Moon rockets are essentially artillery science in many ways, yes, but Jules Verne was still a century away in terms of supporting technologies, raw power, and essential insights into how to do it.

We're on the same page about consciousness then. My original comment only pointed out that current AI have problems that we have because they replicate how we work and it seems that people don't like recognising that very obvious fact that we have the exact problems that LLMs have. LLMs aren't rational because we inherently are not rational. That was the only point I was originally trying to make.

For AGI or UGI to exist, massive hurdles will need to be made, likely an entire restructuring of it. I think LLMs will continue to get smarter and likely exceed us but it will not be perfect without a massive rework.

Personally and this is pure speculation, I wouldn't be surprised if AGI or UGI is only possible with the help of a highly advanced AI. Similar to how microbiologist are only now starting to unravel protein synthesis with the help of AI. I think the shear volume of data that needs processing requires something like a highly evolved AI to understand, and that current technology is purely a stepping stone for something more.