Why is the brain so "efficient"? [closed]

Question

By efficient I mean, that why is the brain able to learn tasks, for example driving, at a much less energy cost than Machine learning models?

From a quick google search it seems like it took about 55 million Kwh to train GPT-4, whereas a human consumes around 3 Kwh in a day (2500 calories, food consumption) with a total of 87 thousand Kwh for a human who lives till the age of 80.

Quite apparently, there is a huge difference in both ability and efficiency.

The reason I am asking this question here and not on the neuroscience stack-exchange is because I am hoping to get a physical perspective.

Answer 1

You are observing that training one large computer model took about 1000 times the energy that one human will burn in their lifetime. But you are forgetting that the human brain has been honed for its current skillset over many thousands of generations, and that there are many thousands of humans whose intellectual output went into training this computer model. "Many thousands" is in both cases an offensive understatement — especially when you consider that the problem-solving and pattern-matching abilities of the human brain are also present, to various degrees, in our various mammalian cousins.

I don't think your question will be answered by some physics digression about the relative energy requirements of transistors as compared with neurons. I think the meat of your question is about deciding carefully which energy inputs to include so that you are really doing an apples-to-apples comparison.

Answer 2

In my opinion there is no physics angle here.

Compared to humans the current state of machine learning is vastly inefficient.

The difference is at the level of information processing.

I think the following youtube video, published in April 2023, is quite illuminating Why AI art struggles with hands

I think the content of that video illustrates vividly that Machine learning is processing information in a way that is vastly different from how biological intelligence is processing information.

Another vivid example is something that Andrej Karpathy described in the April 2019 Tesla autonomy day, discussing object recognition. The engineers noticed: when the object recognition system was presented with an image of a car with a bicycle strapped to the back the object recognition system reported recognition of a car, and a bicycle, as separate objects.

In order to give the object recognition system the capability to recognize that a bicycle is being transported by strapping it to the back of a car they had to augment the data set with a large number of annotated images of cars with a bicycle strapped to the back.

As we know: for us humans: even a person who has never seen an instance of a bicycle strapped to car will instantly recognize it. You are aware of the general concept of strapping something to a car, and you know bicycles. For a human that is sufficient.

It seems to me that in the human brain information is flowing far more freely than in machine learning.

It seems highly plausible that in the brain information is processed both top-down, and bottom-top.

Example:
In the visual cortex large areas responsible for object recognition receive the signals from the retina in such a way that the color information is not there. That is, large areas of the visual cortex receive as input information comparable to the information of a movie in grayscale, only shades of grey. A comparitively smaller area is dedicated to processing color information. Cross-links between the various areas enable the information flow that allows the brain to assign color to the objects we recognize.

(This explains why watching a black-and-white movie is not particularly straining for us. Much of our image processing does not use color information anyway.)

In any information processing there is a trade-off between reliability and speed.

Supposition:
Mental diseases such as schizophrenia are disorders where information is flowing between specialized areas of the brain too freely.

Supposition: All humans are actually teetering on the brink of mental disorder. Evolution has found a way to make the brain extraordinarily efficient at information processing. It would appear evolution has pushed the brain to the edge of a cliff. You are very high functioning, but if for whatever reason you slip over the edge you will tend to slide down ever further

Contrast:
The way that machine learning processes information is robust. Supposition: the very factors that make the process robust also make it inefficient.

Answer 3

That's a very interesting question. The brain and machine learning models are both capable of learning tasks, but they do so in very different ways. The brain is a biological system that consists of billions of neurons and synapses, which communicate with each other through electrical and chemical signals. The brain learns by adjusting the strength of these connections based on the feedback it receives from the environment. This process is called synaptic plasticity, and it allows the brain to adapt to changing situations and acquire new skills.

Machine learning models, on the other hand, are artificial systems that are designed to learn from data using mathematical algorithms. Machine learning models can be divided into two types: supervised and unsupervised. Supervised models learn from labeled data, which means that each input has a corresponding output or target value. For example, a supervised model can learn to recognize handwritten digits by being trained on images of digits with their correct labels. Unsupervised models learn from unlabeled data, which means that there is no predefined output or target value for each input. For example, an unsupervised model can learn to cluster similar images together by finding patterns in the data.

The main difference between the brain and machine learning models is that the brain learns by predicting its own perceptions, while machine learning models learn by optimizing their own outputs. The brain uses prior knowledge and expectations to make inferences about the causes of sensory inputs, while machine learning models use objective criteria and constraints to minimize errors between their outputs and targets. The brain is more flexible and robust than machine learning models, as it can handle uncertainty and ambiguity better than any fixed algorithm.

However, this does not mean that the brain is superior to machine learning models in every aspect. Machine learning models have some advantages over the brain in terms of speed, scalability, accuracy, and generalization. Machine learning models can process large amounts of data faster than the brain can process information from its limited sensory channels. Machine learning models can also be trained on specialized hardware like graphics processor units (GPUs), which draw more power than traditional CPUs. Machine learning models can achieve higher accuracy than the brain in some tasks that require precise calculations or measurements. Machine learning models can also generalize better than the brain in tasks that involve new or unseen data.

Therefore, the answer to your question depends on what kind of task you are interested in. If you want to learn how to drive a car or speak a foreign language, then you might benefit more from using your own brain than any machine learning model. But if you want to perform complex scientific calculations or analyze large datasets or create realistic images or sounds, then you might benefit more from using a machine learning model than your own brain.

As an example, chess bots like stockfish are highly accurate and formidable than GM's.