What Is Intelligence? With David Krakauer

Defining intelligence

0:00That was quite the overview on what it is to be intelligent. It went places I didn't even know intelligence went. Deep. Yeah, and for some reason, I still feel stupid. Coming up, StarTalk Special Edition. Everything you need to know about intelligence.

0:17Welcome to StarTalk. Your place in the universe where science and pop culture collide. StarTalk begins right now.

0:30This is StarTalk Special Edition. Neil deGrasse Tyson, you are a personal astrophysicist. And if it's Special Edition, it means Gary O'Reilly in the house. Hi, Neil. Hey, former soccer pro. Yes. Soccer announcer. Of course, we got Chuck Nice, baby. That's right, Tempest in a teacup. The Lord of Nice. Lord of Nice. All right. Exactly. So, Gary, what did you and your producers stitch together for today? Well, with all this talk of artificial intelligence, has everyone stopped to think, what is intelligence in general?

1:03That's what we're doing today. Everybody thinks they know what it is, and no one can really define it. That's because they're all dumbasses. To Chuck's point, how did intelligence evolve, and why? What chances are there of beings far smarter than us? That's a low bar. Are we? Yeah. If we are the universe's way of knowing itself, is the universe screwed? Neil. I mean, if we're the best it's got to figure itself out.

1:34You know, maybe I want to know that answer, maybe I don't, but we're going to ask it anyway. And that quote, I think, is traceable to Carl Sagan. It might have been in one of the episodes of Cosmos, the original Cosmos. Well, let's find out what our guest has to say.

Evolutionary origins and complexity

1:49You talk about, did intelligence evolve and in what way? We have an evolutionary biologist. Ooh. Yeah, you need them every now and then. Yep. Straighten things out. And he's also president and the William H. Miller professor of complex systems at the Santa Fe Institute. Of course, that would be in New Mexico. Yes. I'm sorry, you mean New America? No.

2:14Well spotted, Chuck. And he specializes in complexity and intelligence. Join me in welcoming David Krakauer. David, welcome. It's wonderful to be back in person. In person. We can verify that. Poke him, make sure he's real. I'm here. Not a hologram. Not a hologram. So, I like the fact that you study complex systems, but you have roots in evolutionary biology. Because that's where we are life forms and that's where it all begins.

2:46So, could you just shed some luminosity on the evolution of intelligence as we think about it? Yeah. Well, okay. So, the first question is, what is the relationship between life and intelligence? Can one conceive of a life that wasn't intelligent? Can one consider or conceive of an intelligence that's not alive? I think the latter, yes, and we might talk about that. But the former, probably no.

3:16And so, as soon as you have life, you have some form of intelligence. Mm-hmm. Okay. So, what is that? And I guess the canonical, simple argument would be, these are some form of organized matter that can sense the environment and act in a way that increases its representation in the world, either through replication or what have you. In the interest of its own survival. In the interest of its own survival. And so, any life form that can sense and behave in a way that's adaptive,

3:50we would have to grant, has some form of intelligence. So, the interesting theoretical problem here is that life feels a bit binary. You don't say that a flea is less alive than an elephant. True. So, that's kind of an interesting fact. Whereas, intelligence is not. So, how does a kind of binary system give rise to a continuous one? And that's kind of the theory challenge for the origin of intelligence. But, tell me again what you mean by the binarity of this. So, for example, I mean... You're either alive or you're not.

4:22Exactly. But that doesn't mean that the level of intelligence you have is dependent on you being alive or not. I see what you're saying. It's a gradation. So, intelligence has a continuum. Yes. You're either alive or dead, does not. Right. Got it. Okay. Just to clarify that. So, that's, okay. So, that's the first question, right? So, over the course of time, what are the forces that have created higher levels of intelligence, if they have? That's an interesting question in itself. Maybe it's entirely context dependent. Maybe there's a sense in which a bacterium in the world is more intelligent than a human destroying a world.

4:52But, you know, one could go there if one wanted. In other words, you would say humans with high intelligence who destroy their world are not acting in the interest of their survival. No, I would say they're stupid. And that would... Definitely stupid. There would be a demerit against their intelligence scale. Right. Absolutely. Yes. Absolutely. And what about this? Because when you said bacterium, is there any kind of collective intelligence where the whole is greater than the sum? For sure. I mean, in a sense, that's our own brains, right? So, we have billions of nerve cells, each of which is quite simple.

5:24Correct. On its own. That's right. And they connect and they create thought, mathematics, music, all the things that we have value. Chuck, that was a stupid question. Well, no, I didn't mean as a part of one entity like our brain. I meant like our gut, where those bacteria are all different types, but they work together to cause us to have emotions and cause us to think certain ways. They're operating in tandem with our brain, but they're all separate bacteria.

5:54Absolutely. I mean, there's this slight paradox that happens. So, this is the world of collective intelligence, like swarms and shoals and schools. Right. Because normally when you think of a collective intelligence, is the ensemble of organisms operating in a way that one individual would not otherwise understand, but the ensemble does. Right. I mean, this is emerging things like flocking. Flocking. Of birds, I guess. Or, yeah, or when you see fish change all direction all at once.

6:25Yeah. Even though, I don't even know if that's like electrochemical or not, but who knows, you know. No, but there's an interesting thing that happens here that's surprising, right, which is that brains, made of very simple cells, produce extraordinary thoughts. Whereas human societies, kind of made of extraordinary individual thoughts, produces a total shit show in the aggregate. So, there's an interesting relationship between parts and holes. Right. If you have very smart parts, you don't necessarily have a smart hole. That's true. Because there's nothing less intelligent than a mob of people with pitchforks and tiki torches.

7:00True. It seems to be inverse correlated with the number of people who are part of the mob. Right. Exactly. Yeah. They act less and less intelligently. Less and less intelligent, but more and more as a unit. Yes. Isn't that funny? Okay. Like, the less intelligent that mob is, the more in constant they act. The more monolithic. Yeah, the more monolithic they are. The more monolithic that is an intent and action. Right. And that is, in some sense, gets back to this question of evolution. Because in evolution, you call these major evolutionary transitions.

7:32So you go from a single cell, like a prokaryotic cell, like a bacteria, bacterial mat, kind of a collective, a single eukaryotic cell to a multi-cell, to a tissue, to an organism. So that is the story. And in every case, it's not clear whether the aggregate is somehow more restrictive, less intelligent than the parts. So, if you're talking about around evolution here, when it comes to our human intelligence,

Language and signaling systems

8:01what's the turning point when we get complicated language? Or was it something else? Well, I mean, that, you know... Wait, let me prepend that. All right. Because you're saying, well, we have language and, you know, other animals don't. Or do they not have language? I mean, so let's broaden that question to when organisms communicate complex information.

8:28Do we call it language when a bee communicates to another bee in a hive where to go or where to move the camp? And on the limit that we would think of as a limit as our complex language, do you distinguish as an evo-bio person, not evil-bio, but evo-bio person, do you distinguish dolphins squeaking at each other from humans squeaking at each other or not? Yeah, it's a difficult one.

9:00So I think the jury is out. But one quite useful distinction I quite like, so in the biology community, people talk about signaling and communicating as being somewhat distinct from language. And so, I guess with human language, Alexander von Humboldt had a nice phrase that I like here. And he said... Tell us briefly about him. So he was a great 18th century naturalist, kind of invented ecology as we know it, was a great explorer, took extensive measurements, worked on weather systems.

9:30And he was German-French? German. German. German, yeah. And actually, my friend Andrea Wolff wrote his biography, The Invention of Nature. Beautiful book. Plug for the book. And if I remember, I think I own some of his books. Didn't he write about... Cosmos. What's that? He wrote a whole series of books called Cosmos. Yes. And I think one of his books tries to connect our understanding of science to make a better society. I mean, I think he was a civic scientist as part of his intellectual efforts. Oh, he'd be so happy if he saw us today.

10:00I know. Well, but that's an interesting point. I mean, that romantic movement where they were all, in some sense, entangled with each other. You didn't get to just be a scientist or a good citizen. You should be both. Entangled in a good way. In a good way. So tell us more. I'm sorry I interrupted you. So, yeah. So he has this phrase where he says, human language is making infinite use of finite means. Interesting. And what he meant by that is, if you have a grammar, like a rule system, you can construct almost any thought out of it.

10:31Absolutely. And that flexibility that you get with language is not as evident in simple signaling systems, like chemical ones, or even gestural ones, actually. I like that. And how about, now we know whales have a certain type of language. We don't know how extensive it is, but humpbacks call for help when killer whales or orcas are around and the other whales respond. So clearly- They call for backup? They call for backup, you know?

11:03And they'll even attack the orcas when they're going after seals. They're not just going after the other whales. So that's language. How are monkeys warn each other about jaguars who are in the forest? Just to be clear, jaguars, if memory serves, can climb trees in ways that lions cannot. So a monkey is not entirely safe in a tree. And that's exactly why they warn each other. They have the concern. Right. So what I'm saying is that can be considered signaling, but it can also be considered language.

11:36Well, okay. So, all right. Go ahead. So, yeah. So people, so there's an American philosopher, Charles Peirce, and he made a distinction. So I'm going to get into the signal thing. Okay. And he made a distinction between what he called an index, an icon, and a symbol. So the point is this. So vervet monkeys, Cheney and Sefer famously worked on these different signals. Birds. So if you say falcon, or they emit a sound, they crouch down. So there's a correlated behavior, right? And if there's a kind of terrestrial predator, they'll climb a tree and so on.

12:07So they have those. And in the kind of signaling literature, that would be described as an index, right? There's a kind of mapping one-to-one. That sound always means, watch out, there's an aerial predator. Gotcha. Now, with humans. Pavlov knew all about this. And Pavlov, completely, absolutely. Yes. So you could say that's how it's acquired, maybe, or it's innate and so on. Sure, right. Exactly. Or that's where it is. It doesn't even have to, it can be just a sound, like the bell. Anything. It can be a good sound. Right. Okay. Now, in the human case, you can substitute any sound for any appropriate response of

12:41almost arbitrary complication. Correct. And that's something you learn in your lifetime. Wow. And I think that, so it's not that they don't have something that's embryonic of our ability, but what we do is quite extraordinary. Oh, yeah. I didn't think of it that way, but you're absolutely right. Because now I'm thinking about, in all the movies, they're like, and here's the signal. Caw-caw! Right. And that means, take out your math textbook. So, yeah. Okay. All right. Well, that's very good.

13:19Mmm. Quasars. The word is a pseudo-acronym for quasi-stellar radio source. What's a quasar? These are objects that are among the most distant things known in the universe. How do we know about them at all? Because they're really, really luminous. Some of them are as luminous as 500 trillion suns. That's why we can see them to the edge of the universe.

13:53If you like that fact, you can find 4,999 more in Lost in Space. 5,000 facts to help navigate the universe.

14:09Lost in Space is now available wherever you get your books. Does the expression of complex language predate the development of what we think of as our intelligence, or did it follow it? Good question. Oh, wow. Good question. Why don't you have answers to this? Yeah, I got to say. Why don't we invite you on the show? I don't know.

14:39To humiliate me. I don't want you to hear that. I just had a good question. I want you to say, I have an answer to that. Yeah, I don't. These don't, you know, the reality is, you know this. They don't have final, conclusive answers. Okay. They're not even close. But I would say that, okay, so I'm going to define it. I think I did last time I was talking to you guys. So intelligence for me is making hard problems easy. Okay? That's basically it. Across the animal species, every species, even plants. So, you know, finding an efficient way of solving a problem, not an inefficient way of solving. We'll get to AI in a minute, which is all about inefficient solutions to problems, but that's something else.

15:12So I think that's everywhere. All life forms find elegant ways of solving problems in their world. Now, the question is, what does language add above and beyond an innate repertoire or even a simple Pavlovian association? And I think the possibility for almost arbitrarily complex behaviors, which don't necessarily produce intelligent behavior, right? They can produce warfare and so forth and strife. But that's the thing that we do. And I think it somewhat decouples from the question of intelligence, right?

15:43It creates great intelligence, but it also creates great stupidity. All right.

Intelligence versus consciousness

15:47How are we distinguishing between intelligence and consciousness? Wow.

15:54Yeah. We're just barely getting intelligence here. You could throw consciousness right on top of it? I actually think consciousness is easier than intelligence. Because. Easier. And I read some of your work and you cite the fact that mathematicians will tell you their unconscious mind, subconscious, unconscious, will solve the math problem, but not your conscious mind. Yeah, interesting. Yeah. So between intelligence and consciousness, how are we distinguishing? So, okay, I want to say two things. You're talking about the old edge, sleep on it. Yeah, sleep on it. Exactly, sleep on it.

16:24The Kikuli problem, exactly that. So let me do a bit of both here. I'm not sure the best way to do it. So on the consciousness side, I quite like this way of defining it and Neil Seth, who works on consciousness, does. He'll say intelligence is about problem solving and consciousness is about being and feeling. So I consider consciousness very primordial, right? In other words, any organism that lives in an environment that it has to sense and maintain its integrity in some way. It could be a single-celled organism.

16:56It could be a cat. It could be a whale. They will manifest in consciousness. It's about feeling and the sense of self that's required to survive, right? If you didn't have a sense of self, it's like, oh my God, my foot's gone. It doesn't matter, right? It sort of matters. And intelligence is about problem solving. Okay, now, so that's the first point. Now, unconsciousness that we were talking about is very interesting, right? Which is, for most of our evolutionary history, we didn't have language. And so somehow we were having to solve problems in the world as every other animal does without that kind of declarative self-awareness.

17:26And so, in that sense, there's more, if you like, neural resources available to solve problems unconsciously than in that tiny little thin layer of cells that we call the frontal cortex that we associate with conscious deliberation. Wait, wait. Just because early on we didn't have a language doesn't mean the thoughts were subconscious. Well, that's also an interesting question, right? So for most people— Would you stop telling me I have interesting questions? Well, you do, man. And so this is the thing where a lot of people would associate self-consciousness with deliberation, which seems to depend on the possession of language.

18:02Yes. Yes. But that's only because we have the language that we have, okay? Because we don't know if a lion is deliberating, you know, or if other apes that were a part of that family, if they actually look up and go, damn, you know, if I didn't want to throw my poop right now, that'd be a very beautiful thing to look at. You know, we don't know that. You don't know. But the fact that we have this inner dialogue is the reason why we believe that we are, I think, therefore, I am.

18:38Right, because we've all seen Animal Kingdom videos where two animals, typically the same species, there's a standoff, and they sort of pace back and forth. You know something's going on in their head. Something has to be going on. Something's like, should I do this? Right. Would I make it out of this alive? You don't want none of this. Don't make me. You don't want none of this. You're not built like that, son. Don't make me put my foot up your ass. Exactly. You can't tell me nothing's going on in their brains. No, no, no. I wouldn't say that for a second.

19:08I think that, so the question of intelligence, as I said at the beginning, is ubiquitous, and degrees of it, and high degrees of it in the non-human world. Absolutely. But there's an interesting fact, right, which is that there is a tighter coupling between signaling and behavior in non-humans and humans. You could watch humans all day long in a library and have no idea what they're doing or deliberating about. And the examples you gave in animal behavior, there are clues in terms of how they're manifesting their intelligence in their behavior.

19:41Language is this weird thing that's kind of almost decoupling us from our immediate space-time context. And that bit is the bit that feels, and it might not be, somewhat unique to our species. No, and it might not be, but you're absolutely right. And earlier offline, David and I were talking about comedians and how improvisation is this very unique thing that they do. And I mentioned to him that when I'm on stage and I'm improvising, the thing that's weird is I can almost feel my brain being semi-conscious.

20:19There's like a part of my brain that takes over that moment, and I'm not actually thinking. Have you seen The Doctor?

20:29I know. And most people right now are listening like, Chuck, are you ever actually thinking? Is that intuition? Does that become an intuition? Well, no. It's funny because the way I look at it is people say that you have a subconscious mind or an unconscious mind. And what I feel when I'm in that moment is that that part of my being, my subconscious, is accessible to me. So that it's kind of a slight overlap in the Venn diagram of my consciousness and unconsciousness, where my executive function is able to dip into the subconscious and pull something out.

21:11That's the way I describe it.

Expertise and the unconscious mind

21:13No, and I think that's, I mean, that's the, I mean, there's an interesting point that's not talked about enough, this distinction between, if you break intelligence down a little bit into skill and expertise. So expertise, the essence of it is practice that gives rise to automaticity, that you don't have to think about it. If you're a great athlete, right, we were talking about that, you're taking a shot. The last thing you want to do is think about it. Think about it. Because you're not going to make it. It's exactly the same thing. You're not going to take the shot. It's exactly the same with math, right? You're learning mathematics. You get it, stay out of your head. Exactly. So there is this very thing that happens.

21:44It looks as if you need language, that early deliberative process. Yeah. And then you practice out of the language. And then you're returning to a state that's not even linguistic, right? So it's almost pure program, pure reflex. But you have to do it via language. So we learn mathematics, like we learn sports. Right. But you practice it enough, you're not thinking about, oh my God, how do I integrate that function? You just do it. And I think expertise is, in fact, paradoxically, not thinking. Yes. Right. Because you've built the muscle memories. Right. Or the conception memories.

22:16Yeah. Right. So you're not consciously thinking. Right. When you're in that zone, your subconscious is feeding you. It's feeding. In the moment. Right. Without forethought, it's happening in the moment. Right. Yeah. So our conscious mind has access to our senses. Mm-hmm. All the whole sensorial palette. And it's a survival, but our subconscious, our unconscious mind, does that have access to the senses? And is it then, doesn't want anything to do with our survival? It's got another executive sort of mission statement. Oh, I like that. Because, you know, when you're dreaming, you're not really responsive to touch.

22:49I mean, nobody, you know, it's, you're disconnected. You can play music. The dream is in its own space. It's almost like it's making its own sensory world. Well, that's what, so actually my colleague, the novelist, Cormac McCarthy, he wrote this beautiful essay called The Kekule Problem. Yes. Right. The discovery of the structure of the benzene ring. And there was kind of a dreamt solution. The snake eats its tail. The snake eats its tail. And Cormac's point was very interesting. He said, the unconscious mind evolved long before language emerged.

23:20Because other animals presumably have an unconscious too. Right. And it doesn't want to communicate to us in language. So it communicates with us visually. So, for example, the Ouroboros, the snake that eats its own tail. And so, Kekule wakes up and he says, wait a minute, what's going on? What are you trying to tell me? Why can't you just tell me? And the unconscious says, I don't do language, you moron. Right. I do symbols. I do visual intuition. Yes. And then he imposes on it the language of modern chemistry through language.

23:50Wow. Why are we programmed to absorb information that way in a better way? Because that's the problem-solving aspect. Obviously, if we're asleep, there's no noise. The brain is quiet. Well, you value judging it, saying it's better. I would say it's different. Okay. I take that point. And as an educator, educators know some people are visual learners. Some are rote learners. So what you just said, it cracks this nut of human cognition for me. Because, for example, when I'm speaking, typically I use my hands.

24:24And not just in an Italian way where you put emphasis like this. I will gesture an image of what I'm also saying. I'll say Earth is going around the sun. So that there's another way it's entering their mind. And I find it to be much more potent that way. Accessing both parts of that brain. Yeah, we're totally multimodal. I mean, exactly as you say. I mean, it's visual, olfactory. Right. And linguistic, they all buttress one another, right, and consolidate.

24:57But this question of, I mean, the argument would be that what we're calling for at the present moment, the unconscious, whatever that means, these words are all a little bit fluid, is older. So to the evolutionary question, it's been around for much longer. Ancient animal brain. It is. It is exactly that. So animals are calculated trajectories using rules of thumb heuristics all the time. We then give it this additional gloss. But a lot of the calculation, and this has been written about endlessly by mathematicians reflecting on how they solve problems,

25:28they're not necessarily solving the problems deductively, symbolically. They go to sleep, dream clues to the solution to the problem, and then wake up and impose on top of it this secondary armature of language, which is useful because you can communicate it. How could I communicate to you? Well, I dreamt of this snail that's sort of eating a leopard. And you'd say to me, I don't know what you're talking about. So the symbolic thing doesn't naturally lend itself to communication in the way the language does. Okay. However, you know, for the longest time, we did communicate in symbols written form.

26:02So before we had alphabets, which everybody does now, which some of the alphabets of the world right now are pictograms. They're not actual. Well, Chinese. The Chinese are kanji, right? So, you know, for the longest time, we drew pictures to communicate our thoughts. And once we figured out words, that's where we are now. But getting back to your point of the source of where an insight might come, there's the famous discussion of the mathematician Ramanujan, if I remember his name correctly, who, he was Hindu.

26:34And he would say, the solution just came to him. And it was, you know, Vishnu or Krishna putting the solution on his tongue to be shared with others. Well, he was wrong. It was Jesus. Jesus. You're only encouraging him. So in what way does one's cultural bias influence what they think is the operating force on their thoughts?

27:06That's a very, you know, that's a very difficult question to answer, right? Because presumably, I would guess that the underlying unconscious process is quite universal. And then the secondary form it takes when we communicate it. On this show, I think when he says universal, he means earthwide. I mean, yeah, I do. I mean global. I mean global. On this show. I mean global. Yeah, terrestrial. For all life. Yes, good. For all human life. Human. The earth life. Earth life. And this secondary cultural layer that we impose, whether we call it, you know, Jesus

27:41or Vishnu or what have you, or the Buddha, that's where the variability probably exists. But I would guess that the underlying dream state or unconscious solution is probably quite universal. Interesting. So let me get back to that dream state. Do we know the title of the short story that became Blade Runner? Do androids dream of electric sheep? So that brings us. Do androids dream of electric sheep? There you go. Yeah. Now somebody was high. Well, he was. High. He was. Philip K. Dick was always high. Oh, awesome. Philip K. Dick.

28:11I love it. So that gets us to the question, will AI never be us because AI doesn't dream?

Large language models and energy

28:20Ah. And will solutions available from our dream state never be available to AI? Hmm. So there's a few things to say about this, right? I hope more than a few. There are many things to say about this. So we started with the evolution story, and I made the claim, right, that sentience or consciousness or sensation was primordial, that all beings have it. That is definitely not true of NLLM. Okay. So point one.

28:52The large language model. The large language model, right. It has a completely different path towards its intelligence, if you like. Secondly, it immediately starts with language. It doesn't have an unconscious. There is no unconscious. Right. So it's just got a linguistic foundation right at the beginning. Thirdly, it's a collective linguistic. I mean, it's not an individual language learner. It's every book ever written. That's right. So there are some really fundamental differences. Including 14 of my books that are now in a class action suit. I bet. Just thought I would tell you that. Absolutely. So these are important differences.

29:25So start with the last thing that we got, right, which is language. And ignore what every other organism has, which is this, if you like, homeostatic, sentient form of existence, pre-language. Right. Wait, wait. So where are you taking us with that? You can't just say that and not take me somewhere. Okay. So now this opens this whole debate about what we mean by an LLM, a large language model,

29:55an AI being intelligent without being alive. And I'm asking you, do they dream? They definitely don't dream. Definitely don't dream. So does that limit where it can take us relative to where we can go ourselves? Okay. So we were talking about this before. I mean, you should repeat that earlier offline, that point you made, because I think it's an interesting one about what you can do with language. So offline, when Dave and I were talking, I said that with language, I can explain anything

30:30to you, no matter what it is, that if I can find the right words, and you know those words, right, it doesn't make a difference, what it is that I'm trying to convey to you, I can do it. And that is extremely powerful. And then you said what? Yeah, so we were talking about the recent thing in the last week that everyone's talking about, which is LLM solving a mathematics problem, the singularities of the Navier-Stokes equations, this kind of stuff, right? This dynamics of fluids.

31:02And there was this long-lost, you know, an outstanding question with a prize behind it about some properties of these equations, which is, do they blow up and so on? Do they have singularities in them? And so out comes an LLM recently, and at considerable expense, I mean, tens of millions of dollars, thousands of agents, it finds a solution. But it's doing it by being fed text. Not geometry, it doesn't have a visual apparatus, right?

31:33So what does that mean, that we can solve math problems by only being trained on language? And that's what we were talking about. Right. And I think what we've discovered, the interesting thing about purely linguistic intelligence, if you like, an LLM, is that you can solve almost any problem with it, but you solve it very inefficiently. Right. And then you gave the example of eating soup, which I thought was brilliant. Well, yeah, you could eat a bowl of soup with a fork. Slowly. You could. Slowly. There you go.

32:04Enough time, enough energy. You can solve math problem with language. Enough time, enough energy. And human intelligence is based on a tough world where you don't have enough time and you don't have enough energy. That's why we invented all these other languages to solve problems. It's a very important distinction. Plus we have, you know, a billion years testing ground to make it happen efficiently. Exactly. Because if we only got smart doing LLMs, we would have burnt our energy load, you know,

32:36hundreds of thousands of years ago. We would have never gotten to where we are. Yeah, we'd have a lot of knowledge sitting in the dark. But that's an important point about why I don't like calling them intelligence, because they're very capable. They can solve problems. But for human beings, you don't call someone intelligent just because they're capable. If I said, you know, Gary, could you pass me that glass of water? And you got him, ran around the room 20 times and then gave it to me. We'd all say, that's kind of weird.

33:08What's Gary doing? What's Gary's problem? What's his problem? He achieved the, solved the problem, right? He gave me the water. Yeah, at the end, you got your water. At the end, I got the water. And I think a lot of what they're doing now is getting up and running around the room 20 times. And people are calling it intelligence. Because it's solved for the benchmark. Just call it Gary. No, just call it capability. So that could implode, because it could exceed our capacity to deliver it the energy it needs to solve that problem. Exactly. So there could be an upper limit, a functional upper limit, short of tapping the sun directly,

33:43like we tap a keg. But see, if we were truly intelligent, if we were truly intelligent, then we would use this as a motivation to create that energy. No, I got a better answer. Go ahead. If the AI were truly intelligent. Oh, God. It would figure out how to do it with less energy. There it is. No, you're right. That's it. I got a mic drop. No, you got a mic drop there. What happened to my plastic mic? Well, here it is. You dropped it. Okay. Wait, and mic drop.

34:13Mic drop. There you go. By the way, do you just carry around a microphone? No.

34:19No, I just have stuff in arm's reach in my office. You discuss things called complementary and competitive cognitive artifacts.

Complementary and competitive tools

34:28There's way too many syllables. No. I didn't invent my fault. Yeah. And where does AI sit in which column? Yeah. Cognitive and... No, complementary and competitive cognitive artifacts. As two separate... Yes. Got it. So take us there. Okay. So this goes back, in fact, to a French philosopher, Henry Bergson, who debated relativity, with Einstein, as you know, right in the 20s. But in 1907, he writes this book, Creative Evolution, right? And he says, we shouldn't be called homo sapiens, wise humans, right?

35:02Which came from Linnaeus. We should be called homo faba, humans that make things, humans that make tools. And Bergson's whole theory was what makes humans different to the extent that we are is we not only make tools, but we make tools to make tools. True. Right? So that has been somewhat neglected in this discussion of intelligence, right? Because in some sense, you can't really think of human intelligence without thinking about painting, symbols, writing, various kinds of calculating tools and artifacts.

35:36So that's the background. And that has its own development. Quick question. Yes, we have opposable thumbs. Yes, we make tools. Why should that alone distinguish anything when we don't know if whales had opposable thumbs, would they be making tools themselves? They just can't. But we do know that if we did not have opposable thumbs, we would make opposable thumbs so that we could make other stuff. I'm just saying, just because they don't doesn't mean they wouldn't if they could.

36:13That's true. I mean... That's a good sense. And there's a lot of debate about this. I mean, there was a lot of early discussion, I remember, on language, which is whether or not, the reason why other primates don't speak is there's some anatomical constraint. And if that wasn't in place, perhaps they would have evolved language. Oh, the position of the larynx. That sort of debate. Exactly. Evolved to be in a lower position. Exactly. So breathing. It was a genetic mutation, basically, that enabled this. That's nature. So that's where it begins, right? This idea that humans make tools.

36:43So I got very interested in that idea. And there's archaeologist, French archaeologist, André Goulran. And he said, what are tools about? And he said, it's an additional memory system, is what he said. He said, we've got genetic memory stored in our genomes. We've got learned memory stored in our brains. And you've got cultural memory stored in language, art, stuff. And so he actually argued, went further than Bergson. And he said, to be anthropogenic is to be technogenic.

37:16Wow. He views them as indissoluble. Look at that. That somehow technology and humanity are kind of wrapped together in some way. So that's where I jump off with this kind of mathematical theory of tools. And the basic idea is that, right, there are tools which I call complementary, things that enhance our ability to reason, like a map. We talked about it last time. I think you might even have an abacus in here. Yes, there is an abacus. Yeah, around somewhere.

37:46Okay, but that's a good example of what I call complementary. And it helps you reason. If you don't have it, as we know from studies of abacus schools, these kids can then do that calculation. Don't even need it. That's right. Don't even need it. It's in their head. So this is the key point. The abacus is a scaffolding for thinking better. Look at that. Right? So now in the competitive case, these are tools that don't scaffold anything. They replace it. So take a GPS versus a map. Ah, interesting.

38:17So the idea is so you can develop this kind of theory. Oh, no. And it says that if you outsource completely, you lose the ability to reason. We're becoming dumber and dumber. The smarter we become. The smarter we become, the more stupid we become. Yeah, because we're replacing the accrued knowledge. Oh, no. Wait, wait, wait, wait, wait, pause, pause for a second. What? I have to restate what you said just so I believe I understand it. In the case of the abacus, the abacus is a means to calculate,

38:50and this becomes autonomic within me, and then I do it, but the act of doing it is implicitly in reference to the abacus that trained me. Now, also, we have at least two or three generations of people still on Earth who know how to read maps, and they can translate what's on a map to where they are in the world. There are no maps anymore. It's just GPS barking at you, turn left, turn right, so up ahead on the right.

39:21And you're saying we are handicapping a generation of people who do not know how to read maps, even how to orient a map or what the legend is on a map. They have no idea. But why is that a bad thing? Okay, so it's interesting. So here's, so we've got... Did I get it right? I think I got it right. So we need more vocabulary here than just intelligence, right? So we have, let's just introduce some terms. One is capability. So using a GPS or using a map both make you capable.

39:52They both help you to navigate. No problem. So benchmarks on AI are testing capability. They can do it. They can calculate. They can solve problems. They can navigate. Good, right? So that's capability. Then there's competence. Different. Competence is, okay, give me the GPS. Now navigate. And that reveals dependence. And so what we're doing is we're moving into a world to when you say, does it matter? As long as you always have it. Yes. And you're not interested in competence,

40:24but only capability. Oh. Right. And the problem is that gives you wise to a world of total dependence on the provider of your capability. Correct. And that provider, who will not be named in this conversation, because we know who they are, don't necessarily have our best interests in mind. Absolutely. So I went through this intellectually some years ago. I'm old enough to have, I think I'm older than you, to be formally trained, formally trained on a slide rule.

40:55Okay? It must have been the last year that happened because between that year and the next, the four-function calculator dropped in price from $200 to $40. So it became the price of like a textbook. And so thenceforth, they stopped teaching you the slide rule. Yeah. And I got a slide rule here too.

41:15This was my high school slide rule. All right? Now there's a lot of sort of mathematical insights into the design of a slide rule. And you learn about logarithms implicitly. And then I, this is made of plastic, but earlier I have a vintage model. You know, this used to make these out of elephant tusk. And I think this is just wood. But anyhow, in my high school, you had a holster with your,

41:46and the bigger the slide rule, the more precise you can be in your measurements. So we'd be walking down the, this math class ain't big enough for the two of us. But here's my point. My intellectual challenge was, if I never see a slide rule again, because a calculator renders it obsolete, what am I missing? And I said to myself, if I'm always near a calculator,

42:17You're missing nothing. I'm missing nothing. And let me move on to some other thing that requires my intellect to solve. So that was my, that was my coming of age to embrace a calculator and release the slide rule to the dust men of history. So yeah, in fact, when Hewlett Packard made their first, affordable calculator, all of the slide rule companies went out of business. The same year, 1973. Yeah, right. That's when, that's it. That time. That's it.

42:47So there is something missing though. So right. So one possibility, it's a bit like vitamins, like vitamin C. You might say, you know what? I can't synthesize it myself, but it'll always be in the environment until you sail and get scurvy. So one possibility is, we do find ourselves in a case or a situation where we don't have access to the technology and then we're useless because we've outsourced it all. Apocalyptic earth. Yeah. We're just dumbass earth. Dumbass earth. Well, you know, a power cut. A power cut, yeah.

43:18Doesn't take much. Okay. But the other one that's interesting is that the intuition that you built which relates exponentials to multiplication and what the slide rule does, that kind of intuition that came out of using that is useful beyond that, solving that particular problem. Completely. And that's where it becomes deep for us because we don't know whether or not some of these things that we're learning by using these analog devices doesn't somehow diffuse

43:48into everything else we think about and develop a stronger intuition. And that's beyond losing the capability because there's a power cut. Absolutely. Okay, let's slow this forward. Yeah. How many generations, if just one generation, will it take if we outsource all of these sort of capabilities to an AI for us to be as dumb as a rock? Yeah. So in the... Yeah, that's a depressing question. It is. So I can tell you that in the mathematics there's a key distinction to be made and that is that

44:18if you learn first, let's just do it hypothetically, the slide rule and then move to the calculator, your memory kind of helps you out. Yes. So if you take it away, you still have in some sense some intuition for how that works. But if you began with the calculator, that is you never learn the logic behind the calculation, then you're lost. Yeah. And I think what we're entering now is a world where all of these kinds of things are being outsourced. Being outsourced. And you'll never acquire the...

44:49Here's what I found. I'm sorry. I was just going to say, it sounds to me like what you're saying is these analog devices are mapping onto our brain. That's what they are doing. Exactly. Whereas the other devices are replacing the brain and creating dependency. That's right. Because just not to get inside baseball here, but a slide rule implicitly trains you to think about what's called significant figures in a calculation. I think about that all the time, Neil.

45:19No, stop. I'm just saying.

45:23Significant figures.

45:26So you do this and there's only a certain number of figures you carry with you into the next calculation. Is it 3.67894 or 3.68? Okay. Because you can't carry that many decimals forward by this device. But it tells you that often those other decimals are not relevant to the major point of the calculation you're conducting. I saw someone do a calculation on the slide rule

45:56and they somehow believed that all the digits that were returned on the calculator were important and wrote down every single digit without having a clue how irrelevant they were. And so that was just ignorance. It is. This idea, I think, I don't know if it was Bertrand Russell who said, so approximation is the supreme sophistication. Learning when something matters and when it doesn't. Yes. When these decimal places, when you can think about powers of 10 or approximation is a part of our intelligence.

46:27In graduate school, we have a recurring, it's like a seminar. It's called Back of the Envelope Universe. Right? You just ask a question. Blunt questions like, how many pianos are in the world? Well, how would you estimate that? You say, well, how many have you seen? What have you been to? How many people? Are they making more? You don't have to know the exact number we get, but get an approximate number and know how to make approximations. If you don't know how to do that, you're a victim of false precision in the world

46:57distracted by what you think is real when in fact it's a distraction on your path to the truth.

Approximation and intelligibility

47:05Oh.

Approximation and intelligibility

47:06And this gets to a concept which I think is for humans as important as intelligence and that is the notion of intelligibility. Oh. Not intelligence, but intelligibility, which is about understanding things. Mm-hmm. And that's the other point. If you're in the world of capability, prediction and utility are all that matters. In the human world, science is about making the universe and the world intelligible. We want to understand it. Different game. On that point, if we don't outsource the capabilities

47:36and we retain a large chunk of what intelligence we possibly do have. Ha, ha, ha. Wishful thinking. I know. Are we going to, at some point, solve the problems, the big problems? The universe. Why, how, what? Are we going to get there or are we just going to be clutching at thin air? If we fully outsource, you mean? Yeah, but if we retain, if we haven't outsourced completely and if we retain some intelligence, do you think we're capable enough? Because you have said human intelligence potentially unlimited.

48:08Yeah, what do you mean by that? Because we keep developing that. Yeah. So, this gets back to Bergson and the others on tool use. So, humans are always hitching limits. Yeah. I mean, Neil just talked about the slide rule. You can't do that with your fingers. No. Okay. And this gets to why we use tools. There are things we can't do. We build them. We understand how they work and they solve problems that we couldn't in some raw state solve. So, in that sense, an LLM, an AI is just a tool like any other. And it's important to understand what kind of tool it is.

48:38Mm-hmm. Because this tool is a tool that needs tons of data. Most of our tools don't. No. Good point. Right? I don't have to feed the slide rule to get my answer correct. A little bit. And they're very good, and we can unpack this, at doing high-dimensional problems where lots of variables matter. Yes. So, think about protein folding. Yes. Right. It turns out protein folding, lots of variables matter. Mm-hmm. Right. And so, doing the molecular dynamics, the full computation. The predictive modeling, yeah.

49:08It's just too computationally expensive. Yeah. And so, LLMs come along, these models, actually, they're not LLMs, they're different kinds of networks, deep neural networks, and they're good at doing high-dimensional problems. And so, they help us. So, they do protein folding. And a lot of problems are going to turn out to require high-dimensional representation. Lots of variables matter. Mm-hmm. And they're going to help us solve the problem. Just to be clear, so everyone's on the same page, when you solve a problem, the number of variables can be thought of

49:39as a dimension, how many dimensions need to be accessed to get the solution. Mm-hmm. And multivariate problems require vastly more computing power to solve. However, is that really intelligence or just brute force? Yeah, so tell me the difference between the AI solution to the protein folding. Previously, we had really powerful computers. Give them the protein folding problem. What did the AI deep learning

50:10neural nets, what was that able to accomplish that just sheer computing power wasn't prior? So, there's two factors here. One is, and I'm going to go a bit nerdy on you guys. Bring it on. Okay. We like it. This is a nerd-safe space. Okay, this is, okay, I'm going to say it, is that when you do these full physics simulations of a protein to get it to fold, so you're looking at all the bonds formed between all the atoms. Yeah. This has something called a scaling relation, which is that the amount of computation

50:41increases non-linearly. It goes up quadratically. Yeah. So, double the size more than doubles the amount of computational effort. Now, it turns out because of some architectural feature of these AI models, when you increase the scale of a problem, it scales linearly. Ooh. It has a much more efficient scaling property, this architecture. So, things that would not have been computationally feasible with the full physics become feasible with the deep neural net.

51:13That's a very important point. Well, that answers it right there. And that's basically the answer. That's AlphaFold 3. These are AlphaFold. The Dennis Hed Pessibus Company. We've had a couple of guys from there on the show before. Now, the other factor, though, is not just the scaling, which is slightly more subtle, is that you can feed all sorts of different data into these models. So, in the physics simulation, it's just the physics. But in the folding, it's not just the physics and chemistry, it's the evolutionary history. And so, you can put all sorts of

51:45different data and drop it into these black boxes. And that's the second source of that advantage over physics. Wow. Wow.

52:05I'm Brian Futterman, and I support StarTalk on Patreon. This is StarTalk with Neil deGrasse Tyson. Occasionally, I lose sleep

Protein folding and scaling relations

52:20on this very thought. Are humans intelligent enough, however you want to define it, to completely understand the universe with no stone left unturned? And if we're not in the individual, in the collective, might we be? I don't have to invent calculus because Isaac Newton did. So, I stand on his shoulders. And we just keep ascending this ladder incrementally. Can we incrementally become super intelligent?

52:51Or is this the work of an AI that can train on itself and then just bullet past all of us to contemplate the cosmos? So, I think we already know that we have limits. I think the protein folding case gives us that answer. That there was no theory. Well, that was a scale problem. It was a scale problem. Not a how deep it was you're thinking. But I would argue that all of our hardest problems are scale problems. And then, let's give an example because it relates

53:21to this concept of emergence. So, we wouldn't have been able to do simulations of large numbers of particles unless we had fluid dynamics. Right? So, we found a way of making it more comprehensible. And we don't have to track every particle. To say that more precisely, the laws of fluid dynamics allow us to not have to track every single molecule of the fluid. We treat it in an ensemble. Like the gas laws. The gas laws are blunt equations

53:52that describe pressure, temperature, volume of gas. We're not tracking what the particles are doing. In principle, if you did, you should get those formulas out. But we don't have to because we have the macroscopic description. So, continue. Exactly. And most of human understanding, once we hit this barrier given by the lower level, in that case, particles, but you could be this case. It could be, I'm feeling unhappy, or I love you. This is a macroscopic description. Very macroscopic.

54:23Of a very large chemical system inside my body. Right? And that's what we do every time. And we consider that understanding, a form of understanding. Now, it turns out there are problems like protein folding that we couldn't do that. There was no macroscopic description. We had to worry about all the molecular interactions. Oh. You see? And so, this particular tool helped us to solve the problem where there wasn't an emergent theory that would solve the problem. Okay, so now this is AI being our

54:54agent in that quest. It's not AI running amok becoming a super intelligence on its own. No, that's right. It's AI as a tool once again to deal with irreducibly high-dimensional systems. Which I'd like to think our future relationship with AI is precisely that in every way that best serves us, not one where it becomes our overlord. In recent decades, there's been a lot of discussion, especially in modern astrophysics, about, from information theory. Is there a limit to how

55:24much information is in the universe? Can we somehow create more than that? Does that information go away? Because the entropy seems to be a clock that we cannot stop. Would the universe approaches ever greater entropy in the future than it does in the present or in the past?

The industrial revolution of information

55:43Do any of your

The industrial revolution of information

55:44calculations touch on this? Yes. So two different ways. One is about the fundamental limits to computation, what gets called the thermodynamics of computation, and that takes us to quantum and all that. So right, there is a fundamental energetic limit. So that would be, the quantum computer would be, in principle, the most efficient computer we can conceive. Exactly. Whether or not we're there yet. Exactly. And then there's the other entropic limit, which is the cost of calculating with a classical

56:14von Neumann computer, the kind of computers that we all have are iPhones and so on. So we have these massive data centers that are consuming megawatts of energy. Big gigawatt. Big gigawatt. They're boiling the planet, all to make your emails better. But it's okay because they're going to solve climate crisis. Eventually. This is fascinating. That's the whole thing in itself. Once they've melted the planet, for self-survival purposes,

56:45they're going to figure it out. They're going to have to work out. And it's not just being a co-pilot to your rating. So that's the other entropic problem. So again, I mean, a long story here. Maybe I want to, can I just give a little narrative here? I think it's useful, historical one. You're the guest. Okay. So try to make sense of what is going on in the current moment in terms of these kind of energy guzzling AIs. There's a useful comparison to the industrial revolution and to ecology.

57:16So most of the history of life is predator and prey. That is how energy got transferred. You ate something.

57:25In parallel, in the Carboniferous period and the Mesozoic period, back there hundreds of millions of years ago, fossil deposits are being left in the earth, becoming coal seams, becoming petroleum and natural gas. And that energy is not accessible to us. Just there. Then the industrial revolution happens.

57:49And we get steam engines. And they get more and more efficient. And what steam engines do is they liberate all of this energy that was buried in the earth. Just to be precise, only when they discovered coal could do that. Exactly. Beautiful, clean coal. You can't beat it. Otherwise you just burn wood. That's right. Vastly less efficient. Yes, the energy packing capacity of wood versus coal is no contest. No contest. I think the calculation is you could burn all

58:19of the standing forests of the world and provide enough energy for the current population for six months. Wow. That's it. That's pretty serious. That's it. And by the way, before we knew about thermonuclear fusion, we said, well... It's a hoax. No. The sun, you know, the sun is like really efficient and we knew there's a lot of carbon in the universe. It must be a lump of coal. It must be a big lump of coal. Because it was efficient and it could go a long time. Our first calculations were maybe the sun is made of coal. Man, somebody should tell our president that

58:51right now and maybe we'd get to renewable fuels. Continue, I interrupted. Right, so we discovered super energy-dense, power-dense materials. Power means you could liberate it fast, like in a nuclear reaction. Okay. So the world changed. It went from being networks of predators and prey, highly constrained by interaction, to all of a sudden this massive, high-energy-dense archive that we could access. Okay. That's the energy world. So let's talk about the information world.

59:22Nearly all the information we have, we get from talking like this. It's a bit like ecology, predators and prey, right? It's like, oh, I'm going to tell you a fad. You could read a book. Problem is with reading books, you read like, I don't know, maybe 250 words a minute. Over your lifetime, if you're lucky, you read a few thousand books. That's how it works. So what happens? So a thousand books, let's use the number, a thousand books. So just like in the Cambrian, right, or the Carboniferous rather, and in the Mesozoic, where fossils were being deposited in the

59:52earth, we've been depositing texts in libraries, starting in ancient Summa all the way to the present. So we have now order hundreds of millions of books deposited. And it's mostly in the present. Mostly in the present. The rate of book publication has grown exponentially over the decades. Yeah. However, the knowledge also covers all of our past, because we've discovered our past in the present, and deposited that information in the depositories you're talking about. Exactly, but here's the thing.

1:00:23So this is why the LLM is interesting. The LLM is like the steam engine. I see where you're going. Because what happened is, it's mining, exactly. It's liberating that energy, which is our literature, which is our language, which is our thoughts and cultures and everything that we are, all in one depository, that it can now go in the same way we went in and dug for coal. It can go in and get all of that very, very quickly and turn it into

1:00:54power. Exactly. Wow. Exactly. But this time, conceptual power. Right. Mechanical power. Exactly. So there's this strict analogy between, if you like, the fossil deposit and, as you say, the cultural deposit. The steam engine liberated the energy deposit. The transformer architecture liberated the conceptual deposit. Look at that. Now, both of them produce what economists call euphemistically externalities, right? In other words, they pollute. Mm-hmm. So that's the problem with the Industrial

1:01:24Revolution, right? Right. It produced its smogs, right? Right. It generated a huge amount of entropy in the environment, just as these data centers and transformers are producing a huge amount of entropy in the environment. Right. So there's this very interesting parallel. And wait a minute. And the culture.

1:01:41Look at that. They're creating entropy not only in the environment, but they're creating entropy in the culture itself. Yes. Wow. Yes. So that's a really interesting point. In the conceptual space, the disorder is not just thermal, it's knowledge-based. Yes. It's basically crap. No, it's misinformation. No, we call it AI swap. Yeah. Look at that. So there's a very weird thing going on here. And just to be clear, in the Industrial Revolution, not all coal or oil deposits

1:02:11were equally as refined, equally. I mean, different sources of oil in the world are not as good for- It's not a uniform- No, absolutely. It's not a uniform deposit. Deposit. Yeah. No, correct. No, you're absolutely yeah. You have light sweet crude, you have tar sands, you have- Yeah. Yeah, go ahead. Now, look what happened. So we created, so out of the Pennsylvania, out of these deposits, come all the big corporations of another era of BP, Shell. Right. And the steel companies that

1:02:41use the coal. Exactly. So now what's happening? U.S. steel. Well, we have our analogs. We have Anthropic. We have OpenAI. Yeah. These are the mining companies of our time. Look at that. Right? Generating the kind of entropy that you were discussing. Yeah. Now, what's weird here that is worth discussing is that coal is depleted. Natural gas is depleted. But in principle, knowledge is not. But there's a difference, which

1:03:12is that we can foul that knowledge with the entropy in concept space. Right. That's your point. Now, you could have done that with coal deposits, but it would be difficult. Yeah. And why would you? Why would you? Right, right, right. But now, unfortunately, we can take this knowledge, synthesize it, and put back crap. Right. And that's a, so it's different and similar in rather interesting ways. Well, is the crap because it doesn't know how to do it, or is the crap because so much of what we do have in the library is just horseshit?

1:03:45I mean, just because it's a book doesn't mean it is correct or even contributes to the greater good of us all. And the LLM doesn't know that because it's not trained at a university to be able to sift the wheat from the chaff. It just, they're just words statistically associated with other words and outcomes. It's a probabilistic aggregation. Oh, I like that. But the difference is the crap that's in our libraries

1:04:15is an example to us of our journey and how we got to where we are. So they become guardrails so that these are the mistakes, the thoughts, the things that we know, that is horse crap. Whereas the LLM, without, if we didn't know those things, it would give us that and we just say, oh, here we are. Wait a minute. If I say, what should I do with a witch? It goes back and finds all the literature on what you do with witches. Which is witches? And say, oh, you should obviously burn them. And burn them.

1:04:46And this is how you burn them. Right. Whereas when you go to the library, you find out, oh, the witch trials were an attack on women. Because you have context. Tell me about context. Well, this is, I've been working on papers on mathematical ability in these machines based on this notion of the chronology of the development of the concept. So one really important difference between humans and these things is that we learn at school using a

1:05:17curriculum. Right. Everyone, you typically learn what a number is before you learn like a natural number or an integer before you learn a fraction. And then you learn fractions typically before you learn pi, real numbers. And you learn that typically before you learn algebraic geometry or group theory or something more refined. Okay. So there is a sequence. It's understood we learn incrementally using a curriculum. These models learned everything all at once. There is no curriculum. Another question is, what does that mean for how we do math and they do math?

1:05:49And it means lots of things for how we do math and they do math. And one of the things it means is this, which is that they basically find very complicated, what we would call heuristics, rules of thumb, approximations actually, for solving problems that we solve exactly using algorithms. Let me make that explicit. So when I learned how to do long division, right, assuming as I did it right, I'd get the right answer. Right. You have to learn how to do it right, but it's always be right regardless of the scale.

1:06:20Well, the current best models, LLMs, will not get it right regardless of the scale, guaranteed. They'll get it wrong at large scale. Long division. They'll get long division wrong at large scale if you have too many digits, because they're using heuristics, right? And so part of our training was to learn these rules piecewise through a curriculum. Their mode of learning is giving rise to a situation where, despite the fact they look great, at large scales, they're going to

1:06:50confabulate. Wow, look at that. And make wrong answers. And we're not going to be smart enough because we've outsourced to them to know the difference. Well, this is what Jeffrey Hinden was saying, that they're overconfident. Yes. So they're expecting now us to just take that for granted as the answer. Yeah, I always loved Douglas Adams, like Hitchhiker's Guide to the Galaxy. Yeah. And they build this supercomputer and they ask you, what's the meaning of life, the universe, and everything? So what's the answer? And the answer is, like, 42. Yeah. And you think, what the what does that mean?

1:07:21What's it mean? We are now in a world where we could be told that 42 is the answer and we wouldn't know what it meant. We wouldn't know it, right.

Universal Darwinism and alien intelligence

1:07:28Do you think the universe will allow a super intelligence to be created, to evolve, and therefore solve the mystery of the universe and hopefully it's not 42? And that presumably would be an alien somewhere. Yeah. So we, so I have problems with the concept of super intelligence, I should just say. If you buy my definition, which is making hard problems easy, it's not clear to me that the path we're on now is about making hard problems easy.

1:07:59I think it's about solving some hard problems, but doing it in a very convoluted way, which I do not consider super intelligence. That's number one. Yeah. In terms of, I'm subscribed to the idea of kind of universal Darwinism. Now, I do mean universal.

1:08:15Granted. Right. That any... You need permission and I just gave you permission. I'm just saying, and that in most life forms, however different they would be, would be living under conditions of scarcity. That's sort of prerequisite for evolution in the theory, actually, because you have to compete for scarce resources. And so, oddly enough, they will look more like us in their intelligence, then they might look like LLMs. Now, it could be that that alien intelligence is all that's left once those AIs have devoured the

1:08:46life forms that built them, in which case they will be more like LLMs. But I think it's an important point to note that under the conditions that evolution takes place, you want to be intelligent by having a small amount of data and quite restricted experience for the world and making a decision relatively quickly so that it actually enhances your survival and not using megawatts of energy to come to the solution. So we can imagine a planet where the environment

1:09:17shifts with some frequency. Like, let's say you could program this so that you're always forcing an evolutionary change. The organism never stays happy. It finally settles in. You give it some assault. Make sure you don't kill them all, because that's what happens most of the time. But there's some portal that it can move through and start evolving. Is it inevitable that that will lead to what we call intelligence?

1:09:47Or just more appendages or more senses or more... Does that give us the super intelligence that we're thinking, the biological super intelligence? So, the open question is why we didn't just stay bacterial. That worked pretty well. They outnumber us. It still works. It still works. Still working. We're dependent on it. Yeah. And the way to think about this is that there is a fundamental limit to how information can be acquired by

1:10:17an evolutionary process. Because a mathematical theory tells us that you can only put one bit of information into a genome every generation. And as you get bigger, you get more energetically efficient, but your generation time gets longer. So, at large scales, it takes more and more time to get a bit by selection into the genome. Why? Because the way bits get put into life forms is through differential survival.

1:10:48Yes. So, imagine you had to learn But why is it only one bit? Why can't it be a suite of bits? Yeah, because selection would get confused about which bit it put in. It would be confused about which bit was important for the survival in the next step. Got it. So, for it to know the difference, it has to be one end. So, it's like a brick at a time, basically. Brick at a time. It's a brick at a time. Okay. So, when you're very small, like a microbe, a brick at a time can be fast, but as you get bigger and longer lived, that's slow. So, what did we do?

1:11:18We evolved brains. So, brains can add bits more quickly than genomes can add bits. That's why we have brains. So, there's two things going on. We get bigger to get more efficient energetically, but it makes us slower evolving, which makes us have to have brains to pull information from the world when the world changes. So, there is a kind of underlying ratchet towards what we would call higher complexity. And, therefore, towards greater intelligence.

1:11:48Wow. That's great, though, because that means out there somewhere is probably another intelligence along the same track as we. But, maybe there's a variable that they didn't encounter that keeps them from killing each other, destroying cultures, and enslaving one another, and all of these stupid things that we do, and maybe they're just so much more advanced because they don't retard their

1:12:19own advancement. Although, everything he said about a species moving through derives from, in most cases, competition. And that competition is what creates that other stuff that I was just... And that's our reptilian brain. God damn it! God damn it!

1:12:40You were onto something for a little bit there. I've never had something!

1:12:45So... Well, it does make it sound like there's some kind of plan. Plan? Yeah, rather than it's just random occurrence. Plan? Some kind of... Well, there are trends. You could say there are trends in evolution. I mean, the question to your observation, Chuck, is, at this point, where we could transcend the competitive basis of evolution by building AI, could we... That's the way... Could we? That's the pathway. That's the pathway.

1:13:16Unfortunately, it's not what we're doing. Yeah, it's not what we're doing. Right. Ugh. Man. Man, this is... I tell you, it's so much easier to tell dick jokes. It's probably not. It's probably isn't. It's probably not. One last point. How long have you been at the Santa Fe Institute? About 20 years. Oh, that's a long time. It is a long time. A big part of your career. It is a big part. It is my career. And they're famous or infamous for... Famous.

1:13:46You didn't hear what I was about to say. For dangling people at the edge of what is known and exploring places that are just on the surface crazy sounding but might be something profound. So that's still

Asking good questions

1:14:02kind of an operating principle? Yeah, I think the history of science is risk and then it got institutionalized. Yeah. And it became a bureaucracy that was enforcing norms. And SFI wants to be heterodox. SFI, the Santa Fe Institute. And that's a different environment. To be in a high-risk environment is a little bit going back to the time of early science. I was about to say, you're just taking it back to when science was kind of the Wild West

1:14:33and people were just like, hey... Try this. We should do this. Try this. Try this. Smell that. Smell that. Cook that. Right. Exactly. And so that sounds like it could be quite fertile for people on the frontier with powerful imaginations for what could be true. Yeah, I mean, I think now because of all these tools that are solving technical problems, what we have to cultivate is amazing question askers. That is what remains for humans.

1:15:05Right? Because it's like, you don't need to be good at arithmetic once you've got a calculator. Probably don't have to be good at calculus anymore. But you have to be able to ask good questions. And that environment is something different. So on that point, let's land this plane on a quote from a poem by the German poet Rainer Maria Rilke, I think I'm pronouncing it right. Okay. Here it goes. As quoted in my alien book, Take Me to Your Leader. Be patient toward all

1:15:38that is unsolved in your heart and try to love the questions themselves. That's a great quote. Yes. That's beautiful. And right after that, she said, we have ways of making you talk.

1:15:58You couldn't help us. Professor, thank you. Thank you so much. Thank you for yet another tour of duty on StarTalk. Yeah, man. It's wonderful to be with you guys. Yeah, too much. Too much to think about. Now we got to think about it. I can't even. I'm out for two months now. I got to do it. Thank you. How do we find you? Are you online? Oh, you find me. You find me at the Santa Fe Institute. You can find me at davidccracauer.com. There you go. Okay. Find me anywhere. Just keep up with you because you're doing fun, interesting things.

1:16:29Thank you. And I have a special appreciation for your evolutionary biology background because I work here as an astrophysicist in a museum famous for its evolutionary biology. And so I have an extra little osmotic attachment to your work. Well, thank you. It's wonderful being with you guys. To your work. Your people keep him working.

1:16:50Chuck. Always a pleasure. We're good here. Gary. Pleasure, my friend. All right. This has been another installment of StarTalk Special Edition. And we're going to call this the intelligence installment. Are we or are we not? Until next time, keep looking up. We'll be right back.