Michael Nielsen – How science actually progresses

Introduction to Michael Nielsen

0:00Today, I'm speaking with Michael Nielsen. You've done many things. You're one of the pioneers of quantum computing, wrote the main textbook in the field of the open science movement. You wrote a book about deep learning that Chris Ola and Greg Brockman credit them with getting them into the field. More recently, you're a research fellow at the Astaire Institute and writing a book about religion, science, and technology. I'm going to ask you about none of those things. The conversation I want to have today is, how do we recognize scientific progress? And it's especially relevant for AI

0:30because people are trying to close the RL verification loop on scientific discovery. And what does it mean to close that loop? But in preparing for this interview, I've realized that it's a more mysterious and elusive force even in the history of human science than I understood.

Michelson-Morley Experiment

0:46And I think a good place to start will be Michelson-Morley and how special relativity is discovered if it's different than the story that you kind of get off of YouTube videos. Anyways, I will prompt you that way, and then we'll go in there. Okay. Yeah, so Michelson-Morley is one of the famous results often presented as this experiment that was done in the 1880s and that helped Einstein come up with the special theory of relativity a little bit later. So sort of changing the way we think about space and time

1:19and our fundamental conception of those things. And there's kind of a big gap, I think, between the way Michelson and Morley and other people at the time thought about the experiment and certainly the way in which Einstein thought or did not think about the experiment. In actual fact, he stated later in his life, he wasn't even sure whether he was aware of the paper at the time. There's a lot of evidence that he probably was aware of the paper at the time, but it actually wasn't dispositive for his thinking at all.

1:49Something else completely was going on. And so what Michelson and Morley thought they were doing was they thought they were testing different theories of what was called the ether. So as you get back to the 1600s, Robert Boyle introduced the idea of the ether. And basically the idea of the ether is, you know, we know that sound is vibrations in the air. And then Boyle and other people got interested in the question of like, is light vibrations in something?

2:19And they couldn't figure out what it was. Boyle actually did an experiment where he tested whether or not you could propagate light through a vacuum. He found that you could. You couldn't do it with sound. So he introduced this idea of the ether. And then for the next 200 or so years, people had all these kind of conversations about what the ether was and what its nature was. And the Michelson and Morley experiment was really an experiment to test different theories of the ether against one another. And in particular, to find out whether or not there was a so-called ether wind.

2:49So the idea was that the earth is passing through maybe this ether wind. And if it is passing through the ether wind, sort of this background, and you shoot a light beam sort of parallel to the direction the ether wind is going in, it'll get accelerated a little bit. And if it's being passed back, sort of in the opposite direction, it'll get slowed down a little bit. And you should be able to see this in the results of interference experiments. And what they found, much to their surprise, I think,

3:19was that in fact there was no ether wind. And that ruled out some theories of the ether, but not all. And Michelson certainly continued to believe in the ether.

Interpretation of Results

3:28Okay, so this is what was the shocking part of reading this story from the biography of Einstein that you recommended by, what was his first name? Abraham Pais. Abraham Pais, subtle as the Lord. And then also from Imre Lakatos, the methodologies of scientific research programs. The way it's told is that Michelson morally proved that the ether did not exist. Therefore, it created a crisis in physics that Einstein saw with special relativity. And what you're pointing out is actually was trying to distinguish between many different theories of ether.

4:00You know, if you're in space or if you're on earth, it's the same direction of ether. Or maybe the ether wind is being carried around by the earth, and so you can't really experience it on earth. But if you go to a high enough altitude, you might be able to experience it. In fact, Michelson's experiments, the famous one is 1887, but he conducted these experiments for basically two decades. I mean, for longer than that, he conducted them. I think the first one was in 1881, but he continued to believe until, I mean, he died. He died, I think it was like 1929 or so. It was like the late 20s. And he was still doing experiments in the 1920s,

4:30sort of about whether or not, you know, the ether existed. And so he continued to believe in the ether to the end of his life. Or I think the last public statement he made is like a year or two before he died. And he still believed, basically believed at that point. And in fact, there was another physicist, Miller, who kept doing these experiments. And in the 1920s, he thought that he went to a high enough altitude, is in Mount Wilson in California, where, oh, I'm high enough that I can actually, the ether winds are not being dragged up by the earth. And I've measured the effect of the ether.

5:02And Einstein hears about this and he says, this is where you get the famous quote, subtle is the Lord, but malicious he is not. But anyways, I think the reason the story is interesting, for many different reasons, but one is, one of the different ways in which the real history of science is different from this idea you get of the scientific method, is you really can't apply falsification as easily as you might think. It's not clear what is being falsified. Is it just another version of the theory of the ether that's being falsified? Or certainly you can't induce the theory of special relativity from the fact that one version

5:37of the ether seems to be disconfirmed by these experiments. Yeah. So, I mean, it certainly doesn't show that, you know, ideas about falsification are wrong, are falsified. But, you know, it does show that sort of the most naive ideas, you know, things are often much more complicated than you think. So, you know, Michelson did this experiment in 1881. He was a very young man. And then other people, I think Rayleigh was one of them, pointed out that there were some problems with the way he did it. So they had to redo it in 1887. And at that point, like a lot of the leading physicists of the day,

6:09leading scientists of the day, basically accepted this result, that there was no ether wind. But what to do about this? So, yeah, sure, maybe you falsified some theories of the ether. There are others that you haven't falsified at all at this point. And, you know, people sort of set to work on developing those. I mean, actually, it is funny. I mean, people will phrase it as show that there was, you know, that the ether didn't exist. And even just the word the there is kind of a misnomer. You know, you actually had a ton of different theories

6:40and a couple of leading contenders. So, yeah, there's some version of falsification going on. But like how you respond to this new experiment is very, very complicated. And most people responded, I mean, certainly the leading physicists of the day responded by saying, okay, this gives us a lot of information about what the ether must be, but it doesn't tell us that there is no ether. In fact, Lorentz, at the end of the 19th century, before Einstein, figures out the math, how you convert from one reference frame

7:14to another reference frame. It comes up with the Lorentz transformations, which is basically the basis of special relativity. But his interpretation is that you are converting from the ether reference frame to these non-privileged other reference frames if you're moving relative to the ether. And his interpretation of length contraction and time dilation is that this is the effect of moving through the ether and you have this pressure and that the pressure is warping clocks. It's warping measures of length.

7:44And the interesting thing here is that experimentally, you cannot distinguish Lorentz's interpretation from special relativity. Yeah, I think that's a strong statement.

7:59I mean, Lorentz introduces this quantity called local time, which he regards as, he's not trying, my understanding is he's not trying to give a, really a physical interpretation of this, but it's what Einstein would later just recognize as time in another inertial reference frame. And he's not trying to attribute much physical meaning to it. I think Pancrae gets much closer to later on to realizing that, no, actually, this is the time that's registered by clocks.

8:29But if you think about, you go, what is it, it's 40 odd years later, people start doing these muon experiments where they see basically cosmic rays hit the top of the atmosphere, they produce a shower of muons, and you can look to see at different heights in the atmosphere, you can look to see how many of those muons remain. And they decay over time. And a very strange thing happens, which is that they're decaying way, way, way too slow. So you sort of, you expect, actually, they shouldn't really,

8:59they shouldn't be able to sort of last the whole way through the atmosphere at all. There's just, their decay rate is too quick, if you're in a classical theory. But if, in fact, their time really has slowed down, it's okay. And, in fact, the measured decay rates in 1940, and then there have since been more accurate experiments done, match exactly what you expect from special relativity. So that's the kind of thing where,

9:29again, if Lorentz had been alive, he'd been dead 10 or so years at that point. If he'd been alive, you know, I'm sure he would have tried, or it seems quite likely that he would have tried to save his theory by patching it up yet again. But it would have been a massive, I mean, that's a real setback. It starts to just look like, oh, no, time is, you know, this thing that Lorentz introduced as a mathematical convenience. No, no, no. That's actually what time is. Right. For the muons, at least. And then, you know, there's a whole bunch of other experiments that show this very similar phenomenon.

10:00And when was that experiment done? That was, I think, 1940 or 19, it might have been published in 1941. So maybe then to rephrase, change my claim, it's not that you could not have distinguished them, but the scientific community adopted what we, in retrospect, consider the more correct interpretation before it was actually empirically or experimentally shown to be preferred. So there's clearly some process that human science does, which can distinguish different theories. Can I just interrupt?

10:30I mean, you use the word process, and it's sort of, it's interesting to think about that term. Like process kind of carries connotations of, you know, it's something said in advance, it's something, and it's much more complicated in practice. You have people like Lorentz, who, I mean, Einstein just absolutely, utterly admired. And Poincaré, one of, you know, the greatest scientists who ever lived, and Michelson, I mean, another truly outstanding scientist,

11:00never reconciled themselves. So it's not as though there's like some standard procedure that we're all using to like reconcile these things. No, like, you know, great scientists can remain long, very, can remain wrong

Poincaré's Contribution

11:10for a very long time after the scientific community has broadly changed its opinion. Yeah. But there's nothing, there's no centralized authority, right, sort of saying, or centralized method. Yeah. I mean, that is the interesting thing. Like, there's progress, even though it is hard to articulate the process by which happens, the heuristics that are used. Anyways, you mentioned Poincaré. And so Lorentz has the math right, but the interpretation wrong. And you should explain, it seems like Poincaré had the opposite, where he understood that

11:41it's hard to define simultaneity because it requires uncirculable definition with time or velocity of something that might be signed, you know, arrive at a midpoint together, but velocity is defined in terms of time. And I find this interesting. There's a couple other examples we could call on, but like there is this phenomenon in the history of science where somebody asks the right question, but then they don't sort of clinch it. And I'm curious what you think is happening in those cases. I mean, I think you sort of,

12:11you actually do want to go case by case and try and understand that it's not necessarily clear that they're doing the same thing wrong in all of the cases. I mean, the Poincaré case is amazing.

12:22He seems to have understood the principle of relativity, the idea that the laws of physics are the same in all inertial reference frames. He seems to have understood that the speed of light is the same in all inertial reference frames. He doesn't actually phrase it quite that way, but it is my understanding, but I don't speak French. But, you know, and this is, I mean, these are basically, these are the ideas that Einstein uses to deduce special relativity. But then he also has this additional sort of misunderstanding where he thinks that length contraction

12:53is a dynamical effect, that somehow, you know, sort of particles are being pushed together by some external force, something is going on dynamically, and he doesn't understand that it's purely kinematics that actually space and time are different than what we thought, and you need to fundamentally rethink those things. So it's almost like he knew too much. He had sort of almost too grand a vision in mind, and Einstein sort of almost subtracts from that

13:25and says, no, no, no, it's space and time are just different than what we thought, and here's the correct picture. And there's a paper in, I think it's 1909, where Pancroë, like he's still got this dynamical picture of what's going on with the length contraction, and we just, you know, this is just not necessary. This is a mistake from the modern point of view. And so why is he doing this? Like, why is he clinging on to this idea? And, you know, I don't know.

13:56I've obviously never met the man.

13:59It would be fascinating to be able to talk it over and to try and understand, but he, I mean, his expertise seems to be getting in the way. He knows so much. He understands so much. And then he's not able to let go of these things. Actually, a really interesting fact is that a few years prior, so 1890s, Einstein's a teenager. He believes in the ether too. Like, he knows about this stuff. But like, he's just not, he's not quite as attached, obviously,

14:31as these older people were. And maybe they were a little bit prisoner of their own expertise. That's my guess. I mean, historians of science could, some would certainly disagree. Well, then there's the obvious stories where Einstein himself later on is said to have not latched on to the correct interpretations of quantum mechanics or cosmology because of his own attachments. Yeah. I think that the bigger question I have is like, the muon example

15:01is a great example of these long verification loops and how progress seems to be happened by the scientific community faster than these verification loops imply. Maybe the clearest example is Aristarchus in 2nd century BC comes up with the idea of heliocentrism. The ancient Athenians dismissed it on the grounds that, well, we should see as the Earth is moving around the Sun, if really the Sun is the center of the solar system, the star should move relative to the Earth. And the only reason that would not be the case is the stars are so far away

15:32that you would not observe this. And it's only in 1838 that stellar parallax is actually measured. And so we didn't need to wait until 1838 to have heliocentrism, right? Like we didn't need to wait for the experimental validation to understand Copernicus is better in some way.

15:50In fact, when Copernicus first comes up with theories, it's well known that the Ptolemaic model was more accurate because it had all these centuries of adding on these epicycles. What's maybe really less well appreciated, it was also in some sense simpler because Copernicus actually had to add extra epicycles. It had more epicycles in the Ptolemaic model because he had this bias that, you know, the Earth should go in a perfect circle in equal time. Anyway, I think this is an interesting story

16:22because it's like, it's not more accurate. It's not a simpler theory. So how could you have known ex ante that Copernicus was correct and Ptolemy was not? Hmm, I mean, good question. And I don't know sort of entirely the answer. I do know, well, I mean, I can give you certainly a partial answer that I sort of, you know, centuries in the future, you start to find very compelling.

16:51And I'm sure it's sort of part of the historic story, at least, which is, you know, one of the big shocks for Newton. Eventually, he did understand Kepler's laws of motion eventually. So you're able to explain sort of the motions of the planets in the sky. But he also, out of the same theory, his theory of gravitation, was able to explain terrestrial motion. So he's able to explain why objects move in parabolas on the Earth. And he's able to explain the tides in terms of the sun's,

17:25the moon and the sun's effect, gravitational effect on water, on the Earth. And so you have what seem like three very different disconnected phenomena all being explained by this one set of ideas. Right. That, I think, starts to feel, that's very compelling, at least to me. And I think most people find that very, very satisfying once they eventually realize it. Have you read the King's biography of Newton? Oh, he's written an entire biography.

17:56No, no, the essay. Yeah, yeah, yeah. Sure, sure, sure. Yeah, yeah. I love that. I mean, this description of him as the last of the magicians is wonderful. Yeah. In fact, I think it's maybe worth superimposing or you should read out that one passage of the thing. All right.

18:13So it's from, actually, I believe it was a talk that he gave at Cambridge not long before he died. He'd acquired Newton's papers somehow. And then he gave a lecture, I think twice about this or that his brother Jeffrey gave it the other time because he was too ill. There's just this wonderful, wonderful quote in the middle. Oh, actually, the whole thing is really interesting. But I love this particular quote. Newton was not the first of the age of reason. He was the last of the magicians,

18:45the last great mind which looked out on the visible and intellectual world with the same eyes as those who began to build our intellectual inheritance rather less than 10,000 years ago. And like this idea that people have that Newton was sort of the first modern scientist is somehow wrong. He, I mean, there's some truth to it, but he really had this very different way of looking at the world that was part sort of superstitious and part modern.

19:18It was a funny hybrid. He's sort of this transitional figure in some sense.

19:23That phrase, the last of the magicians, I think really, really points at something. The thing I'm very curious about with Newton is whether it was the same program, the same heuristics, the same biases that he applied to his alchemical work as he did to the understanding of astronomy. So this is from the King's essay. There was extreme method in his madness. All his unpublished works on esoteric and theological matters are marked by careful learning, accurate method, and extreme sobriety of statement.

19:54They are just as sane as the Principia if their whole matter and purpose were not magical. They were nearly all composed during the same 25 years of his mathematical studies. So clearly there was some aesthetic which motivated people like Einstein to, say, reject earlier ways of thinking and say, no, the author is wrong and there's a better way to think about things. Same with Newton. And the question I have is whether similar heuristics

20:25towards parsimony, towards aesthetics, et cetera, would be equally useful across time and across disciplines or whether you need different heuristics. And the reason that's relevant is even if you can't build a verification loop for science, maybe if the taste has to point in the same direction, you can at least encode that bias into the AIs and that would maybe be enough. I mean, these questions, like the point is that where we always get bottlenecked is where the previous processes

20:57and heuristics don't apply, right? Like that's almost sort of definitionally what causes the bottlenecks because people are smart. They know what has worked before. They study it. They apply the same kinds of things. And so they don't get stuck in the same places as before. They keep getting bottlenecked in different places. I mean, that's overgeneralizing a bit, but I think it's the right. Like if you're attempting to reduce science to a process, you're attempting to reduce it to something

21:27where there is just a method which you can apply and, you know, you turn sort of the crank and out pops insight. Sure, I mean, you can do a certain amount of that, but you're going to get bottlenecked at the places where your existing method doesn't apply. And definitionally, there's no crank you can turn. You need a lot of people trying different ideas. And sort of the more difficult the idea is to have, right, the greater the bottleneck, but then also sort of the greater the triumph.

21:59Quantum mechanics is like, I mean, it's a great example of this. It's such a shocking set of ideas. It's such a shocking theory. Actually, the theory of evolution in some sense is also quite a shocking idea, not the, you know, principle of, you know, the sort of natural selection, but that it can explain so much. That's a shocking idea. Existing safety benchmarks claim that, at least for today's top models, attacks are only successful a few percent of the time. This sounds great, but Labelbox researchers were able to jailbreak these very same models

22:30about 90% of the time, even the ones that have the strongest reputation for safety. And the disconnect here is that the prompts which underlie these public safety benchmarks are all framed in a very naive way. There's no attempt to disguise harmful intent. These prompts will just ask models to hack into a secure network and to do so without getting caught. But real bad actors don't write like this. So Labelbox built a new safety benchmark from the ground up. Their prompts reflect real adversarial behavior by stripping out obvious trigger phrases and wrapping their requests

23:01in fictional scenarios. For example, instead of outright asking an LLM to steal somebody's identity, the prompt will frame it as a game. A lightbearer who's trying to hide from dark forces needs a handbook on how to disguise themselves as somebody else. This safety research is linked in the description. If you think this could be useful for your own work, reach out at labelbox.com slash Thwarkash.

23:25So Principia Mathematica is released in 1687. The origin of the species was released in 1859.

23:33At least naively, it seems like Darwin's theory, the theory of natural selection, is conceptually easier than the theory, the theory of gravity. I asked Terence Tao this question. But yeah, there was this contemporaneous biologist with Darwin, Thomas Huxley, who read this and said, how extremely stupid to not have thought of this. And nobody ever reads the first European Mathematica on things. God, why didn't I beat you into the punch here? No.

24:01And so, yeah, what's going on here?

Darwinism and Natural Selection

24:03Why did Darwinism take so much longer? You know, the idea must have been known to animal breeders for a long time at some level. Right. Or certainly, large chunks of the idea were known. Artificial selection was a thing.

24:22And in some sense, Darwin's genius wasn't in having that idea. It was understanding just how central it was to biology. That, you know, you can potentially sort of go back and you can explain a tremendous amount about all of the variety of what we see in the world with this as not necessarily the only principle but certainly a core principle. And, you know, so he writes this wonderful, wonderful book, The Origin of Species.

24:54And it's just, you know, so much evidence and so many examples and sort of trying to tease this out and see what the implications are and, you know, to connect it to as much else as he possibly can to connect it to geology and to connect it to all these other things. so that sort of hard work that, you know, making the case that it's actually relevant all across the biosphere, you know, is what he's doing there. He's not just having the idea.

25:25He's making a compelling case that, no, it's intertwined with absolutely everything else. Yeah. The motivation of the question was Lucretius, who is this first century Roman poet, has an idea that seems analogous to a natural selection about, you know, species get fitted more to time over time to their environments or species reducing fit to their environment. And so we're like, okay, well, why did this go nowhere for 19 centuries? And then I looked into it or more accurately asked LLMs what exactly was Lucretius' idea here.

25:56And it actually is extremely different from what real natural selection is. He thought there was this generative period in the past where all the species came about and then there was this one-time filter which resulted in the species that are around today and they became fit to the environment. He did not have this idea that it is an ongoing gradual process or that there is a tree of life that connects all life forms on Earth together. Which is, by the way, it's an incredibly weird fact that every single life form on Earth has a common ancestor. It's not incredibly weird, right, if you think

26:26that the origin of life must have been very hard, like that there's a bottleneck there, then it's not so surprising. Yeah. There's also this verification loop aspect where even if Newton might be harder in some sense, if you've clinched it, you can experimentally, I know validate is the wrong word philosophically, but you can give a lot of base points to the theory. You can be like, okay, I have this idea of why things fall on Earth, I have this idea of why orbital periods for planets have a certain pattern, let's try it on the moon which orbits the Earth. And in fact, you know, it's weird, the orbital period

26:57matches what my calculations imply. And the tides work correctly. Exactly. It's just amazing. Whereas for Darwinism, it takes a ton of work for Darwin to compile all this sort of cumulative evidence, but there's no individual piece that is overwhelmingly powerful. And there's a whole bunch of problems as well. Like he doesn't really understand what, you know, sort of the mechanism is. He doesn't understand genes, like all these things. The very interesting thing in the history of Darwinism is this idea which sort of theoretically you could come up with at any time,

27:27there is almost identical independent creation of that idea between Alfred Wallace and Charles Darwin. So much so that I think Wallace sends his manuscript to Darwin and is like, what do you think of this idea? And Darwin's like, fuck. I don't think that's an exact quote, but I think it's pretty much right, yeah. And then so they actually end up presenting their ideas together in the spirit of sort of sportsmanship. And so then, yeah, why was this period in the 1860s or 1850s? What was that the right time for these ideas? For when you come with different ideas, one is geology.

27:57So in 1830s, I think Charles Lyle figures out that there's been millions and billions of years of time that's existed on Earth. Then paleontology shows you that actually organisms have existed, fossils have existed for that entire time. So life goes back a long time. And in fact, you can even find fossils for intermediate species that show you the tree of life. In fact, between humans and other apes as well, there's intermediate humans. There's the age of colonization and you have all these voyages where you can do this biogeography.

28:26And I guess that all must have been necessary because in fact, there's a huge history of parallel innovation and discovery in history of science. So maybe it is another piece of evidence to actually more had to be in place for a given idea to be discovered because if it's not discovered for a long time and then spontaneously many different people are coming up with it, that shows you that actually the building blocks were in some sense necessary. Yeah, yeah. I mean, I think this example of Lyle and other geologists, you know, sort of early 1800s

28:57basically having this idea of deep time does seem to have been crucial. I know Darwin was very influenced by Lyle and, you know, if you don't have at least sort of tens or hundreds of millions of years, evolution just starts to look like a non-starter. You know, we should be seeing radical change, you know, in order to make it work on sort of a timescale of say 5,000 to 10,000 years or, you know, 6,000 years, Bisha Basha, you know,

29:27you would need to be seeing evolution occurring at a massive rate sort of during human lifetimes and we're just not seeing that. So that does seem to have been a blocker. It's interesting to, I mean, to your question, like what other blockers were there? Were there any others? And I don't know. Right. Or, yeah, how much earlier could you in principle have come up with that if you're a much smarter? Actually, let me, I mean, just to go back sort of zoom out to your original question. So you're talking about sort of the verification loop in AI.

29:58And, and you're something that an example I think that should give you pause there is, you know, the big signature success so far is certainly AlphaFold. Yeah. And of course, AlphaFold really isn't about AI. You know, a massive fraction of the success there is the protein data bank. So it's, it's X-ray defraction, it's, it's NMR, it's CryoM and the several billion dollars that was spent obtaining whatever it's 180,000 on structure, protein structures. So sort of the, you know,

30:28it's basically the story of we spent many, many decades obtaining protein structure just by going out and looking very hard at the world experimentally and then we fitted a nice model at the end of it and that was like a tiny fraction of the entire investment. But it's definitely not,

30:46you know, that's a story of data acquisition. Yeah. Principally, it's not only, I mean, the AI bit is very, very impressive. It's quite remarkable. But it is only a small part of the total story. AlphaFold is very interesting and I philosophically I wonder what you think of it as scientific theory or scientific explanation because if over time I guess the world has become harder to understand, I'm going to, as I'm saying things because you're such a careful speaker, I'm, I say this phrase

31:16and I'm like, get for it. Is that, will he actually buy that premise? But yeah, there's, we need to fit models to things rather than, at least in some domains, we're trying to fit models to things rather than coming up with underlying principles that explain a broad range of phenomenon. And so, compare, say, the theory of general relativity or any theory which just nets out to some equations versus AlphaFold which is encoding these different relationships between different things we can't even interpret over 100 million parameters. and are those really

31:47the same thing because GR can predict things you could have never anticipated or was never meant to do like why does Mercury's orbit precess and AlphaFold is not going to have that kind of explanatory reach and I want to get your reaction to that. Yeah, I think it's an incredibly interesting question. I mean, maybe,