AI ‘scientists’ promise to accelerate research — how do they work?

Introduction

0:00Welcome back to the Nature Podcast, this time the AI scientists that could generate your next hypothesis and using LiDAR to see round corners. I'm Benjamin Thompson

0:36and I'm Nick Partridge-Howell.

AI in Science

0:45In the most recent era of AI, chatbots have had a huge impact on science. Many researchers use them to help with coding, writing of their articles and even idea generation. Simultaneously, some researchers have raised concerns about energy consumption, that over-reliance on these tools could affect our cognition, and that hallucinated references to papers that don't exist are polluting the literature. Love or hate AI, the advances in this field

1:19continue apace. And now there are several new AI scientists in the pages of Nature this week that could go even further, being designed specifically to assist scientists in a cornerstone of scientific work, coming up with testable and rigorous hypotheses. And in this case, they haven't just got ideas from chatbots that have combed the literature. These new AI scientists have been put through their paces in the real world and have come up with novel hypotheses

1:49that could help treat diseases. One of these AI scientists goes by the name of Robin and comes from Future House, a non-profit organisation that seeks to speed up the process of scientific discovery. So fundamentally, we are bottlenecked by, you know, the availability of human talent, right? By the number of extremely brilliant human scientists working on a particular problem. This is Sam Rodriguez, founder and CEO of Future House.

Future House AI

2:17And so our goal at Future House, can we figure out how we can use AI to remove talent as a bottleneck in scientific research, right? To allow us to consider way more ideas than we have ever been able to consider before, to do way more data analyses, and just generally do more work. The other AI scientist is called Co-Scientist and comes out of Google DeepMind. I think one way to think of this is a system that intends to give scientists superpowers. This is Alan Kaffee-Lay-Kasingham, one of the team behind Co-Scientist.

2:49The volume of new scientific knowledge is itself growing at an exponential rate. So that means that actually for scientists and scientific teams, the idea of not only mastering that field, but also traversing all other potentially relevant fields, in then creatively coming up with new ideas to solve the grand challenges of our time, is a really daunting task. So that's at the heart of the Co-Scientist and why we call it a Co-Scientist, not an AI scientist.

3:20Ultimately, both of these systems aim to come up with new hypotheses in a particular area of science, and develop experiments that could test them. The two systems work in a similar way, by harnessing the power of large language models to create multiple AI agents. These are tools that can operate autonomously to complete tasks, pulling in other systems, including other AIs, to help them do so. For Robin, the system from Future House, the AI scientist uses agents to scour through huge

3:53databases, including hundreds of thousands of openly available research articles, patents, and clinical trial data, to come up with new hypotheses. In this case, the team focused Robin on identifying existing drugs that could be repurposed to treat a different disease to the one they were originally designed for, something the researchers do a lot to try and speed up getting useful medicines to patients. It's also something that the team believed that the AI is particularly well suited to. The way that Robin works is that you take a particular disease that you're interested

4:25in treating, and Robin goes and considers thousands or tens of thousands of different, like, potential biological mechanisms, right, that you could use in order to address the underlying causes of that disease. And the thing about this that's very cool is that you can imagine that if you want to go and consider a hundred or a thousand different ways of treating a disease, and then for each possible mechanism for treating that disease, you want to go and consider a hundred or a thousand different drugs and whether that drug could have that mechanism. That's way more combinations than a human can possibly consider. To complete this process, Robin called upon the help of other

5:00AI tools that the team had developed to scour through text, crow and falcon. You may be detecting a bit of a bird theme. Ask them questions and come up with potential mechanisms and drugs to test. Then there would be a tournament of ideas where the best hypothesis would be selected based on the strength of the scientific rationale and supporting evidence in the literature. Google DeepMind's co-scientist system works similarly, scouring the literature but also using their past work on developing AIs to win games,

5:33to power a tournament to select the best hypotheses. In this case, the ideas would also be allowed to evolve with co-scientists iteratively improving on each idea, then ranking them to present to the human scientist. To come up with its ranking, it focuses on ideas that are new but also realistic. It doesn't suggest wacky things that have no grounding in science, but at the same time it doesn't suggest doing studies that are pretty much just confirming what's already known. But as any scientist listening knows, the key for any hypothesis is testability. And with both co-scientists and

6:09Robin, the teams tried them out in the real world, generating real hypotheses that researchers would go out and test. Robin was put to task to find potential drugs for dry age-related macular degeneration, a leading cause of loss of vision. It started by looking through the literature to find that phagocytosis, where immune cells engulf other cells, could be beneficial to treat macular degeneration. And then separately, it was known in the immunology literature that there's this

6:39particular category of drugs called rokinase inhibitors, and that those molecules can actually stimulate phagocytosis. So Robin, by going and sifting through all these hypotheses and identifying, oh, we should be looking at phagocytosis, and then, oh, there's this category of drug that improves phagocytosis that has not been considered before, was able to make that connection, which we then were able to go and test initially in cells, and then later in animals. Robin suggested using a drug called riposudil, a pharmaceutical that has been approved for treatment of a different eye condition, glaucoma. And the test that the human scientists conducted

7:14showed that it appeared to increase rates of phagocytosis, which could potentially help with dry age-related macular degeneration. For co-scientists, it was actually tried out by many independent scientists working separately from Google on a variety of biomedical topics, including Gary Peltz, who works on liver fibrosis, a disease caused by a build-up of scar tissue on the liver. Gary was, similar to Robin, looking to repurpose drugs that could help with this condition. Here he is speaking at a Google press conference.

7:46It identified three drugs, and I was quite surprised by what it identified. When I tested the drugs in the lab, of the three drugs, two of them worked very well. And it was quite striking that one of the drugs was a FDA-approved drug for another indication, and it blocked about 91% of the damage caused by the fibrosis. When I saw that, it was really quite striking. I kind of fell off my chair

8:18and was really surprised. But co-scientists didn't just look at drug repurposing. It was also used to generate hypotheses for how flu responds to different hosts. In this case, birds and humans. As another of the independent scientists at the Google press conference, Claire Bryant, explains. So it generated and ranked a series of hypotheses, some of which I had already considered, but some which I hadn't. So then I was reading through the outputs when I was on a train to Brussels, and I had that kind of aha moment, because co-scientists had prioritised a protein that I

8:48really hadn't been focusing on at all. And it connected it to several signalling pathways I was already investigating. I spent the rest of the week itching to get back to the lab, frankly, to give it some more data. So both Robin and co-scientist could be used to generate new hypotheses for scientists to go out and test. But they have some differences. One limitation of Robin was that it did not produce experimental protocols, while co-scientists did. However, according to its creators, the unique thing about Robin was that it was able to refine hypotheses based on its own data analyses, whereas

9:23co-scientists took feedback from scientists to help it refine hypotheses further. For example, Robin devised follow-up experiments based on the data provided to it from the human's tests on macular degeneration. And this was actually the first example that I'm aware of, of this like, you know, lab-in-the-loop kind of iterative cycle of discovery, which is the thing that ultimately led to the specific hypothesis that Robin was able to come up with.

AI Limitations

9:47One thing that has plagued large language models that ultimately powered these AI scientists are hallucinations, where AIs can come up with false or misleading information not grounded in any fact. Both teams think that their new system have ways of tackling, while not completely eliminating this. Co-scientists used a team of AI agents, one that was coming up with hypotheses, and then others that were critiquing and verifying them to make sure that they are grounded in reality. Whereas Robin

10:17used a system which prevented hallucinations by making sure it only answered questions from information it actually had, essentially stopping it guessing. The team suggests that this system hallucinated less than humans do. Also, the AI scientists can accidentally propagate erroneous findings. They're kind of only as good as the literature they have access to. In this case, largely open access papers and other articles that the scientists involved would have. So it's possible that they may find a paper that seems to have reasonable results, but actually aren't reproducible. That said, these AIs

10:53were initially created about a year ago, and this field moves fast. The teams think that they can address limitations like these, and Future House have already unveiled a new AI scientist called Cosmos.

11:07AI scientists like these promise to accelerate scientific discovery. But they also raise the question of what roles humans have anymore in science. As a nature editorial opines, human researchers should advocate for their own importance by bringing their expertise to guide these AI tools to do the best research for humanity. On that, the teams behind these tools seem to agree. Here's Alan again. I think scientists have always been the people who kind of chart the path into the future. They take us into

11:41fields of kind of completely new knowledge through their mastery of their own domain and their creativity. And I don't think that essential description changes. What I think is changing is that the capacity of every scientist will hopefully increase. Like there should hopefully be a democratization of capacity. And it can allow more productive scientific activity by more great scientists and elevate like what each scientist is capable of doing. At the end of the day, at the moment, one of the major

12:13things that humans have that the models do not have right now is good taste. So, you know, you have to decide which problems are important to tackle. And that is something today that the models are simply not great at. Now, will we get models with great taste eventually? Maybe. But I still think that fundamentally at the end of the day, humans are going to be serving as resource allocators, right? So you have to decide which questions you want to answer. And that is something at the end of the day that humans are going to be responsible for. That was Sam Rodriguez from Future House in the US. You also heard from Alan Calfi-Lakers-Ingham

12:50from Google DeepMind in London, here in the UK. Gary Peltz from Stanford University in the US, and Claire Bryant from the University of Cambridge in the UK. For more on this story, check out the show notes for some links. And if you have opinions about AI, we'll also put in a link of where you can do nature's poll all about it. Hate it or love it, we'd love to hear from you.

Seeing Round Corners

13:14Coming up, we hear about a study that could, one day, allow smartphones to see round corners. Right now, though, it's time for the research highlights with Sharmini Bundel.

13:29Tardigrades, the miniature creatures known for surviving in extreme conditions, have now been found to withstand extreme heat under unusual circumstances, when they're dehydrated. A team of researchers wanted to understand how these hardy animals, nicknamed water bears, withstand various levels of heat. First, they tested well-hydrated tardigrades, finding that they couldn't survive at 45 degrees Celsius for even an hour. But drying the creatures out before heating resulted in 90% surviving, with some even able to weather temperatures

14:03of up to 85 degrees Celsius. The results seem to indicate that when the tardigrade is drier, heat travels more slowly through its body, perhaps protecting internal parts from damage. You can find that research in the Journal of the Royal Society Interface.

14:23Antarctic ice contains evidence of our solar system's journey through an interstellar dust cloud. As the solar system travels across the galaxy, the Earth mops up interstellar dust, including the rare isotope iron-60, a signature product of supernova explosions. Some of this isotope is preserved in ocean sediments and ancient snow. Researchers melted 295 kilograms of Antarctic ice cores, formed from snow that fell between 40 and 81,000 years ago. Then they counted the iron-60 atoms in the meltwater.

14:57Their measurements suggest that iron-60 was deposited on each square centimetre of Antarctica at rates as low as one atom every five years. These rates are lower than those found in measurements from more recent periods. The authors say this suggests that during the last 40,000 years, the solar system has been crossing a region of space relatively dense with interstellar dust, called the local interstellar cloud. Before that, our solar system was in the outskirts of the cloud, where the dust is less dense. You can read that paper in physical review letters.

15:35Next up on the show, a team of researchers have demonstrated a way that might allow mobile phones to one day see around corners. Using technology to peer where our eyes cannot is known as non-line-of-sight imaging, or around-the-corner imaging, and it has lots of potential applications in fields like robotics. In this case, a team of researchers are testing whether something that's already built into a lot of phones could be used to accomplish this feat. This technology is called LiDAR, which fires

16:13out pulses of infrared light and measures how long they take to come back after hitting nearby objects. It's similar to how bats echolocate, but using light instead of sound. LiDAR is typically used in phones to measure distances, which is useful for things like focusing the camera. But could it be used in a different way, to identify objects that can't be directly seen? Previous lab-based experiments have shown that round-the-corner imaging can be achieved by bouncing a high-powered laser off a wall

16:49at an angle towards a target object. Some of the photons hit the target, bouncing off it to return to their point of origin, where they are picked up by a detector. But these photons don't all return at the same time. The differences in return time builds a picture of the shape and position of the target. Of course, unless you're a James Bond villain, carrying a high-powered laser about in public is neither safe nor practical. So, in this case, the team have turned to phone-based LiDAR.

17:23However, phone LiDAR has several disadvantages. For example, the light it produces is a lot lower powered, and the sensors involved are a lot lower resolution. These are some of the things that the researchers have looked to overcome in their new work, which is published in Nature this week. It's worth noting that in this work, the team didn't use mobile phones, but did use consumer-grade LiDAR systems of comparable capabilities. One of the researchers behind the work is Siddharth

17:54Somersen-Durham, from the Massachusetts Institute of Technology in the US. I called him up to hear more about the research, and he explained an early experiment to see whether the team's approach would work at all. Whenever we're dealing with these sensing problems, the first thing we want to test is, is there any information in the data that we're collecting? Can we see the signal that corresponds to the thing that we're trying to image? And the way we did that initially is, we used the kinds of materials that you would find in stop signs, like very reflective, very bright surfaces. And we kind of saw something that we're like, okay, this might be it, but we're

18:30not 100% sure that what we're seeing is what we're expecting. So the other thing that we did, which really allowed us to confirm our suspicions, is we took this bright reflective patch, and we started to move it back and forth. So we had this LiDAR system that's pointed at the wall, and the patch is next to it, but the camera doesn't see the patch directly. So we just moved this patch back and forth, sort of perpendicular to the wall. And then we looked at the raw data, and we saw something in the data that sort of oscillated at the same frequency that we were

19:01oscillating the patch in the scene. And once we saw that there was a correspondence, we're like, okay, there is information. So the proof of feasibility has been established at this point. And so you had this early experiment then, and you could tell that an object was moving in space. But there's many more experiments in your paper as well. What else did you get this system to ultimately do? There were four major capabilities that we showed in the paper. The first one was that you could just track objects around the corner. So you point a camera at some surface, and you might have someone

19:32around the corner, and you can just track the position of that person around the corner. The second thing we showed was you can get the shape of objects by using camera motion. The third thing is tracking multiple objects. So doing something like hand tracking, or tracking multiple people moving in the same room. And the last thing that we showed is using the hidden object to figure out where the camera itself was. And the reason we thought that this might be interesting for something like robotics is because in robotics, you have a robot that often

20:02knows the environment that it's in, but it doesn't know where in the environment it is right now. And sometimes you have parts of the environment where you see a very clear landmark. And based on that landmark, you know where you are. But if you don't have a landmark inside your line of sight, you're not going to be able to figure out where exactly you are. Like you have a much better chance of detecting a landmark if you could just extend your field of view. So that was the goal. Can we just use hidden landmarks to figure out where we are? And all of this came out of a very simple model,

20:33which we showed in the paper. Well, let's talk about that then. So what was it that you did that enabled you to achieve these results? The most important thing here is motion. That was kind of the core of the paper. How can you model motion efficiently in a way that's conducive to the kind of processing you would want to do on a mobile device? So that's where we came up with this motion model, where we were able to simultaneously model the effects of the object shape, the object's position, and the camera's position. And we call this the aperture sampling model. All of the capabilities

21:04that I've mentioned are related to these three properties. Then it became a question of dealing with the noise and the resolution. So if you're looking at just a single frame, you are going to have a very hard time seeing what's happening. But if you look at things happening over a sequence of steps, then okay, like you could see that there's motion. So the basic approach that we took is we need a way to enforce coherence in these measurements over time. Like if your object is at a certain position at a certain time, it's not going to suddenly jump to another position, which means that if you have a certain measurement at a certain time,

21:37you don't expect a very different measurement in the next time step. So having a way of modeling that coherence, and for something like reconstruction, just being able to move the camera around, that in and of itself allows you to solve a lot of problems. One thing that I think we have to make clear, though, I suppose, is that these aren't photographs. These are a suggestion of what's there. Let's say you and I are standing on different sides of the room with the divider between us, and you point your device at the wall. It isn't a photograph

22:08of me, right? What sorts of things does one see using this equipment? Yeah, if you're hearing this, and then you go see the result, you should calibrate your expectations that you're not going to get an iPhone quality megapixel resolution image. What you can expect is oftentimes with these signals, first of all, you can say whether or not there's something there around the corner. That's the first order thing that you can get. The second order thing you can get is, is this thing moving or not? And the third order thing is maybe you could say something high level about it. Like, is it a rigid body object? Is it non-rigid

22:39body? How big is the object? And then at the last level, you could say maybe something about the shape of the object up to the resolution limit of the sensor. And you are by no means the only team investigating seeing round corners, essentially. How does your work differ to theirs, would you say? Yeah. So there's a lot of great work in this area where people use other types of sensing modalities. Some people try to use like just regular cameras to see around corners. Some people try to use Wi-Fi to see around corners or sound. There's sort of

23:12different trade-offs associated with each of these techniques. With things like Wi-Fi, it can penetrate through materials. So you can actually see through walls rather than around corners. The trade-off is there. You just lose resolution. So you might not be able to localize as accurately as you do with something like LiDAR. LiDAR is, I feel like, a very good trade-off because it's widely available and you're getting a good signal out of it. But it has to be said that a lot of your tests, you're using, you know, a perfectly flat white wall, you know, relatively simple shapes that you're scanning. What does this work not do? Where else does it need to go?

23:44Yeah. This work is by no means comprehensive. Like, we've sort of used simpler scenes. We've made assumptions about how we can capture the data. And there's a lot of things that happen in the real world, which are not yet reflected in the method. And so there's a lot of follow-up that needs to be done. So yeah, one example you brought up was like this white walls example, right? White walls are sort of friendly surfaces. One, because the geometry allows you to reflect light very nicely. But also, you tend to reflect most of the light that gets sent out. The implication of it not being white and planar is you're just going to get less light back, which doesn't mean that it's

24:17impossible. It just means that you're going into an even harder regime. But there's still a signal. So we sort of need better algorithms that can make sense of this very, very noisy data. And I think that's where things like machine learning and AI come in. Like, just have them see this data, learn to make sense of it. And I think you're much more likely to get better results. And I wonder as well, I mean, have you and your colleagues discussed the potential ethical issues associated with this as well? Because it's something that you can imagine is something that needs to be discussed. It's a very valid question. So I mean, the field I work in is computer vision. And it feels

24:49like almost anytime there's a breakthrough in computer vision, there's going to be a lot of ethical implications. Like if this technology just continued to develop, of course, you want to make sure that it's used for the right purposes. The good news is, because the sensors themselves are not the best quality. Again, it's not like you're getting a full photograph around the corner. But yeah, as to getting other higher order information, like in my opinion, like there definitely needs to be a discussion around that. But I'm not sure what the exact discourse for that should be. And in terms of uses, once this technology is out there and maybe developed a bit further,

25:21as you say, what do you see this being used for in an ideal situation? The use case that I would be most excited by is putting this into various robotic platforms, putting this into like self-driving cars, or even like these little delivery robots. So something like accident avoidance, search and rescue, where like, maybe you need to look through areas very quickly, and you don't have time to physically go check every part. Like if you could just use this laser to more quickly search the area, I think that could also be very impactful technology.

25:53But I will say most of all, I think just making this technology widely available is the application in some sense that we're most excited by, because there will be things that people think of, which we never thought of. And I think that's really the part that's most exciting to us. I can imagine folk itching to give this a try. Can they go out and install it on their phones right now? What needs to be done if that's not the case, you know, to get this technology out of the lab and available for people to try out? So unfortunately, you can't download it on your phone, because that would require like these companies

26:27to release their raw data, which they often don't do. But what you could do instead is you could buy these sensors for relatively cheap, like the sensor itself is maybe 1020 bucks. And then you maybe need to buy like a microprocessor, which is maybe another 20 bucks maximum. And then we have the code online, you can just download that code. And then you can basically run it plug and play like you don't need to do any calibrations, you don't need to set up any hardware, you just buy the camera, point it and then shoot, and then you're pretty much all set. And how excited are you to see folk figuring out what they can do with it?

26:58Oh, I'm super excited. Yeah, I already have people reaching out, like asking how this works. And usually the people who worked with this in the past were like imaging people like myself, they're experts in building imaging systems. But now I have people coming to me from fields like robotics and UI, UX and AR. And they're like, how can I get involved with this? That's very exciting. That feels like a really pivotal moment in this field of around the corner imaging. Siddharth Somersaduram from MIT there. To read his paper, look out for a link in the show notes.

27:28And that's all for the show. If you'd like to reach out to us, you can. We're on social media at naturepodcast, and we're also reachable by email, podcast at nature.com. I'm Nick Patridge Howell. And I'm Benjamin Thompson. See you next time. Thank you.