For Jennifer Chayes, math is a 'native tongue'

Mathematics and the shape of networks

0:00Hey, it's Flora, and you're listening to Science Friday. When you pick up your phone, a lot of the code that describes how social networks work, how apps predict what music you might be interested in, code that underlines new innovations in AI, traces back to kind of esoteric-sounding advances in mathematical theory and data science, ideas dealing with phase transitions, graph limits, and graphons.

0:30These concepts can be hard to wrap your head around, but fundamentally, they describe how massive networks behave. One of the people who helped create that field of math is Jennifer Chase. Her game-changing ideas in the mid-2000s helped lay the theoretical foundations of machine learning and modern AI. She spent 20 years at Microsoft building interdisciplinary research labs. These days, her interest lies in using machine learning to work on topics like material science, cancer immunotherapy, ethical decision-making, and climate change.

1:06She's also dean of the College of Computing, Data Science, and Society at UC Berkeley, the first new college at Berkeley in more than 50 years. And she's with me today. Jennifer, welcome to Science Friday. Thank you so much, Flora. I'm thrilled to be here. We're thrilled to have you. So you're clearly a math person. And when math people are portrayed in movies or on TV, they're often shown with this far-off expression and, like, numbers scrolling over their field of view. That's how they see the world. But what is it like for you? Do you see the world through math? And can you describe what that's like?

1:44For me, it's actually sense-making. It's how do I make sense out of the world. And for me personally, it's very interesting because my mother could not add fractions. My grandma got married at nine. And when I was three or four, I couldn't read at that point. I wasn't taught to read until I went to first grade.

2:14But I had this attempted sense-making in my head, which was bizarrely, I later recognized, mathematical. I would ask myself questions as a little kid and try to figure out how things behaved. And I had some wonderful neighbors. And I had some wonderful neighbors who would give me and my brothers candy and cookies, the wife would. And I heard the husband and the daughter talking about math when I was about four.

2:50And I asked, can I just sit here? And then I asked, can you ask me some questions? And it was almost as if, oh, this was my mother tongue, but I hadn't heard it articulated, almost as if I was a baby that had gotten that and then been moved someplace else without it. And then I was hearing it. And so for me, trying to understand, you know, like little almost word problems, like how, like, oh, if you're walking this fast, how long would it take to get across?

3:25You know, and so for me, this other vocabulary, this other way of making sense of the world was always there from my earliest memories. That's so fascinating. I want to follow up on some of your story in a bit.

Phase transitions in large language models

3:50But when you think about how you use math to make sense of the world, when you think about your work, your body of work, is there a big idea that connects all of it together?

4:01So networks is an idea that connects a lot of it. And I also think about phase transitions, which, you know, we all know is like water freezing or boiling or magnets magnetizing. But what happens in networks is that there are qualitative changes in the behavior of networks when certain parameters undergo phase transitions.

4:37We actually have seen a profound phase transition recently in chatbots. There are underlying relationships among words and concepts. And in the beginning, you know, we could complete spelling, we could complete sentences, we could probe locally in the domain.

5:19And then at a certain point, these started producing information from very remote parts, apparently unrelated parts of the network. So I think that's a profound phase transition, which we experienced in the network underlying some of the large language models. Is it a change in sophistication?

5:52No, I think it's a change in what parts of the network can be accessed. So let's talk about a really simple thing like water boiling, okay? And you start to see these little pieces where there's a little bit of boiling here and there, but it's not really going across the system. And then it goes across the system. There are little pieces.

6:23And then all of a sudden, there's order across the entire system. And those are the phase transitions one usually thinks about in physics or in chemistry. But there are transitions on information networks. So when I have a few sparse connections in a network, things remain isolated. Or it could be a network of human beings and a virus, and it remains localized.

6:54But then at a certain point, it gets transmitted across the system. And I believe something similar is going on underlying some of the knowledge systems that are embedded into the frontier models. So you're describing this with mathematics, which, you know, to me as a non-mathematician feels so abstract. And at the same time, this work feels so grounded in our lives.

7:25Do you feel like your brain works in both modes? Like the abstract mode and the tangible one? Definitely. It definitely works in both modes. I feel almost as if phase transitions and networks and, indeed, mathematics are, for me, a mother tongue. And so they are, I mean, I'm mixing metaphors here, but they are the lenses through which I see the world.

The creation of graphons

7:58That's so fascinating. I mean, you co-founded this field of math, the field of graphons.

8:04What problem do they solve? What do they help you make sense of? So sometimes there is a behavior of a network, be it a big network of people. It could be something like a social network like Facebook. It could be an actual human network. A network is any time you have entities interacting in pairs. And if I want to understand how that network is going to behave, the way that preceded graphons was I looked at each of those entities interacting with each other.

8:45That is as if, you know, I wanted to understand how molecules of air, you know, of oxygen and carbon dioxide, we're acting in this room by following every one of them. You know, we have thermodynamics, which tells me things about how pressure is related to temperature. And that's not by following every little entity.

9:20Scientists have realized for years that we can discuss the collective behavior without discussing each individual piece. And so, you know, when I started looking at networks, I went to some of the leading experts in graph theory. And I asked, well, how do you look at properties when networks get really large? How do you look at overall properties? You must have a limit of networks.

9:52And they said, no, we don't have that. You have to follow everything. And I said, that doesn't make any sense. Because you can't see the forest for the trees, you know. And there are properties that are forest properties. You don't need to know every tree in the forest to know that the forests on the Olympic Peninsula in the Pacific Northwest are different than the forests in upstate New York. Yes.

10:22And, I mean, it's not just that there are different trees. There's a different collective behavior of that forest. And so, we asked, what if you took a limit of a network where you had, like, an infinite number of pair-wise interacting entities? Every entity didn't interact with every other one. And then we realized, oh, you could take a limit so many different ways.

10:55You could take a limit that is so weak that everything converges to the same limit. You could take a limit that is so strong that everything converges to a different limit. Neither of those are very useful. And so, we came up with, like, six different ways taking limits that we thought were meaningful and that all gave the same answer.

11:26And that answer is the graphon. Hmm. Is it gratifying to see it, like, blossom into a whole field that kind of underpins, like, much of our lived experience? It really is. And it's particularly wonderful for me to see especially young scientists come along and do things with graphons that I never would have imagined. There's a company called Graphon, which some young guys started, which is using graphons to interpret video really wonderfully.

12:04And I'm on the board of that company, but I never imagined that anybody would use it for something like that to interpret video much better. Hmm. We have to take a break, but when we come back, I want to ask you a little bit more about your story. Is that okay? Absolutely. Stay with us.

12:31Hey, Flora here. Before we get a break, have you seen videos or workouts or massages promising to cleanse you or debloat you through lymphatic drainage? We're working on a story about the science of the lymphatic system, and we want to hear what claims you've seen, if you've tried anything, and what questions you have about the lymphatic system. Give us a call. 877-4-SY-FRY is our number. 877-4-SY-FRY.

A stormy childhood and dropping out

13:00I want to ask a little bit about how you grew up. Tell me a little bit about, yeah, tell me a little bit about your childhood. You mentioned mother tongue. Math is a mother tongue. Yes. Math is a mother tongue.

13:23Strangely came to me very, very early in my life. Math was always my happy place. It was a place of calm in a little bit of a storm. My childhood was a little stormy. I was a pretty rebellious kid. And I dropped out of high school, and I lived on the streets of New York City in the early 70s.

13:53Wow. Yeah.

13:56So pretty stormy. You know, I hung out in the West Village, which is not like it is now, and occasionally in the East Village. And I made friends with people who would, you know, help me take care of myself. And I was kind of there, and then I would go back home, and then I would go there. What was the impetus? Were you feeling like you wanted to be free? Did you not like school? Did you want to be out of your house? I actually, I loved school.

14:29I mean, I loved studying.

14:32The house was a little rocky. And I wanted to be out of the house. And I think it was also, you know, some of us have more hormonal fluctuations than others as we go through adolescence. I mean, now we have science helping us to understand that a little more, but I think we've all known that. And so I landed up going back to school, to a school for dropouts that was formed by the public school system in a church basement with volunteers teaching different subjects, which was great.

15:08And they taught us history by reading Supreme Court cases and English by acting in plays. And I taught the math because they didn't have a volunteer who could teach math. So I taught the math, which was great for me. So you're in school, but you're also teaching school. Yeah. But, you know, math is just, as I've said a few times, it's my happy place. And, you know, I was friends with the other kids there.

15:41And so the fact that I loved it so much was probably good for them, because they may not have thought they would love it.

15:53What did you learn from those years? Those years, I feel, were essential to who I am today. They taught me to embrace risk. They taught me not to be afraid of failure. I worry for kids now because they have to be so perfect. Kids with my background today might not get into a good college.

16:31I just aced my boards because I was a good test taker. And so in spite of all this rockiness, I got into a good number of good colleges. And I've always felt like I can go out on a limb trying to realize a vision that may be high risk in some ways, because if I fail, I can always get back up.

17:05You know that because you've lived it. I know that. I know that. And, you know, I try to instill that in my mentees. I've had a couple of hundred mentees over the years. And I try to get them more comfortable with taking the risks they are really very well qualified to take. Yeah, that's a hard lesson to learn, I think.

17:39I think it's a hard lesson. And I think that one of the reasons that, you know, I did biology and then physics as an undergrad and I did physics in grad school and I was a professor of mathematics. And then I ran computer science labs and did economics and biology and chemistry and all these different subjects is because, you know, there's something nice about being an expert.

18:12But there's also something thrilling about being a novice and I wasn't as scared about being a novice as some people who have not had to pick themselves up and figured out that they can do that.

Building interdisciplinary research labs

18:31You know, as you just mentioned, you created multiple Microsoft research labs and you brought together mathematicians and computer scientists and social scientists. I feel like now interdisciplinary work is kind of trendy. But what was it like when you were building these institutions? Well, it's really interesting when I got my Ph.D. in 83 and I wanted to do mathematical physics and everybody kept saying, no, no, you have to choose math or you have to choose physics.

19:03The mathematical physics community was very small and they said you're going to have to be in one kind of a department or another kind of a department. I tried to build interdisciplinary efforts within a normal university system and I realized the funding situation nationally and many other things were not aligned with interdisciplinary work.

19:34One of the reasons I landed up going to Microsoft, I had I had gone to grad school at Princeton in physics with the first CTO of Microsoft and he got it into his head that I should go there. And I knew no computer science and I didn't understand why he wanted me there. And he said, just come visit. And when I visited, I said, can I start interdisciplinary labs in anything I want? That's a big ask, first of all.

20:04It's a big ask. It's a big ask. I was really excited when they said yes and you can hire six professors away from academia. And I asked for six sabbatical visitors and six postdocs. And they said, yes, yes, yes. I called my husband and I said, I want to go to Microsoft. And he said, are you crazy? I mean, he said, did they offer you a lot of money? And I said, oh, my God, I forgot to ask about money. Or, you know, I didn't ask to have a lot of stock options.

20:37But what I asked for was, can I have complete scientific freedom? Yeah. And they said yes. And my husband and I landed up going there. We were going to look for academic positions together, but we landed up going to Microsoft. And Nathan Mirvold and Bill Gates were right. And I was wrong. I thought there was no relevance to computer science. And in fact, there was tremendous relevance to computer science.

21:10And it was really easy for me to learn computer science because it's all math. You must have been the only woman in the room a lot in your career. I think I read that you were the only woman getting a Ph.D. in math and physics at Princeton when you were.

21:28People have asked you if you faced discrimination. And I think in the past you have said you've said no, which is I find unbelievable, actually. Tell me about this. So I thought the answer was no. And I never felt it at the time, I think, because probably I had blinders on. I mean, there were things that were said that I realized in hindsight are just horrendous and things that were done.

22:03But I really did not recognize it at the time. I think it was just my coping mechanism. And it was only many years later when I was a professor that I saw these brilliant women who would tell me that when I would suggest that they might want to go to grad school, they would say, oh, no, no. I'm not good, I'm not good, not good at math. I just work hard. Or my brother's the one in the family who's good at math, you know, when someone is the top person out of 250 students.

22:36And so then I realized there was a problem.

22:41I didn't realize it for myself. I realized it when I saw it in others. And, you know, so over my career now, I've mentored a couple of hundred people, the vast majority of whom are women or other minoritized groups.

Generative AI and material science

23:01So I want to talk about AI. Your work was really foundational for the AI revolution. You've said that generative AI inspires you and scares you. So can you tell me a little bit about both of those feelings?

23:18Yes. Generative AI absolutely thrills me in so many ways. It is going to enable things in science, which will have ramifications for biomedicine and health and sustainability and human welfare in many ways. It can also be the great leveler of playing fields.

23:50You know, if I had generative AI way back when and some of the phenomenal personalized tutoring programs that are coming out now, I could probably teach some math to my mom. It could be such a great leveler of playing fields. I'm also, of course, worried that the results or the benefits of it will fall unevenly if we don't build it in the right way and there are safety problems.

24:31Give me an example of where you think this is, you know, really powerful, that really excites you.

24:38So about four years ago, I started working with Omar Yagi, who last fall won the Nobel Prize in chemistry. He knew no AI, and I knew essentially no chemistry.

24:55And we worked also with my husband, Christian Borgs, and with other people. And when ChatGPT came out, we asked, could we synthesize these amazing materials that Omar had created 30 years earlier called MOFs, Metal Organic Frameworks? Could we synthesize them faster? Metal Organic Frameworks are the most porous materials known to man.

25:28A gram has the internal surface area, because it's so porous, of a football field. So they can do amazing things. They can capture specific molecules out of the air, like carbon dioxide. So these are amazing materials, but it would take about three years when he had an idea to figure out how to synthesize it. So we used AI to take that down to a couple of weeks.

25:58Typically, failed experiments are not as useful as they could be, because you see something fails, but you don't really take advantage of what that is telling you. So the right way, I believe, to do AI for science is to couple the AI to experiments, to have the AI help you predict what the next experiment should be, and then deliver an answer that goes right back into the AI automatically.

26:39And each time we have a negative experiment, we rule out a huge part of the space, and rule out another huge part, and another huge part, and we get to answers. And that is, I believe, the way AI for science is going to happen. Hmm. You know, you've built so many institutions over your career. What kind of new institutions do you think we need to help us utilize AI to make it safe, to make it leveling, and not, you know, creating even bigger gaps?

27:14So I'm happy you asked that. That's actually why I came to Berkeley seven years ago. Well, AI was not quite where it is now, but it was moving pretty fast. And I feel that we all need to inoculate ourselves against the harms of AI, and I believe it is teachable. I interact with my chatbots probably differently than some other people do because I'm always asking them to verify things, always.

27:53I say, give me references, verify, tell me how I can check this whenever I ask a question, and in that way, you inoculate yourself against some of the misinformation. And I believe we can also teach everybody how to use an agent and give themselves the resources that only the really privileged and wealthy had in the past. This is something, I mean, we're creating material not only for Berkeley, for our college, but for the community college system in the state of California, which has 2 million students.

28:35You know, I graduate 2,000 undergraduates a year from my college. That's 2 million students, a million graduating a year from the community colleges. So I believe it can be a huge leveler. We have to teach people how to use it and how to be the partner to AI. So I really want to see after people learn the basics, for which we're going to go back to blue books, then we need to teach them how to work in AI hybrid teams and push the boundary of whatever AI can do in whatever their discipline is.

29:12And so I believe there are going to be many, many new jobs at these boundaries. Okay, last question. You've lived such a fascinating life. Leave me with your best life advice.

29:28My best life advice is to, well, it's based on the fact that opportunity knocks at the most inopportune times. Grab the brass ring. When you see it, grab it, even if it's an inopportune time. You know, my mother died right before I was supposed to interview for this job.

30:01Grab that brass ring. I love that. Thank you. Jennifer Chase is the Dean of the College of Computing, Data Science, and Society at UC Berkeley. This has been such a privilege. Thank you. Thank you, Flora. It's been a privilege for me, too. This episode was produced by Charles Bergquist. If Science Friday makes your network of media better, consider leaving us a rating or review on your favorite podcast app.

30:33Thank you for listening. Have a great weekend. I'm Flora Lichtman.