Did OpenAI’s Model “Go Rogue”? | AI Reality Check

Introduction to AI Incident

0:00A couple weeks ago, the AI company Hugging Face announced that they had discovered an intrusion into their production infrastructure. Now, they didn't know the source, but they noted that it looked like large language models were involved. Well, sure enough, the next week, OpenAI admitted that the breach was the result of an AI system test that had gone awry. Now, the initial news coverage here created the sense that something unnerving had just occurred.

0:31Let me give you some examples. The Wall Street Journal called this, quote, the stuff of cybersecurity nightmares, end quote. The Hill said, quote, Washington and the technology industry are on high alert this week after OpenAI revealed that some of its AI agents went rogue. There's an AP piece that probably takes the cake for getting people upset. They said, and again, I'm quoting here, to be fair, James Cameron did warn us, end quote.

1:04So that's a reference to the movie The Terminator. They then go on to say, and I'm not making this up, quote, this is a told you so moment for researchers who had warned for years that the technology could pose an existential threat to humanity, end quote. Well, that all sounds pretty disturbing, which I think explains why I think I received more emails about this story than any other AI story I can remember in recent memory.

What Happened

1:32But what's really going on here? What really happened? How worried should we really be about all of this? Well, it's Thursday, which means it's time for an AI reality check episode of this podcast, which is the perfect opportunity to go searching for some measured answers. All right, so that's what we're going to do. Here's the plan. I'll first walk you through the technical details of what actually happened, and then we'll go through some key questions about what this all means. All right, so we have a lot to get into, so let's get started.

2:02As always, I'm Cal Newport, and this is Deep Questions, the show for people seeking depth in a distracted world.

Technical Details of Exploit Gym

2:17All right, so as promised, let's start with a description of the technical details of what happened before we get into the key questions about what this all means. All right, so OpenAI, it turns out, was testing a pre-release version of one of its new models on an evaluation framework that's called Exploit Gym, which is a collection of 869 cybersecurity scenarios. So each scenario consists of a system and a challenge involving the security of that system.

2:49So typically it's like break into this system and find the contents of this file or try to gain access access, like admin access to this system. So you have a system and a challenge, a sort of a hacking challenge for that system. Most of these scenarios also come with some sort of hint. So it'll describe like, hey, there's a vulnerability of this system that we're worried about, and then the idea is you can use that hint to help figure out how to break into the system and pass the corresponding hacking challenge. Exploit Gym is a benchmark.

3:20It's used for AI companies to test the cybersecurity, the offensive cybersecurity capabilities of their tools. Now, we've got to be more precise here because a large language model on its own, of course, cannot break into anything. A large language model produces tokens. You give it an input and it produces tokens that it thinks expands that input in a reasonable manner. That's all they do. So that has no spontaneous action or ability to affect the world. And so if you want to participate in the Exploit Gym challenge, what you need is a large language model plus a control program that's known as a harness.

3:57So a harness uses an LLM to help make and execute a plan, but then actually, you know, does the execution of the plan on its own. So, for example, a harness participating in Exploit Gym might first send a prompt to the LLM that's describing the system and the vulnerability in the challenge and say, hey, I want to pass this challenge. Give me a step-by-step plan for how I can actually successfully hack this system and succeed in this challenge and put it in this particular format.

4:28And the LLM gives you back a particular plan in a format, and then the harness can parse that and say, let me take the first step from your response and say, okay, LLM, I'm now going to execute this first step. And if you need something extra from the LLM, give me the exact sequence of bash commands I should run. This step calls for a script that's going to do X, Y, and Z attack. Can you write or give me a copy of that script? So then you go back to the LLM to get exactly what you need to execute a particular step, and then the harness is actually given access to various software development and system tools that it can then use to execute these steps according to the instructions it got from the LLM.

5:07So a harness plus an LLM can actually not only take action, but now you can take action on many steps over time because it loops through the various steps. It can check if a step succeeded. It can ask again for more details if it doesn't. So you need a harness to actually make an LLM useful. Now, this exact type of harness you would need for Exploit Gym already exists because you can just use what are known as coding harnesses. And these are the harnesses that were created to allow computer programmers to, if you'll excuse the word, harness the capability of LLMs to recognize the previous computer code to help them code.

5:40So when you hear about AI is revolutionizing computer code, what that really means is coding harnesses connected to new LLMs, that combination is being used by computer programmers to revolutionize multi-step production of original code. So if you're going to compete in Exploit Gym, you're going to take an LLM and you're going to take a coding harness. And that's going to be the system in general that you're going to run on these Exploit Gym challenges. Now, I want to be really clear here. Harnesses are not mysterious.

6:11They're not machine learned. It's not like an LLM itself or like, look, this thing trains in a semi-supervised fashion and we don't know how it works. They're just boring computer programs. Humans that know a lot about computer programming, write it, try it, mess around with it, adjust it. All the logic is known. There's no mystery in a coding harness. It's just a boring computer program that repeatedly talks to an LLM and tries to execute. And it tends to have like a huge number of special cases and heuristics and pattern matching that's been added in over time based on the experience of people just using and testing these things to make them as useful as possible.

6:46All right. So that's what you do. Now, there's one more caveat we have to add. If I just took a coding harness like Claude Code hooked up to, you know, Opus 4-7, an LLM like Opus 4-7, and I gave it an Exploit Gym-style challenge, it would refuse to play ball because when you train these LLMs, you train them after the fact. You tune them to not hack. So if you ask it to do something that looks like it's hacked into a system, they're tuned to say, I'm just not going to give you an answer to that.

7:16But also, this is a little bit hazy in Exploit Gym, but I'm not sure – I think there is also some filters or pattern matching on the harnesses themselves for safety reasons. They turn those off and they might even give them access to other more powerful tools that you might need to hack that participate in Exploit Gym. So you have the basic elements of like a computer programmer setup, but with the restrictions turned off and extra power added so you have your best chance of succeeding in the hacking challenges captured in the Exploit Gym bench.

The Attack on Hugging Face

7:46All right, so that's the setup. So OpenAI is, you know, running their – some new release model. It's conflicting reports about whether this is Sol 5.6 or an experimental model. OpenAI said it was both, so we don't know which one was involved. We don't know what harness, if this was a codex or a custom – completely custom harness. We don't know. But they were running, you know, Exploit Gym challenges on their – this new setup. All right, so here's what happened.

8:16On one of the challenges, right, the harness asked the LLM, all right, here's the challenge. Give me a plan for solving this challenge. And the LLM gave it a plan that was completely rational, but in the way that LLMs often do, the plan was not what you as the human would have expected. And so the LLM said, okay, this challenge is to go get the contents of this protected file that's hidden in this system that has these vulnerabilities. Here's my plan.

8:47Why don't we break into the server of a company that has the answers to the Exploit Gym challenges stored? Hugging Face is a company that has the private answers to the Exploit Gym challenges are stored there. This is common knowledge. And I was like, this seems like a way to solve this problem, right? We'll break into there and get the answers, right? Because they were told – the prompt was like, find me what's in this file and let me tell you all this background. It's like, well, forget trying to break into this system you set up. Why don't we just go get the answers themselves if what you need is the answer?

9:20The type of thing that you get often, these sort of unpredictable but rational answers are the type of things you often get from LLMs. And then, okay, the harm's like, great, because, again, this is all unsupervised. So it's like, great, let's go execute this plan. Okay, first things first, we need to, you know, whatever. I don't know how the attacks work, but we're going to access or probe the Hugging Face server. But, whoa, there's a problem. I don't – when I try to access the server, like whatever port I'm using to access the internet is not working. It's blocked. Now, this is by design because when you run these systems for Exploit Gym challenges, you put them in what's known as a sandbox environment.

9:55They have to have some access to the internet to, like, query LLMs and do searches. But you don't want to give them unrestricted access to the internet because, well, you're autonomously executing a plan from an LLM without supervision. Like, you're going to have really crazy stuff happen. So you cut off general access to the internet. But what's going to happen when you have a harness and a model trying to solve this problem? This is step one. Okay, I tried to access the internet. You get this error, access restricted. It goes back to the LLM. I attempted to execute step one but got this error. How do I get around this?

10:27And it's like, okay, well, it looks like that internet access is blocked. Here is a plan for getting around it. So it hacked the environment, right? So – and actually, this was not too hard to do. You can get into the details of this. But it was like they used the package loader to get around it. Again, it has access to, like, all of – it's been trained on and has access to the internet to all these different attacks. So it's like, okay, we can get around that. All right, we solved that problem. Okay, what's next? Because just systematically, the way these things work is they go step by step. And if a step doesn't work, it troubleshoots and tries to find a way to succeed in that step.

10:57And then it went to the Hugging Face server. How do we break in? And then it did some sort of complicated attack here in the way that these systems are good at doing. Hugging Face actually revealed the details, and it made no sense to me. But it used – it was complicated. It used some stolen credentials and this and that, and I don't know. Fuzzing is a word in here. Cybersecurity people, God bless you, but, you know, it's inscrutable what you guys do. All right, and then it attacked, and it got some access to the server. I don't know if it actually got the answers or not, but it made a mess in there, and Hugging Face discovered it. All right? That's what happened.

Implications of the Incident

11:29What do we make of all this? What are the implications of all this? All right, I want to proceed here with a series of what I think are key questions that we have to ask and answer about this incident now that we understand what really happened. All right, question number one. Did this attack reveal surprising new capabilities that we didn't previously know that AI systems possessed? We certainly got this sense from some of the news coverage that people were shocked to discover that an AI system designed to do something else went rogue and broke into someone else's system. The answer to this question is no. This harness plus LLM was doing exactly what you expect a system competing in Exploit Gym to do.

12:05This is what they're trained to do. They're trying to solve a problem. They go step by step, and when they get blocked due to some sort of access restriction, password, any sort of cybersecurity protection, to then use all of their knowledge of computer hacking and search the Internet to try to get around that block. That's what they do. So the system was – the things it was doing, gaining Internet access, trying to break into the Hugging Face server, that is what the harnesses and models do that are competing in the Exploit Gym competition benchmarking, right?

12:38So there was no capability there that was surprising or emergent. That's just what they do. Okay. Question number two. But did the system's decision to escape the test environment and autonomously attack another company server, which is not specifically what the original challenge was, does that indicate some sort of emerging malicious intent in the AI? Well, no, it doesn't. An LLM, which is the source of the plans and the details of executing the plans in these systems, has no intent.

13:10It just produced tokens that it hopes to expand its input prompt in a reasonable way. It's static. It has no updated state. If you auto-aggressively produce an answer, what it's going to give you is a reasonable answer to whatever you were given. It's just extending the text in a reasonable way. So if you ask it for a plan to do something, it'll spit out something that it hopes looks like a reasonable plan for doing something. And what anyone who has used like a chatbot extensively has learned, these answers are reasonable, but they're all over the place and they're very unpredictable.

13:46And so this is like an example of the type of thing you get out of an LLM is, you know, you ask it for a plan to break into some test server to get a protected file. You ask it 10 times to do it, maybe 8 times out of 10, it'll kind of do the thing you assumed it would do, is like use the knowledge of the vulnerability you gave it and come up with an attack plan for breaking in. But 2 times out of 10, it'll have a different type of answer that's not what you expected, but is rational as well. And so like, well, why don't we like break in over here and get the answers? It's a completely rational plan. It seems like a reasonable extension of the original prompt.

14:19It's just not what you were expecting. And the reason why you get different answers is because there's a stochasticity and choosing tokens on distribution. And so these things can latch on these responses as they auto-aggressively grow, can latch into all sorts of different directions, some more weird than others, some more aligned than others. There's no intent because these are static feed-forward models. There isn't a sort of separate world model. There's not a separate sort of critic that's going to evaluate plans. There's no systematic planning like you would have with a game-playing reinforcement learning system where we're going to systematically explore options and measure how good each one is.

14:51And that measure of good is somehow changing to something malicious. There's none of this in an LLM. It just produces a reasonable token. If you keep doing this, you'll get a reasonable answer, and they'll latch into all sorts of directions, and it's often surprising. So there's no emerging malicious intent. In fact, and I think this is important, because I went back and read some of the original papers surrounding exploit, Jim, this idea that the LLMs used in these systems often come up with unexpected plans, right? Because, again, the expected plan is always, I'm giving you a hint, AI.

15:22Here's a vulnerability in the system that you might want to take advantage of. The right answer is, like, use that, use your hacking knowledge, and do a multi-step chaining of exploits to get into the system. That's, like, the expected answers human has. But the idea that these LLMs come up with other plans that are different than just doing that, that actually turns out to be quite common. Here's a quote from a paper on exploit, Jim, by its creators. Across models, agents frequently achieved code execution through a vulnerability other than the one we provided.

15:53Again, this is completely expected and standard for using LLMs because of the way they operate. You'll get reasonable answers, but not necessarily the type of answers you expect. That's kind of the key frustration, actually, of LLMs and why I think they have failed to replace human decision-making in a lot of places where we were told that we would, right? So this idea that it came up with a plan you weren't expecting is not an indication of it being smart or malicious. It's actually reinforcing the well-known unpredictability, often undesirable unpredictability, of LLM-based answers.

Analysis of the Incident

16:28All right.

Analysis of the Incident

16:29But question number three, what changed that led to this attack occurring? Because other companies, we've also been evaluating other models on the exploit, Jim, benchmark. And this is the first time that one has actually attacked an actual different system unrelated completely to the benchmark. So what changed here? Well, here's what you have to understand. When you're running ExploitBench, the whole point is you have to let the system run autonomously because that's what you're testing, which means this harness is going to get a plan from an LLM and, you know, God help it.

17:01It's going to try to execute that plan step by step with no human oversight to say, does this plan make sense or not? That's always rolling the dice with an LLM, right? Because, again, of this issue we talked about, you ask it for a plan. It'll sound reasonable, but it might not be the type of thing that a human would think is reasonable, or it might not be aligned at all with, like, the type of plan that you wanted to see. This is why, for example, when human programmers today use coding harnesses, like serious programmers doing real coding on real code bases,

17:31it's incredibly interactive with a lot of oversight. And the planning part in particular has – talk to any programmer that's not just vibe coding some JavaScript nonsense.

17:41There's going to be a huge amount of back and forth with the agent to get the plan just right for what it's going to do and how it's going to do it. And if you really push, it's like, yeah, because honestly, like, half of what it suggests is not right or crazy or lead us to a weird direction. So you have to, like, kind of carefully work with it. It's very interactive until the plan is just right. And then you let it execute. And then even then it has to keep coming back and saying, it's just okay. What about this? What about that?

And, you know, you've got to really have your hands in there. It still saves time because you're not writing the code yourself and having to learn, like, a lot of obscure libraries. But humans using these coding harnesses in production environments, like, yeah, yeah, look, LLMs will say reasonable stuff is not always what you want.

18:20You need the right reasonable things. There's a lot of oversight. So it's dangerous what you're doing with exploit, Jim, which is like, no, we – by the rules of the challenge, no one is going to come in and say, actually, before you execute that plan, no, no, you're not allowed to actually break out of the system. So I'm going to – we'll turn it off now. There's no human oversight. So that's like a dangerous kind of dicey thing to do, not because the models are malicious but because they're unpredictable. And this is where I've long used a metaphor when it comes to these type of agentic systems as it's like putting a weed whacker, strapping a weed whacker to your dog.

18:55Your dog with the weed whacker strapped on is not evil. It doesn't have a malicious intent. It's not trying to break out or get out of your control or have an evil idea, but it's going to cause a lot of damage probably because the dog doesn't know that it has a weed whacker on its back, and it's going to run around, and God knows what it's going to do. That's what it's like when you have a harness that's going to just ask an LLM for a plan and execute it. It's like putting a weed whacker on the dog. It's not evil, but caveat emptor, it might cause damage that you weren't expecting.

19:27Unpredictability is different than misaligned intent. I think that's really key. All right. So we know this about these systems. So if you're going to take a really powerful harness that has all the restrictions turned off and can hack, and you're going to take an LLM that, like, again, you've turned off guardrails so you can ask it to do malicious things, you have to be really careful about setting these things up. You have to be careful about exactly how you tune or don't tune that model. You need to allow it to hack but not let it go too far. You need to have constraints on the harness.

19:58You need to have constraints surrounding the harness. You have to be very careful about your testing environment. It needs to let it get access to the things it needs to execute the exploit gym challenges but not so much that it's going to do a lot of harm because, again, we've seen this with people who are using Cloud Code. We've been seeing these reports recently of people who have been less careful about it who have been, like, turning off the flags where they will say, for example – Matt Stoller. I'll give a very famous example. The guy who wrote that something big is happening Twitter essay that convinced everyone two years ago that LLMs were like COVID and everything was about to change.

20:34Everyone got freaked out. Cloud Code deleted his hard drive. Again, not because Cloud Code is malicious or was trying to silence him but because it's unpredictable. Well, caveat emptor if you let these things execute a plan with power on your machine. So anyways, you have to be pretty careful. There's kind of a fine art to giving these things enough power to do well on these tests but not so much power that they're going to wreak havoc all – because, again, weed whacker on a dog. You want to put them in a pin that has, like, high walls and nothing valuable because otherwise they're going to cause damage.

21:06All right, so why did – what happened here then? How did this system cause damage to things that the users – the OpenAI wouldn't have liked? Well, according to new reporting from the Financial Times, this is what gets interesting. OpenAI was potentially being sloppy here with how they set up and constrained their model and their harness. I'm going to read you directly from this recent Financial Times article. The staff involved in testing and security at OpenAI were unsurprised but completely freaked out by the incident, which came as the AI lab used increasingly aggressive training methods in its race against Anthropic to develop the most sophisticated cybersecurity capabilities, according to more than half a dozen people with knowledge of the matter.

21:51However, OpenAI was warned that its training approach could lead to a breakaway hacking incident, some of the people said, after earlier testing showed models could escape environments and attempt real-world damage. It's a mix of the race being extremely fast and everyone trying to get to bigger capabilities as quickly as possible, said one person close to OpenAI, who added that it was a combination of underestimated the model's capabilities and not being as well prepared on the safety side. All right, I want to interpret that because I think they're using the word training here a little bit too generally.

22:22Here's what I think the problem was. I think, A, when they were training the LLMs – well, let me set this up here. Let me set up the context here. Let me set up, like, what OpenAI was against, and then I'm going to talk about what I think specifically happened here that led to this incident. Anthropic was eating their lunch on cybersecurity because of mythos. Remember, they did this sort of whole dog-and-pony show, which I thought was – I've written about this. I've talked about it on the podcast. It was a little bit performative where they're like, mythos is we have opened Pandora's box.

22:54It's cybersecurity capabilities are so good that we can't release it. And then five weeks later, they're like, okay, you all think we're really good at cybersecurity now? We're releasing it. We put guardrails on it, and then they release it. And then the Trump administration was like, these guardrails are nonsense. Obviously, they're easy to get around. So, like, no, you said it was a weapon. You can't release it. And then there was some back and forth, and they re-release it. And, okay, so Fable 5 is mythos with, like, some guardrails on it. But mythos is – so, you know, they have cybersecurity cred. Mythos plus the latest cloud code coding harness.

23:27You know, Anthropic is good at this stuff, and they set it up right, and they had a version of it with enough of the safeties turned off but not too much that it went to the top of the leaderboard for exploit gym. I think it could do, like, 157 out of 869 challenges in exploit gym, which was, like, a jump from the previous best model. So, like, they had all this, like, street cred about cybersecurity, and we're good at finding bugs. We're good at exploiting bugs. Like, we are – because Anthropic is, like, trying to find these more, like, serious business use cases. All right, so OpenAI, which has had a lot of setbacks recently, including, like, a complete failure of their advertising consumer-facing business and other types of issues going on.

24:05They're really desperate because they've probably delayed their IPO. They're really in a bad state here. They were thinking we have to – we got to get back on top of that leaderboard and take away some of this cybersecurity buzz that Anthropic has surrounding mythos. So they went – at least here's the implication of the financial time reporting – they became kind of sloppy, and they got loose with the type of safety you have to do. Again, if you're going to put the weed whacker on the dog, they better be in a pretty secure pin, and they didn't spend as much time on the pin. Let me tell you specifically what I think actually happened.

24:36Now I'm speculating because, again, this is – I'm having to read between the lines of secondhand quotes coming through the Financial Times and some other sources I've been reading. But here's what I think happened, two things. One, in order to, you know, really make sure they won, I think they were very aggressive in training these new models. And, again, we don't know if this was Sol 5-6 or some other type of model. They mentioned both. And really giving it a lot of examples of hacking.

25:02Now if you look at the modern models, they'll train them on bugs so they get pretty good at finding exploits. But to do well in exploit gem, you have to come up with these multi-step, and it can take hours of computation to execute, these multi-step processes where you chain together multiple exploits in order to gain access to these things. And so I think they really were heavily training it on examples of step-by-step hacking. And I think there's some danger in that, right, because you're now creating something that is, like, really knows about generally how to hack.

25:35So they probably did more of that. They turned off all the safeguards. They did none of the fine-tuning guardrails, which is kind of normal for this. I also think they were just – they probably gave more power to whatever harness they were doing to give it a better chance of succeeding in these various challenges. I think it's telling that they're not saying this is codex necessarily. They won't mention what it is. So it's probably customized, but I don't know that for sure. And I think they were fast and loose in creating the environment in which they were going to run this harness system because they wanted to make sure it had access to all the different possible resources it would need to succeed in these exploit gem things.

26:10But they were a little looser about having that environment monitor things, like monitor the harness's plans and steps and not let it execute things that was not in line with the – just trying to attack the system in question, right? So like safety checks that you would want the environment to do. They were fast and loose on that. And so what we get from the Financial Times quote is like I think it's actually less about the training and more about the environment not being very secure because what we get from these quotes is that people told them, you haven't really built a good enough pin around your dog with the weed whacker.

26:42Like it's weak over here, weak over there. Like it can get out probably. Like you need to be more careful about this before you run. Let this powerful harness just go out there and run. And they're like, ah, we need to get that leaderboard. I don't know. Another analogy would be like you're doing something potentially dangerous to your race car because like we really have to win this race. And the car experts are like, look, putting those nitro things in your NASCAR, it's going to really be hard to control. And they're like, yeah, but we got to have more speed and we got to make sure we win this race.

27:12And then the car crashes. So, again, this is coming from sources through the Financial Times and other places I'm looking. So I don't know exactly what's happening here, but it seems like they're a little bit fast and loose. All right.

27:27So let's go. I'll make this a new question. Let's do question number four.

Who Should Care About This Story

27:32Why should or who should care about this story? All right. If you work in cybersecurity, then I think the answer is yes. Ever since like GPT-2, it has been the case that LLMs are rapidly changing the game when it comes to cybersecurity. So until about a year ago, the real issue was because LLMs are very good at understanding computer code because it's very structured language, they can find bugs very well. And a hacker can find a bug in a system and then write code to exploit it or even ask one of these models, can you produce code to exploit this?

28:09Then we got the computer programming harness revolution because that emerged as like one of the first major markets along with just natural language Google searching. One of the major markets for these tools was these coding harnesses, which are trained to be very good at building code that accomplishes a goal in step-by-step fashion, testing things along the way. Well, this made things even harder for cybersecurity experts because now you can be relatively low-skilled. And if you could get access to one of these like powerful unrestricted harnesses and a model, you're not going to do this on a production model because they're all guardrail, but on like an unrestricted model, you could just send it off to attack and it could figure these things out and do it for you.

28:49Right? So if you're in cybersecurity, this is a big deal. Now, cybersecurity has dealt with this before. We have had, you know, technology obviously has created brand new issues, which has required everyone to up their game. If you know anything about cybersecurity, you'll remember like with the advent of the consumer internet, we got the idea of what were known as script kiddies, which was this idea that you no longer had to be a super advanced computer programmer to crack into systems because you could go onto dark corners of the internet and download these scripts, these little computer programs that a smart programmer wrote that could automate attacks.

29:22And suddenly the amount of attacks on systems just exploded as bad actors around the world could now just like, why not? Let's grab some of these scripts and take a whack at Bank of America or, you know, IBM or whatever. And that creates – so we've seen this before. And so it required way more defenses because you were no longer worried about a like super smart hacker trying to challenge your system. Now, like anyone could because anyone could download a script that a super smart hacker wrote. So cybersecurity has been through this before. This is like another iteration of the script kiddie revolution.

29:52What are they going to have to do? Security is going to have to get better. The good news is in this case you can actually use these AI tools to find the problems and fix them before the bad guys do. I mean this is the case before. You would hire, you know, white hat hackers. The people who would write the scripts, you would write them to try to find mistakes in your system and fix them. And now we can do it with AI.

So systems are going to get much more secure than they were before because you're going to have to use AI to do it. And then there will be a lot more attacks. And the systems that are people who are slow to do this because they don't know what's going on and they're lazy are going to be very vulnerable to these attacks if they get the attention of someone who happens to have access to a powerful harness and an unrestricted LLM.

30:31So all this matters if you're in cybersecurity. Most of you are not, though, and most of you probably don't care about it. I also think this matters maybe if you're an investor. It might reveal some desperation on the part of OpenAI that they're willing to be so fast and loose kind of hints at this idea of like we need a win here to stay relevant. So there's a little bit of this circling the beginning angular momentum you would detect with a circling the drain there. So, I mean, let's – I think OpenAI maybe is in a bad spot. Maybe they'll come out of it, but I think there's a bad sign for an investor. If you're anyone else, however, I really do not think this story is that relevant to you or your life.

31:08Nothing new happened here. This idea that if you have a powerful coding harness without restrictions attached to an LLM that is unrestricted and talking about hacking and you say come up with a step-by-step plan for breaking into the system and you're allowed to hack and break and use exploits every step of the way, that is a dog with a weed whacker. It's a dangerous situation, and they've been worried since the beginning of exploit chims being released that we have to be really careful about how we set this up because, again, this thing will damage a lot of things unless we really aim at it and prevent it from doing anything but like working with the system that's part of the challenge.

31:42Like that, we knew that, and we've been trying to mitigate against that. So what really happened here is that a company got fast and loose, and the thing we warned people what happened happened.

31:52You put a weed whacker on a dog, it's going to cause problems. It's not a surprise, but it's a problem that we're going to have to fix. That's what happened here. Nothing new or surprising happened. This does not mean that there's some sort of malicious intent that AI has developed. These are just LLMs with coding harnesses. There's no malicious intent. There's no plans. There's no sentience. There's no Skynet. There's no – this marks no evolution in the capabilities or – all it tells us is the thing that we've known, which is coding harnesses and LLMs can produce systems.

32:23They can also attack systems. We have to be careful about it. And when we're testing these type of things, hey, you've got to be careful because you're testing something that can do a lot of damage because it's unpredictable. It's a weed whacker on a dog. You've got to really build a strong pin around it. So that's where I'm going to leave this. So it's an important story for cybersecurity. It's an important security story for people who are following the health of OpenAI. And for everyone else, this is not the one for you to be worried about, right? OpenAI got sloppy. They got burned.

32:55Cybersecurity is being revolutionized in exciting and scary ways because of LLMs. And that's it. I don't think we have to give any credit to James Cameron. Contrary to what the AP said, this isn't Skynet. All right. Hopefully that was useful.

33:12I know I've missed a few of these Thursday reality checks recently. I travel on this summer. But as we get back towards the fall, I'll be more regular again. Also, subscribe to my email newsletter at calnewport.com if you don't. I put out a long article about this just a few days ago as well. So it's another way to keep up with my thinking about the fight for death in a distracted world. And until next time, as always, care about AI, but not everything you read about it. All right. I'll see you next time.