Monologue: Jacob Coxon and the AI Safety Grift

Anthropic insiders and AI doomerism

0:00This is an iHeart Podcast. Guaranteed Human.

0:05Hey, it's Kelly Rowland. You may not know this, but I have eczema, so I get how it can steal your time. But why let eczema take over when you can talk to your doctor about ebglis? Ebglis Lubrikizumab LBKZ, a 250 milligram per two milliliter injection, is a prescription medicine used to treat adults and children 12 years of age and older who weigh at least 88 pounds or 40 kilograms with moderate to severe eczema, also called atopic dermatitis that is not well controlled with prescription therapies used on the skin or topicals or who cannot use topical therapies. Ebglis can be used with or without

0:37topical corticosteroids. Don't use if you are allergic to ebglis. Allergic reactions can occur that can be severe. Eye problems can occur. Tell your doctor if you have new or worsening eye problems. You should not receive a live vaccine when treated with ebglis. Before starting ebglis, tell your doctor if you have a parasitic infection. Paid partnership with Lilly. Respect your time. Ask your doctor about ebglis and visit ebglis.com or call 1-800-LILLY-RX or 1-800-545-5979. Healthcare can feel complicated. That's why Optum uses technology to connect the

1:10people and processes that make healthcare easier, more affordable, and more effective. We're making it clearer for you to know exactly what your benefits cover. And to help you better manage your health, we're coordinating care between your doctors and your technology. We believe better, simpler healthcare is always possible. That's healthy optimism. That's Optum. Visit Optum.com to learn more. Nothing quite pops like the fun freeze-dried crunch of Eminem's Popped Caramel. Except maybe a pop concert performed live with you and a friend in the crowd. That's why Eminem's Popped Caramel is

1:45teaming up with iHeartRadio, who's giving away a trip for two to the iHeartRadio Music Festival in Vegas this September. Enter for your chance to win at iHeartRadio.com slash MMS. Eminem's, it's more fun together. Available at your local retailer. No purchase necessary. Ends 8-20-20-26. Must be 18-plus. 50 U.S. and D.C. resident only to enter. See official rules at iHeartRadio.com slash MMS.

2:06This Labor Day at Lowe's. Get up to 45% off select major appliances. Plus, deals on select materials and tools to keep the job moving. Right now, get a free DeWalt 20-volt max battery 2-pack. When you buy a select DeWalt 20-volt max tool. At Lowe's, we have what you need to keep your job moving. Valid through 9-13. All supplies last. Selection varies by location. See Lowe's.com for more details.

2:36Call Zone Media. Hello and welcome to this week's Better Offline monologue. I'm your host, Ed Zitron.

2:47Better Offline.

2:50I've now had quite a lot of emails about the post by Evan Hubinger and Jacob Coxon of Anthropic saying that they, and I quote, earnestly believe AI could kill all humans with a higher than 10% chance that it happens within the next decade. I want to leave by saying that these men, regardless of their intentions or cynicism, are beyond loadsome. They are disgusting to me. I find them vile. And I find it equally vile how many people are falling for their garbage. If they truly believe what they're saying, the idea that they're continuing to work on a product

3:22that could destroy humanity makes them want to be war criminals. Coxon, who recently quit Anthropic, did so in an extremely public way that has been covered by much of the mainstream media. Claims that he quit, to quote the Wall Street Journal, because he doesn't want to participate in an industry-wide rush to build AI systems that can improve themselves, which, as I will refer to later, is referring to recursive self-improvement, which nobody has proven actually is possible. In an interview with Wired, Coxon spoke at length without really explaining what it was he was scared

3:54about, outside of one moment where Max Zeff asked for specifics about his concerns, at which point Coxon incorrectly describes the hack as AI doing this all of its own volition, front of the hugging face attack, only for him to respond when pressed about whether this is companies just moving recklessly fast, that he didn't want to focus too much on the hugging face attack, because there's plenty of evidence that the companies don't know how to align the models properly. In other words, the moment that Coxon was asked to get specific about what the companies are doing wrong and where they're making mistakes, he punts to a non-answer. By the way, Coxon's solution to all of these

4:28problems is that the AI labs should agree to not rush towards recursive self-improvement. That's still a theoretical term for an AI that can train itself that everybody's talking about like it's real. In other words, Coxon is suggesting that Sam Altman and Dario Amadei put their clammy hands together and agreed to delay breeding the Grinch. We're not getting a Lorax, folks. We're not making Shrek. We will not let the fairy godmother into your house, okay? Godzilla will not happen. Yet the most laughable part of the interview was, when asked about why he kept working at OpenAI and Anthropic for years, when Coxon said that it's also not hyperbole that we could

5:04cure cancer, because OpenAI, by stealing someone else's work, I'm not getting into it, it's all over the subreddit, solved the Navier Stokes problem, adding that there really is no reason that we, referring to the AI industry, can't transfer that to scientific domains like biology. The kind of thing you say when you're just making shit up for attention and you're wrong, you're just fucking wrong. Those are two very different things. A mathematical problem is not the same as biology. What are you talking about, Jacob? Why are the journalists just printing everything he says without pushing back? It drives me insane. Nevertheless, people

5:38have been saying, oh, oh, well, he's leaving because he's so concerned. He's got to tell everyone because he's so concerned. He's so worried. No, no. The piece ends with Coxon saying that he'd like to do some sort of independent commentary on where things are going, which means a sub stack, and he'd like to do something like AI 2027 or move into some kind of accountability role. Yay! Yay! There's the grift. Wee! Congrats, everyone. Congrats on helping Jacob Coxon get a new job. Congrats. Congrats, everyone. You did it. You helped elevate a real piece of shit.

Media failure and the AI grift

6:12Okay. My dear friends in the media, my esteemed colleagues who allegedly have been doing this for years, Anderson Cooper, CNN, Wall Street Journal, all of you, everyone, how many fucking times are you going to fall for this? Why do you believe these people other than that some of them are saying vague and scary stuff and that they happen to be at the companies? Why do they never really say what it is they're scared of? And why, when they even try and get specific, not that they do very much, do they never blame the companies? I say it again. Nobody has actually achieved recursive

6:48self-improvement, nor have they shown any proof that it's even possible. But the fact that Coxon is bringing it up in the run-up to Anthropic's IPO, and as the rest of the AI hogs oink about it non-stop, makes it impossible for me to believe that Coxon has anything other than the most cynical intentions. People say, oh, he gave up his options in Anthropic, as if that matters in any way, shape, or form. He was barely there a few months, he probably didn't have that many, and saw an opportunity to get a bunch of attention. He was at OpenAI for years and likely has a ton of

7:21options from there too. Why in the world does this keep happening? This is how bubbles form. This is how grifters grift. It starts and it finishes with the media industry giving up on their jobs. It starts and finishes with the most important journalists in the world failing. And I think everyone failed here. And I am disgusted and outraged to see this happen again. It's exactly what Matt Schumer did a

7:51few months ago with something big is happening. It's the Citrini memo about the global intelligence crisis. It's the same shit. It's science fiction dressed up as fact. And the fact it works is only because to quote Ed Elson, we have this cult-like worship of the wealthy, where we believe that the people at the companies and the people will move around from these companies in a way that's only honest, in a way that's only true. When in fact, these are some of the most cynical people in the world, which is pretty obvious in the fact that they go, oh, I'm afraid that this stuff is going

8:26to destroy the world, but they keep working on it. And I don't give a shit if he's not working at one of the companies. If he goes into an alignment role or a non-profit for this shit, it's the same thing. It's a way to keep milking the cow. And Coxon's entire argument centers around this nebulous idea that things are moving really fast, but never really crystallizes on what it is that's moving fast, nor does it ever hold the companies in question accountable. He told Wired that it was

8:56important to ask executives on the record to give an actual probability for extinction in the next decade. And I want to be clear that this statement means absolutely nothing. And anyone taking the question or the answer seriously is a goddamn mark. What does the answer even mean? What's the difference between a 10% and a 30% chance? What does 10% even mean? What does any of this mean? Ah, who cares? Put the AI hype in the bag. Who gives a shit, right? Coxon claims that this is not a marketing stunt. By the way, the biggest sign that something is a marketing stunt is when someone says that.

9:29And that executives and senior researchers couch their phrasing in the press to sound sensible, which is hilarious because I've been hearing these kinds of threats for years. Dario Amadei, Wario himself, said to Axios last year, or maybe CNN, that there was a 50% chance of AI wiping out all of white-collar labor, or maybe it was an AI will definitely wipe out 50% of white-collar labor. But the fact that it's not really clear is kind of the point I'm making. It's just saying stuff. In any case, Hubinger, who still works at Anthropic, followed up on his post agreeing

10:02about the dire threats of AI, but adding that the risk from the present models is low. But don't worry, though. His worries were around superintelligence arising from recursive self-improvement, which is happening faster than we thought, referring to something that has not happened yet.

10:22To be clear, there is an actual threat from LLMs. Anthropic and OpenAI are using hundreds of billions of dollars' worth of infrastructure provided by the largest companies in the world to brute force hack, using LLMs bouncing off of each other because they can't find any other uses at scale. These models are doing exactly what they're trained to do, which would make everybody ask why the fuck they're being trained to do it.

10:55Healthcare can feel complicated. That's why Optum uses technology to connect the people and processes that make healthcare easier, more affordable, and more effective. We're making it clearer for you to know exactly what your benefits cover. And to help you better manage your health, we're coordinating care between your doctors and your technology. We believe better, simpler healthcare is always possible. That's healthy optimism. That's Optum. Visit Optum.com to learn more. All right, everybody. I'm back talking to you about Quince, one of my favorite clothing brands,

11:29and I've been feeling a little dangerous, which is why I'm eyeballing one of their cotton mesh stitch sweater polos when summer ends and the temperature begins oscillating here in New York City. I'm also likely to pick up one of their cotton knit blazers for my next trip to the stock exchange and really can't recommend their stuff enough. Quince focuses on high quality wardrobe staples made with premium materials like 100% Mongolian cashmere, organic cotton, and merino wool. Everything at Quince is priced at 50 to 80% less than similar brands, and they work directly with

12:00ethical factories and cut out the middlemen. So you're paying for high quality, not brand markup. I love their stuff. I bought it before they advertised, and I'll keep buying it still. Find your next full favorites at Quince. Download the Quince app for app-exclusive offers or go to Quince.com slash better. Get free shipping on your order and 365 day returns. Now available in Canada and the UK too. That's Q-U-I-N-C-E dot com slash better.

12:30This is Matt Rogers from Las Culture East. That's with Matt Rogers and Bowen Yang. This is Bowen Yang from Las Culture East. That's with Matt Rogers and Bowen Yang. You know when people try a new food and suddenly it's like, okay, hold on. I got a new favorite food. That's the reaction a lot of people are having when they first try Kewpie mayo. Yeah, it's the one with the red cap and the little baby on the bottle. You've probably seen it at the grocery store. And this mayo is different. Most mayonnaise uses whole eggs. Kewpie only uses egg yolks, which gives it this rich umami flavor. It's smoother, deeper, almost buttery. Once people try it,

13:02they start putting it on everything. Egg sandwiches, fries, burgers. Chefs use it. Restaurants use it. People who really care about flavor use it. Put it on just about anything. Then you'll understand. Kewpie, the original Japanese mayonnaise. Instagram teen accounts have automatic protections for what teens see and who can contact them. Plus, time management tools.

13:24And Instagram will continue adding built-in safety features to help create age-appropriate experiences. Learn more about teen accounts and Instagram's ongoing work to protect teens online at instagram.com slash teen accounts.

Reckless hacking and corporate irresponsibility

13:40As I went into with Cal Newport a few weeks ago, this is not autonomous hacking by agents that escaped sandboxes. It's anthropic and open AI throwing unlimited compute to make LLMs that can quote, do cybersecurity and having really terrible security practices and not being able to train them in a way that was consistent or safe. And then they still use them. They still treat them as if, oh, they'll just work it out or just, maybe they don't give a shit.

14:15Maybe they don't care. I think that's, that's probably the most likely outcome here, by the way. To be clear, whatever anthropic and open AI models that were responsible for the hack are very, very dangerous, but not because of any sentience or magic or thought. These are LLMs talking to LLMs to decide what they should do next, prompting each other again and again and again, and having unlimited resources to do so. If we weaponize billions of dollars to sink into the automated scripts that hackers have used in the past, we'd get probably much the same results and be sending people to jail. Somehow, despite years of these dire warnings, the warnings

14:51in question never include anything about what's happening today with really any specificity. There are no scaragrams about how anthropic and open AI have near unlimited resources to commit to dangerous, reckless experiments that amount to felony hacking. There doesn't seem to be any interest in holding these people accountable. There doesn't actually seem to be any interest in stopping anything. It just seems that we all want to have a jerk-off theater around scary things that nobody actually wants to define. Even when Coxon discussed the hugging face attack with

15:21Wired, he framed everything in terms of helplessness, calling the LLMs an agent swarm and saying that one of the agents did this hack as part of a general strategy for understanding more about the grader, versus a piece of software with poor security controls and unreliable automations pursuing a task in an unexpected way. Coxon's language intentionally minimizes any kind of responsibility or accountability on the part of the AI labs, choosing instead to put the blame on powerful AI that they can't understand. There really is no economic reason to do

15:53cybersecurity stuff with LLMs, outside of the fact that these companies are hitting diminishing returns in coding, and that there are exhaustive troves of vulnerability data online for them to train their models with and do exactly what they've been doing. Brute force hacking of GPUs has existed for over a decade, albeit with different techniques, and the innovation here is a direct result of the unbelievable resources handed to these two irresponsible, disgraceful companies. But China might do this is not an answer to why are American companies doing this, because this is

16:23clearly a situation where OpenAI and Anthropic have built something they were aware would mindlessly bash its head and millions of dollars of compute against the problem until it broke through. Make no mistake, dangerous AI is already in the wrong hands. Those of Anthropic and OpenAI. If a regular person did the hugging face attack, they'd be in jail, because this seems like, based on discussions with experts, a clear case of firmly hacking. And if this wasn't the result of powerful AI, everyone involved would be in shackles. These companies act as if they're being forced to build

16:59these tools and run these experiments, when they're actually acting with complete autonomy, far more than they should ever have been allowed to have, with resources that I've repeatedly said and will say again are virtually unlimited. They act as if they have no responsibility or ability to stop their own experiments, or monitor them, or really do anything but continue to feed them training data and watch what happens in awe, but also write long blogs about how they're scared, and also fucking hack people.

17:27Let me be very clear about this. Anthropic and OpenAI have repeatedly and flagrantly engaged in what appears to be illegal hacking of multiple different systems, and have done so using AI GPUs, sold by NVIDIA, and powered by infrastructure built for them by Amazon, Google, and Microsoft. They are the ones making every one of these choices, and they will continue to do so every time we elevate the voice of a cynical grifter like Jacob Coxson. The people that work at these companies are willingly engaging in acts that, if not criminal, are morally and ethically bankrupt.

18:02And the people who keep giving these dire warnings about the dangers of AI never seem to give a shit about what's actually happening. Always keeping your eye on some non-specific harms in some non-specific future, all while saying that they alone are the ones who will protect you from whatever it is that might not happen. That they're blowing the whistle on companies that have paid them hundreds of thousands of dollars. And indeed, we don't know. And I think there are actually very good questions to ask about what his compensation from Anthropic was, and whether it's ongoing. I ask this because a lot of people from Anthropic are very supportive of a guy who just quit. A guy who just

18:37quit and is basically accusing the company of creating something dangerous that will destroy the world. Isn't that strange? To Coxson, Hubinger, and any other AI doomer, I have to ask, what is it I am meant to do with this information? What is it any of us are meant to do? Are we meant to be scared of you? Are we meant to sign up to Claude Max? Are we meant to invest in the Anthropic IPO? Are we meant to regulate this stuff? How? Why do you never say that? Why do you never say what it is we're meant

19:12to do to stop this? Why do none of you fucking people ever have any kind of suggestion or call to action or anything other than some sort of self-serving or company-serving diatribe about how powerful and scary AI is? And if you hear any loathing in my voice, that's because I find you loathsome. I find what you're doing disgusting. I think you are cynical. I think you're malcontents. And I think

19:42you're bad for society. And the fact you're being elevated is dangerous, but not for the reasons you're saying, but because it proves that our media ecosystem barely has object permanence. Congratulations on exploiting it. You're a fucking asshole. I think the answers to these questions are actually pretty simple. You don't actually give much of a shit and you want some attention. If you actually feared this stuff and felt a moral obligation to tell people, you'd be specific about what it is we should be scared of and indeed tell us what we need to do next. Every time it's the same, sorry, fucking story.

20:19Oh, we're building something so scary and powerful and we can't stop it. Nevertheless, we're going to keep building it for some reason we never discuss, but when it destroys humanity in some way we can't explain, you'll thank us, I guess. Great. Thank you, man. Thank you for taking up the airwaves. Thank you for distracting from the real problems. You guys are horrible. You guys are genuinely kind of evil. If you wanted to do something about this, you'd do something about this. If you cared, you'd care.

20:50You'd be saying we need to regulate in this way. You wouldn't be doing this mealy-mouthed horse shit of, oh, maybe they'll stop doing recursive self-improvement, a thing that doesn't exist. Weird that you don't talk about what's happening today. Weird that you don't talk about stopping the hugging face attack of the future. It's always the positive. It's always the positive in, even when being negative. And it's hard to see this as anything other than cynical marketing and attention seeking from two guys who should spend more time working on building technology than posting on

21:22social media about how scary things are getting. I don't like them, I don't trust them, and the whole thing's a fucking grift. Next week, I'll be back with Cal Newport and Adam Becker to talk about

Rationalist origins and next week preview

21:33the people that inspired this AI doomerism, the rationalists, and how to push back against their noxious bullshit. I'll catch you then. I love you all.

21:49Healthcare can feel complicated. That's why Optum uses technology to connect the people and processes that make healthcare easier, more affordable, and more effective. We're making it clearer for you to know exactly what your benefits cover. And to help you better manage your health, we're coordinating care between your doctors and your technology. We believe better, simpler healthcare is always possible. That's healthy optimism. That's Optum. Visit Optum.com to learn more. Most dog food brands don't really want you seeing how their food is made. Just food for

22:25dogs is the opposite. They actually invite you in. You can walk into any of their kitchens and see real human-grade ingredients like chicken, beef, carrots, and peas being prepared right in front of you. It's real food made in real kitchens. Nothing is hidden behind labels. And that kind of transparency says a lot. Nothing to hide. Everything to love. Go to justfoodfordogs.com and get 50% off your first order.

22:52With LPL Financial, we provide the services to help push you forward. When it comes to your finances, your business, your future, the only question should be, what if you could? Pit advertisement. Anna Kendrick is not a client of LPL Financial LLC and receives compensation to promote LPL. Investing involves risk including potential loss of principal LPL Financial LLC member FINRA SIPC. Hey, it's Kelly Rowland. You may not know this, but I have eczema. So I get how it can steal your time. But why let eczema take over when you can talk to your doctor about ebglis? Ebglis Lubrikizumab LBKZ, a 250 milligram per two milliliter injection, is a prescription medicine

23:25used to treat adults and children 12 years of age and older who weigh at least 88 pounds or 40 kilograms with moderate to severe eczema. Also called atopic dermatitis that is not well controlled with prescription therapies used on the skin or topicals or who cannot use topical therapies, ebglis can be used with or without topical corticosteroids. Don't use if you are allergic to ebglis. Allergic reactions can occur that can be severe. Eye problems can occur. Tell your doctor if you have new or worsening eye problems. You should not receive a live vaccine when treated with ebglis. Before starting ebglis, tell your doctor if you have a parasitic infection. Paid partnership with

23:56Lilly. Respect your time. Ask your doctor about ebglis and visit ebglis.com or call 1-800-LILLY-RX or 1-800-545-5979. This is an iHeart Podcast. Guaranteed Human.