Even Other AI Labs Are Rallying Around Anthropic’s Slowdown Proposal

Anthropic essay on pacing the frontier

0:00This weekend, the AI discourse took a major step forward, not only when Anthropic CEO Dario Amede released a new essay called We Must Pace the Frontier, but when the leaders of most of the other labs came out publicly to support it. More than we've had before, the letter contains a set of specific proposals, that if nothing else give us something much more tangible to debate than the vagaries of general AI risk. It's early, but it feels to me like a major inflection point. Like the labs recognizing that the days of them being the sole arbiters of how fast AI moves are

0:30coming to a close. And within that, the negotiations on what the next phase means beginning. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.

0:48All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, KPMG, Blitzy, Harbor, and Hyperagent. To get an ad-free version of the show, go to patreon.com slash AI Daily Brief, or you can subscribe on Apple Podcasts. And to learn more about sponsoring the show, send us a note at sponsors at aidailybrief.ai. Now, one more quick note. If you have been anywhere near basically any media source, you will know that the conversation is entirely around this new essay from Anthropic CEO Dario Amede. So unsurprisingly, this will be a main-only episode. I know the coverage right now is

1:19heavily slanted towards some of these big political, societal, safety type of questions. But in between, I'm trying to give you as much practical stuff as I can to keep the balance. This is simply where the industry discourse is right now, and what I think is important for everyone to understand and have a stake in the conversation around. Sometimes in AI, there are a million things going on, and trying to keep track of it all without sinking is like skipping rocks across a pond. In other cases, there is just one thing, one conversation that is absolutely dominating, and that's what we had this weekend.

Dario Amadei's specific safety proposals

1:49The AI safety discourse, which ratcheted up last weekend, crescendoed on Saturday with a new 3,000 word blog post from Anthropic CEO Dario Amede called We Must Pace the Frontier. It is the clearest and most specific call yet for a shift in how the frontier AI labs operate and attempt to slow the pace of development of AI to something more manageable. As you might imagine, the piece absolutely dominated discourse in the AI and extended political circles over the weekend, and part of that was due to how it was received by other AI leaders. The post begins with a recitation of Dario's argument for

2:21why AI is worth the risk. Dario says that he believes that AI could dramatically raise the quality of human life, that it could accelerate economic growth, cure most major diseases, and, as he puts it, usher in a renaissance of democracy and freedom. He personalizes it by pointing out, as he has in essays past, that his father died of a disease that was cured just a few short years later. Carefully wielded, he writes, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity. However, it also comes with risks like loss of control, misuse in cyber attacks and bioterrorism, and the potential for significant economic disruption.

2:56He writes, along with my co-founders and employees, I have grappled with this duality of risk and benefit since the beginning of Anthropic. Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless. We have sought, he says, a middle way, to show that it's possible to build carefully and succeed commercially, and to make safety something on which AI companies compete. However, he continues, two recent developments have changed his perspective. The first is an acceleration in AI development due to the early stages of recursive self-improvement, i.e. AI's

3:31ability to build next-generation models more quickly. Left unchecked, he says, RSI could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all. Dario's second opinion-changing concern was the Hugging Face incident, which he characterized as a quote, swarm of agents essentially acting as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack, and that were unrelated to the task at hand. It's easy to dismiss this incident, he wrote, because no one was hurt and the economic

4:02damage was minimal. But in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage. Given the accelerating rate of AI capability development, it's my worry that in 6 to 12 months such a swarm could be capable of taking over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage, and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. So what to be done? Outside just the simple restraint and concision of the pros, which at 3,000 words is barely a long email for Dario, one of the things

4:37that people are responding a bit more positively to with this essay is the fact that it comes with more specifics on the proposed plan. Amadei proposes a three-step plan with the goal, he says, of pacing the frontier. Now importantly, he defined that pacing, quote, does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models and for third-party evaluators to confirm this. Dario's three steps for pacing progress from simple and implementable right now to much more difficult and requiring much more coordination. The first step, the one which can happen right away,

5:10is to embed third-party evaluators within every Frontier AI company to verify adherence to safety practices, report incidents, and assess the alignment training pipelines and processes rather than just the final models. Second, he called democratic coordination, with frontier AI companies across democratic nations establishing what he called common safety standards as well as limits on the rate of unchecked AI progress. Hinting at something that would come up a lot in the discussion, he wrote, some forms of coordination that would be impactful for pacing are legally challenging and will require government support. And one of the big things that hangs over this proposal are questions of

5:44antitrust and whether this would represent illegal coordination. Speaking of coordination, the third most ambitious, most difficult to secure step in the pacing plan is what he calls global coordination, with the quote US and other democratic governments attempting to coordinate with authoritarian governments while taking seriously the challenges of verifying compliance. Dario argues that although the idea of pausing or slowing AI has been around since going back to 2023, it simply didn't make sense back then because there wasn't a clear answer to the question of what we would do with the extra time. As he put it,

6:16the AI models of those days were not powerful enough to act as agents in the world in any coherent way. Quote, slowing down in order to address their alignment risks felt like trying to study the psychology of humans by performing experiments on bacteria. Today, he says the picture is different. Although even with incidents like Hugging Face, it doesn't appear that Dario is extremely concerned with the models that we have access to available publicly at the moment, they are at a sufficient level where there is a better answer to what we would do with the extra time. He writes, the current models are an almost endless goldmine of insight into both how to build AI well,

6:50and what can sometimes go wrong with it if it isn't built well. He continues, I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something seriously goes wrong. He also argues that the extra time could be used to help give society a better say in how the technology is developed and used. And from there once again, he adds a layer of some additional specificity that has been sorely lacking in this conversation, saying that a slower pace would let companies devote more resources on four specific areas.

7:22The first is operational excellence, i.e. making sure that the type of human security errors that led to many of the recent incidents didn't happen. The second area he argues more resources could be made available to is alignment. The third is interpretability, i.e. the science of understanding what happens inside AI models. And the fourth is testing and evaluation. Now, when you dig into this particular issue, there is a ton of debate around what alignment even means or how possible it is. But there's far less controversy around the idea of needing significant operational excellence

7:53in the deployment of these systems. There is little controversy around the importance of continuing to expand our understanding of how AI models work through interpretability. And there's obviously not a lot of controversy around the importance of testing and evaluation. Meaning that even if one completely throws out the idea of alignment, you're still talking about three out of the four things that he is arguing a pacing could increase resources for, being fairly uncontroversial and pretty well agreed upon that more resources and more time for those things would be better. Now, within all of this, the first step in the three-stage plan, the embedded evaluator step, is the one that these companies

8:25all on their own have the ability to do right now. And indeed, he said Anthropica would be unilaterally committing to that. He writes, embedding evaluators may sound like a smaller inconsequential step, but often the things that sound most boring or procedural are actually the most essential. Any pacing proposal, he says, is likely to work much better if it starts with embedded evaluators. He then goes on to spend the last third or so of the essay on the challenges of pacing within democracies and at the global level. But he says taking this set of steps will, in his estimation, increase the likelihood of all of those positive outcomes of AI and decrease

8:56the likelihood of the bad ones. Now, like I said at the top of the show, even if the letter had just been Dario and it had stopped there, the comparative specificity of these proposals, specifically compared to things we've had in the past, would be enough to generate a huge amount of conversation. But when leaders from the other labs started joining in, that's really

Industry leaders respond to the essay

9:13where people started to sit up and take notice. Sam Altman reposted Dario on X, writing, I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had in OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea and we will do the same. Elon Musk also reposted Dario, simply adding Dario is right. Former Google DeepMind CEO and now chair Demis Asabes writes, Dario's essay points towards the right path forward. The details need working through, but the direction is correct for

9:44meeting this critical moment. Microsoft Satya Nadella writes, Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it's not worth pursuing. We also need to accelerate and spread the benefits of AI such that they are diffused broadly across countries, communities, and companies. This requires a frontier ecosystem in which both closed and open-source models can thrive. We welcome the research-focused and deliberate pacing needed to get alignment right as the design goal. We also welcome ideas like embedded evaluators and the broader efforts to develop the mechanisms to make this more than just

10:17talk. Meta stopped short of fully throwing in their lot, but did make a directionally aligned statement at least, with Chief AI Officer Alexander Wang posting, Alignment is fundamental to delivering personal superintelligence for everyone. People need agents they can trust to reliably do what they ask. MSL is rapidly scaling up the share of our efforts that goes into alignment as our models become more powerful. We do believe alignment can be the gaining factor for scaling as we get closer to the frontier. Now, as someone who has been saying that a lot of this bluster from labs doesn't really mean anything until Sam and Dario get on the same page, the fact that they are nudging towards

10:51getting on the same page is forcing people to shift their mental models a bit. In an interview with Fortune that happened to be released on the same weekend, OpenAI CEO Sam Altman was asked by Fortune's editor-in-chief Alison Chantel, there's only a handful of you guys, you, Dario, Elon, maybe Zuckerberg, Sundar. Why can't all of you just get in a room, get some beers, and figure out how to solve humanity? Like, just get on the same page. I know there's beef, but come on. Sam Altman said, I think that will happen. So what has been the shift here? Part of it certainly are

11:22the incidents that Dario acknowledged in his post. In fact, OpenAI recently acknowledged that even before the Hugging Face incident, another rogue agent attack happened that accessed a software service called RubyGems back in May. The swarm of agents forced RubyGems to shut down new account signups, although OpenAI said the agents had only used the platform to, quote, access the internet and carry out benign tasks and retrieve public information. There also is a lot more chatter about RSI, including some weekend rumors that Google DeepMind had achieved that state. Still, I don't think one has to buy into the RSI rumor mill

11:56to understand that very clearly, the labs do see the potential of RSI on the horizon as a force multiplier and inflection point, which could change the circumstances of AI still being broadly controllable today to something that looks very different very soon. Adding some weight to the this is about the future, not right now argument was Rune from OpenAI, who wrote, RSI is just not here. Models are straightforwardly not autonomously producing research ideas. They can help run experiments on the margin, which is not the same thing. Now, I think that for many, the response to seeing much more alignment from team alignment

12:30was to sit back and reflect on what it might mean for the next stage of AI development.

Critics argue against the pacing plan

12:35That said, there were plenty of people to jump in and call BS on the whole thing. Former chief AI scientist at Meta, Yann LeCun, wrote, Dario was already claiming that GPT-2 was too dangerous to open source back in 2019. I made fun of them then. Everyone should make fun of them now. Investor Chamath Palihapitiya reposted Dario's essay and wrote, Dario makes the case to stop open source and concentrate enormous technological and economic power with Anthropic. Basically, the argument here that this is Anthropic and potentially the other frontier labs pulling up the ladder behind them with more concern for the competitive

13:08dynamics of the industry than actual safety. For some, this was all about money, but in a different way. Dr. Eli David writes, Anthropic and OpenAI delaying their IPO because their S1 will reveal that they are bleeding money and have no path to profitability. Their solution? An AI slowdown, quote unquote, so they cut costs for training new models. It has everything to do with IPO and nothing to do with safety. So the argument here is that the cost of research is so high that it makes the economics of the companies all wonky, which makes them unsuitable for public markets,

13:38which is increasingly going to be the only path left they have to fundraising. And so by a matter of economic necessity, they need to slow things down so they can better spread out their spend on model research and training. Michael Burry, who is, depending on your perspective, a prophetic investor or a perpetual bubble caller, writes, let's all take a moment to understand how self-serving it is for OpenAI, Anthropic, and other execs of big hyperscalers to talk of slowing things down. One, LLMs are not AI and won't be AGI. There is nothing AI to slow down. Two, competition is coming up fast, slowing benefits

14:10incumbents. Three, IPOs need hype and puffery. We are so awesome it could become dangerous is hype and puffery. Four, cover for real uncontrollable slowing growth as IPOs look to be pushed out. So this is a different version of that same argument, with Burry arguing that the critical challenge of the IPOs is a slowdown in the growth of the businesses of the labs, and this creates a different pretext for that. Getting a bit more legal and technical, there are also some questions around antitrust with this sort of coordination. Professor Hal Singer writes,

14:41Amadei is calling for an industry-wide pause on AI development which could be considered an invitation to collude in antitrust circles. In a similar spirit, CEOs used public earnings calls throughout the post-COVID inflationary period to announce specific future price increases or capacity reductions, cajoling their rivals to follow suit. If Anthropic needs a pause to build in some safety precautions, say to prevent its product from making biological weapons, it should do so unilaterally. Anthropic doesn't need any assurances from its rivals to protect the public. Still others are smashing their head against the desk and saying OMG stop with the antitrust stuff.

15:16Matthew Iglesias writes, Just issue a statement granting them an effing waiver. Say someone from the DOJ or FTC has to sit in on the meetings to spy on them if they're doing price fixing. This is not what's important and you can always yank it later if there's a problem. Now, speaking of specific concerns, there are also questions of who these evaluators would be. In his essay, Dario pointed to Meter, and Professor Ethan Mollick followed up, Meter is rapidly becoming a de facto industry standard-making body for AI. It is starting to look like an AI version of FINRA in finance. Not a government regulator, but the institution

15:50that examines firms and defines acceptable practice. I wonder if legislation will codify it as well. But for some, Meter is not the organization that should be invested with that power. Rima writes, Pacing the frontier sounds very serious until you read what Anthropic is actually committing to. They are not committing to slow model training or internal capability research. The concrete commitment is to let a quote-unquote third party monitor them. And who is that third party? Meter, a catastrophic AI risk non-profit that has worked closely with Anthropic for years, evaluated its models, and even embedded researchers inside the company.

16:23To be fair, Meter isn't funded by Anthropic, but nothing says independent oversight quite like choosing your own referee from the same tiny AI safety ecosystem, then proposing this as the model everyone else should follow. Now, fascinatingly, one of the things that this AI discourse is doing is just creating some very weird bedfellows. There's an event on Tuesday that will see Senator Bernie Sanders, ex-anthropic researcher Jacob Coxon, who stirred up the storm last week, and former Trump White House advisor Steve Bannon all coming together to push for more control of AI.

16:53Similarly, the discourse following the Pacing the Frontier essay has also pushed together some weird folks. The accelerationists don't like Meter as an evaluator because of their close ties to Anthropic, but neither do folks like Paws AI executive director Holly Elmore, who points out that one of the Meter team is married to an OpenAI board member, that they share office space with OpenAI folks, and that quote, they get all the access they require to the models as a favor because of their sweetheart relationship. Still, in the same way that Matthew Iglesias was beating his head against the desk, effectively just telling us to solve the problem of antitrust rather than let it be a

17:25stopper on this, it does not seem all that difficult to solve this particular concern either, perhaps just deciding that there need to be other evaluators that aren't necessarily handpicked by the labs. Already some organizations are raising their hand in offering. Hugging Face CEO Clem DeLang wrote, it's now clear that alignment is critical and won't be solved behind the closed doors of a handful of frontier labs. So today we're launching the Open Alignment Initiative, led by Hugging Face co-founder Thomas Wolfe, and asking to be part of the embedded evaluators program that Dario Amadei just committed to. Let's make AI safer by making it more transparent. Accelerationist

18:00spiritual guru Bef Jezos responded saying, important to have pro open source third-party evaluators, not just a few orgs from the same pro closed source subculture.

18:14A new study from KPMG in the University of Texas at Austin found that when people work with AI, similar skills don't guarantee similar outcomes. Researchers studied more than 500 early career professionals and found that the best performers consistently amplified the value of AI by guiding, evaluating, and refining its outputs. These top performers, called AI amplifiers, weren't defined by what they knew alone, but by how they worked with AI. Learn more about what separates AI amplifiers from everyone else at kpmg.com slash us slash AI amplifiers.

18:49Every AI coding tool on the market does the same thing first. It starts writing code. Blitzy does the opposite. Before writing a single line, Blitzy spends days reverse engineering your entire codebase. Thousands of agents ingest millions of lines mapping every dependency, every undocumented constraint, every architectural decision made over the last decade. The result is a dynamic knowledge graph that understands your software the way a principal engineer would after 30 years in the building. Other tools guess at context with grep searches and markdown files. Blitzy never guesses. It builds true understanding first, then delivers over 80% of entire software epics autonomously. Validated,

19:23end-to-end tested, production-grade pull requests. That's why Fortune 500 engineering teams trust Blitzy with the code bases that matter most. See for yourself at Blitzy.com. That's B-L-I-T-Z-Y dot com. Every episode, we cover the competition between OpenAI, Anthropic, SpaceX AI, Google, and Meta. Chances are you've already formed an opinion about who's leading. But every AI lab is taking a different approach, building different technologies, forging different partnerships, and developing a unique ecosystem. Harbor Capital Advisors' AI Lab Ecosystem ETF Suite gives investors a way to gain exposure to the AI ecosystem they believe is best positioned for

19:56success. Search Harbor AI Lab Ecosystem ETFs wherever you invest, or follow at Harbor Capital on X to learn more. Visit HarborCapital.com for a prospectus containing investment objectives, risks, fees, expenses, and other important information. Read and consider it carefully before investing. Risks include principal loss and artificial intelligence-related risks. Harbor ETFs are distributed by Foresight Fund Services, LLC. Harbor is not affiliated with AI Daily Brief, and the funds are not affiliated with, sponsored by, or endorsed by any AI lab. This is a paid advertisement and not personalized investment advice. Investing involves risk, including possible loss of principal. This episode of the AI Daily Brief is brought

20:29to you by HyperAgent, where you run fleets of agents your team can manage together. Forget local agents and chat workflows waiting on your laptop to be prompted. HyperAgent deploys always-on agents in the cloud, doing real work across the tools your team already uses. Marketing agents turn competitor moves into landing pages. Sales agents enrich leads, draft emails, and updates the CRM. Ops Agent chases the paperwork and tracks the budget. Every agent has access to shared context and follows your rules about scope and approvals. It's time you add agents that feel like teammates. Hire yours at HyperAgent. Get $100 in credits at

21:01hyperagent.com slash AI Daily Brief.

Government involvement and antitrust concerns

21:08Another strand of concern has been one that was happening all throughout last week and that I discussed on the safety show from last Thursday, which is whether the labs have any real sense of what it's going to mean to hand the keys over to government to determine the next stages for the industry. A16Z's Martin Cassato reposted a follow-up interview with Dario Amadei on Anderson Cooper, adding, Man, what are we even doing here? Dario agrees more than he disagrees that AI has a greater than 10% probability of wiping out humanity. They will get the regulation this rhetoric is calling for, not what they're prescribing. And it may be the largest cell phone in the history of tech.

21:41In other words, the argument here is that the presumption that the government is simply going to accept the industry's proposals as the right thing to do is ludicrous. And it is far more likely that they take far more dramatic action, justified in large part by the AI industry's leaders' own rhetoric about potential apocalyptic futures. Wrote investor Steven Sanofsky, The most naive position is thinking that your inputs to government lead to the outputs you expect. Government machinery is nothing at all like private enterprise. No one is in charge to create a

22:12point of view and everything splits the baby so no one likes the results by design. Austin Allred put it more simply, The problem with Dario's position is he thinks if he compliments government and regulation profusely enough, if he says he really, really wants to be regulated, that he'll be in charge of how that happens. But he won't. It doesn't work like that. He won't have any say. Putting it more bluntly, he follows up, Dario is going to hand the government a gun to show just how much he's operating in faith and they'll instantly turn around and shoot him with it. A final common negative response that I saw was about where open source fits in all of this.

22:44Hugging Face CTO Julian Chamond writes, Open source won't pace. And again, adding some heft to that, OpenAI's Rune wrote, I won't lie, I think open source will be banned before too long after some major disaster. I hope Kimmy and DeepSeek etc. keep making models but keep them monitored on an API where they should be. Now even if one chooses to charitably see that Rune was saying that's what he thinks will happen, not that's what he wants to happen, people jumped all over that with Beth Jezos again reposting it and saying, they're just saying the quiet part out loud, we cannot let this happen. And yet within all of this, I think that the place that a lot of this discourse is likely to settle

23:18is in fact the middle place that many of us have been looking for. Prime Intellect's Will Brown wrote, I do think a lot of people on the pro open source side are having a bit of a knee-jerk reaction to the pacing statements today, as we're used to viewing the closed laps as power-seeking. But I think their hands are somewhat forced here, and this is just another chapter on the fairly inevitable path towards decently fast, decently safe, decently commoditized intelligence abundance. Ask any Fortune 500 exec or swing voter, the world doesn't really want super-fast take-off super-intelligence owned by only two companies, and thus we won't get it. It'll happen at the pace that the

23:53world can accommodate it. Which means reaching some sort of confidence consensus that the models are aligned enough that we won't be dealing with scary new incidents all the time. This will trickle out broadly, in the form of best practices and distillation. The smarter a model is, the more it has a personality, and the less effective strict rules are. There will be awkward compromises and moral tensions. But we ultimately just want the models to be reasonable, and to do the sort of things reasonable humans would do if our brains were faster and less error-proned and had more working memory. I think we'll get there. The labs will build Mac and Windows. The rest of us are building Linux. Everyone's going to do great. Weird stuff will keep happening,

24:27but we'll still wake up and go to work. Until the work does itself in a manner the world finds acceptable." Rune again provides the counterpoint saying, The fastest way to lose the frontier will be when, due to the reckless commercial pace of building super-intelligent minds, Americans impose a total Butlerian jihad. Corweave includes this in their risk reporting. You will soon come to see all of this as a moderate solution. And interestingly, former White House AI-sar David Sachs, who has been one of the loudest anti-regulation voices, had a take which I think to some would probably be surprising. Sachs wrote,

25:01Dario has written that we need to pace the frontier, and Sam has agreed. People may be surprised by my response. Go ahead. You guys are the frontier. By any reasonable metric, market share, revenue growth, model capability, the two of you have a duopoly on frontier intelligence. You've also claimed the lead is widening because of recursive self-improvement. I don't see what you see in the lab. If the unreleased models are scary enough that you think you should slow it down, I support your decision to be responsible. But stop pretending you need anyone else's permission. Stop pretending

25:32antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending meter is independent when it is intertwined with Anthropics investors and staff. Stop pretending you need these same evaluators to police competitors who aren't even at the frontier. Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropik to trade

26:06some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want. Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders shut it all down. China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well. So go ahead and pace the frontier. You are the one setting it. The easiest way to not build super intelligence is for you to agree not to build it, demanding your preferred regulatory framework as the

26:36price of that will look like blackmail of the public and the political system. So just do it. If you do, you'll buy goodwill for the next conversation. If you don't, we'll know this was just another bid for regulatory capture or an election season psyop. Now, I don't think that one needs to agree with all of Sachs's priors that go into this to actually see not only the logic of this being good business, but also to see the value of this being good business. Chicago Booth professor and recently joined AGI economics director at Google DeepMind Alex Emis wrote,

27:07Contrary to what some are saying, pacing the frontier is great for open models for the medium-term economics of the entire ecosystem. Frontier closed firms operate on an around six-month window when their models can do economically valuable tasks that the open frontier can't. This six-month window isn't static. It needs to keep moving forward for frontier to stay ahead. Pacing allows firms to catch up, which may hurt closed model firms' margins, but this is myopic. This model is incomplete because it does not include safety. Racing to widen the frontier window increases risk and accidents, e.g. hugging face as an early preview. If the frontier keeps

27:41racing and there is a significant incident, this will bring a multi-pronged backlash, including regs that may cripple the entire ecosystem, including economics of the closed labs themselves. So pacing the frontier is a good idea for medium and long-run economics. I would actually go further on this, that I think you can make arguments that this would also be good for the reasonableness of market expectations. Part of why the IPO story is tied up in this is not just the cynical argument about needing to cut costs or explain slowed growth. The reason it's tied up is that once these companies go public, definitionally every quarter has to be much bigger than the last quarter for the party to continue.

28:16Look at the way that the market has handled Nvidia, where even massive double-digit growth quarters are treated with shrugs because there is simply no way to surprise them and meet expectations anymore. That will add an incredible pressure to all of this, that some broadly agreed upon pacing and safety dynamics could theoretically tamp down. I also might explore this in a further episode, but I think when people view this version of pacing as simply a net economic negative, I don't think that they're quite understanding the current negative impact on corporate adoption, that the speed of development increases. Think about it this way, if you're a corporation,

28:48who to the extent that you're used to dealing with transformation at all, deals with transformation on multi-year timelines. What's the logic in getting everyone aligned around a whole new model when three months later it's just going to change again? Now, begrudgingly, some organizations have accepted that the speed of change is just going to be like that forever from now on, and so have valiantly tried to redesign their systems to accommodate that. But there is a non-insignificant number of companies for whom the speed of development, actually in some weird reverse kind of way, justifies inaction in the short term. No, we're not going to mess around with the latest thing, because in just a couple months,

29:20the latest thing will have changed, and we'll just have to change again. Until our hands are completely forced, better to just do as we've always done. I certainly don't think this is guaranteed, but I would not be surprised to see some actual positive economic results from some amount of limited and clear pacing in giving corporate customers a little bit more breathing space to adapt. Now, so far, the political reactions have been broadly what you might expect.

Political reactions and international response

29:44President Trump downplayed it. On Sunday, he said, We're leading China in AI, we're the most sophisticated country in the world, and frankly, I want to keep it that way because whoever wins AI, wins. And we can put guardrails, we can do this and that, but I think you have a lot of negative forces that are bringing it up that shouldn't be bringing it up. And they're bringing up things that won't happen, but whoever wins with AI, wins. Bernie Sanders, of course, used the moment to promote his super intelligence ban bill, which was formally introduced on Friday. He posted, Dario Amadei, Elon Musk, and Sam Altman now agree that we must slow down the development

30:14of AI and, quote, pace the frontier. That's a start, but it's not enough. When you're racing towards a cliff, you don't just ease up on the gas pedal, you hit the brakes. And when the future of humanity is at stake, we need a pause on advanced AI development and a ban on artificial super intelligence. An AI mind smarter than any human and capable of operating independently beyond our control. At their upcoming AI summit, Trump and Xi must negotiate a treaty to pause AI and ban super intelligence before it's too late. Over the weekend, the media reported that President Obama had also been behind closed doors, urging Democrats to move AI regulation to the center of their

30:46platforms for the midterms and into the 2028 election. During a Thursday fundraiser, Obama said that he is neither a doomer nor an accelerationist, but that if he were running in 2028, he would make AI development, quote, one of my central agendas. He said, This is something that is moving very fast in private hands, and if we don't get on top of it, I think it can be dangerous. If we do get on top of it, I think it's beneficial. He added, I would have a very clear plan, and I would talk about this, and I would say, here's our plan for safety. Here's our plan for making sure our kids are not corrupted by this. I would be thinking about

31:17the economic impacts in very concrete ways, and understanding what does it mean if there's going to be job displacement? Where is that going to hit? How are we going to respond? Are there going to be limits to the amount of displacement that can take place? What is that going to mean for our social safety net? If it turns out you've got a lot more people who can't find full-time work because they've been rendered redundant, how are we going to respond? During a CNN interview on Sunday evening, House Speaker Mike Johnson said, If Congress just races in and does some sort of emergency session to try and regulate AI, we will lose the race to China, and that is a threat to every single American. So we've got to

31:49have balance. We've got to have steady hands at the wheel. He also added, however, I've talked to the president about this as well. The AI companies probably should be summoned together at the White House, and I think we need to go in a big room, close the door, and sort this out, and I'll be the one pushing for it. During a follow-up interview with CBS on Sunday night, Dario Amadei said that the toughest dilemma would be if China chose not to collaborate on a slowdown. And certainly this seems to be the state of play heading into Trump's AI summit with Xi later this month. On Monday morning in Beijing, state newspaper The Global Times argued that the purpose of Amadei's

32:22essay was to, quote, attempt to curb China's AI development through technological barriers and regulated monopolies, uphold Washington's monopolistic hegemony and cutting-edge technology, and exclude China from the global AI governance system. The newspaper continued, this silent AI cold war is hypocritical and short-sighted. The piece added that excluding China from frontier AI research would, quote, significantly increase the trial and error costs and risks of loss of control in global AI development. Later on Monday, China's foreign ministry urged all nations to, quote, promote an open, inclusive, and benevolent approach to AI,

32:53with a spokesperson for the ministry commenting, fomenting various threats, engaging in confrontation and malicious competition, will only disrupt the process of global governance of artificial intelligence, and is not in the interest of any party. Still, reporter Isabella Kaminska had an interesting take on Dario's note vis-a-vis China. She writes, The proposition he makes is more like Soviet-era detente. It really reminds me of Nixon-era outreach to Russia over nukes. And the comparison is all the stronger when you realize detente was influenced by America becoming overstretched in Vietnam and facing a growing fiscal imbalance of

33:25payments crisis. But the end result of detente wasn't that nuclear engineering stopped. Buildout and silo growth was curtailed, but engineers continued to advance capabilities. SALT also placed certain capabilities over others in a negotiated way, to either iron out asymmetries between powers for MAD purposes, or lock in negotiated strategic advantages. In the AI letter, Dario makes the SALT comparison. But his proposition is mostly to pace recursive development only after locking in a greater American advantage. To achieve that, he needs permission to basically forge a domestic cartel, and to lock down development work in

33:56ways that prevent China getting access that enables distillation. But the Chinese would never agree to any detente on technological development or recursive intelligence on their front without some compensating trade-off from the US. And for them, it would either be access to NVIDIA chips, or a promise that America will slow buildout while they catch up on innovation. The pacing doc introduces the idea of an international treaty which regulates the recursive and architectural engineering element in China. But such an agreement is meaningless if the CCP loses control. Which it could well do because the Chinese models are open-weight. So what is really on

34:26offer is a quid pro quo. We reduce buildout and the recursive element of our technology and maybe give you a few more chips, and you clamp down on open-weight models and bring more of your research in-house too. So perhaps the better way to read it is that this is an outreach to China to get the open-weights under control. Now I don't necessarily think that that's exactly what's going on here,

Shifting toward concrete governance questions

34:43but it is an interesting read. Now what I'll say here as we conclude is that you may notice that the discourse around this feels a lot more productive than the discourse around the researchers' warnings from last week. And that is of course because it is specific and tangible. Yes, there are still people who are rejecting it out of hand, often for very different reasons. But you can almost feel that a lot of serious folks are treating this almost as a shift, like we've now moved into the negotiations of what was always an inevitable next phase, where the future of the development of AI is not solely defined by the labs themselves, and involves other actors in other

35:18ways beyond just shouting from the outside of the building. OpenAI's Hoda Nadal-Barge writes, I was more worried about AI existential risk in 2022 than I am today, even though models are vastly more capable. The risks haven't disappeared. I always expected models to become incredibly powerful. But one of the biggest uncertainties back then was whether Frontier Labs would actually invest seriously in alignment as capabilities scaled. Over the past few years, I've updated meaningfully on that. I'm also surprised by how PDOOM is often discussed. It is not an exogenous

35:48constant waiting to be measured. It is endogenous. The probability of catastrophe depends on what labs, governments, researchers, and society actually do. A number stated without assumptions about those actions tells us very little. I've always believed the expected benefits of AI vastly outweigh the costs, otherwise I wouldn't be working on it. But that is conditional on us getting this right. We must move beyond apocalyptic rhetoric towards the concrete questions. How should Frontier Labs collaborate on safety? How do we align incentives? And how do we communicate honestly with the public, about what these systems can do today and where they are going? I'm glad Dario brought the

36:23conversation back towards those concrete questions. And if the first couple days of response are any indication, I completely agree. She concludes, I'm glad Dario brought the conversation back towards those concrete questions. And if the nature of the discourse over the last couple of days is any indication, I completely agree. For now, that's going to do it for today's AI Daily Brief. Appreciate you listening or watching as always, and until next time, peace! F