OpenRouter: from Seed to Stripe — with OpenRouter’s Alex Atallah & AMP’s Anjney Midha

Starting OpenRouter and PubSub principles

0:00Okay, we are here in Andrzej's house, which is where all great startups in San Francisco start. Howdy.

0:10And congrats on Cursor, Mistral. God knows what else, you've got so much stuff going on. There's a lot going on. Well, OpenRouter has probably been the most, I would say, one I'm excited about recently. Yeah, and we have Alex, first time on the pod, but you've been in the EIE a few times. I appreciate every time you've shown up for the community. Congrats. What a journey. When I was looking back at your past posts, one of the earliest principles that I saw you write as a product person is PubSub as a product principle.

0:42And I wanted you to maybe explain how you think about what should exist in the world. Yeah, the PubSub piece, which was early 2023, I didn't think about it until we talked like 10 minutes ago, is about how there is like a way of thinking about products as an intersection between subscribing to data and publishing data. And marketplaces are an easy, easy, easy, easy example of this. You have suppliers that are publishing some kind of product to a skew.

1:16And the skew is kind of like a PubSub topic that a consumer is subscribing to and just going to consume whenever they want. And humans consume in a very, like, discrete, ad hoc way. It's not very scalable. You know, all their attention is on the topic when they're buying the thing, and their attention is nowhere else when that happens. Agents and consumers of inference don't act like that. They're consuming continuously, and they're changing the skews that they consume from all the time.

1:48So open router is sort of like a blend between a, you know, a normal API experience and a marketplace where we create model slug. We have the auto router. We have all kinds of, like, product skews that you can subscribe to. And then you can, like, continuously add, like, derive value and make decisions based on those consumers. Yeah. This is something that was more consensus now, but not consensus when you guys started, which was that there is such a demand for swapping models and changing things out, and that people would not use the native SDKs.

2:25I guess, for each of you, what was your sort of realization moment that this would be it?

The Alpaca moment and early open models

2:30You've given a talk at AIE about Alpaca as, like, one of your inspiring moments. Alpaca, I can, like, rehash the Alpaca moment for a sec. Like, like, the very beginning, at the end of 2022, OpenAI was the only game in town. There was, like, OpenAI, Cohere, and then a smattering of, like, early attempts at open-weight models. Yeah. When WAMA came out in January of 2023, it was like, whoa, really exciting. This is really big. It outperforms GPT-3 on one or two benchmarks.

3:04But you can't chat with it. It wasn't actually an engaging model, but it seemed like someone just needed to fix a couple things and do some RLHF on it to get it all the way there. And Alpaca was the first model that I saw that did that. It only took $600 to do. A team at Stanford generated a bunch of synthetic data, fine-tuned WAMA, and made Alpaca, 7 billion parameter model. Or was it, maybe it was 13 billion parameters. And it was so good.

3:34Like, I was just, like, on an airplane using it. I, you know, in many cases, like, you could not discern a chat GPT versus an Alpaca result. And I figured if it was this easy to make a model, one, we have a whole new way of monetizing data for the first time. You can just, like, take really valuable data and turn it into a service in $600. And that cost will probably go down over time. Two, when you say, sorry, when you say monetizing your data as what eventually will become an MCP endpoint or as a training data for a model?

4:11Yeah, training data for a model, like an abstract way of saying, like, hey, I have this data. Compress it into a model. Like, it makes sense for me in my product. But, like, I could repackage it in the form of a model and sell it. And so it's just a whole new business model for the economy. It also, of course, provides, like, you know, a way of following what Frontier Labs are doing, but in a way that, like, a single developer or a small team of developers can roll on their own.

4:44And so whenever you have an example of that, like, a breakout app that's doing really well and then some kind of framework for imitating it in your own flavor, you have an immediate ecosystem of, like, an immediate ecosystem, like, should arise. Because there's just a huge gap between the, like, decisions that the single company is making and all of the variations in those decisions that, like, a wider ecosystem can create themselves. And so then, you know, you need a marketplace to, like, discover all of those services and all of those products.

5:20There wasn't any place on the internet that, like, was, like, a home base for LLMs in terms of seeing how much they were being used and seeing who was using them and why. The closest would be Hugging Face. Hugging Face was the closest. They just started Hub, like, a few years ago before that. Yeah, and Hugging Face also didn't have the closed source models. Yes. And they didn't, you couldn't use the models at the time.

5:43And there wasn't data about who was using them. There are, like, a bunch of differences between Open Router and Hugging Face. And those differences felt really critical to me, especially when I was just trying to learn about LLMs and, like, why people are choosing, like, these different little ones that are emerging over time. Got it. And then, Anj, no stranger to wanting more model diversity, at the time, you know, you're a couple years into your anthropic journey, which we've covered in the previous podcast as well.

6:13What was your introduction to Alex?

Stanford roots and platform security at Discord

6:15Well, the introduction was, I think, 13 years before that. Oh. But the Open Router handshake actually happened right over there, if you remember. Yeah. Which was, Alex and I met, I believe it was sophomores now, if I remember, at a Stanford Review meeting for the first time. I think so, yeah. Yeah. So, Stanford Review was the libertarian newspaper on campus at Stanford that Peter Thiel started back in the day. And for whatever reason, Alex and I both showed up to one of the meetings.

6:47And I remember the editor-in-chief was a mutual friend of ours. Lisa was really a really great editor-in-chief. Part of an editor-in-chief's job is to assign responsibilities to people and make sure the work gets done. And I may be misremembering the details, but I remember wanting to, it was kind of surprising to me that at the time there was no dedicated technology section in the newspaper.

7:12Because it's political, right? It was primarily, it originally started as like a political, yes. Yeah, states and all those things, yeah. But to take us back in time, you may remember this, but there was this technology kind of legislation that was being debated called the Net Neutrality Act. And net neutrality is like inherently this political concept, right? It's about the regulation of internet broadband access. And so there was a community of us who were kind of technologists, but also debating the politics of the technology.

7:44And I thought the review would be a great place to like write about that. And I was working on, I think a net neutrality article. And I remember proposing, well, maybe you should start a technology kind of section. And Alex was one of the only people who said, yes, that would be cool. And so I forget whether we ended up writing stuff together, but that's when we first met. That was 2011 or 12, I forget which year it was, it was one of those. Yeah. It was at Old Union, if I remember correctly, that's where we used to meet. But, you know, along the way, Alex and I have had a chance to hang out often.

8:18And probably the time when we had the most professional overlap was when I was running the platform at Discord. And it had become this explosive kind of platform for crypto. Yeah. And NFTs in the middle of the pandemic. Which also, by the way, you were in charge of safety and security as well, right? I was the head of platform, which meant all of the crypto, DAO and NFT launch security debugging fell on me. And the phishing and... The phishing, the social engineering attacks, that Katana DDoS that we were getting hit by.

8:53But so on the time I started teaching security at Stanford, CS153. And Alex was at OpenSea at the time, and I was trying to figure out how we could defend against all these attacks that we were... And at peak, I forget if you remember how much NFT volume was running through Discord. But it was like a meaningful amount of like several billion dollars in NFT volume of GMV, so to speak, were running through the platform. And it was all coming from OpenSea. It was these like buy, sell, trade servers. I mean, the D in DAO is Discord.

9:23Yes.

9:25So that's when I think we had hung out professionally. But a year after that, OpenAI gave Discord early access to GPT. Sorry, GPT-3. No, it was GPT-3.5, actually. GPT-3.5, which is the RL version of GPT-3. And that's around the time we made a Discord bot with OpenAI for internal deployment. And that's when I realized we would need... Like, since I was part of the deployment team... What was the use case? There were two that were... And there's actually a post now called Discord is your place for AI with friends that somebody sent me recently that I wrote and published in 2023.

10:01But there were two use cases. One was Clyde, which was like a first-party friend inside of Discord that could help you set up your Discord server and talk to you about onboarding and get your friends to hang out more. And then there was content moderation. And one of the realizations we had with content moderation was it would refuse to moderate. Like, we just refuse our prompts because the RL, the post-training was... We were very early in the post-training era.

10:33And it would just... Our prompts would trigger it. It's like guardrails. And we told OpenAI, hey, guys, we need access to the weights because if we're going to be doing content moderation at scale, we have 250 million monthly active users. We need more reliability that the model will do what we need it to. And they said, well, sorry, guys, that's not how this works. We're a closed-source company. And so that was my first realization that we needed open models. And the enterprises would need more control over capabilities. And then ultimately would need some kind of control plane or management system to orchestrate these open models.

11:05But there were no good open alternatives until maybe six months later when Llama came out. And six months after that, I led the series into Mistral, which was started by Guillaume and the Llama team. And around that time is when I remember hearing about Alex launching OpenRouter and going, these worlds are going to collide. And I don't know when it'll make sense to team up. But Alex was so early and could see... I think he was totally right about this ecosystem starting with Llama that then needed like an easy layer to manage for it.

11:38Especially for... I was approaching it from the enterprise perspective because I had been that... As the VP of platform at Discord, it was my job to ensure that when we deployed models to like 250 million users, they did what we wanted them to. And that was very hard. Because if you outsourced it to the labs and they controlled the guardrails and their guardrails or their safety policies forbid the model from responding to your prompts, that was quite catastrophic. Yeah, but a moderation is a thing that they want to support. And obviously, beyond that, OpenAI would work with you, presumably, to give you a moderation endpoint, which they offer for free.

12:16It was an interesting use case that they... So they did give us a moderation endpoint. However, as you guys know, every Discord server is like a mini deployment of itself. And so the use case was instead of having human moderators that have to interpret the norms of the community, you just give the... They often, like every subreddit, Discord servers, public ones, have their own rules that the user... Oh, yeah. The users create... We run the links based on Discord. Yeah. And then humans used to read those norms and then enforce it every day manually, like observing each message in these communities.

12:51And these communities have like millions of users. So we had a 5,000 plus person team globally on the Discord content moderation team. These are outsourced contractors who had a really tough job. And so the idea was instead, if you could give the norms of that server to the LLM, then the LLM would do custom moderation for that server. It's almost like a, like in-context moderation for that server. And many of those servers' norms just violated OpenAI's rules. And so that, it was like we had our own custom eval.

13:22So each server had its own custom eval. But at the time, OpenAI's evals, we were all so primitive in our thinking about how to deploy these LLMs that often the post-training prompts were super heavy-handed. It said, oh, anything about Harry Potter, anything that has trademarked content, you know, don't refuse. And if it was a fan, Harry Potter fan community, this is a real use case that had a content moderation. The LLM would just refuse. And that was just not precise enough. Another one that we heard was like if someone was trying to write like a detective story and there's one chapter with a lot of violence, like maybe someone kills someone.

14:03The LLMs would just refuse to like help with that part of the story. Yeah. And then they, like these would be like, okay, this is not like structurally inherent to LLMs. There must be like some choice out there so that I can like switch to another model when I'm getting like a refusal or a bad result from the main one that I have. And that like tension also drove me for a marketplace. Yeah. I think that is well accepted now.

VC skepticism and scaling laws

14:31What was it like back then when you were raising or, you know, starting this? Did people get it? You know, what was the, some of the struggles? Basically, I like, I like getting stories out of him about how other VCs don't get it. So like anything, anything you want to, you want to talk about, you know, now that, let's call it the, you know, that the early journey of open router is done. You can obviously talk about some of the early day stuff.

14:55Well, I was going to say that like the, the biggest objection we got is, is big model win, which is all the value. Scaling laws. Yeah. Scaling laws. And natural network effects are just going to kind of accrue to one company, which will be like, it'll be a Google style monopoly, just like how Google won the search market by a large, large margin. And you'll just be fighting for scraps at the end, basically. That was probably the biggest objection we got.

15:26But, you know, it is interesting that Google won the search engine race with such a huge margin. You know, I think like, had there been more interesting benchmarks or had like search engines been, you know, had people like seen them a little bit more like LLMs where there are services that you can build companies on top of? That might not have been the case. But LLMs don't merely have a user interface. They're also like ways of building entirely new businesses. And, you know, a Google level monopoly would be like the Dutch East India company times, you know, quadrillion in magnitude because the whole economy ends up like depending on the one monopoly as well.

16:08So, it didn't seem like, you know, like would be a really crazy outcome if that happened. And it's also less likely because the economics of like creating good competitors are much like much more decentralizable. Everything Alex said is true. And I came at it from a completely different perspective, which is. Yes, this is why we're here. So, the scaling laws were never like, in my mind, were always a feature, not a bug for why OpenRouter would be very valuable.

16:38Because, you know, I was one of the first investors in Anthropic. And it was obvious to me that other researchers in our friends group, I went to grad school for machine learning. And I just had a lot of friends in the ML community who it was very obvious to us that the bitter lesson holds. And so, I was like, oh, like, fantastic. Now, we have at least two proof points that compute scaling works. It was OpenAI and Anthropic. And by the time I think we decided to team up on OpenRouter, I had already invested in Mistral and Black Forest Labs and Luma.

17:09So, there was multiple model companies and teams that I was working with. But you did other modalities, whereas this is literally text. Across different modalities, yes. Text. Exactly. And it was so obvious to me that an ecosystem of different kinds of models were being created. And that this whole narrative of, like, only one company will dominate, like Google, was, like, maybe true. But, one, I don't believe that. But, two, there was so much extraordinary innovation happening across several different research teams. But the shared problem I was noticing across all of them was often, you know, the research teams were fantastic at figuring out how to reason about new capabilities.

17:48They think in terms of capabilities, but never, like, are not developer mindset-oriented. Like, what happens after the training is done and the checkpoint comes out? Like, you'd be shocked how, like, similar their early pre-training teams at OpenAI, sorry, Anthropic, BFL, Mistral, WERB, in their, like, default approach to taking their research out of the lab and kind of scaling their impact. Which is often, oh, the checkpoint is done, put it out as an API, done, and then there'd be crickets.

18:19It's, in the case of Claude, the first Claude checkpoint was actually done a year before they released it internally. And then ChatGPT came out and we decided, okay, yes, it's a good idea to release a Claude version externally. And they had no plan. Like, no plan for how to get developers to actually try it out. And so, if you go to the Claude1 blog post, you'll notice there, like, three kind of developer examples for users of the API. And one is a Discord bot, and the second is Vivian, my wife's startup called Junie Learning.

18:50And then there was, like, Notion, because these were all friends of, like, the Anthropic team. Because that's how, like, last minute the planning was around, hey, once the model's done training, how do you get it out to the world? There was no distribution platform that understood what developers needed. All the key management, provisioning, like, simple, like, you know, endpoint management, versioning control. Like, all these things that scientists and researchers go, I mean, that's plumbing. I don't really think about it. Implementation detail. Right. And instead, Alex came at it from that perspective. And so, you know, it was so obvious to me that, like, every single lab I was funding would spend, like, literally sometimes billions of dollars into training.

19:28And then a checkpoint would be done. And there'd be crickets, like, during early access, because they were like, oh, that's right. Like, it's hard to use a checkpoint to make anything. You actually need a whole bunch of plumbing around it to make it usable by a developer. And so, by the time, I think we think it was so obvious to me that a distribution platform like OpenRouter was critical to having the ecosystem if we wanted there to be competition to Google. Like, unless, you know, with Google, DeepMind is done training a new checkpoint. And then they push a button and it gets blasted out across all their surfaces from Google Docs to, you know.

20:01Everywhere, even if I don't know. Everywhere. You want to know about, like, on Android. Like, overnight, they can deploy a new checkpoint to, like, a billion devices, right? And that invisible infra advantage, distribution advantage, most people don't realize. But until OpenRouter showed up, you had to think about all of that yourself as a model lab. And it was very daunting. You know, at Anthropic, I think it took more than 12 months to get to our first 10 million in revenue. And in contrast with Blackforce Labs, I remember the early days, you guys had a conversation with the BFL team.

20:33And it was so simple for OpenRouter to say, oh, no problem. Like, the day you launch, we can send a million developers to you. You know, that was crazy. That was like a step function change in, like, power. Is that a real number, million? I think today it's like four million. How many developers are on OpenRouter today? Over 10 million, but, like, it's hard to, you know. Yeah, I don't know how to, we do a lot of, like, you know, account deduping work, but, you know, no one.

21:04If you could get 1,000 developers, just to put it in context, if you get 1,000 developers who actually try the model on day one after you release it, and just, like, do inference and give you feedback, that's 1,000 more developers than they knew how to get to on their own. Well, you know, BFL had a reputation, but yes. They had one with stable diffusion. And with Mistral, I don't know if you guys remember, but the first checkpoint they released was, like, torrents. It was, like, torrent weights. Yeah, they just put up a magnet link. Yeah, there was no API.

21:34Because they didn't, they weren't in for people. You know, like, it's like, okay, download these weights, and you guys go through. He has a story on his side, yeah. Yeah, I mean, in addition to the, like, building a really good developer experience around it, the marketing that we do, like, for different models is totally different and perceived totally differently from the marketing that a model lab does for itself. Yes, 1,000%. We are, like, a neutral layer looking at this market like it's a big, dark room with all the corners completely obscure to users, and users are walking into the room

22:07and, like, feeling around and trying to figure out what objects to grab off the tables and, like, build into their companies. And it's an insane way of working. Like, models are not products where you can just enumerate all their features onto a web page. They're all black boxes, including the open-weight ones. So you need to, like, shine lights on all corners of this room so that people can see what makes this model good. And you need the company shining that light to be a neutral third party,

22:40which is what we specialize in. So the, like, in addition to developer experience, there's also, like, a very important, like, marketing and product packaging component and a way of, like, routing and discovering models becomes, like, critical to your go-to-market as a provider or a model lab or a server tool and more in the future. And this value, to your earlier point, about how many VCs, like, you know, just don't... One of my biggest frustrations is that venture capitalists,

23:12many of them, like, just don't have any operating experience in the field. You know, so unlike a traditional investor who's just maybe come up through the ranks as, like, an associate working on financial modeling or maybe hasn't been a real operator in the field for, like, more than 10 years, which is a big part of the industry now, I had just arrived at A16Z, like, a year after running the platform. And so I knew what the challenges were of, like, building a real great developer experience and actually, like, being able to create a working piece of software with a model.

23:44And there were a few, I won't name names, but there were investors who were looking at Open Router and, you know, felt at the time, like, when I would compare notes with people, that it was just, I quote, unquote, just a marketplace. Yeah, just a thin layer, just a proxy, just a... Correct, a wrapper or whatever on other people's APIs. And I was like, you have no idea how strategic the value that Open Router has created by being able to orchestrate even three APIs in production. The amount of both engineering work and community design

24:17that goes into getting that actually live and running in production at the scale the Open Router team had started just doesn't happen by default. And that was one of the things that stood out to me about Alex from the earliest days. Like, he just understood, like, these, from a systems perspective, like, how do you get these flywheels going? Like, that stood out to me with OpenSea when we were working together on the NFT integration at Discord. Like, Alex had a level of systems thinking around how you get these flywheels going that most scientists and machine learning people just don't think of.

24:49Like, we often think in terms of pre-training, mid-training, post-training. It's a linear stage. It's this linear pipeline. It's no loop, yeah. Yeah, it wasn't until much later that the modern context feedback loop cycle really got standardized in the industry. But at the time, if you remember, machine learning was like, like, mostly we did a lot of ML, like, when I was in grad school on a laptop. So you just, like, downloaded a data set, ran some ablations, and you looked at the loss curves, and you're like, great, I made AI. And the idea that you have to, like, deploy those capabilities, collect feedback trajectories, then, like, put those into a continuous loop,

25:21like, came much, much, much later. And it was very counterintuitive to the traditional AI mindset. I do remember doing the investment phase for OpenRouter. I just didn't try and re-educate a bunch of other VCs on why it was not just a marketplace. I was like, you know what? I'm just going to invest. And I'm going to, like, take the opportunity to partner with Alex. And if not, no other VCs get it, that's totally fine. Because at the time, it was not obvious, I think, to several other investors that, like, OpenRouter

25:51was not more than just a wrapper around APIs. And that infuriated me. And I was like, you know what? I don't have time to debate you. I'm just, we're going to invest. And then I think, like, a month later, Matt Murphy marked it up by 10x. Like, I think, I forget what the exact post-money was and so on. But, you know, to his credit, Menlo Ventures realized, okay, there's actually much more strategic value here as well. Maybe you didn't hear all these conversations behind the scenes. But that frustrated me a lot. You know, there was a lot of this, like, opining about wrappers. And if you're like, oh, an app is just a wrapper

26:22on a model, then, like, and, you know, OpenRouter is, like, this wrapper on top of other APIs. And this is the most stupid reductive framework. And so it's clearly somebody who has no experience deploying products in the field. It's the thing you dismiss other things with. Like, everyone's a wrapper on everything, right? Like, there's some point, some wrappers have value. I mean, investors are wrappers on LBs, right? Like, metric wrappers. So, I mean, yeah, it's all wrappers all down to bare metal, I guess. And energy. When I started the whole AI engineer, I guess, the coining in 2023,

26:52like, that was the number one pushback, is that this is no value. You should actually just train models. And, yeah, I mean, obviously this is, like, you guys are one of the testaments to the fact that you can actually build very valuable wrappers, but also very valuable model companies. It's so hard to be, like, the day a model launches, the fact that you have an open router endpoint for that model frequently at the top of Hacker News on day one, people don't realize the amount of work that goes into accomplishing that.

27:23And open router used, like, that would happen over and over again. And I remember going, people have no idea how hard that is. You know, that's not... Yeah, we've covered some of the inference engineering that goes behind some of the... Yes. With Base 10 and all those. Well, today you have, you know, all those, like, cool codename things that people guess what OxyAlpha is and all those things. But I guess one of the things that you're teasing is how do you get that initial flywheel going, right? Because today you have your scale and your reputation and all these things. So obviously you drive an immense distribution.

27:53But when you're early on,

Bootstrapping communities and user experience

27:54when it's mostly... The bootstrap, yeah. What is the bootstrap like? I mean, to bring it back to early Discord days, I think we, like, initially connected with... This is an OpenSea story, technically. But we initially connected when you were at Discord and we talked about, like, the Axie Infinity server. Oh, yes, yes. This server was, like, the biggest server at the time at Discord. That's right. And you were kind of, like, constantly bumping up the limit. The limits on the server. Oh, my God. For those of you,

28:25like, 10% of the Philippines was on that server. That's a... It was, like, a meaningful contributor to the GDP of the country. It was an NFT, like, crypto game. It was, like, a Pokemon breeding thing. Yeah, similar. Yeah. There was battling, there was breeding, and then there was, like, a marketplace for trading. Play to earn as well. Yeah, play to earn. And, like, the graphics were really cute. And fun. And you kind of, like, you, you know, you get kind of emotional about your Axie that you make. So, to, like,

28:55start a community like that, which we had to do many times at OpenSea with basically every early project for us to create a marketplace for it, we need to make sure that, like, the community actually wants it. Right. And it's kind of like building something that people want and going and telling them about it. Like, you can do that on a one-on-one basis. But there's way higher leverage to do that in a community where everyone can talk to you at the same time. So, we spent a lot of time, like, building things

29:25that the community really wanted. We did the same thing for OpenRouter. And, you know, like, the Axie community was one of, like, a zillion communities we did that with. And Ange, like, saw us doing it. And, because you could just see people sharing OpenSea links constantly in that Discord. Like, users sharing links is a really clear indicator that, like, something important is going on. So, we spent, you know, a lot of time, like, first figuring out what the gap is in the technology

29:56that people care about. Like, what was the actual problem that needs to be solved? You know, in early LLM days, it was, you know, OpenAI refusing to finish the prompt or, like, to, like, complete the task. It was also, you know, inability to customize models. And so, there are communities that, like, are just completely blocked on that issue. And those are the communities that are most useful to, sort of, learn about and dive into

30:27and explore. Something that really struck me at that time, as I was just hearing your talk, I remember noting how, you may not remember this, but we were, we had these, like, working Zoom calls that we were doing a sprint around for, like, this OpenC integration with Discord. And, you know, we'd get, it was myself, my engineering team, I think you were there. And I remember, you know, Alex in the middle of one of those calls, just, like, there was, like, silence.

30:57You know, we were all like, oh, yeah, this totally makes sense, let's do this. And then there's some, everybody aligned. And Alex was like, no, this makes no sense to me. And everyone's like, I remember going, what? Like, it works. Like, you click on a link and then it bounces you out to, like, OpenC. And he was like, it's not a good user experience. Yeah, we should not do this. And I remember going, you know, he was the only person out of all of us to actually raise his hand and go, yes, it made sense from a technical

31:27implementation perspective. Like, we were bouncing the user out into OpenC. And so it kind of checked the box of the product manager's requirements on both sides. But Alex went one step further and was like, you know what would be better, guys, if we just embedded the experience right here inside of Discord. So the link opened up as an embedded iframe and you can just check out right there. And not one person on the call, there were like seven of us who had met like, you know, week after week. And it's the guy who doesn't work for Discord. And it's the guy who doesn't work for Discord. Like, technically you benefit if they bounce.

31:57Exactly. And that was like adversarial. To keep the user inside of Discord would be adversarial to OpenC. And yet Alex put that user experience first. And I was like, that's special. Wow. Because it's very hard to have somebody who's technical like Alex and understands the developer flow, but also understands the best user experience and wants to prioritize that. And that's two sides of the flywheel that you can get spinning. Like, it's often hard to stop. And you just reminded me, like, that one was one of those moments where I go, I realized I got to be better at user experience because I should have been the one who came up with that. And I didn't.

32:28And I learned from you and I think that went into one of our case studies for the PM training program at this point. I don't know if it's there. You need an Alex, this is the conclusion. Yeah, you need an Alex. And this is why, nobody should be surprised why Stripe decided they had to buy OpenRouter because it's a really rare combination of people who understand the machine learning community, the developer experience, and the end user experience. And putting all that together has resulted in this extraordinary scale

32:59that very few other marketplaces have been able to achieve over the last five years. Yeah, well, we should talk about the other reasons for acquisitions, which you've written about. I want to sort of proceed somewhat chronologically as well. So, there is a point that, you know, one of the questions that Dave from HF0 sent in was when did you know it really started to work and you brought up Mixed Draw? I don't know if you want to bring up that stuff. Oh, yeah. Which obviously you overlap with, so.

Window AI and the crypto dress rehearsal

33:26Yeah,

Window AI and the crypto dress rehearsal

33:27the MOE was, I don't know when, I mean, there's no like one moment where I was like, oh, this is, you know, officially starting to work. It was like moments of kind of increasing connection. Like really super early on. Oh, yeah, but well, yeah, so before OpenRouter, I wanted to like explore a bring your own model experiment. And, which anyone familiar with crypto is like, yeah, Phantom and all these things. Yeah, yeah, so it felt like doing a MetaMask analogy for AI would be kind of a fun way

33:57of exploring that. And, at the time, there were no AI apps. There were probably as many AI apps that were like hitting AI, like hitting an LLM via an API call as there were like games just doing it in JavaScript. You know, basically like there was a moment in time where it could have been the case that web apps call LLMs through the browser, like through some kind of desktop managed app

34:28that is controlled by the user. And of course, there are like, I think many reasons that that did not happen. But back when the days were that primordial, I built a Chrome extension called Window AI and With Plasma. With Plasma. I had come across early on and I was like, who's going to actually use this? You did. Plasma had a couple, like, I think Phantom was using it. There were some other like real companies using it. There was like a shim, basically. React for Chrome extension.

34:59It compiles to all these. Yeah, kind of like Next.js for Chrome. Cool. Next.js, Next.js. Okay. And yeah, built Window AI on top of it. The creator of Plasma like started contributing code to Window AI in GitHub and that turned out to be Louis Vichy who is the co-founder of Open Router. You have told me this is how you met Louis. Yeah. Okay. So, that allowed users to kind of like configure which model they wanted to use for a web page in their browser and then like the app

35:29would just call out to that model when it needed to do things. You know, not the right form factor for LLMs. But, you know, it's like fun experiment. You learn a lot and like I, you know, open sourced it and, you know, the main learning is like, okay, this has to be an API and it has to look a little bit like there has to be more of a developer experience here and more of a discovery experience as well. Like, I don't know where to use these models and a little Chrome extension is not going to help me discover it's not enough real estate.

35:59I need more space. I need visuals. I need graphs. I need, you know, examples. I need images. I need to like, I need to be able to like explore both as a human and as an agent. So, that's kind of how OpenRider came to be. You know, a meta point that I think is underappreciated but Alex is reminding me is that we were quite lucky that we were so, we were like adjacent to the crypto community in those days because in hindsight, crypto ended up being kind of like a dress rehearsal for generative models,

36:31right? If you think about the Axie experience, you know, Alex is totally right. There were not that many AI apps at the time and while I was dealing, you know, my job was to be the head of platform at Discord which meant to be a general purpose place for communities and friends to create, for developers to create apps and bots and, you know, other services that could be deployed across Discord. and while 80% of the attention at the time was being spent on crypto because that's where all the NFT volume was, there was like 20% of my time I was spending

37:01with a friend who would get hotpot with me and ask me for, we'd play Magic the Gathering on weekends and he was working on a little Discord bot that could take a text input and turn it into an image and it was called Midjourney. Is that David? It was David Holtz who was a good friend and David and I have both been sort of failed AR VR founders, you know, in the last, before that and I remember this, you know, Midjourney was one of the fastest growing communities we had after Axie Infinity started to peter off and many of the,

37:32the, like, the abstractions and the infrastructure decisions we made to scale Axie happened just in time because, you know, Axie did this and then fell off a cliff and then as Midjourney was taking out we like explicitly decided to help David make the server, the Midjourney server the primary place for interaction with the model because it was very hard for people to understand how to use the model if they couldn't see other people using it and copy them and so the single player Midjourney web app on its own like Midjourney.com had like terrible retention

38:03because people would show up they'd see this empty field it's kind of like Dolly 2 and they would type in like cat or dog and it was like paralyzing for them to have this blank canvas that they had to fill because they'd never used an AI model before but instead in a Discord server you could see other people using it and riff off of their prompts and the engagement was off the charts and so scaling you know Midjourney from zero to like 10 million monthly actives was a much smoother approach post Axie Infinity and so Don't forget the best of four pictures and you choose one which is the feedback loop The RLHF feedback loop

38:35which by the way separately Tom Brown David and I used to play Magic the Gathering on weekends and so it was one group of friends who would hang out and these concepts were all being discussed all the time but you know there was I think there were few of us who bridged both the crypto worlds and the AI worlds and compared to crypto where the question was always what's the use case you know for this technology there was never any need to ask that for AI because the use case was so visceral it was like I can create now anything I can imagine I can write novels

39:06I can code and the infrastructure that those of us who believed in the distributed systems like value of crypto like the censorship resistance part found this use case that was explosive and I think between mid-journey you know Claude was a Discord bot pre-launch you know that we were using internally as an LLM Eleven Labs had a TTS model that we had on Discord as well like Discord became this petri dish for like early apps to innovate and I don't think it's a coincidence that they found a home there before Open Router gave the world like a public

39:38home store or like a you know storefront Discord was this like almost kind of petri dish storefront that was kind of like piggybacked on the infra we built for crypto communities and then I think Alex was one of the first people to realize wait a minute like these apps need their own home on the internet and then Open Router to me was a continuation of that of that community's needs and of course there was the crazy distribution that you enabled for a lot of these developers so then my question is

40:08how come you were my perception is Open Router is not that Discord centric right you have a Discord yeah and you use it to engage your community but it's not like Mid Journey where like no that is like the primary way people experience Open Router yeah Mid Journey like it really helps us see visually really quickly how people are using the model and how to prompt it and I think that is partly why the server was so critical it's like it is the user experience it actually adds a ton yes and you can go

40:38the whole mile with just like prompting via Mid Journey like via the Mid Journey Discord server getting your images and then sharing them and having fun for Open Router for LLMs like you need a lot of user experience around LLMs to make them like really usable and yeah but like seeing the examples of other people is also not as useful because it's a lot of stuff to read it takes a long long time you need like code base integration not possible to do in a Discord server you need

41:08or technically it's possible I shouldn't say that it's just not a great great developer experience you need like you need governance for at the point where you got code base integration now you need governance for managing the LLMs that have access to it the data policies which teams all that stuff needs a lot more than a Discord server can provide so it's just like it's not the right well in addition you're not wrong but also there's the very important distinction that you know Midjourney was an end user application

41:39and you know that's why Discord which has 250 million monthly end consumers you know it made sense for Discord to kind of to be a host for that application experience what I knew was going to happen soon after Midjourney found explosive product market fit because we I think when Midjourney launched from launch to 100 million revenue run rate it was less than eight months and shortly thereafter Stable Diffusion launched and you know

42:09all of us used to hang out in the Discord server it was the it was the it was the it was the it was the it was the it was the it was the the image community that spawned Stable Diffusion and so when Stable Diffusion came out I realized oh now other people can build their own Midjourney because until then Midjourney did not have an API so they were a full stack company right they were training their own models and they were deploying them as an application but if you want to build your own Midjourney there was no API of that quality and I think Dali too was still quite primitive

42:40like Midjourney actually had great quality and then when Stable Diffusion came out suddenly there was this new person who could there was new this new capability in the world which is a developer could create their own Midjourney and that I think created the need for something like open router because then you need an API if you were if you had the kind of creativity of David Halls and you had Stable Diffusion as the model and you wanted to put these things together how could you do that without having to figure out how to host the weights and what open router the shape of open router enabled is is that right

43:11when you have open models alternatives to closed sort of applications open router's value in the world becomes extraordinary because now any developer can just show up and use the API did you just say the shape of open router oh no I'm misaligned now I've been over trained I've been using cloud way too much haven't I cloudish is what people are saying cloudish oh god I got untrained myself okay and I just

The Mistral price war and model speed

43:39want to cap off the Mistral side my TLDR is there was a Mistral price war is what they called it right like roundabout in Europe since 2023 or 4 December they launched Mistral 8x7b and the price went down like 80% to me that's very positive because it's like the first like real competition to to host Mistral is there more yeah that was I'm like trying to remember it all the things that happened like we saw that model come out and immediately

44:10saw people say that it was the best model in the world yes like this was to my knowledge the first time an open weights model was called that in real seriousness it's hype right is it you know it was it was hype it was hype it was also like hype from AI influencers at the time and there were many examples where it was like outperforming GPT-4 so people really wanted to try it

44:40out and see is this going to be true for me too and if so at what price and the like inference landscape was really messy yes we cleaned it up it allowed like providers to compete on price so we could give users the best price in one spot and so it was I think the first clear example of like a provider marketplace working in a way that adds value to developers Sean you may not remember this but I think we met

45:10for the first time a few days after Mixtral came out at Neurips at a luncheon yeah that's where I also met BFL as well and Guillaume was there you were there too and we had just announced the Mixtral investment and Guillaume was over there and I remember turning to Guillaume and asking him like how are you feeling after the launch of Mixtral and 7B and you know him in his typical French fashion was like I mean it's an okay model it's not that good and I was like it was so

45:41you know in contrast but I I remember him also saying that part of the reason he felt a lot of people thought that it was better than GPT-4 was because of the speed you know it was an MOE model that they had like absolutely kind of figured out how to make super efficient it was on the periodo frontier and this is an important thing about LLMs right sometimes when they're faster you think they're smarter even though like if you did N of you know these common like evals that are seven you do seven tries and I

46:12don't actually remember I think we should go back and figure out what the data says but I wouldn't be surprised if it turns out on an N of seven attempts GPT-4 was smarter on evals but the perception of on like or correctness would be smarter or more accurate but you know people like from a human preference perspective felt that it was faster because it was smarter because it was so fast and actually most queries do not take that level don't take that that's true this is the start of humans as router which then eventually

46:43becomes open router as router of like the auto mode because humans are the routing mechanism like I will ask the fast model first and then if not good enough I'm going to upgrade manually but then he's going to auto it I hadn't thought of it that way but that makes which then there's a lot more techniques like fusion is the thing that we should talk about before I move on to those things I just want to close off the sort of early years one thing that I observe which you are also an investor in arena right and we talked about mid-journey having that feedback loop

47:13of ABCD and choosing that being very important and you understand the flywheel so how come you didn't build arena and how come arena didn't build open router

OpenRouter versus Arena

47:22well arena started before open router right they had they had the school project and then they became a company and I know you had some arena experiences like the heads up comparison type things but you never really went as hard as arena did in doing heads up experiences and elements actually did have a router project based on element arena elos which they never commercialized it's hard to do a company that does both because one company is taking data

47:52and selling it and the other company really can't by default so you know I think there is like a branding reason that there are two companies here like when you set up open router there's no training there's no prompts aside from what your provider policies set like open router can't see your prompts or completions if you want to see that as an org you have to opt into it and enable it and so we're like pretty conservative

48:23and careful about data policy and security and privacy and El Marino is like their business model is like oriented around the labs and you don't give it for free they give it for free yeah I mean we do give some we like have free endpoints too but like those free endpoints I think we're not collecting any prompts we're not like monetizing the data unless you you know opt into it for some reason this this compares I mean you're not the first person that asked me this and Alex knows this but I you know I was the

48:54first CEO of Arena for the first five months when we were helping Anastasius and Waylon kind of spin out of Berkeley and I did invest in that before Open Router but it was very strange to me the comparisons that outside folks would make between the two projects because the missions were completely different the founding entity for Arena we called it the AI Reliability Institute because it was actually there as an eval service like the data so to speak that

49:24they were originally kind of offering the labs was how do you make the evaluation of models more reliable than kind of like the state of the art at the time which is like really just finger in the wind that's kind of what Anastasius and Waylon's PhD work was as scientists at Berkeley was on statistical methodologies for sort of correcting eval estimates based on intrinsic biases and how you collected the data style control

49:55and stuff like that which is very much like if you're a scientist and you're trying to kind of the highest expectation customer for Arena was always like a post training and like a research at a lab whereas the highest expectation customer from my perspective that Alex really understood and the mission was to serve was like a developer who then takes the result of the research and then produces an application that's deployed

50:26to the world is actually a completely different problem and person that these two teams were focused on and so from the outside in actually I don't know if you remember this but I have a distinct memory of a few weeks before we did the term sheet together for Open Router I had given you a call because we were trying to get a pooled data set together from Open Router and from Arena to create like an open source repository of prompts I mean these projects were so kind of different in their goals that it was totally normal to me

50:57be like oh yeah let's call Alex and see if you'd want to team up on pooling data because they're so different we actually don't have that kind of data at all we didn't have API prompts we didn't have like what developers want to do with the models which is very different from what researchers inside a model lab want to do before releasing the model does that make sense and so to this day I think you see that this difference even though at a 30,000 foot level you could I guess you could kind of conclude that Arena and Open Router are adjacent

51:28but the roadmaps the missions and so on at the time at least were like in very different sort of directions that ideal customer I get I totally get that as a founder I want to own everything right so like this is clearly the adjacency and I'm like I'm gonna explore that own everything meaning like you don't know what to do yet so you want to like make sure you catch PM you want to own the entire infrastructure space and so you kind of expand to whatever demand you can capture yeah I

51:59think that's that's hard you know in reality because serving multiple customers is clearly you know this is the focus right yeah I still think even in the age of AI like focus is underrated and critical not just because you end up with a better product by focusing your humans on it but also because the world knows what your focus is the world can map like oh I have this issue which brand out there is going to help me with that issue this is the brand that's known for that

52:31focus so like if I want real attention on this issue like this really matters to me I should go with the brand that cares the most about it to underscore Alex's point about how important focus is in the early days of Anthropic it was not easy to like people think that the early days of Anthropic were like super easy because they were on the GPT-3 guys who left but it was actually very competitive the company was starting 10 billion dollars behind open AI right and so to get to the frontier like the big question was what do we want to be known for what's

53:02the mission and the mission was the AGI pair programming and so to the exclusion of all kinds of other things that were really shiny at the time like image models and video models that were getting lots of you know momentum the Anthropic team was like we just got to focus on coding like that is the core capability that we're focused on today you can see the results right it's a trillion dollar company within five years and that focus I think like the focus on who your highest expectation customer is and how you exceed their expectations because exceeding anyone's expectations is hard and doing

53:32it for multiple customers is so even more difficult is part of the reason why OpenRider succeeded and Anthropic as well was the focus on coding that early though or did it come later literally from day one it was AI pair programming is responsibly commercializing an AI pair programmer was the seed memo that was when I invested right we actually kind of like refined that memo a lot well you got to ask Dario and Tom for permission on that but it's an extraordinary piece of writing that they had put together and AI commercializing and responsibly

54:03commercializing an AI pair programmer was the mission from day one and I would say there were maybe like a couple moments in the company's history where they did experiments to kind of see if little detours made sense like a general chat bot like Cloud AI when ChatGPT was really taking off but at the end of the day especially once they got their significant pre-training computer online I think all the main evals of the company for example have always been coding evals long horizon agentic programming from day one that was always the plan

54:34Cloud Instant came out and Cloud 2 came out I remember the marketing mostly being focused on pros like this model right better long context long context this directly affected me because I know something on it what did you make? small developer which was my devin before devin oh yeah yes that's small yes and you know so I think there's all that really like good like focus is another thing that is a question that people do want to ask you know

55:04you could have built any other things like and obviously OpenRod was working working working were there other ideas that you wanted to pursue that you turned down

Roads not taken and fine-tuning experiments

55:13you know just paths roads not taken we made a couple prototypes for things that we didn't launch one was a fine tuning model as a service lots of that open pipe and all those things but it was it kind of was in a very consumery form factor where you would give us a YouTube video or two or three we would then extract all the transcripts from it and try to fine tune a model to talk like the person in the YouTube video or the people in the videos that you sent so like a really really

55:43easy way of creating a fine tuned model based on like some kind of videos that you like that would be so useful we made it too it was like it was nobody used it it was we didn't actually like test it with that many people because the model marketplace was our main focus and it was like growing and we were building more conviction in it over time just as a creator I've been pitched many like I have

56:14500 hours of recorded voice of myself make a thing of you charge access to it works for OnlyFans doesn't work for us as regular people I think mostly it's just a glorified rag bot whether it's in the weights or it's outside the weights doesn't really matter you're just doing rag on the videos and people ultimately always just want to find the source video that directly answers it my use case was mostly to practice with myself because I often like to see what like the way I practice for a job interview or I'm hiring a

56:45candidate or public speaking or whatever I wish there was like a good mini-me that I could like critique because it's kind of hard to pull yourself out I would never offer it to other people in service pick your top five mentors that didn't talk to them instead of talk to yourself that would be cool too yeah that was the use case we were aiming at I see it's like you want to create an experience like AI Steve Jobs and AI Steve Jobs was the initial use case even though it's not allowed that's a common prototype talking about

57:15adjacencies fine tuning as a service as part of the router service is something that I would typically think about as well right like why don't you do that because if people are running already their inference through you store everything log everything fine tune to a smaller model that is cheaper faster all these things that's within your control right you didn't do that but like other people would have pitched that in the general state of an infra startup yeah I think you were just maybe a little bit early because today that's an extraordinarily fast growing segment like you know from Astral where they do a lot of enterprise deployments it's often fine tuning

57:45custom models for ASML or whatever but not as a router they're just like I come to you because I like your Mistral models I want custom Mistral model right it is not I want to run all my OpenAI prompts get to store all my results and then just move off of OpenAI right they're not doing that as like a way to export off of dependency on a frontier lab I have not seen that yet yeah which was your kind of your I mean we decided really we like leaned into our focus and figured

58:16that like there are like we just saw the ecosystem develop over time all these inference providers that do want to help companies do that like it makes sense for us to partner with them and to like give users lots of choice and to like you know figure out what makes them what gives them competitive advantages it's a whole new business basically and there's there's value in being a neutral marketplace that just kind of like works with those companies could you share a little bit

58:46to Sean's point like how you prioritized what are some ways you prioritize features because you've always done it so elegantly I never you know it just happens and you make all the right decisions that always have product market fit from the outside looking in but consistently you seem to have prioritized you know a lot of hit features that worked and maybe I have a sample set bias or whatever but Sean can you list what you think hit features worked like the leaderboards like leaderboard okay yeah you know like from day one charting BYOK but like he had

59:16like plugins you know he had like I think there was a whole thing I want to get into about like completions versus check completions versus completions and then also let's call it the rise of the reasoning models and how you deal with those multi-modality all those things BYOK that was a huge one there was one like I think it was in early 2024 very early 2024 we thought it might be interesting to fuse the results of multiple

59:46models together and we launched a prototype called MOM mixture of models that let you like pick a couple models we'd pick them for you and then it would fuse the results together at the end and it would show you all the intermediate results in this like big Kanban board looking product what does the fusion at the end another model another model the smartest of the three of the set so this is like a council idea it was a model it was like a very early LLM council this is a multi-agent

1:00:17swarm as they would call it at one of the frontier labs in the early days you know yeah like some of those ideas are like going the right direction but the devil's in the details there's a lot of like product refinement needed to make them really work they take your focus away from you know whatever else you have going on and there's a lot of like community building and learning that you need to do and the technology might be too early so there are like all kinds of reasons they might go wrong and in our case the technology was a little too early in other

1:00:57because the best model was so far ahead of options two and three at the time you know over time the top three or four LLMs have gotten closer together still neurodivergent but like all capable of inserting like pretty interesting ideas like RL has basically like expanded the surface area of creativity for machine learning researchers within each lab and so they can diversify the reasoning power of different models more effectively at least that's

1:01:28my theory for why Fusion works better than it used to early 2024 so the technology was a little bit too primitive the form factor was not right and so we would have had to go through a couple more iterations and so we decided to just delete all the code and then years later middle of 2026 or early 2026 we're like let's bring it back like

1:01:56promising for Fusion the models now have like two three four top frontier models that are all really good and like I'm frequently trying to like consult multiple models to get the best results and then I ran a little personal experiment where I was like I'm going to do an architecture plan for a code change I'm going to give it to all the models I'm going to fuse the result and I'm going to ask all the models if the fused result is better than the individual result

1:02:27each model came up with and they all said yes that the fused result was better and this happened a couple times and I was like okay spot check pretty good we should benchmark this and that's how we built Fusion and it came on your Fable so you were like this is Fable level yeah let's start leading

Milestones and coding agent growth

1:02:45up to this year should we haven't gone to this year can you mark out the main milestones in the journey I think it seems like your promise was routing you decided the business model very early you take a cut and what are the major milestones that inflect the growth right like you're growing like 9% week on week now is the official number in terms of token volume I think that sounds about right yeah so just like can you mark out the sort of brief

1:03:16history of open router up to the acquisition let's call it we're just talking about you know people are you have sort of your birth moment with the Michelle stuff where people are really competing you have your state of AI thing where it's very cute you have 100 trillion tokens ha ha ha because now you're doing 10 a week you know we're doing 10 a day 10 a day now yeah more so yeah you do this in 10 days like what are the major points

1:03:46there you know I just want to like there's a smooth curve but like you feel the infections a lot of this is kind of oriented around model launches we had you know a huge focus on pros all the way up through May of 2024 because coding was just not there and no apps were able to build much on top of it so you know a diversity in models but not a wide diversity

1:04:17in use cases Dream Tavern was one of our top apps at the time the creator of Dream Tavern now runs product at Cognition Devon the then we in the middle of 2024 we saw Claude Sonnet 3.5 that came out incredible leap forward encoding and we saw the dynamics of apps building on top of us change we saw a huge surge in volume in

1:04:47users using open router and this is when people started to look at the money they were spending and get a little bit like whoa what's going on I might need to think about more cost efficient but equivalent models and shortly after that I think it was after Sonic 3.5 mixed role 8x7b came out and everyone was like what this is the model like the open weights community delivered and so it was really good timing from the strong basically all of

1:05:18Anja's portcullers are just helping you it takes an ecosystem to grow an open router you know yeah that was the yeah it was like this early ecosystem it was like a swing action where like model labs would come up with some sort of frontier innovation like usage would surge then users you know look at their invoices 30 days later and like whoa what's going on here and then open weight models would deliver like cost effective options

1:05:49to three months later we saw that happen several times one thing also did with the coding agents was that you broke out which are the top coding agents and they love that leaderboard the client versus the root code versus the what have you yeah yeah like client was like the top of our leaderboard at the time we we then at the end of and I'll skip forward a little bit the end of 2025 there were quite a few coding apps on the leaderboard but they were all

1:06:19IDs or terminal based agents and at the end of 2025 we saw OpenClaw appear and OpenClaw was like particularly interesting because one it was like a new form factor that like brought in a new type of user not just a developer but like

1:06:41an internet creator came to AI for the first time and it also had an interesting architecture where it was like calling your chosen model for these heartbeats to see if it was still alive in addition to actually using the model for real tasks and the heartbeats are like they're kind of you don't want to pay a lot of money to do a heartbeat so the auto router that we provided was really really useful to this wide

1:07:11range of users all of a sudden and so we just saw it rocket exponentially and then we saw open cloud blow up and a couple other apps lean into that new paradigm and do something similar Hermes came out and really leaned into things like the auto router and built a really good community and leaned into basically skill management and making it really easy and effective for people to set

1:07:42their memory in the agent and build really good skills which another thing you never did memory skills sandboxes all these adjacent things you could have done could have but like it's hard to bet they're also very there are things that really matter for the developer use cases that were coming out at the time developers wanted to architect those things those are critical to building a good user experience it's been hard for companies to find abstractions

1:08:12that work for all developers on the memory layer there are some like Mastra has done a pretty good job for example but developers have lots of varied preferences for them and then we saw the way our leader board has changed over time is kind of like a movie of how the AI space has changed over time if you just sort of like go to the way back machine and look at the rankings leaderboard and the

1:08:42apps leaderboard over time it sort of shows you what's happened in AI over the last couple of years to me the coming of each moment was Andre Karpathy was like I no longer read Local Lama because I just go to open routers leaderboard which I remember I think he probably said sorry guys I'm going to send a bunch of traffic to you so I was going to bring it into the Stripe thing how does that kind of conversation start we had this long standing relationship with Stripe though from

1:09:13you know like many different projects that we had worked on with them we invest you know a lot of effort in countering abuse token

Token fraud and trust challenges

1:09:24fraud and token fraud can you give some numbers just to so people understand I think I like I posted about this we we block 10x as much dollar volume last month as the month before and the types of token fraud are diversifying quite a bit you know there are like fraudsters going after typical stolen credit cards but there are also you know people trying to resell traffic against the terms of service there's like

1:09:54hacked accounts there's people who just lose act you know like their whole company is compromised and they don't even realize it and we help them regain control and detect it there's there are accounts that are like reselling inference on the side there's there are accounts that are dealing with you know like an accidental runaway agent and they don't realize it not not a hack but it's something that blows up and and the company doesn't want it and so

1:10:24our trust and safety team like works a lot on all of these like categories of problems and and helps block it and detect it and so we've built these you know we have models around them we have we worked closely with Stripe for a while on this and I think it's going to become a huge problem in the ecosystem like we're already seeing a lot of companies start to see these fraudsters like spread and look for other ways other you know

1:10:55other than open router to other fraud vectors and if you're making a gateway or selling like generalized inference you are a target for fraud if you're selling very discreet like intelligence products intelligence products that are like doing something pretty specific but not like you know just reselling inference with some added capability then you're way less likely to get these fraudsters so it I think we'll see companies also move away from just reselling inference with some

1:11:26sort of like added capability and move towards sort of like discrete tasks and charging for those tasks and charging for those enhancements and letting people bring their own inference like in a third-party way whoa okay and yeah obviously you would power that but people will pay for outcomes or per task I think people will pay you know I think like the data dog pricing page is a good look

1:11:56at like the future to come it's like companies like infrastructure companies will like charge for different types of events that they're providing and there'll be lots of like continuous pricing models that look like that and of course there'll be like if you go down towards consumer apps you know simpler pricing more subscriptions you know fewer events to worry about and ones that like are not focused on just adding a markup

1:12:27on top of inference not just because fraud is hard but but also because the pressure from the labs and from like good inference providers to like do a commit and then bring your inference elsewhere is going to be very high any comments two one you know I I think Alex has done a very eloquent job of describing something you know counterintuitively I knew would be a thing at scale like four years ago because of discord and the particular experience

1:12:57that taught me this was you know as we started scaling my journey you know one of the primary ways that we used to give away or like get people to try midjourney early on to get to their first 10 generations because you know 10 generations of 10 images generated was roughly the magic moment activation point we found like once you done 10 you were like this is extraordinary but for that we had a free trial with midjourney and one day I woke up because they had a platform and had to monitor all these dashboards I had

1:13:28three missed calls from David and it turns out there had been this flood of new users overnight and we were like this is great and he was like no actually we shut down the free trial and I was like why is that and he said look at the geolocation IP addresses and somebody in China had started to resell mid-journey free subscriptions with the free trial as a way to like basically you know it was fraud abuse right and even for a specialized model like mid-journey yeah and that was actually an application so this

1:13:59idea that I think the big picture realization I had back then was hey there's a new type of unit of value that's being streamed across the internet called a token and over the next 10 years the entire internet value chain was going to have to deal with the fact that the more valuable tokens got the more bad actors were going to try to get their hands on those tokens and anytime you scale something and the payload gets more and more valuable

1:14:30more bad things people try to get access to that value and so it was very obvious to me back then and so to this day I don't think Majority has ever turned on the free trial since then because it was really not an easy problem to solve in terms of trust and safety that's why I started teaching the class security at scale at Stanford that and the anthropic learnings to me it was clear that the need for security at scale was going to be enormous a few years from then because

1:15:00if you just do the math think about if online payments started roughly in the 80s and 90s and grew to over a trillion dollars over the next 10 years and we needed to build entirely new payment solutions to deal with online fraud where we are today is roughly there on tokens but over the next even five years we're expecting the token economy to get to roughly five trillion dollars and over the next 10 years I'd be

1:15:31trillion dollars of token flow and so if we were starting to see such aggressive abuse and fraud at subscale mid-journey at this point was less than 300 million revenue run rate a year I just realized we were going to need entirely new systems to deal with the fraud that was going to happen for trying to get into the token flow and so I forget the board meeting it was when you brought up Stripe wanted to partner up and it made so much sense to me because Stripe

1:16:02Radar when I was a Kleiner 10 years ago we invested in Stripe and the whole pitch that Patrick and John communicate so eloquently was like unlike traditional payment tools like Braintree that do a seven day verification KYC and AML to get the fraud out of the way we actually just bite the fraud cost up front as

1:16:27is over time we collect all this data on the developers and they did five years later they launched Stripe Radar and Stripe really today is a security company people think it's a payments company there's lots of other payments providers today that give you cheaper payments transmission but the reason Stripe keeps being the dominant one here in Europe is because they have extraordinary fraud detection that they've built over the years this is the same story with Elon and Max Levchin and you know

1:16:58I think the story shows up over and over again where every time you have value streamed across the world in large amounts you need new protection and security infrastructure to fight to keep the bad guys out and allow the good people to have their transactions happen really fast and so I think this is why from my perspective the stripe and open router story is a security story for the internet ecosystem without a partnership like that it becomes very hard to defend the quality of experience and the speed and all the good stuff without letting the bad guys get in

1:17:29the way the second is that there's this underappreciated thing about the fact that you need all the bad things that Alex described as being perpetuated by humans right now is going to be perpetuated by AI agents over the next 10 years right so think about the recursive scale we're about to see of bad actors it's not just bad human beings it's all the bad agents that are going to be attacking the token flow and it's very hard if you're

1:17:59a researcher or an AI lab to reason about that problem because the only data you have is how agents you're training are going rogue but that's just a fraction of all the bad behavior on the internet that we're going to see and so what you need is defenders new sheriffs in town which cowboy has that can see all the bad behavior from AI agents across the ecosystem from different model labs and different post-train deployments and different developers and take all of that data and say we're going to build a shield for the

1:18:29entire token economy because without that the amount of fraud we're going to see of this 10 trillion dollars in GMV and global GDP growth is like a huge percentage of that I think is going to be fraud abuse and we might never get there if people just don't trust tokens and I don't think this infrastructure exists so you have your work cut out for you at Stripe but I don't think people have realized the scale at which agentic fraud like bad behavior perpetuated by AI agents is about to hit us like a tsunami yeah I mean

The Stripe acquisition and future roadmap

1:18:59there's a lot to dig into there I want to give you the last word we do have to wrap what can people expect from open router and Stripe I mean I think this is a really good way for us to accelerate go to market and to go up market more quickly it's also you know as Anj eloquently described this is there's a really clear better together story here when it comes to improving trust and safety and making it really easy to like accept tokens and let people bring their own inference to your

1:19:30app and to help developers just like build on top of inference going forward we have a really strong brand with open router and we're keeping the brand so like open router like as a product and the roadmap and the name and the brand like you know is staying the same and so what like you should expect you know in the next six months is that most things will be like what we would have done had we been independent except everything will be moving faster

1:20:00and that that's kind of like our you know near term goal longer term hopefully I can comment on it soon but I can't now okay well we'll hopefully do a follow up at some point but thank you for being so generous with your time and congrats on the partnership I mean this is one of the most beautiful bromances I've seen in AI just starting from Stanford to here lots more to do lots of sheriff policing to do of the token economy

1:20:31we need new sheriffs for sure yeah awesome thank you thank you