How Mistral Is Building Frontier AI for the Enterprise | NVIDIA AI Podcast Ep. 301

Introduction to Mistral AI

0:00The benefit for everyone involved, really, is that we will have a new open source frontier model that everyone can build off on.

0:13Welcome to the NVIDIA AI podcast. I'm Noah Kravitz. My guest today is Tim LaCroix. Tim is co-founder and CTO of Mistral AI. And we're here to talk about Mistral's philosophy on open models, their collaboration with NVIDIA and the Nemetron Coalition, and their new framework, Forge. Tim, welcome to the NVIDIA AI podcast. Thank you so much for taking the time to join us. Thank you for having me. So maybe we can start with you telling the audience a little bit about Mistral and about your role there from the beginning as co-founder and, of course, as CTO.

0:47Yeah, sure. So we started Mistral about two and a half years ago with Guillaume and Arthur. And at the start, we were all three of us fresh of our researcher's role in big tech. And what we knew how to build were models. And so that was what we started with. And we showed the world that we knew how to train models, that we could be efficient with the infrastructure, and that we could deliver high-quality models, that we decided to release open source.

1:19So that was our claim to fame. But our goal as a company was to provide the value of these models to enterprise. And so quickly, we realized that just chucking weights over the world wouldn't achieve that. And so we went ahead and built a service part of the company that would go with our customers and help them realize value with those models. We also started building a platform to influence those models and enable our customers to really actually use them.

1:55And that platform has grown a lot over the years with the industry, really, with the rise of connections and the need for more context. We've added MCP connections. We've added a lot of niceties to the platform to handle authentication and things like this. With the rise of agentic AI, we're also adding a lot of hosting capabilities. So our customers will require to easily deploy things like sandboxes for their MCP microservices.

2:30They might also need some sort of hosting and auto-scaling there. And so we're really building up those platform capabilities in a way that stays something that we can deploy on-prem for the customer and where they have full control. And so through this, we've also seen the need to extend at the lower layer into infrastructure. And so June of last year, we also announced Miss World Compute, which is our initiative where we're building our own infrastructure and setting up our own data centers.

3:07And we've started to train on them. And it's infrastructure that we can also ship to our customers. How long has the company been around now? Two years and a half. Two years and a half, man. It's a lot packed into those two years and a half. It's been quite a journey. We started with three of us. And today, we're north of 700 employees. Amazing. So alongside all of this, we've also grown and managed the company. It's been interesting.

Open Models and Innovation

3:34Yeah. Yeah.

Open Models and Innovation

3:35What is it about open models that really accelerates global innovation so much? So especially around pre-training, I think there is a lot of wasted resources because everyone is taking the same raw data, which is the data that's available on the web, and doing their best to compress it into a fixed amount of weights. And so there is certainly a lot of know-how into how to select that data, how to curate it, how to do the right optimization procedure, how to train models at scale.

4:11But essentially, everyone is doing the same thing and ending up with pretty much the same artifacts. And so one of the main frustration with Guillaume when we were working together at Mela was that the entire world of research couldn't really benefit from this and couldn't really work on top of all of this effort because models weren't open source. And no other academic lab had the resources to create something like this. And so by creating models and releasing them as open weights, we're still free to work on the licensing and free to provide a software platform and services around it.

4:49And we can still build a business. But we also enable the entire community to really create around those models. And it's one of the most exciting things I've seen is how quickly the open source community around open weight model has grown and the crazy things that it has built, both in terms of the creativity of it, but also the quality of the infrastructure that it now provides. I want to ask you about the Nemetron Coalition.

5:20It's something that Mistral has joined. And I'm wondering what your perspective is on it, what you think it's going to bring to bear both for Mistral, but for the coalition and the industry as well. Yeah. So for us, working with NVIDIA isn't really a new thing. And we've already trained a model in the past, which was called Mistral Nemo 12B. And that was our first experience training models together. And so our teams know how to collaborate. And what this will provide for us is NVIDIA's expertise in larger scale infrastructure, because NVIDIA has more resources and more experience in running those large data centers at scale.

6:03We have a lot of expertise in the various ways to pre-train models. We have expertise in multi-modality training as well. And so as we speak, our teams have been running experiments and exchanging on what model it is that we want to train together and release to the community. The benefit for everyone involved, really, is that we will have a new open-source frontier model that everyone can build off on. When it comes to customizing, or as Mistral says, tailoring AI models, can you speak a little bit about, I mean, the importance of it, I think, as we're talking about large language models and world models, but now small language models and models that are kind of tailored to really do a specific purpose or work in a specific sector or industry.

6:56Talk about Mistral's philosophy and why tailoring models has always been such a core part of what you do. Sure. The reasoning behind it is quite simple. It is that when you think about agentic system or automating workflows, not all of the intelligence in all of the step has to be this big, very powerful thing. However, a lot of the steps are going to be repeated when you go at scale, they're going to have to run fast, they have to run cheaply, they have to be efficient.

7:32And so once you reduce the domain of decision that your model has to make, or you reduce its input space and output space, then you can really reduce the size and energy that it requires. And so there are a lot of customers with whom we work to specialize models to make them faster and cheaper to run. But customization is also about extending the capabilities of models that are already there. So typically, if you go to different areas of the world where English might not be the main language, then it's beneficial to also address this in the training mix and maybe continue pre-training some models to really add to be some Southeast Asian languages to the mix to get a model that is now fluent in that language.

8:22You know, you mentioned this kind of the bias towards English and a lot of these models and the importance of, you know, in your example, with a Southeastern Asian language of, you know, pre-training them all that way. Do you do a lot of work? Do you think a lot about kind of these less serviced languages and cultures and this idea of needing to preserve intelligence through models? So the kind of thing that comes through in work? So this is something that we work on with clients from these areas of the world where for addressing their own customers or for their own needs, they will need to improve the capabilities of the models in those areas.

9:04And so they're usually more adept than we would be at finding good source data in those languages. And so we help them with how to address the right data mixture to really make the models better through that data. Right, right.

Mistral Forge Platform

9:20So I want to ask you about Forge and about how Forge works together with NVIDIA technologies and the Nematron family. But maybe first you can start just in case all the listeners aren't familiar. Just describe what Mistral Forge is a little bit before you get into it. Sure. So Mistral Forge is a platform that we're releasing, which is really the distillation of all of our training capabilities in-house. And so this is always a set of things, a set of capabilities. So you have the training framework, so really the part that takes inputs, provides gradient steps to the model and updates to provide a better model.

9:58And so all of this mechanic, all of the hosting, all of the runtime checkpointing and all of this is something that we provide from our own training capabilities. But it's also the tooling around it. So the data pipeline infrastructure, the evaluation infrastructure, having something that we can use with our customers that's close to what we use for our internal research is also a lot easier on us because we can validate the results. We know what to expect if there is an evaluation that is interesting to us and where we're failing for some reason, then we can transfer it.

10:36And so it's really valuable for us. And it's been something that we've been using with a few companies in different industries. So typically in manufacturing, you would have companies that have a gigantic amount of specifications for what they're building and what their engineering is doing. Having a model that's fluent in that company's domain is really helpful to provide assistance. As I mentioned, we've done a lot of customization around languages and one particularly requested use case is also around customizing to code bases that are private and have never been seen on the web.

11:15So if a company develops a very complex domain-specific language, most modern language models will struggle to provide assistance with this because they just haven't seen a lot of it. Whereas if we work with the company, deploy our solution and specifically train our coding models on that company's code base, which can be massive, then we get improved performances with the same style that the company expects, with the same guidelines and fluents and maybe languages that don't exist anywhere else.

11:47Right. Is there a tension that you find with enterprise customers between open source and whether it's a real tension or maybe it's just in the expectations, but around things like performance and, you know, using open models and staying true to that, but also staying, you know, up to date with what's, I mean, you know, better than I do, just a furiously moving industry. How do you sort of balance that? So it depends.

12:18For some customers who are running typically in air-gapped environments, they don't really have much of a choice. Because they'll have to run the models themselves. And for this, I mean, my personal conviction is that we as a company, and in particular with collaboration with NVIDIA, will be able to push the frontier of open source model to be able to match what other companies are doing. And if we weren't in this for this, I wouldn't be doing this job, really. Of course not, yeah. And so I truly believe that we can provide models that are frontier in their capabilities, and maybe we'll be six months late, but a lot of the customers that are running with us are fine with a six-month delay.

13:04If that means that they completely control the models, if they can customize it, if they control its runtime, there are also many benefits to it. And so one thing that is important for us and for a lot of our clients is to really know what the gap is and whether we're addressing it and whether we're, you know, we indeed are only like six months late, which is completely acceptable. And so to address this, we can potentially provide evaluations through other third-party models. It's also helpful to build solutions with the latest technology when it's available.

13:38I mean, we can't stay state-of-the-art forever in all of the domains. And so it's completely fair and fine for our customers to go through other models and other providers to build their stack. What I want them to be confident in is the fact that there will be missile models or other open source models that will be able to provide those capabilities quite soon. And it's typically one of the benefits of the coalition with NVIDIA.

Enterprise Customer Needs

14:07What are you seeing, Tim, from the customers themselves? What are your enterprise customers thinking about or what are they looking for, you know, maybe this calendar year from their AI investments? I mean, everyone's looking for value in solving use cases. When we engage with an enterprise, we often try to target an iconic use case, something that's really hard and that really provides value. This lets us dive really deep into what the enterprise in question does.

14:43And it also lets us set up a lot of infrastructure to be able to solve that use case. And this is really important for us because in doing so, not only do we get a deep understanding of this company, but we also enable a lot of further progress once that use case is in prod and to completion. So typically, interfacing with all of the company's connectors, setting up a system for a sandbox, setting up all of the company's context that will need to be accessed.

15:16All of this is reusable, right? So whenever we set up with a customer, we try to develop all of our use cases in a way that compound for that customer. And so that it's value that accrues for them. So that's the next use case is going to be easier. And the use case after that, even more so. Right. Absolutely. And there is a lot of work that goes into this. And sometimes it's plumbing work. Sometimes it's work around defining the right roles and the right access control lists. All of this ensures that after this, when people are adopting more and more AI bottoms up,

15:51they will do so safely and easily. Did Mistral see benefits in training your model using Blackwall? Yeah. I mean, definitely the GB200s, which we've been using since June 2025, I believe. We quickly saw a 2.5x improvement, at least like out of the box, when training a large sparse mixture of experts models, especially. And yeah, no, it's definitely been a great acceleration for us on that.

16:23Class of models. And yeah, I think we're seeing further improvements with the GB300s as well. Yeah. Great. How is adopting NVFP4 precision impacted model efficiency, throughput, and also cost in your inference pipe lens? Yeah. I mean, it's always great to see that there seems to be no limit to how much we can compress the models, really. And it's a pleasure to see that NVIDIA is addressing this with hardware that can natively run those operations. And so typically, on infrastructure that supports it now, we run a lot of our inference in NVFP4.

17:01The challenges that we've seen with it is really around the manipulation of the attention and the longer context, where that's where things start to break down for us. But it's also part of the game to address this and make models or quantization better at addressing those issues. Kind of along those lines, is there a big hurdle in front of your team right now that you're really focused on getting around? I mean, there are many big hurdles.

17:32I tend to focus on all of the things that we haven't been doing and all of the things that we should be doing better. I think managing inference at scale is something that everyone is doing with infrastructure that's shifting, infrastructure that's new. The open source world is also going at full throttle. And so keeping up with this, maintaining our inference both stable and correct has been quite a fun challenge. The main thing that's keeping me awake and thinking is really how do we make the permission system of AI agents something that's not a headache to configure,

18:16something that's simple and natural to configure, but also robust and something that people can trust. Because in a way, it's easy enough for someone to configure an open claw or a Nemo claw in a way that's going to work for them. And they'll likely be safe. But also as the CTO, I worry around how do I set that up in the most efficient way in my company in a way that's respectful of all of the data. And typically one of the challenges is that we often think about what an agent is going to be able to read.

18:53We more rarely address where it's going to write the results. And so thinking about audiences and what restrictions we should put depending on all of the content that went into the thought process and into making up the results is something that I think is not addressed wildly in the industry right now. And is that just a matter of just things move fast and it takes time to sort of, in retrospect, kind of put these guardrails and protections in?

19:23Yeah, I think it's something that's also, it's one of the benefits of the open source community, really. They create a lot of amazing things. And typically with this claw technique, I think what we saw was how important it was to have something that has a consistent context that's always on, always working for you and always available. Now, it's our role as well to understand the needs of our clients, their requirements in terms of security and governance and understand their fears and let them really address those fears in a way that's understood by everyone.

20:00And that lets all of the people then build on top of it in a way that's where they feel no restriction because they know everything is safe. And that's where you really see explosion in adoption and use cases, which is really a beauty to see. So what's next for Mastral?

20:22Specific to the NVIDIA collaboration we talked about and also just on the company's roadmap for the year. Yeah, so on the collaboration, I mean, I guess our teams are already at work collaborating on figuring out what architecture we're going to train, what techniques we'll adopt, and working on the data mixture together, figuring out the scale and the model. And so this is going to go on through 2026. For Mastral as a company, it's going to really be about execution in the agentic world.

20:55So really providing a platform that's easy to use, easy to deploy, and maintain for our customers and enabling a lot more people to build on top of it. So the platform has been getting more complete, being like a real tool that we build on. And so now we want to also propagate this to the world through partners and enabling others to build on top of our technology. Excellent. Tim LaCroix, thank you so much for taking the time to join the podcast.

21:25And, of course, best of luck to you and everybody at Mastral. Yeah, thanks a lot for having me.

21:33Thank you.