#244: How Alibaba.com Is Building the Future of Agent-to-Agent Commerce

AI agents in B2B commerce

0:00And there are going to be endless back and forth conversations, different time zones, different cultures and languages in there that AI can help a lot. So we believe that the B2B will go to A2A, the agent to agent. That will be the future. Welcome to AI Transformations, a special series from the Artificial Intelligence Show. I'm Mike Kaput, Chief Content Officer at SmarterX and Marketing AI Institute, and I'll be your host. Every business's AI journey looks different. In each of these episodes, I sit down with leaders who have lived through real AI transformation, including the actual stories, the pain points that push them to act, the moment things started to click, and the results they can point to today. So join us as we accelerate AI literacy for all as part of our AI Transformations series.

Alibaba.com AI transformations

0:53Welcome, everyone, to episode 244 of the Artificial Intelligence Show. I am Mike Kaput, co-host of the Artificial Intelligence Show and Chief Content Officer here at SmarterX. Now, today we are continuing a special series that we've been running called AI Transformations. This is where we spotlight how real companies are driving real change using AI. Now, we actually launched AI Transformations with this initial six-episode run where we had some select stories we knew we wanted to tell. But there are far more than six transformation stories worth telling. So we are continuing the series and bringing you more of these conversations directly as part of the Artificial Intelligence Show.

1:44Now, the premise here remains the same. AI transformation can feel abstract. Plenty of companies talk about it. Far fewer show what it actually looks like inside real organizations with real teams, real constraints, and real business outcomes. So in these episodes, we are talking directly to the leaders doing the work. We're going to explore their particular AI transformations, what AI work looks like inside the company, what got difficult along the way when it comes to AI adoption and innovation, and what kind of results AI is starting to unlock.

2:19So today, the leader we are talking to is Quo Zhang, who is president of Alibaba.com. Quo is here to talk about how Alibaba.com is using AI and agents to rethink how entrepreneurs and small businesses participate in global commerce. So before we get into today's conversation with Quo, this episode is brought to you by Mekon, our marketing AI conference for marketing and business leaders happening October 13th to the 15th, right here in our home base of Cleveland, Ohio.

2:53We are expecting thousands of AI forward leaders here. We are less than two weeks away from Mekon, if you are listening to this when it drops. So if you've been thinking about joining us, this is the time to make it happen. We are going to bring together so many people for three days of keynotes, sessions, workshops, and conversations about how organizations are adopting, operationalizing, and scaling AI. So you'll hear real-world case studies, practical frameworks, and direct lessons from the people doing the work inside companies today.

3:26If you are a podcast listener, use the code POD100 at checkout for your Mekon ticket to save $100 on your pass. Now, what's really cool is that using this code will also reserve your place at our private lunch for podcast listeners, where Paul Reitzer, my other co-host, our CEO and founder, and I will spend an hour answering your questions about AI, your company, your career, whatever you're trying to navigate in the wide, wonderful world of AI.

3:56So seats are limited. The offer closes when the room is full. So use that code as soon as possible. Go to Mekon.ai, M-A-I-C-O-N.ai to register. And we hope to see you October 13th to the 15th here in Cleveland.

Global commerce platform overview

4:13So today, I am joined by Kuo Zhang, president of Alibaba.com. Now, when most people hear the name Alibaba, they probably think about the much larger Alibaba Group. Alibaba.com is a specific part of that organization, and it is where the entire company began. Alibaba.com launched in 1999 as Alibaba Group's first business. Today, it is a global business-to-business commerce platform that helps entrepreneurs and small and mid-sized businesses find suppliers,

4:45source products, conduct transactions, and manage different parts of international trade. For the fiscal year 2026, Alibaba.com reported that more than 47 million buyers from over 190 countries had used the platform to source business opportunities or complete transactions. It also had more than 240,000 paying members. Kuo has served as president of Alibaba.com since 2017. He began his career as a computer scientist at IBM and Microsoft before joining Alibaba in 2011,

5:17where he worked on merchant platforms and tools for Taobao. Now, since taking over Alibaba.com, he has helped lead its evolution from what was primarily a directory of business listings into a platform that supports transactions, payments, logistics, marketing, and other parts of global commerce. Now, he is leading what Alibaba.com sees as its next major transformation, the move towards AI-powered agent-to-agent commerce. That journey, which we'll talk about today, began in November 2024 with Axio,

5:51an AI-powered search and sourcing engine built for business buyers. It has since evolved into what you can think of as a team of specialized AI agents for entrepreneurs and small businesses. So rather than answering simply questions or recommending next steps, these agents can break a business goal into specific tasks and help execute them. Things like market research and product sourcing to supplier negotiations, store operations, marketing, logistics, and compliance.

6:22Now, more than 10 million small businesses now use Axio Work to run their operations, collectively consuming 1 trillion AI tokens over a recent three-month period. Now, those are significant numbers, but the bigger story here is what they could represent. Alibaba.com is exploring now whether it can evolve from a marketplace that people navigate into an operating platform where agents increasingly help conduct commerce on their behalf and eventually interact with other agents representing buyers, sellers, and service providers.

6:56So, we're going to explore today how this transformation began, what Axio Work actually does, how customers are using it, and what had to change inside Alibaba.com. Not to mention, we'll chat about what has been difficult along the way and what agent-to-agent commerce could mean for the future of global business. Kuo, welcome to the show. Thank you so much for being here with us. Thank you for helping me. You did a lot of research. We tried. We tried. You know, it gets a lot easier these days when agents can help you with the research, I think.

7:28We'll talk about that, I'm sure.

Axio and AI search

7:30That's true. Before we get into this broader story, this broader transformation, I have to ask about the name of the product. So, Axio, A-C-C-I-O, is Latin for I summon. And Alibaba has actually said in interviews outright, the name is a reference to the summoning charm in Harry Potter. I'm just curious, why was this the right name for what you're building here? Yeah, so Axio, what is it right? It's Latin for I summon.

8:01And also, it's kind of the summoning charm in Harry Potter. It's just you kind of say the name and you can bring it to you. So, we use this name because in the beginning when we design Axio, it's like you have a world or you have some thoughts and we can use AI and global supply chain to help you to make your dream come true. So, it's some kind of Axio.

8:31And also, in Axio, there's AI inside this term. And all the young guys in Alibaba.com that kind of get a vote. And all the young guys love this name. So, we just choose this as the new product name. I love that. That's so cool. So, let's start off at a high level. Maybe walk me through the AI transformation that Alibaba.com is pursuing through Axio work. And maybe talk to me a bit about how this is changing

9:02the way entrepreneurs and small businesses participate in global commerce. Sure. So, we take step by step. So, Axio first launched, I think, two or three years ago. And at that time, AI is not as today. And we're starting from the AI search. You can think about this like a Google AI mode or AI overview. So, when we feel that when the people using Alibaba.com, they are using user search a kind of different way than before.

9:39They tend to use more and more multimodal materials like images, like documentations. And they describe their needs in a kind of sentence, not only a few keywords or queries. And leveraging AI, actually, we have more power to understand the intention behind the scene of the users, the buyers on Alibaba.com. So, I think that's the first step.

10:09And when we kind of launch the product, we are seeing people actually using this online agent, the online sourcing agent, to do many other things other than search. For example, they are using this agent to develop their ideas. They have some ideas. They want to do a market research. They want to see if this product exists. Are there kind of shortcomings of the existing product? And what are the opportunities?

10:41And then they will transform this idea into a design pack. So, previously, when the buyers talk to the suppliers back and forth, many of the times that they cannot describe what they are trying to build. And the leveraging AI, actually, they can generate a very sufficient and professional design pack for the suppliers. When the supplier receives the design pack generated by Axio, they say they never see a buyer, actually, is so professional. And then it reduces a lot of back and forth discussion and dispute as well.

11:16So, that is the beginning of the Axio. So, we start from the AI search to a kind of a sourcing agent. And they are using that not only for inquiries, but also for market research, for customers or suppliers writing, for the product design. And when the AI technology or power kind of moving on, we are using AI to do the negotiations part as well.

11:46So, the AI becomes a real agent, represents the benefit or the interest of the buyers. And you can imagine the seller side has its own agent as well, helping them to do the similar stuff. And for this quarter, we kind of expand the scope of this agent further. We found that it is not only used for sourcing. It can be used as an e-commerce workspace. So, remember, I actually, the first job of mine is in Taubo and Timor, providing all kind of tools for sellers.

12:22And we see that the AI for e-commerce actually has a lot of similarity of the sourcing. So, you need to manage many stores. Like, you have Shopify, you have Amazon, you have eBay, Etsy. And many of the sellers that we met, actually, they manage stores across different platforms. But they want to manage the business as one, not in different spreadsheets and send the P&Ls. And also, when they upload a listing on Shopify, on Amazon, on TikTok shop, or on social medias,

13:00they want to do it once for all, not kind of keep updating different product to different platforms using different images, videos, applying different rules. And this can be taken care of by the agent as well. And the last but not least is that Axio actually is an aggregate of all different agents. And you can put them in a group helping you to do some kind of deep research from understanding the trend to understanding your competitors' opportunities

13:32and also helping you to dig actually who can make this and what's the P&L look like, what's the margin look like. And this together for SMBs like in Los Angeles, you have some ideas. You want to source and you want to list your product in all different platforms. And you want to manage your daily business. So this Axio work actually can help you to manage your end-to-end work, not only on alibot.com, but also on all your kind of storefronts as well.

14:06So this is how this agent evolves. Wow. So it sounds like Axio started out as essentially product search, using AI for better search on alibaba.com. But because of what you were seeing customers, users actually using the AI for, which wasn't just search, it was ideas, it was advice, that it expanded from there to actually become agents that could help with many different parts of the sourcing and e-commerce process when building a business. And it sounds like then that evolved into Axio work being a platform

14:39to essentially help you end-to-end run and build an e-commerce business. Do I have that roughly correct? Yes. And since we launched Axio work this April, so the tokens that used by the users actually is six-fold for the past four months. So that means that they are using these tools much deeper and complex than before. Yeah. That's an insane amount of tokens.

Friction in B2B transactions

15:10So before you built Axio and Axio work, I'm curious, where was the pain point? What was so difficult for entrepreneurs and small businesses? I mean, you kind of mentioned a little bit the agents are advocating or helping the buyers and the sellers, that they're helping communicate between them. I'm curious if you could drill a little more into why were agents, AI agents, the right answer here for the problems that entrepreneurs and sellers

15:41and small businesses were experiencing? Sure. So before we talk about B2B, we can do a kind of cooperation with B2C. This is more easy for the audience to understand. When you buy something, so there are a lot of, I should say, the experiment or explorations for the B2C's agent as well. But majority of them is not that successful yet because of the scenarios or the challenges.

16:15So when you are a consumer, buy some stuff. The first, the product is not that expensive. So decision is not expensive. The second is the decision, how to say, the dimensions that you need to make that decision is not complicated. It's, in general, including the image, title, price, delivery time, probably this is it. And also, many of the time when you buy something, you're killing time. Consuming, actually, is a kill time.

16:47Nobody is using agent to kill time. You don't send agent to cinema to see a movie for you. Okay, you want to kill time yourself. But in the B2B part, when we do the comparison, it's completely different. So the first, the decision is expensive. So on Alibaba.com, every order kind of, in general, is about $3,000. So the decision is expensive. And also the demand, the decision, the process of decision is complex.

17:18So you need to compare at least, you can think about, there's a table with at least 20 columns. It's starting from the name of the product, price, and then go to the MOQ, then go to the certificate, and then go to the kind of delivery time, payment terms, income terms, and a lot of things, other than kind of you buy something for your own use.

17:51You want to buy and resell, or you want to buy because it's part of your product. So it's involved a lot of requirements that is far more complicated than the consumer side. And also, it's a work. The B2B for both sides, they are in a working kind of environment. So they procurement because they need to kind of do a business. And suppliers are the same. So both parts want to use agent to save time, right?

18:24For human beings, we save time so that we can kill time. So agent is designed for save time part. So for the B2B, we think, first, it's solve a lot of problems, can help you to save a lot of time. And also, there are a lot of complicated decisions, which requires back and forth, repetitive work that can be taken care of by an agent. So taking a very simple procurement overseas, as an example,

18:54in general, it takes around 60 steps from you have an idea until you get that product delivered to you. And there are going to be endless back and forth conversations, different time zones, different cultures and languages. In there, that AI can help a lot for both parties, buyer and supplier. So we believe that the B2B will go to A to A, so agent to agent.

19:25That will be the future. So I love that example. There's 60 full steps in that process. Can you, I'd love to hear maybe another example or two from you, just concrete ways, how are entrepreneurs and small businesses using something like Axio work today? I think that really hammers home for people just how much friction there is in this process that agents can help with. Mm-hmm. Okay. Sure.

Real world sourcing examples

19:52So we actually shoot a video for that, if you have interest in the consent. Yes. So just describe a kind of, if you are a guy who is leveraging global supply chain, SSMB. So in general, what's your daily life look like? So you have some ideas. For example, if you want to source a mountain back and resell in UK or in US,

20:22and then you need to describe what you need, like in an email or in an RFQ, and then you copy-paste this requirement to about 120 suppliers with your requirement. Back and forth. And don't mess up with name. Okay. And then you just wait for that reply. Some fast, some slow.

20:52And then you start to manage Excel to compare all the columns, from the MOQ to price to certificate to quality to weight, and you can name it. And that kind of conversation back and forth probably for a week until you gather a spreadsheet with dozens of replies of suppliers. And then you select probably two or three or five of them. You start to further negotiate for further terms.

21:24And then you need to talk with them with how to kind of manage the incoterms, the deliveries, how to pay for them. And when you gather product, you're back and forth on the samples. And these are the kind of daily work life of SMB, who is sourcing, let's say, overseas. And in general, that takes for weeks or sometimes for months. You're just starting from an idea and then find a right guy who can make it for you and then get the results.

21:59So with the help of AI, actually, what do you need to do is you tell the agent what your needs look like and what you are kind of looking for, what's your budget look like. And then the agent start looking for the suppliers who specifically match your needs and help you to communicate back and forth with the suppliers. And they can generate the projects for you with all the columns that you need, probably just within a day, if you're lucky, in hours.

22:33And starting from there, that you can start to push forward other communications and the agent can help you along the way to finish the kind of contract in the end. So with the right MOQ, right price, and with the right shipment method, incoterms, and payment terms. And then we can leverage trade assurance as actual B2B payment, help you to protect the entire order. So that kind of streamlined the B2B business a lot easier than before.

23:05So I'm just using sourcing as an example. There are many other examples as well. So you can imagine that for SMB in general who don't have the global sourcing experience, very limited time and resources, and don't understand about the terms. And in many cases, they don't know how to write a design pack for your product. The agent can help you to handle all that parts. And you're only focusing on the most important question,

23:38like what problem you are going to tackle? Who are your customers? And that's agent to help you to manage the rest. I love those examples. That just shows there's so much opportunity here to dramatically streamline and speed up the trade process, especially in B2B with all these tasks that frankly, like businesses, entrepreneurs, even SMBs, or people who are very expert at sourcing, nobody wants to be spending hours, days, weeks on these tasks.

24:10They want to get back to the business of selling. So it sounds like agents can really streamline that process. I'm curious, at a high level, with whatever you can share, what kind of technology is powering Alibaba.com's AI transformation, including Axio work? We've talked about agents kind of broadly. Can you maybe go into a little more detail there?

Architecture of Axio work

24:31Okay.

Architecture of Axio work

24:32We have a formula, actually, in general to describe what the agent or Axio is composed of. So agent equals to model times harness times context. So why is it multiplied? Because each element is zero. The entire equation is zero. So model actually is, how to say, the model includes a lot. It's intelligence power. And we are using, we're leveraging all the frontier models in the world, including the queen.

25:07And also we do some post-train on the queen, small size model to make sure that it has the same power of the big size frontier model because of the post-training that we use. And also we do a lot of serving stuff as well to make sure that we have better cache rate and we have a better performance of that model. And harness actually is the key as well. So harness contains all this logic, the loops, the gutrails, the tools that you need to connect.

25:41Like we manage to help the users to manage many platforms. Then that you need to have tools to connect to all these platforms, to aggregate data, help you to upload listings, do the marketing research, so on and so forth. And the last but not least is the context. The context, including all the real business data, the conversations you have, and also the data from platforms.

26:14The broader the data is, the kind of more broader view or full understanding of the background and the trend and the category, actually you can have a better decision and the result out of that. So this is the kind of the formula that we are composed and we kind of keep improving on to get a better agent. And to make sure that we are continually improving the system, we invented a bench.

26:50We called Commerce Agent Bench. So you know there are all kinds of bench, including mathematics, coding, so on and so forth. But when we talk about the business environment, a little bit messier than all the bench before. And we are kind of distilled from millions of conversations. I mean the real conversation between the buyers and the sellers and different execution workflows as well.

27:20And we abstracted 107 different tasks to make sure we cover all the categories where people are doing business on platform. And we're using this bench kind of to test all the results using different models and leveraging our harness and using the real context of the real business to make sure that we can evaluate the result actually is measurable, is reliable, and most importantly is affordable to all the SMBs.

27:54I really like that framing. That's an elegant way to put it. It sounds like you have your model, which is doing various different types of post-training on some version of Alibaba's Quen family of models, which are extremely formidable. That's kind of your raw intelligence multiplied by a harness, which is kind of the proprietary secret sauce, bit of code, logic, guardrails, essentially all the machinery or architecture around the model to help it be agentic in the way you want.

28:27And then multiplied by that and then multiplied by the context, all the business data, the domain expertise Alibaba has, et cetera, that's a really powerful mix. And it is why you cannot just duplicate something like this in a different business or a different context. There's these three elements, it sounds like, working together to make all of this possible. Exactly. So what we've talked about is amazing. I mean, there's really cool results by saying, okay, applying agents to the e-commerce process, especially a B2B process through Alibaba.com, is producing awesome results and things for customers, for buyers, for sellers.

29:07But this also, you kind of alluded to this when you said A to A, like agent to agent commerce. Like stepping back a bit here, how do you think AI agents are actually going to be transforming commerce at large? It seems like there's some big implications here. How is this changing Alibaba.com's own business?

29:26So first of all, we think of in the future, what is going to look like. So we think A to A is the answer. Since the both party, I mean, the suppliers and the buyers both want to save their time and get efficient result as much as possible. So there are needs, and also there are network effect. So when you get a network of suppliers and you have a network of fulfillment supply chains, and you have all demands in the world, you know that every small and medium business all over the world, when you ask them, do you need global supply chain?

30:13Do you need kind of a genetic team to help you to save time? I think it's a yes for all the companies that we meet. And why they do not use the global supply chain or do global business? It's because of their size, power, limited of resources. If we can kind of give them this power to access, I think every small SMBs can unleash their full power.

30:43Since they understand the local business, they understand the local needs. And that means that we can have a more broader audience of our customers, the users. It always comes first from the demand side. When the demand side is getting larger and larger, we can kind of aggregate the supply side all over the world as well. Since all the suppliers, they want to get more business. And the more that they review, actually encrypt by agent or provide their real business data, provide their real kind of manufacturer capabilities,

31:21that is more easier for them to connect to this network. And when the agent-to-agent kind of working together, it's more like a multiply. It's not like I search engine. I recall some of the results. For example, I have hundreds of thousands of suppliers. I only give you one source and using this kind of ranking mechanism. And you can only talk to dozens of them since you don't have enough time. But when the A2A network work together, it easily helps you to kind of map with a much huge network.

31:57And it's much easier for you to kind of get the best match with your requirement, with your budget, with your timing. And it's much easier for us to kind of get a better result out of the demand and supply requirements in the world. So I think if we think about it that way, that is much bigger than what we are doing today using the search engine-like business.

32:25So the technology is really one part of a transformation like this, both in the industry and with what you're enabling for customers. I'm curious about looking at Alibaba.com. What had to change inside the company as you move from experimenting with AI to building and operating products and workflows entirely around agents? That probably has some implications for how your company operates, how talent works. I'm curious if you could unpack that for us.

Startup velocity within Alibaba

32:59Sure.

Startup velocity within Alibaba

33:00So I just try to tell you as much as possible. Well, the first thing is that Axio operates as a startup team or company within Alibaba Group. Okay. You can imagine that Alibaba.com has its own pace, has its own business model. And also, since we are a public company, that it has its own plans as well. Well, Axio or AxioWork is a completely different creature that, for example, Alibaba.com has a feature update by weekly or by month.

33:41Since that system already exists for 26 years, as you mentioned, it started from 1999. So every feature that you add on, you need to be compatible with the existing features and does not impact the customers who are not familiar with this. You do need to have a lot of A-B tests and have a lot of kind of zone-off for any feature goal line. So what Axio and AxioWork is, it has a version by daily basis.

34:12So every day, it has a new version. Oh, wow. Yeah, every morning that I get up, the first thing is that I upgrade this software or this agent to see what's new there. Since the agent, actually, many of the features, they write themselves. It gathers all the information, the requirements from all this kind of daily execution environments. And it's ranking. Actually, people have to rank and they fix the bugs and kind of a lot of coding is written by themselves.

34:43So it's just evolved in a different pace. And also, what do we need to integrate two together is, think about Alibout.com is part of harness of this agent, right? We need to get this website first to get it all this services client. It can be invoked. It can be used, leveraged by this agent easily.

35:16So it's not only designed for human, but it's also designed for agent as well. And these two toolkits need to work simultaneously. And now we are kind of integrating Axio back to Alibout.com. We call it the AI mode, meaning that all the users can use Alibout.com as an agent for them. So we started from a startup team to kind of create this business. And also, we are kind of using the existing engineering team to kind of give the whole platform an upgrade.

35:56So that it's easy to be used by the agent. And now we are integrating the two products together. So all the users on Alibout.com are easy to kind of assess to Axio as well. So it's a step-by-step approach within Alibout.com. That's really fascinating. So it sounds like Axio work is almost like its own innovation hub or lab or business within Alibout.com, where it sounds like Alibout.com, because rightly so of the legacy and the history with customers

36:28and the need for continuity, is shipping updates at a bi-weekly or even monthly rate, whereas Axio work is literally updating almost every day or a version every day. It sounds like it is also updating itself. And then you had mentioned CLI or command line interface to then it's actually starting to learn or you're implementing having it work to talk back to Alibout.com, right? To start integrating there so you can incorporate agentic features more into the main business.

37:02Do I have that right? That's incredible. Yes. And today, since Axio is integrated in Alibout.com as AI mode, now Alibout.com is operating daily basis as well. Okay. Okay. So it's made it possible to then ship faster based on what AI and agents are now enabling. Yeah, exactly. So a couple more questions here for you, Kuo, as we wrap up here. I'm curious, as you've gone through this incredible journey with Axio work and Alibout.com generally,

37:39what has been hardest or messiest about going from all these AI experiments to actually building agent-based products and operations? Anything jump out to you as a particularly big obstacle? Yes. I think the biggest challenge is everything is just moving so fast, right? Like previously, it's easy for us to predict what it looks like for three years. Like you have a three-year business plan and you executed quarter by quarter.

38:14But now, in the AI era, three years means forever. So after three years, actually nobody can predict what will happen. So think about three years ago, nobody can understand or can predict the AI actually can penetrate coding, let's say, for 95% probably. I did not see any coder now don't use any kind of AI to do their coding in daily basis.

38:44And you can imagine that in all different business categories, the penetration will be same or even higher. If we find out a way that using a genetic team is far more efficient than the way that you do business before, I think the adoption rate or speed is going to be faster than ever. And also the technology is evolving faster than we expected as well.

39:18From the hardwares to the models to all these kind of agent design systems, if you ask me the formula of an agent, let's say six months ago, probably I cannot answer that question. So now more and more we understand what's the kind of underneath technology, technology were the key factors and how we measure the success of the agent. Actually, we now can answer your questions. And if I have this knowledge, let's say travel back to six months ago,

39:52a lot of things that we can do differently. But yeah, we just, we don't know what we don't know. And now I think there's a lot of things that we don't know yet. And we just need to keep, keep learning and keep on the pace of this kind of AI era to make sure that the thing that I think what we do right is we found the right problem to solve, right? If the problem is not a real problem, it's a fake problem,

40:22then no matter what you do, actually, in the end, it's not going to get any business impact. If it is a real business problem and it can solve a lot of issues for those SMBs, we think that there are going to be a huge potential of this AI that is not unleashed yet. That's incredible. I love how you've broken that down. So, you know, final question for you, Quo. I'm curious, I want to circle back to something you mentioned. You talked a bit about Commerce Agent Bench.

40:55And I'm curious, can you talk a little bit more about this? Because the kind of issue here is as agents actually start executing real business tasks across knowledge work, not just commerce, I'm curious, Alibaba.com has this kind of interesting, unique approach. For measuring how or whether or not agents actually get the job done. So Commerce Agent Bench is this open sourcing benchmark that you had mentioned for evaluating AI agents. Can you just tell us a little more about that?

41:27And we'll kind of wrap up here just talking about how Commerce Agent Bench makes it easier to understand are agents doing the right work and are they performing well?

Commerce Agent Bench metrics

41:39Yeah, sure.

Commerce Agent Bench metrics

41:40It's a, so if you are familiar with this idea of Pareto Frontier. Yep. Pareto Frontier, yep. Yeah, everybody's talking about that. So think about this is a new system. It's a kind of new framework for the commerce environment of Pareto Frontier. So you use this model to solve mathematics, it's perfect. But when you use this model to solve, let's say, the real problems in the business world,

42:13many of the cases fail. So this kind of commerce workbench is just for you to mirror the real business scenarios with real business case and real business context and with the real harness, which is Axel. And using that kind of test, we have 107 tasks yet. And that number is keep evolving. If next year you ask me, probably it's going to be 300 and more.

42:44And we'll just keep evolving this worksbench to make sure that we can capture more and more complicated problems in the world. And now we actually see it in different dimensions. The first is the pass rate. So make sure that this agent can perform in each of the scenarios. If it cannot deliver or the result is wrong, the pass rate is zero. It's only one or zero.

43:15And then we also consider about the time. How much time and steps that the agent can give the result with a certain model. And also we consider about the price. So at what price that you solve these problems? So in the end, only the affordable AI is a useful AI for SMBs. So you will have the best model, but it's not affordable to do daily basis work. It's not useful as well. So we are kind of redesigning the framework, thinking from the business perspective for SMBs

43:51to make sure that we are routing to the best model fit for the right problem. So that's why we are kind of inventing this. And we are not only kind of competing or comparing model with model in the same harness. Like we put all the frontier models, including the queen model in a harness of the Axio. And you can compare which model solve which problem more efficiently.

44:24And we can also use this to have a benchmark for the agent as well. For example, with the same 107 tasks, we're using Axio, we're using Codex, and we're using Cloud Code to solve that 107 same task. Axio actually can have a higher pass rate with less than half of the price. So meaning the SMBs with the same budget, you can actually do twice of the job.

44:55So that's another way to see how we can leverage this workbench as well. So we're just to understand which model fit for which problems. And we can help the SMBs routing through different models to solve different problems and get the best result. I love that. And just to kind of reiterate here as we wrap up. So when we talk about the Pareto frontier, this is basically where, just to kind of clarify for anyone who hasn't heard that term, or if you want to connect the dots here, that's when we're really just trying to determine what are the best tradeoffs among different qualities of an AI model, right?

45:35Like we all have encountered this in one way or another. You want to go use, say, GPT-6 Astra or Fable 5 or something. It is really good at certain things, it's bad at others, and it costs a lot of money. There's different tradeoffs, right? It could be slower in some ways or faster in others. That's exactly what we're talking about here. So Axio work, the reason this benchmark matters is because you actually sound like you have a way to actually look at over 100 different tasks that are actually important to businesses.

46:05And you're able to benchmark the agent on those exact tasks from a Pareto frontier perspective. So it is the best tradeoff between speed, reliability, price, performance for those tasks. Do I have that correct? Exactly. Okay, great. So just for everyone listening, that is kind of exactly what we're getting at is what's so important with these types of developments as we move into this agentic era is the ability to actually say conclusively, can this agent actually do the work?

46:41So, quote, thank you so much for sharing Alibaba.com's journey, for sharing all about Axio work. This honestly feels like looking into the future. I can't wait to see how A to A, agent to agent commerce evolves. Thank you again for being here and for sharing Alibaba.com's AI transformation story. Thank you for having me. Thanks for listening to our AI transformation series from the Artificial Intelligence Show. To keep learning, visit smarterx.ai, where you'll find on-demand courses, upcoming classes, and practical resources to guide your AI journey.