Meritocracy, Exams, and the Knowledge That Matters (with Daisy Christodoulou)

Welcome to EconTalk

0:00Welcome to EconTalk, conversations for the curious, part of the Library of Economics and Liberty. I'm your host, Russ Roberts of Shalem College in Jerusalem and Stanford University's Hoover Institution. Go to econtalk.org where you can subscribe, comment on this episode, and find links and other information related to today's conversation. You'll also find our archives with every episode we've done going back to 2006. Our email address is mail at econtalk.org. We'd love to hear from you.

0:36Today is July 21st, 2026, and my guest is Daisy Christodoulou. Daisy is the Director of Education at No More Marking, a provider of valuation software for schools. She was last year in March of 2025 talking about education and feedback. Daisy, welcome back to EconTalk.

Meritocracy and Civil Service History

0:56Great to be here, Russ. Now, our topic for today is the intersection between meritocracy, education, governance. But we're going to go back in time and look at some crazy historical moments in civil service history, and we're going to start with a failed Chinese civil servant exam taker, Hong Xiu Xuen. Why is he interesting? So Hong Xiu Xuen is really interesting because he's probably the instigator of the worst civil war,

1:31maybe the worst war in history. It's not very well known in the West. And in fact, one of the leading American historians of this war, this civil war, has said that he didn't even read about it till he was an undergraduate. So it's remarkably kind of not very well known in the West, given how significant it is and how major factor it is in Chinese history. And Hong Xiu Xuen is basically a failed civil service candidate. So this all takes place in the mid-19th century in China. And at that time,

2:03China has these very rigorous exams to become a civil servant. So if you want to become a civil servant in 19th century China, you sit these very, very tough exams, they have multiple stages. So you will sit some that are just, you know, kind of close to your village, you will sit some, you'll go to the provincial level. And by the mid-19th century, these exams have been around for centuries. And China's population has been increasing over that time. But the number of openings, number of job openings available for the exam takers hasn't gone up. So it's very intense competition. And you have

2:39all of these incredibly bright, young Chinese men studying really hard to try and pass these exams. And they're tough exams. And you have to go away for the provincial ones. And they're three days and you're sitting in cells working away at these exams. And if you fail them, you can take them again. And they don't happen every year, they happen every couple of years. And what you have is you have, you might see some parallels with today, you have students who are maybe the brightest student in

3:09their village, maybe even the brightest student in their in their province. But they get to the stage where they're competing against all the brightest students from all over China, aren't very many jobs to have. So you can be very bright and spend years studying and still fail and not get the job. And you have people who take them, carry on taking them into their 40s, 50s, 60s, kind of keep going back and taking them. And Hong Shiao Shen takes them quite a few times, these exams, and he fails them. A third or fourth time he fails them, he's absolutely devastated. And he has what we may call now a kind of

3:46mental breakdown. And he goes back home to his village, and he's seeing visions. And in the time previous times when he's taken these exams, he's also come into contact with some Christian missionaries, and he's picked up a few of their tracts. And so after this final failure, he's got these tracts on his mind, and he sees these visions that tell him he's the younger brother of Christ. And he's inspired to kind of start some kind of millenarian kind of revolution. And his first few followers are people like him, his

4:19cousin, some friends, people who have also taken these exams and not succeeded. And he basically moves across the countryside, kind of picking up an army. And the army grows and grows. And as I say, he becomes the leader of what is known now as the Taiping Rebellion, which, as I say, is one of the worst civil wars in history, maybe around 30 million people die. Eventually, it's put down and he's killed, but not before enormous kind of death, destruction, suffering. And I think approximate

4:55cause of this terrible war is the frustrations, the pent up frustrations of these young men who are trying to take these exams and failing and not getting the jobs that they hope. But I also think you can look in a way at what's the ultimate cause of this revolution. Because the exams that Hong Zhurikian is taking, the exams he's taking, they are full of kind of the ancient Chinese classics.

5:26And by the time he comes to take them, they're quite stereotyped, and they're quite rigid. And you have to, there's a particular format called the eight-legged essay, where you have to write an essay that is set out in a really precise and specific way. There's nothing on them about the kind of modern Western knowledge of scientific and maths that's transforming the rest of the world. So you can make another argument that there's certainly the students who failed these exams, it's not great for them. But for the students who pass them and become the bureaucrats who are running

5:57China, they're really lacking in the kind of knowledge that is transforming the rest of the world. So you can make a case that these exams play a very important proximate and ultimate role in one of the worst civil wars in history. So we could spend an hour on the 19th century Chinese civil service and how it was, the entry to it was control. But we were not going, we are not going to.

6:27We're going to divide it between the Chinese civil service in the 19th century and the British civil service in the 19th century. No, we're actually, we are going to touch on the latter, but we'll move on to other things. But it's a fascinating story. It raises a question which I don't think you talked about in a preliminary essay you shared with me, which is why were people so eager to be a civil servant in China in 1850, say? And it raises a general question, which we're going to spend some

6:57time on, which is the prestige and pay and honor that's associated with the civil service differs wildly by culture and country. And the significance of the civil service, the bureaucracy, I think varies quite a bit. So let's start with China. Why were people so eager to have these jobs?

The Appeal of Chinese Civil Service

7:22So look, I don't want to over claim and pretend to be an expert in China. My entry into this has been through exams. My specialism is assessment and exams. But I think, obviously, when you just see the sheer numbers of young boys and it is boys who are studying for these exams, you can read some of the literature that the failed candidates leave behind. For whatever reason, it is hugely prestigious. I think it's partly, there aren't many other, there aren't many other jobs or outlets for if you are,

7:58kind of, you have academic tendencies and you enjoy reading and you enjoy studying. So there's not much else that necessarily you can do that will give you a living. And in fact, when these candidates fail, a lot of the time, the only option for them is to go back to their village and become a teacher and to try and teach the next students who are coming through and see if they can get them to succeed. So there aren't necessarily many other things you can do. And I think, yeah, you know, the prestige and the power. The civil service in any society is powerful. They do have powers. And for young men

8:35from lower, you know, lower social backgrounds, again, in any society, you would say it's one of the few ways of advancement and the few ways to develop yourself. You said something kind of shocking to me a minute ago. You said something about how the bureaucracy runs the country. Now, the bureaucracy often thinks they run the country.

8:58And people are often under the illusion that even a dictator runs the country when a dictator can't run the country either. But certainly in modern democratic states, the bureaucracy doesn't run the country, although they may run parts of it more in a more serious way than the legislature runs the country or the so-called president or whoever is the leader, the prime minister. But I want to go back a sec and just talk about your specialty, which is exams. Because on the surface, if you think it's important that the

9:35bureaucrats who are, quote, running the country are skilled and capable, an exam has an appeal. Because otherwise, it's who you know, and so on. And what's remarkable about the Chinese exam, and will prove remarkable about the British case when we turn to it, is that the content of these exams is not what you or I would design to make sure that the best ended up on top. So talk about what

10:07happened in England around this time, and the reforms that were made, and how peculiar they were, and strange.

British Civil Service Reforms

10:15Absolutely. So, by really a kind of historical coincidence, a historical irony, as this terrible civil war is breaking out in China, and almost literally as the news of it is making its way back to Britain, Britain is engaged in a big national debate about how it should recruit and select its own civil servants. And obviously Britain at this time has an empire. So, in particular, it has India. Now, India at this point, you know, is in flux as well. But essentially, Britain is either directly

10:50or indirectly ruling vast swathes of the Indian subcontinent. And also, you have a lot of changes going on at home, and you have a lot of growth in the home civil service, as you might call it. So, there's an increasing debate about what's the best way to pick all of these administrators and to recruit them. And as you say, one problem you get when it comes to recruiting bureaucrats and recruiting administrators who are going to run the show, whether you think they're really running the show or not. I think that it's fair to say they have at least some opportunities for kind

11:22of power and influence, even if they're not necessarily always running things in the way they might think. The risk is that you get people just appointing on for patronage, basis of nepotism, people giving jobs to their mate's nephew. So, that's a risk. And if you want people who are competent to do the job, as you say, one thing you can do is say, we'll have an exam. But it really matters what is on the exam. And on the Chinese civil service exam at that time, as I say, is really a lot of ancient classical Chinese knowledge and languages and this eight-legged essay.

11:58And Britain are kicking around ideas and saying, well, what goes on? If we're going to have an exam, what should go on the exam? And there've been a few kind of experiments prior to this time, and there are a few places where exams are starting to be used. But there is this major point in 1854 where a major report is written that proposes there should be one central exam for applicants to the civil service, the British civil service, and that this exam basically should

12:30consist of a lot of knowledge of the classical languages of Greek and Latin. And these are very prestigious at the time in Britain. They're taught in a lot of the public schools, the great public schools. They're taught at the famous ancient universities, Oxford and Cambridge. And you have two administrators of the time, a guy called Stafford Northcourt, another man called Charles Trevelyan, who writes a report that essentially proposes that these curriculums,

13:04with the heavy emphasis on ancient Greek and Latin, should become the basis for the exam that selects for civil servants.

13:15And it gets, the report basically gets implemented? Yeah, pretty much. Yeah. So the report is commissioned by William Gladstone, who's obviously quite a famous prime minister eventually. So it's commissioned by Gladstone. It's written in 1854. It is just a report. So it isn't immediately an act of parliament. But gradually over the next decade or so, most of its recommendations do become law and are implemented.

13:45And to the extent today that it is now regarded, even in Britain today, as the founding document of the British civil service. So a very senior official currently in the British civil service, actually, I think he lost his job a couple of months ago, but a very senior figure, Ollie Robbins, has said very recently, Northcourt Trevelyan, the Northcourt Trevelyan report, as it's known, is the founding document of the modern civil service. So it's been hugely influential. And it does come in for its fair share of discussion.

14:22And it's become something that's been discussed a little bit more lately, as I think people wonder about modern Britain and modern Britain civil service and whether modern Britain civil service is as effective as it should be. And if some of the problems of that do go back all the way to this Northcourt Trevelyan report. So I want to set this next part up a little bit with some background and some things to think about for listeners and for you and me, Daisy, because you could take this conversation so far to be about some obscure exam in 19th century England and who cares about Northcourt Trevelyan.

14:59But here's what's important. And I want to bring in a couple other things. America is is pretty disdainful of its bureaucratic system. Americans do not look at, quote, bureaucrats, the polite term is administrators, the more negative term is bureaucrats in, say, health and human services or various Department of Education. They don't have a lot of prestige. And a lot of people would say they don't do a very good job.

15:32At the other end for both the U.K. and the United States, you have the military and the standards and and excellence is totally different.

15:48One might suppose that's because one's a matter of life and death and one isn't. So indulging in hiring students who are good at Latin and Greek, you could say, well, who cares if the Treasury Department has a lot of those? It's not so important. But when it comes to military affairs, you kind of want them to be able to know what they're doing. So you'd want a different test for the Royal Navy or the U.S. Air Force or the Marines or the U.S. Army. Now, before we get to I just want to put that on the table in terms of the importance of this.

16:20But I want to also talk about this reform, because as you point out, this is extremely interesting to me. Basically, they took this prestigious gate of getting into the civil service and the gatekeeper became, are you really good at what we teach at the two most prestigious universities in the land? And it's hard not to view this as a public choice story, meaning the field in economics that assumes policy decisions are made, often driven by self-interest and not the national good or the public good.

16:59So basically, they basically circled the wagons, Northcutt and Trevelyan. Forget what they said about it. What they actually did in that report was encourage the British Civil Service to keep hiring people like themselves, people who've gone to Oxford and Cambridge, who are really good at Latin and Greek. And as you point out in your essay, which is hilarious, people at the time said, you know, it really doesn't matter what they study or what the criteria are or what the subject matter of the test is. It just has to be hard.

17:31And that will get the people who are good at hard things and they'll be skilled. They'll be the most skilled people because they could do the hardest things, even if it's nonsense. Thomas McCauley, the great historian, says, yeah, you know, if they studied astrology or alchemy or the Ptolemaic system of astronomy of the solar system instead of Newton, they're hard to master. And that's what counts. So I've thrown a lot on the table here. Take a crack at reacting to that.

18:01Sure. So I basically agree with what you said, your view of Northcote Trevelyan. But what I will point out to you, so what you've said, this is a public choice theory. This is basically a way of getting people kind of almost like a self-perpetuating oligarchy of getting people who look like you and sound like you and are like you into positions of power and influence and keeping that ticking over. So I would agree with you on that. And as you say, I've got this essay where I kind of make that argument. However, what I will say is that is not the dominant historical opinion of Northcote Trevelyan.

18:37So, as I say, it does have a little bit of fame here in the UK. It's something people kind of think about and talk about. The dominant, I would say, take on Northcote Trevelyan is that it's meritocratic and it's a good thing. Right. And I would say it's only in the last decade or so that there's been a little bit more revisionist scholarship on that. And the book that I draw on to make the I think makes a revisionist argument really well is a book by a guy called Michael Kulikin called No Trades Men and No Women.

19:09And it's a history of the British civil service. And he takes this quite revisionist stance where he says everyone thinks Northcote Trevelyan is about meritocracy because it introduces exams. He said, but it isn't about meritocracy. He has a line where he says, OK, it probably did reduce the opportunities for patronage of the really explicit corrupt type where you pay someone to give your underperforming nephew a job. So it probably does reduce that. But he says it eliminated patronage, but it also eliminated professionalism because actually prior to Northcote Trevelyan, the system is starting to evolve in a lot of important ways to be using exams already, but to be using exams that are relevant to the job you'll be doing.

19:56So I would say when I was growing up, the dominant take on Northcote Trevelyan was, oh, before Northcote Trevelyan, everything was super corrupt. People paid for jobs. Northcote Trevelyan came in and professionalised everything. What I think this guy Michael Kulikin is arguing and what I would agree with and what I think you were saying too is actually what was happening is before Northcote Trevelyan, there definitely was some corruption, but there were also a lot of moves to modernisation and professionalism. There were civil service departments that were using exams that were useful for the specific job that the candidate would be doing.

20:27And actually Northcote Trevelyan came in and said, we're going to replace all that with one central exam that's going to be based on topics that really have no relevance to the job in hand. So it is a revisionist take. It is a controversial one. And there are people I think have grown up with that standard take on Northcote Trevelyan who don't like that revisionism. So it is controversial. But I think this book by Michael Kulikin does a fantastic job of drawing out a lot of the debates. And in particular, he draws out a lot of the importance of a guy who's the master of Balliol College at this time, a very important figure behind the scenes.

21:03He's called Benjamin Jowett. And he is incredibly keen to be getting his classics graduates from Balliol College Oxford into positions of influence all over the British establishment. And he is very successful. And he is a kind of a major figure in the background of Northcote Trevelyan and subsequent developments. So I would agree with you with your how you put it there. But as I say, I would say this is relatively controversial taking, Brim. So talk about the military. And again, I think we can think about bureaucrats who run, you know, say a welfare program or transfer program that distributes money in certain ways.

21:40And there could be it could be run better or worse, obviously. But a lot of times what's at stake in these conversations, and you and I recently had a chance to talk about this a little bit in England, is basic fundamental infrastructure. And so, for example, when I was in England recently, a couple of trains I was scheduled to take were just canceled out of the blue with no real like a half an hour before I got on, just canceled signal issues. I don't know. I don't know if that's a fair criticism. That means it wasn't the system that runs so well.

22:15You evoked pride for the Elizabeth line, which I took without without any problems while I was in in London. And I do like it a lot. And again, we might think about the military where, oh, well, that plane didn't work or this aircraft carrier sunk would be unacceptable. No one would would take it for a minute. And the people who who failed there would would be fired. That seems to never happen with transportation, infrastructure of that kind. There seem to be very little consequences, which makes you wonder whether the people who are in the jobs or have the skills necessary to do it.

22:52So talk a little bit about your your take on on the military and transportation. And you can then you can bring in Samuel Pepys, because if we're going to delve into the into strange moments in British history, we can mention him, although we will not discuss his X-rated diary. But those who are interested can pick it up on their on their Kindle.

Bureaucracy and Feedback Mechanisms

23:15So, Russ, I think at the heart of this is a classic econ talk theme, which we chatted about when I was on an episode before of feedback loops and price mechanisms. So the major problem I think you have with bureaucracies is often there isn't a price mechanism. You don't have those feedback loops. And actually, when it comes to the military, I know what you're saying, that obviously the military, you know, in lots of ways, there's more competence there. But I think you've also got to be aware throughout history and in every single country you can think of, there's a history of military mess ups, too.

23:46Oh, sure. Like, military bureaucrats don't always kind of get it right. And I think what you have in bureaucracies is you don't have a price mechanism. Often there's a good reason for that. So, again, I don't want to get too off topic going into this, but, you know, obviously there are some times, obviously with the military, you understand why the state has to have control of that. And you don't necessarily want to be outsourcing everything to countries that are your enemy, whatever. So in a lot of bureaucracies, you don't have that price signal. You don't have the feedback loops and things can go awry.

24:17And I would say, actually, it's a theme of British military history that often after a long period of peace, the first year of wars go really badly. And a lot of things go wrong. And then you have a different feedback loop, which asserts itself, which is lots of people dying on the battlefield. And that is almost a replacement for the price mechanism. And in a lot of ways, it is a more brutal master than the price mechanism. So the first year of World War I, not just Britain, but every participant country, just have all kinds of crises where the peacetime military bureaucracy was not, and the civilian bureaucracy, was not up to the task of a total war.

25:01And so in Britain, we have a famous thing. We have a munitions crisis. There just aren't enough shells. And this is something that you have to get a mix, as I say, of civilian bureaucrats, military bureaucrats. You have a change in the top of government to handle this issue. So I think you have issues across the board with all kinds of bureaucracies that when there isn't a feedback mechanism, things can go wrong. And then you have feedback mechanisms to assert themselves and people have to scramble to keep up and fix things. And so if we bring this back to the kind of issue of then how do you select, recruit, and not just actually even select and recruit, but like, what do you do once you've recruited your bureaucrats?

25:39How do you check they're kind of doing the right thing and are keeping the military ready for the eventual war or keeping whatever system they have got running? Obviously, you have to be thinking about actually exams do form a major part of this. And again, it might seem like exams are a really modern technology, but they're not. So as I say, in China, they go back centuries. And the reason why exams matter is because if you do want a way of selecting for the competent people who are going to be able to run these organisations well, an exam is not a bad option.

26:14And again, we tend to think of exams in the UK. People talk about Northcutt Trevelyan, 1854 as being a landmark. And people also talk about how exams for the army don't come in until the late 19th century. So there's a set of reforms in the late 19th century after Northcutt Trevelyan called the Wolsey reforms, which bring in more meritocratic examinations. And I don't know, I feel like for those of you who are keen on Gilbert and Sullivan, I think Gilbert and Sullivan are quite popular in the US. I'm a big fan. And there's that song, he is the very model of a modern major general that lists all the things the modern major general has to know.

26:50And that's a satire. So it's late 19th century Gilbert and Sullivan. That's a satire on the Wolsey reforms. So everyone, they're again quite famous and people will look at those and go, oh, so, you know, the army bring in kind of exams in the late 19th century. And the Napoleonic era, so the era before that, officers would buy commissions. So if you wanted to be rise in the army, you actually had to be quite wealthy and you purchased commissions. And that persisted in the army until the late 19th century. So people often say, oh, look, you know, the military has this very kind of unmeritocratic and patronage based system right up to the 19th century.

27:26But that is not the whole story. The army, for sure, has quite a patronage based system up to the late 19th century. But Britain, Britain, the major, the major sort of service in Britain is not the army. The major branch of the military in Britain at this time is the Navy. It's the Royal Navy. France is the great land power. Britain, England is the great sea power. And actually, for that really important service, which lays the foundation of so much of British power, they had exams for much, much longer.

27:59And the guy who introduced exams into the Navy way back in 1677. So almost, I think, over 200 years before they introduced them in the army. And the man who introduces them is a bureaucrat.

Samuel Pepys and Naval Exams

28:12And he's the famous, he's more famous now for his diaries. So he's Samuel Pepys. So Samuel Pepys, very famous now, not just for slightly naughty diaries, but also diaries that he wrote at the time of some great upheaval in London. So he gives us these really vivid first-hand accounts of the great fire of London. And I grew up a 10-minute walk from where Pepys lived and worked. And I actually have an office next door to where his offices were. So I'm very interested in Pepys.

28:43He's a completely fascinating figure. And as I say, he is better known now for these really compelling and vivid first-hand accounts of the great fire of London. And for, as I say, some quite X-rated bits in his diaries. But as a figure at that time, he is incredibly important. And in 1677, after a series of quite bad naval defeats to the Dutch, Pepys introduces oral exams for Royal Navy lieutenants. But because he is concerned about the fact that lieutenants are often well-bred gentlemen with good connections, but they don't really know much about running a ship.

29:21And actually running a ship is really quite hard. And at that point, there's a lot of naval innovation going on. And it's not something you can really just bluff your way through. So he introduces their oral examinations, and it's a panel of three senior officers who will question the candidate very closely on the practicalities of running a ship. And then what you also get come in later over the next few decades, century or whatever, is you also get quite a lot of written examinations come in. So if you look at some of the most famous characters, not actually just in British naval history, but arguably in all British history,

29:55people like Horatio Nelson, James Cook, people like Flinders, Vancouver, Bly, people who go around the world and chart maps of so many parts of the world, a lot of them have taken exams that were either the ones that really Pepys kind of designed or the ones that built on that foundation. So that kind of exam, that really practical examination, cartography in trigonometry, which you do need to be able to navigate in charting and map making,

30:30those really practical examinations are there from really quite an early stage in Britain and play a really central part in the development of one of its most powerful and competent institutions, the Royal Navy. Let's talk about, let's go back to the general idea of, of an exam and what, you know, in the case of the Royal Navy navigation, certain basic math skills. Of course, a lot of it's going to be how to deal with people, which is very hard to take, to have a test for.

31:04You can get recommendations from people who've seen you in action doing similar processes. But, you know, one of the big questions I think in education today is entry exams into college. A lot of American colleges dropped the SAT exam as a gatekeeper, feeling it was either unfair, racist, or ineffective. A lot of people disagreed with that. A lot of schools have brought that back. But I want you to talk about this general question of, of transferable skills.

The Macaulay Debate on Testing

31:38And we, I made fun of Macaulay earlier, but I want to, I want to defend him for a minute and get your reaction. So Macaulay argues that it doesn't matter what you put on the civil service exam, it just has to be hard. And of course, this is, now we would call this a signaling technique. We're going to force people to jump through a bunch of hoops. And it almost doesn't matter, says Macaulay, what the hoops are, because what we really want are people good at jumping. And since it's prestigious, it's going to attract the smart people.

32:11And the smartest ones will figure it out. The content's not so important. What's important is making sure the cream rises to the top. The best people do the jumping, and that's the case. Now, you make the point, undeniable, that, well, wouldn't you want them to learn something that was true? But I want to, and you can elaborate on that, but I want to defend Macaulay for a minute, suggesting that, oh, I don't, I'm not totally comfortable with it, but I'll try. That if you give a really hard exam about something that's really obscure, Sanskrit grammar or Thucydides, which Northcutt himself said was part of the reason he was successful.

32:47He had read Thucydides in Greek, and that helped him build up his brain, and he got smarter. It turns out Thucydides, by the way, a lot of what Thucydides wrote probably didn't happen. He exaggerated. He wasn't exactly accurate 100%. So that makes the argument. Through this difficult parsing of this hard text in a difficult language, you find out who the smartest people are. And that's all you really need. What's wrong with that argument?

33:14So we can view this as a debate through British history between Pepys and Macaulay. That Pepys said, we need an exam, and the exam needs to be in stuff that's useful. And Macaulay said, we need an exam, but it doesn't matter what the exam's on. We just want to find the super smart people, and then they can turn their hand to anything. And I actually just really want to actually read out what Macaulay says, because people might think we're creating a straw man here when we're saying this. And actually, I just want to point out that Macaulay made, he himself made something that sounds like a really extreme straw man argument.

33:50He made the argument, and every time I read it, I am just astonished. I cannot believe that. Macaulay is a big deal, and he's a big guy, and he's very famous. Great historian. Yeah, and he's an MP, he's giving this speech that I'm about to quote from in Parliament. After he gives this speech, another MP jumps up and says, this speech has never been exceeded within these walls for the development of statesmanlike policy and practical good sense. And actually, the nice thing about this is, people are a bit down on our politicians now. I read this speech and the responses it gets, and I think, oh, people must have been crazy, because this speech has no good sense to my mind.

34:27And the speech is mad. Before you read it, I want to let listeners know, I think I know what you're going to read, which is, there's a term in here called a wrangler. Yeah. And a wrangler is a person who's good at wrangling, meaning not animals, which is the American use of the term, but debating back and forth some topic. And there's also a term in here, the wooden spoon. And the wooden spoon, according to Claude, I looked it up, and you can verify it, is an award given to the worst person who passes.

35:01So, you know, the joke is, what do you call the person who finishes last in their medical school class? And the answer is doctor. And, but it turned out in England, it was called the wooden, you got the wooden spoon. It means, yeah, just kind of crossed the line. And we still use wooden spoon in sports now, if it's people finishing bottom of something. So it's your term in English. Okay. Yeah. Okay, so here we go. This is Macaulay. This is Macaulay's take on exams. This is 1833. Whatever be the languages, whatever be the sciences, which it is, in any age or country, the fashion to teach, the persons who become the greatest proficient in those languages and those sciences will generally be the flower of the youth, the most acute, the most industrious, the most ambitious of honourable distinctions.

35:48If the Ptolemaic system were taught at Cambridge instead of the Newtonian, the senior wrangler would nevertheless be in general a superior man to the wooden spoon. If, instead of learning Greek, we learned the Cherokee, the man who understood the Cherokee best, who made the most correct and melodious Cherokee verses, who comprehended most accurately the effect of the Cherokee particles, would generally be a superior man to him who was destitute of these accomplishments. If astrology were taught at our universities, the young man who cast nativities best would generally turn out a superior man.

36:22If alchemy were taught, the young man who showed most activity in the pursuit of the philosopher's stone would generally turn out a superior man. Okay, that's Macaulay done. Now we're back to me. This is just Macaulay getting carried away with his own eloquence. He is literally saying, I would rather people study things that are completely wrong that we know to be completely wrong, like the philosopher's stone. And that's fine, because if they study those and they do well at it, and they cast nativities the best, then, you know, they'll be the smartest guys.

36:54They'll be the hardest working, most industrious. So that's fine. Those are the people we want. What he seems to be completely missing here is that the stuff you study really matters. And that's why I think, go back to where we opened when we were talking about China in the 19th century, if you take the most industrious and hardest working and most ambitious youths of your country and in their most productive years of their life, teach them things that are wrong or just not that useful.

37:27What is going to happen is there will be other countries who take the same tranche of their students and teach them things that are true and useful. And then where are you going to be? Which country do you think is going to come out best? So I really just find it astonishing from Macaulay. I think it's just unbelievably misguided. So I'm going to defend him. Go on. And I can't resist the urge. Yeah.

37:54Well, I am going to resist that urge. I'll tell you later off air. But I'm going to defend Macaulay. Go on. Because behind Macaulay's view, so you're making one point, which is that if you want to incentivize people to master something, you should incentivize them to master something valuable, especially if it might take years, because they could then take that true knowledge rather than just having jumped through the hoops and signal that they're the smartest, they'd actually would have added, besides their raw intelligence, they would have added content that would make them more effective in their job.

38:26So I agree with you 100%. The real question for me, and this is a modern education question, which I know you have views on, isn't the real question, as you describe it, I think, in your essay, turning the brain into this razor-sharp tool? And it doesn't matter whether you study, I'll take a famous example in Jewish life, the Talmud, which is notoriously difficult. So you study the Talmud, and your brain gets more skilled. Okay, you may not agree with what's in the Talmud.

38:56You may think it's nonsense. It's just a myth or superstition. But the act of mastering it makes you, many people would say in Jewish life, it'll make you a better lawyer, because you've studied all these debates back and forth and these cases. And although the cases may not be relevant because you're talking about something that doesn't exist anymore in Jewish life, all that really matters is that you've honed your reasoning ability. And that's what education is about. It's about turning your brain into a powerful tool that you could then, or a searchlight,

39:29you could go across the landscape and turn it on to any topic and illuminate it, because you've got this great brain. What's wrong with that argument? Yeah, so what we are kicking around now, this argument we're having, people obviously were having the argument in the 19th century. So Macaulay did not have it all his own way in the 19th century. People were saying things like I'm saying at the time. And by the time you get to the late 19th, early 20th century, the people arguing back against the kind of just study hard stuff and it will make your brain sharp,

40:03there are people who are arguing back against this.

Abraham Flexner and Medical Training

40:06And one of the things that, and so then they developed basically some empirical evidence on this question, which we can take a look at now. And actually, one of the most important aspects of this debate is that in the late 19th, early 20th century, medicine is really developing as a science. And a lot of people are starting, you know, medicine is becoming more of a career and more of a career pathway. And yet a lot of top medical schools in the US and the UK, and this is very much at this point,

40:37a very big US debate too. A lot of people who want to become doctors are saying, well, actually, like, why do I have to qualify in Greek to get into medical school? Why do I have to qualify in Greek to become a doctor? And actually, is this not a bit of a waste of my time, given that medical knowledge is expanding at such a rate, that maybe my time would be better off spent doing that instead? And so I think one of the best statements of the anti-Macaulay argument comes from an American medical educator

41:09who was extraordinarily influential in early 20th century America. His name's Abraham Flexner. And he writes a fantastic article in the Atlantic magazine in 1917. And he takes on a lot of these arguments through, you know, from the point of view of medical training. And one of the things he says, which, you know, I think really counters that Macaulay argument, is he has this great line where he says, he says, I can compare the champions of mental discipline. So that's the Macaulay type people who think that it doesn't matter what you study,

41:43just matters that you build discipline. He says, I compare the champions of mental discipline to Egyptian priests who planted rows of dead sticks, which for disciplinary purposes, they watered regularly. Had they planted corn, they would have got the same discipline and something more. The corn, for example. So what he's saying is, you've got all these aspiring medics in early 20th century America, and you're saying to lots of them, you need to be studying Greek and Latin before you can look at a medical textbook. And he's saying medicine is hard enough.

42:14Medicine is tough enough in itself. Instead of getting our kids to study Greek and Latin, because that is hard stuff that trains their mind, we could just get them to study the hard stuff of medicine that is useful. So he is a really important figure. And actually, I would say, in practical terms, kind of carries the day in terms of being very influential in the establishment of medical schools in America. And the other crucial figure who lays down a lot of empirical evidence on this point is another American, E.L. Thorndike, who's a pioneer of experimental psychology.

42:45And again, Thorndike is listening to these arguments and sort of thinking, well, can we bring any data to bear? Like, can we do some research? And he lays the foundation for a lot of later experimental psychology. And he does these very pioneering studies on whether skills do transfer across domains. So essentially, the Macaulay argument, if you like, is it doesn't matter what you study. You just need to hone your brain to a razor point sharpness. And in Macaulay's idea, the hardest thing you can study is ancient Greek.

43:17Like, so if you can translate the Melian dialogue, you can kind of basically do anything. Like, it's not just ancient Greek. The hardest bit of ancient Greek is Thucydides. So people are like, once you can translate Thucydides, you can do anything.

43:31And what Thorndike is saying is, well, let's test this. He comes up with a couple of ways of testing it. So one of the things is he looks at the kind of students who are doing the classical languages at school. And he says, well, do they do better on all their other subjects? So that's just a kind of more of a statistical survey. And he basically finds once you control for prior attainment, they don't. The second type of experiments he do, which again have been very influential, is he trains people up in doing one skill really well.

44:01And then he gives them another task to do that's similar but different. And he sees, well, does it transfer across? So, you know, a simple example, but he gives the experimental subjects practice in estimating the shapes of rectangles. And they get a lot of practice at that and they get better at that. And then he says, does that make you better at estimating the shapes of triangles? You mean the area? Yeah, I think the area, estimating the area. So he gives experimental subjects practice estimating the area of rectangles.

44:37And they get better at that and they have the practice and they improve. And then he says, will that transfer over to improvements in estimating the area of triangles? And he finds, no, it doesn't. And even on that very simple, quite close allied skill, it doesn't. And again, I think if we go back to Flexner, are we really saying that if you can translate the Melian dialogue, that is going to make you a really fantastic surgeon? Like, really? So, with your Talmud point, where you say, oh, doing the Talmud makes you a better lawyer.

45:10This is where you get some interesting things about transfer, which in the modern literature would be called far transfer and near transfer. So, near transfer would be when you're transferring, and Thorndike comes up with a term, he talks about where there are identical elements. So, where there are identical elements in the skill you've studied and the thing you are transferring to, then you can have some transfer take place. But when there is not really that much in common, and that's in the modern literature you would say far transfer, it just kind of breaks down. So, I can see that, obviously, the Talmud and indeed ancient Greek, there are obviously some near transfer things that you're going to take from that that would be useful in everyday life.

45:51But are you going to be able to far transfer to lots of other things? I don't think so. So, if it's really important to you to have trained doctors and trained medics and people who understand medical science and you want to train and design a training program for them, or in Peeps' example, if it's really important for you to have trained navigators on board a ship, then you should be coming up with a training program that is quite closely allied to the final skill that it is you want to see. And you should be trying to make sure there are ways that there is that identical elements and that where there is transfer needed, it's a near transfer and you can test that and you can see that it's happening.

46:33So, let me, you know, deep down I agree with a lot of that, but I want to push back on it a little bit, which is I've been fortunate to meet some very high IQ people in my life.

Domain Specificity in Education

46:49And some of them are only good at one thing, but a lot of them are good at a lot of things. Now, I think the relevant point would be, well, they're not just good at those things, they know a lot about them. And those are two very different things. They have a base knowledge of context and raw fact and information that's crucial for allowing that well-honed, razor-sharp brain to do something. And I think when we think about, and now let's move away from civil service and Royal Navy, when we think about what is the purpose of education, let's start with, you know, K through 12 or college.

47:32You know, I like to believe that what I do here at Shalem College is, quote, help my students learn how to think. Now, you could make fun of that and say, well, but that's an amorphous skill that doesn't transfer to lots of things. And yet I think when our students deal with, say, the Iliad or the Odyssey or Plato or Aristotle, they don't read them in the original languages. They usually read them in translation, but they learn some actually, some of the Greek.

48:03I think they get good at understanding things, and they get good at reading, and they get good at raising questions, interpreting text. And those are useful skills, and they do transfer to other things. Maybe they're limited. But I'm curious, given your Thorndikeian pessimism about transferability, if that's the case, should there be anything other than trade schools that teach people expertise in narrow areas where expertise is crucial?

48:36No, so look, so I definitely don't want to say that. I'm definitely not making the argument. I know you don't, Jason. I was just pushing into a corner there. You're right. You're right. And I would agree, I think, with a lot of what you're doing at Shalem College. I think the point I would make is that if you are setting out with a goal to get students to think well, I think that's a legitimate goal. But you have to acknowledge that being able to think well depends on main knowledge. It depends on specific knowledge of a particular domain. So, for example, literature.

49:06I do think literature is a completely legitimate topic of study. And what I would say is if you are studying ancient literature, either in translation or in the original language, if you read a lot of literary texts, you will come to one that you have never read before that's unseen. And you can have a really good go at understanding it, even if you haven't seen it before, because you've got the kind of near transfer from all of the other knowledge of literary texts. And I think that kind of thing is important.

49:39So what I would say is, is that whatever it is we are trying to teach, whether it is literature, whether it is navigation, whether it is good administration, whether it is how to build munitions shells, like whatever the thing is we want people to have skills in. And what we have to understand is those skills are domain specific. And I think what has been neglected in a lot of modern education is the domain specificity of skills.

50:11And in modern education, so if we complete the kind of historical tour of where this is going, in the late 19th, early 20th century, the argument for knowledge doesn't matter. And it's just the kind of create a razor sharp brain. And that school of thought is known as the faculty psychology or mental discipline. And it's known as that in both the US and the UK, and it's influential. And then you have people, as I say, like Thorndike and Flexner come along and say, hang on a minute. And I think what you've had since then in the sort of post-war era and coming up to today is a lot more research on what experts,

50:51what kind of training they do and what kind of skills they have. And again, you've had a lot more research on how even moving slightly outside your domain can really challenge you and your skills can break down. And some of the most famous research was done with chess players by Herbert Simon, who is, again, another American Nobel laureate in economics. He did some really famous research on chess players where he showed that if you show them a chessboard with a bunch of pieces for five seconds, take it away. If those pieces are drawn from an actual game, like a genuine game scenario, their memory is insanely good, right?

51:29They look at it, a couple of seconds, bang, they've got it. They can reproduce it almost perfectly. Chess players, serious chess players. Yeah, really good chess players, chess grandmasters. If you give them a chessboard with 25 pieces, you give them a couple of seconds to look at it, ask them to reproduce it. They're insanely good. But if you give them the exact same task with one small difference, so the pieces are not taken from a chess game, they're just put at random on the board, they cannot do it. So, again, this is disproving that general mental tool idea of mental skill.

52:06Because why is it they can be so good at 25 pieces from an actual chess game, but terrible at 25 pieces placed at random? They don't have, like, a generally good memory skill. What they have stored in long-term memory is a schema of thousands of chess games that they've seen where they know the kind of typical patterns that you would get. And that's what makes it so easy for them to remember and to perform these apparently superhuman feats of memory. And, again, that's a really seminal study, Simon and Chase, skill in chess, been replicated with all kinds of other experts.

52:45And the thing I always say is, you know, people talk about critical thinking as being this all-purpose transferable skill. I mean, the thing I always like to joke about is you can think of a physics professor who's really good at critical thinking within the context of solving a physics problem and really poor at critical thinking in the context of solving a problem in his marriage. We all know that person, right? So, the idea that these skills, you know, you can have this very generic skill like problem solving or critical thinking, it's just not there. One other paper I put in that a lot of Americans have been putting in here, there's a lot of really good American researchers here.

53:19One of my favourite ones on creativity has a little American sporting reference for you in the title. It's by a leading researcher in creativity, James Kaufman, and it's called Could Steven Spielberg Have Managed the Yankees? It's a very good question and he concludes no. Because he concludes that, again, Steven Spielberg's creativity is built up within a specific domain of filmmaking. And when you jump to a very different domain that would require far transfer, like to managing the Yankees, sure, you need creativity to manage the Yankees, but it's just really different knowledge.

53:53The creativity is within a completely different domain. So, and the other one last name check is, the guy I think has done so much work on this is completely brilliant. It's Professor Daniel Willingham at University of Virginia. He's been on the programme talking about some of these issues. We'll link to that episode. Absolutely brilliant. He's been enormously influential for a lot of people here in the UK and he just has brilliant, brilliant, very teacher-friendly articles about how all of this research should manifest itself in terms of how you design a curriculum and assessment. So, I would say the empirical argument, you know, we're arguing back with Macaulay, you know, arguing with him in the grave.

54:29The empirical argument since Macaulay is just completely, I would say, a slam dunk against his argument. Skills are domain specific. It matters what you teach. I'm glad you got slam dunk in there for the Americas. I love it. I think a simpler test, I haven't read the Kauffman article, but it's famously the case that successful coaches in college basketball do not become successful pro coaches and vice versa.

55:00So, successful pro coaches often struggle in college, although, you know, there are obviously exceptions in, and there are other sports as well besides basketball. Like, how far is that? That seems pretty close. And yet, there are some specific skills in there they just don't have or have never honed. And the general skill actually is not so helpful. I want to close with a different, what some might think of as different, but I actually think of as quite related.

Skepticism, Irony, and Cynicism

55:28And I heard you talk about it a little bit recently. Recently, we were in England at this conference together, and it fascinates me. Let me set it up this way. One could argue, I don't believe this, but many people think this is the case, that the goal of education is to encourage skepticism. And you could argue that a lot of econ talk is about skepticism. You know, someone says, studies show about chess players, but, you know, hey, there's publication bias and replicability problems, and maybe that Herbert Simon thing isn't true.

56:02And so, I'm going to be skeptical about it until I get to look closely and see if it was well done. And so, you know, that honing of the brain to be skeptical, one might argue, is a very important part of education. Don't just believe everything you hear. And you could argue that there's not many things more important in education than being skeptical of what you're told. And to make sure that the things that you're told or that you come to believe have real foundations and evidence behind them and so on.

56:34And I struggle with the fact, so I have a part of me that's very much in that direction, in that domain. And yet, at the same time, push too far, that kind of skepticism can lead to what you would call cynicism. Nothing is true. Everything's relative. There are no ultimate values. There's no ultimate truth. You can never know. And therefore, you should just be cynical about everything, not just skeptical. And in my experience, that leads to nihilism, which is not a healthy thing, in my view.

57:07But I heard you say at this conference we were at that very few great teachers, and it's just a casual empirical observation on your part, are cynics. Did I understand you correctly? And then react to what I said. Yeah, so what I said was that every great headteacher I've met, and I work in education, met quite a lot of headteachers, I would say, again, it's completely anecdotal. Is this a, when you say headteachers, do you mean principals in American parlance?

57:37Yeah, sorry, principals. So I would say every great principal I've met, and again, this is totally anecdotal, no data behind this. Maybe I'm doing a Macaulay and going ahead of myself and getting convinced by my eloquence. But I would say every great headteacher I've met doesn't really have a sense of irony. They can have a sense of humour, they can be funny and be witty, but they don't have that sense of irony. And what I mean by irony is that sense of undercutting, of that sense of seeing something, yeah, maybe it is related to scepticism.

58:12To cynicism, I think. It's related to scepticism and cynicism. So I think irony goes too far and it becomes cynicism. And I think the reason for this is a headteacher is setting rules and they can't be undercutting the rules. They have to believe in the rules. And I think I said to you too, when I started out as a teacher, I did an English literature degree, so I need to put my cards on the table. I've spent this whole podcast talking to you about how the study of ancient literature is a terrible idea. And it's basically been my entire academic field of study.

58:44So, yeah, you make of that what you will. But a big part of the study of English literature is irony is a huge part of it. And a lot of the English authors I would say we most admire are ones who are masters of irony. I mean, Jane Austen leaps to mind, but there are plenty of others. And I remember, you know, I came out of education. I came straight out of my degree and was, you know, graduated in the summer and by the autumn was teaching. And it's really hard to sustain a sense of irony in the classroom with teenagers because you have to kind of enforce these rules and believe in them.

59:18And if you don't, and if you're the one who's constantly undercutting, it just gets really difficult. And so I think you can probably as a teacher just about sustain it. But I think it gets harder if, as I say, you're a principal and you're at the top of the tree and you are responsible for upholding these rules. And so I think there is a kind of an issue that a lot of the qualities and in lots of ways I do admire these great masters of irony like Jane Austen. But there are a lot of ways in which it's kind of hard to really construct a coherent worldview on that.

59:49Because as you say, one of the great issues with scepticism is it undercuts itself. And I think the great tension here is that scepticism and rationality have historically given us enormous benefits. So modern science is incredible when it's to a large part built on, you know, not trusting and verifying. Prove it, yeah. But on the other hand, if you just literally believe in nothing and you are trying to prove everything, you will just end up in a reductio ad absurdum. And you will end up really in something, and this again, we're talking about peeps, but you can go right back to the 17th century here.

1:00:24You will end up in a kind of antinomian state of mind where you just trust nobody and nothing. And I think this goes to the heart, the Royal Society, the Royal Society's motto, and peeps was involved, I think, in setting up the Royal Society. The Royal Society's motto is nullius in verba, take no one's word for it. And on the one hand, yeah, fantastic, that's amazing. And the way the Royal Society did that did result in some of the most amazing advances that we all, you know, the entire world have benefited from. On the other hand, actually, it just doesn't work. You can't, you know, to absolutely definitively prove even one of the most basic facts that you take for granted in your life could take a lifetime.

1:01:02So ultimately, we are all taking something on someone else's word. You can't not. But this is the tension. You know, you go too far one way and you're credulous and you just become gullible and you fall for scams and cons. But you go too far the other way and you just undercut yourself and, as I say, become antinomian and the ground beneath you falls away. So there has to be some kind of place that you can find where you do find some secure ground to put your foot on.

But when you find that, I think you do really kind of have to defend it a bit. It's a tricky one. Yeah, I think I first read it in David Epstein's book where he talks about NASA, which, by the way, is another government agency that very much prioritized skill, particular skills of astrophysics and mathematics wisely to keep people alive, but fails from time to time, as we know.

1:01:55And David writes, I think it's where I saw it, on the wall, it says, in God we trust, everyone else bring data. And it's a beautiful idea. And in a certain sense, you want to live your life that way, requiring, you know, take no one's word, prove it. Where's the data? Where's the evidence? And yet, there's a dance as a human being where not just the practicality part, that you can't prove everything and you have to accept some things on faith, but I think there's something more important, which is some things aren't amenable to proof.

1:02:30Some of those are in marriage that you were talking about earlier, how you deal with people, whether it's your spouse or friend or a colleague at work. And this is related for me. I don't know if it makes sense, but I think about my youth, I was brought up on certain types of humor, and they were all ironic. I was, I think, 12 years old in the heyday of Marvel Comics when Spider-Man and the Fantastic Four and others were coming out of the pen and mind of Stan Lee and Jack Kirby and others.

1:03:09And they liked, they were always ironic. Their sense of humor was always about, they were always wisecracking. They were always putting things down and being hip. And I think that's a huge part of American culture. I was part of that as a kid.

1:03:30The humor, I was a big fan of the Firesign Theater, which was this set of crazy recordings when I was in high school of super ironic stuff. And I think, I don't know if it's healthy. I mean, I love my sense of humor. I'm totally happy with it. And I passed, I hope, some of it on to my children through, but part of what they watched was the Andy Griffith show, which I'll just mention is incredibly funny, but also incredibly charming. And there's zero irony in it.

1:04:02It's good-hearted in its nature. It's virtuous in its nature. There's no attempt to show that you're cooler than the people who keep the rules. But all that other stuff's about breaking the rules. And it's a fascinating issue. I think you said it well about, you know, you need a place to stand. And if you're only going to stand on places where you know it's safe, you're in trouble. So I don't know what else to say about it. You can take us home if you want to add anything.

Earnestness and Building Institutions

1:04:31Yeah, I think I agree with that. And I think the other thing we were chatting about when we spoke in person was to what extent maybe I get your point. There's an ironic strain in American culture. You know, America is large. It contains multitudes. But it does feel like maybe there's a strain of earnestness in American culture that often maybe British people maybe sometimes like to mock. Yeah. And I've probably been guilty of that. But then you look at it and you think that earnestness does have its strengths. And as you say, there is a point at which the irony can shade into cynicism and the cynicism can be something that can really stop you getting things done.

1:05:11If you just are always seeing the bad side of things. So there has to be some kind of value to earnestness as a way of, you know, if you want to achieve things and you want to build things, you can't always be knocking things down. You have to want to try to build. And I wonder if some element of earnestness is necessary for being a builder and not a destroyer.

1:05:36My guest today has been Daisy Christodoulou.

Closing and Credits

1:05:39Daisy, thanks for being part of EconTalk. Fantastic. Thank you, Ross. This is EconTalk, part of the Library of Economics and Liberty. For more EconTalk, go to econtalk.org, where you can also comment on today's podcast and find links and readings related to today's conversation. The sound engineer for EconTalk is Rich Goyette. I'm your host, Russ Roberts. Thanks for listening. Talk to you on Monday. Thank you.

1:06:13Bye.