Dwarkesh Podcast
Reiner Pope – The math behind how LLMs are trained and served
April 29, 20262h 13m · 22,975 words
Show notes
Did a very different format with Reiner Pope - a blackboard lecture where he walks through how frontier LLMs are trained and served. It’s shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk. It’s a bit technical, but I encourage you to hang in there – it’s really worth it.
Highlighted moments
if you do not batch together many users, the cost and the economics you get can be like a thousand times worse than if you do batch many two users together.
“The fact that they are charging 5x less for pre-fill than decode does suggest that they are bottlenecked on memory bandwidth to quite a degree”
Transcript
Transcript not available for this episode yet.
More from Dwarkesh Podcast
Adam Brown – A deep but accessible introduction to general relativity
Jul 10, 20261h 38m
Grant Sanderson – AI and the future of math
Jun 30, 20261h 33m
The next big breakthrough will be AIs learning on the job
Jun 26, 202619 min
The data black hole at the center of AI
Jun 19, 202611 min
Ada Palmer – Machiavelli is the most misunderstood thinker of all time
Jun 16, 20262h 8m