Benchmarking AI Models

Linear Digressions by Ben Jaffe and Katie Malone

Mar 30, 202600:29:55Technology

How do you know if a new AI model is actually better than the last one? It turns out answering that question is a lot messier than it sounds. This week we dig into the world of LLM benchmarks — the standardized tests use...

Related Episodes

Continue listening to more episodes from Linear Digressions.

Constitutional AI

Sep 7, 2026 - 00:31:57

How do you teach a model the difference between helpful and harmful when it has no inherent sense of eith...

A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College)

Aug 31, 2026 - 00:40:28

Tom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a re...

Understanding AI Text Watermarking

Aug 24, 2026 - 00:29:57

Anthropic just announced they're baking invisible watermarks directly into Claude's generated text — and...

Better Know a Benchmark: Humanity's Last Exam

Aug 17, 2026 - 00:23:21

Humanity's Last Exam was designed with a bold premise: questions that human experts can answer, but AI mo...

A Scientific Deep Dive into Overconfident LLMs: Interview with Kaitlyn Zhou (Cornell)

Aug 10, 2026 - 00:33:54

When a language model tells you it's absolutely certain, is it actually more likely to be right? Kaitlyn...

Reasoning Models: When LLMs Went Beyond Fancy Autocomplete

Aug 3, 2026 - 00:25:00

Reasoning models don't just answer your question — they *think out loud* first. In this episode we dig in...

Distillation, or, How to Steal a Model

Jul 27, 2026 - 00:23:38

This week we’re covering model distillation: the technique of using a large "teacher" model's outputs to...

Invisible LLM Failures and AI Fluency with Chris Potts (Stanford)

Jul 20, 2026 - 00:41:23

What happens when a Stanford linguistics professor turns his attention to AI chatbots — and the surprisin...