Benchmark Bank Heist

Linear Digressions by Ben Jaffe and Katie Malone

Apr 6, 202600:12:36Technology

What if an AI decided the smartest way to pass its test was to find the answer key? That's exactly what Anthropic's Claude Opus did when faced with a benchmark evaluation — reasoning that it was being tested, tracking do...

Related Episodes

Continue listening to more episodes from Linear Digressions.

Constitutional AI

Sep 7, 2026 - 00:31:57

How do you teach a model the difference between helpful and harmful when it has no inherent sense of eith...

A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College)

Aug 31, 2026 - 00:40:28

Tom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a re...

Understanding AI Text Watermarking

Aug 24, 2026 - 00:29:57

Anthropic just announced they're baking invisible watermarks directly into Claude's generated text — and...

Better Know a Benchmark: Humanity's Last Exam

Aug 17, 2026 - 00:23:21

Humanity's Last Exam was designed with a bold premise: questions that human experts can answer, but AI mo...

A Scientific Deep Dive into Overconfident LLMs: Interview with Kaitlyn Zhou (Cornell)

Aug 10, 2026 - 00:33:54

When a language model tells you it's absolutely certain, is it actually more likely to be right? Kaitlyn...

Reasoning Models: When LLMs Went Beyond Fancy Autocomplete

Aug 3, 2026 - 00:25:00

Reasoning models don't just answer your question — they *think out loud* first. In this episode we dig in...

Distillation, or, How to Steal a Model

Jul 27, 2026 - 00:23:38

This week we’re covering model distillation: the technique of using a large "teacher" model's outputs to...

Invisible LLM Failures and AI Fluency with Chris Potts (Stanford)

Jul 20, 2026 - 00:41:23

What happens when a Stanford linguistics professor turns his attention to AI chatbots — and the surprisin...