How Do You Evaluate An AI Agent? (The Agents Season, Episode 7)

Linear Digressions by Ben Jaffe and Katie Malone

Jun 1, 202600:31:45Technology

Knowing when an AI agent has failed sounds straightforward — until it isn't. Agents have a frustrating habit of finishing confidently while quietly doing the wrong thing, or looping endlessly without ever crashing in an...

Related Episodes

Continue listening to more episodes from Linear Digressions.

Constitutional AI

Sep 7, 2026 - 00:31:57

How do you teach a model the difference between helpful and harmful when it has no inherent sense of eith...

A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College)

Aug 31, 2026 - 00:40:28

Tom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a re...

Understanding AI Text Watermarking

Aug 24, 2026 - 00:29:57

Anthropic just announced they're baking invisible watermarks directly into Claude's generated text — and...

Better Know a Benchmark: Humanity's Last Exam

Aug 17, 2026 - 00:23:21

Humanity's Last Exam was designed with a bold premise: questions that human experts can answer, but AI mo...

A Scientific Deep Dive into Overconfident LLMs: Interview with Kaitlyn Zhou (Cornell)

Aug 10, 2026 - 00:33:54

When a language model tells you it's absolutely certain, is it actually more likely to be right? Kaitlyn...

Reasoning Models: When LLMs Went Beyond Fancy Autocomplete

Aug 3, 2026 - 00:25:00

Reasoning models don't just answer your question — they *think out loud* first. In this episode we dig in...

Distillation, or, How to Steal a Model

Jul 27, 2026 - 00:23:38

This week we’re covering model distillation: the technique of using a large "teacher" model's outputs to...

Invisible LLM Failures and AI Fluency with Chris Potts (Stanford)

Jul 20, 2026 - 00:41:23

What happens when a Stanford linguistics professor turns his attention to AI chatbots — and the surprisin...