How Do You Evaluate An AI Agent? (The Agents Season, Episode 7)

Linear Digressions by Ben Jaffe and Katie Malone

Jun 1, 202600:31:45Technology

Knowing when an AI agent has failed sounds straightforward — until it isn't. Agents have a frustrating habit of finishing confidently while quietly doing the wrong thing, or looping endlessly without ever crashing in an...

Related Episodes

Continue listening to more episodes from Linear Digressions.

Mechanistic Interpretability: How Researchers Try To See Inside Models

Oct 5, 2026 - 00:29:21

As AI systems grow more powerful, they grow harder to understand — and that tradeoff has never mattered m...

LLM As A Judge

Sep 28, 2026 - 00:30:12

Labeled data is expensive, slow, and painfully hard to come by — and when you're trying to evaluate wheth...

The Impact of AI on Podcasting (Harvard Data Science Review Cross-Post)

Sep 21, 2026 - 00:29:38

Originally aired on the Harvard Data Science Review podcast. What can podcasting teach us about AI — and...

Better Know A Benchmark: ExploitGym

Sep 14, 2026 - 00:32:34

When OpenAI's frontier models were caught hacking Hugging Face's servers, most people assumed they were h...

Constitutional AI

Sep 7, 2026 - 00:31:57

How do you teach a model the difference between helpful and harmful when it has no inherent sense of eith...

A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College)

Aug 31, 2026 - 00:40:28

Tom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a re...

Understanding AI Text Watermarking

Aug 24, 2026 - 00:29:57

Anthropic just announced they're baking invisible watermarks directly into Claude's generated text — and...

Better Know a Benchmark: Humanity's Last Exam

Aug 17, 2026 - 00:23:21

Humanity's Last Exam was designed with a bold premise: questions that human experts can answer, but AI mo...