Why AI Evaluation Science Can't Keep Up (with Carina Prunkl)

Carina Prunkl is a researcher at Inria. She joins the podcast to discuss how to assess the capabilities and risks of general-purpose AI. We examine why systems can solve hard coding and math problems yet still fail at simple tasks, why pre-deployment tests often miss real-world behavior, and how faster capability gains can increase misuse risks. The conversation also covers de-skilling, red teaming, layered safeguards, and warning signs that AIs might undermine oversight. LINKS: Carina Prunkl personal website CHAPTERS: (00:00) Episode Preview (01:04) Introducing the report (02:10) Jagged frontier capabilities (05:29) Formal reasoning progress (12:36) Risks and evaluation science (19:00) Funding evaluation capacity (24:03) Autonomy and de-skilling (31:32) Authenticity and AI companions (41:00) Defense in depth methods (48:34) Loss of control risks (53:16) Where to read report PRODUCED BY: SOCIAL LINKS: Website: Twitter (FLI): Twitter (Gus): LinkedIn: YouTube: Apple: Spotify:

You can listen to Why AI Evaluation Science Can't Keep Up (with Carina Prunkl) online on Radio and Podcast. Open the player on this page to stream the available audio.

Why AI Evaluation Science Can't Keep Up (with Carina Prunkl) is an episode from Future of Life Institute Podcast by Gus Docker.

This episode is 54:23 long.

This episode was published on Apr 17, 2026.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from Future of Life Institute Podcast when more episodes are available from the podcast feed.