Distillation, or, How to Steal a Model

Linear Digressions by Ben Jaffe and Katie Malone

Jul 27, 202600:23:38Technology

This week we’re covering model distillation: the technique of using a large "teacher" model's outputs to train a smaller, cheaper "student" model that mimics it. They cover the two big reasons labs do this — making light...

Related Episodes

Continue listening to more episodes from Linear Digressions.

Mechanistic Interpretability: How Researchers Try To See Inside Models

Oct 5, 2026 - 00:29:21

As AI systems grow more powerful, they grow harder to understand — and that tradeoff has never mattered m...

LLM As A Judge

Sep 28, 2026 - 00:30:12

Labeled data is expensive, slow, and painfully hard to come by — and when you're trying to evaluate wheth...

The Impact of AI on Podcasting (Harvard Data Science Review Cross-Post)

Sep 21, 2026 - 00:29:38

Originally aired on the Harvard Data Science Review podcast. What can podcasting teach us about AI — and...

Better Know A Benchmark: ExploitGym

Sep 14, 2026 - 00:32:34

When OpenAI's frontier models were caught hacking Hugging Face's servers, most people assumed they were h...

Constitutional AI

Sep 7, 2026 - 00:31:57

How do you teach a model the difference between helpful and harmful when it has no inherent sense of eith...

A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College)

Aug 31, 2026 - 00:40:28

Tom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a re...

Understanding AI Text Watermarking

Aug 24, 2026 - 00:29:57

Anthropic just announced they're baking invisible watermarks directly into Claude's generated text — and...

Better Know a Benchmark: Humanity's Last Exam

Aug 17, 2026 - 00:23:21

Humanity's Last Exam was designed with a bold premise: questions that human experts can answer, but AI mo...