The Pre-Training Wall and the Treadmill After It

CoRecursive: Coding Stories by Adam Gordon Bell - Software Developer

May 9, 202656:10Technology

I've been confusing Don with frontier-lab links late at night for a bit. Ilya Sutskever told a NeurIPS audience that pre-training as we know it would unquestionably end. There's only one internet, and the data isn't grow...

Related Episodes

Continue listening to more episodes from CoRecursive: Coding Stories.

The Hutter Prize: Compression, prediction, and the limits of intelligence

Aug 4, 2026 - 52:54

I texted Don about a side gig: €5,000 for every 1% you shave off a zip file. It's a real contest. For 20...

The Bitter Lesson: The history of reinforcement learning

Jun 13, 2026 - 01:00:01

I've been trying to understand how machine learning actually works. Not use it, understand it, down to th...

Story: The Aging Programmer

Apr 2, 2026 - 41:52

Kate Gregory has been writing C++ for over forty years. Books, keynotes, a consulting firm she built from...

From Hacker News to TikTok - How Algorithms Learned to Hook Us

Mar 2, 2026 - 41:32

Corey told me about his AI cat reel problem. He found these AI-genearted cat videos hilarious. Who makes...

Notes: The Universal Paperclip Clicker

Feb 4, 2026 - 11:05

Multiple VS Code windows. "Agent stopping" in a robot voice. A laptop stand on the treadmill so Claude ca...

Story: Inside Early Google - Race Conditions, Java Pain, and the Birth of AdWords

Jan 2, 2026 - 37:40

Ron Garret left JPL for a 100-person startup he'd just discovered on Usenet. Four a.m. alarms. Burbank to...

Story: The Bug He Couldn't Name - A 15-Year Fight Inside One Developer's Mind

Dec 2, 2025 - 44:27

Imagine facing a problem you can't name, something that feels bigger than any bug you've ever had to fix....

Story: Godbolt's Rule - When Abstractions Fail

Nov 4, 2025 - 44:13

What do you do when your code breaks and the only fix is to dig into the runtime below? Matt Godbolt live...