The Startup Powering The Data Behind AGI

In this episode of Gradient Dissent, Lukas Biewald talks with the CEO & founder of Surge AI, the billion-dollar company quietly powering the next generation of frontier LLMs. They discuss Surge's origin story, why traditional data labeling is broken, and how their research-focused approach is reshaping how models are trained. You’ll hear why inter-annotator agreement fails in high-complexity tasks like poetry and math, why synthetic data is often overrated, and how Surge builds rich RL environments to stress-test agentic reasoning. They also go deep on what kinds of data will be critical to future progress in AI—from scientific discovery to multimodal reasoning and personalized alignment. It’s a rare, behind-the-scenes look into the world of high-quality data generation at scale—straight from the team most frontier labs trust to get it right. Timestamps: 00:00 – Intro: Who is Edwin Chen? 03:40 – The problem with early data labeling systems 06:20 – Search ranking, clickbait, and product principles 10:05 – Why Surge focused on high-skill, high-quality labeling 13:50 – From Craigslist workers to a billion-dollar business 16:40 – Scaling without funding and avoiding Silicon Valley status games 21:15 – Why most human data platforms lack real tech 25:05 – Detecting cheaters, liars, and low-quality labelers 28:30 – Why inter-annotator agreement is a flawed metric 32:15 – What makes a great poem? Not checkboxes 36:40 – Measuring subjective quality rigorously 40:00 – What types of data are becoming more important 44:15 – Scientific collaboration and frontier research data 47:00 – Multimodal data, Argentinian coding, and hyper-specificity 50:10 – What's wrong with LMSYS and benchmark hacking 53:20 – Personalization and taste in model behavior 56:00 – Synthetic data vs. high-quality human data Follow Weights & Biases:

You can listen to The Startup Powering The Data Behind AGI online on Radio and Podcast. Open the player on this page to stream the available audio.

The Startup Powering The Data Behind AGI is an episode from Gradient Dissent by Lukas Biewald.

This episode is 56:15 long.

This episode was published on Sep 16, 2025.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from Gradient Dissent when more episodes are available from the podcast feed.