119. Jaime Sevilla - Projecting AI progress from compute trends

There’s an idea in machine learning that most of the progress we see in AI doesn’t come from new algorithms of model architectures. instead, some argue, progress almost entirely comes from scaling up compute power, datasets and model sizes — and besides those three ingredients, nothing else really matters. Through that lens the history of AI becomes the history f processing power and compute budgets. And if that turns out to be true, then we might be able to do a decent job of predicting AI progress by studying trends in compute power and their impact on AI development. And that’s why I wanted to talk to Jaime Sevilla, an independent researcher and AI forecaster, and affiliate researcher at Cambridge University’s Centre for the Study of Existential Risk, where he works on technological forecasting and understanding trends in AI in particular. His work’s been cited in a lot of cool places, including Our World In Data, who used his team’s data to put together an exposé on trends in compute . Jaime joined me to talk about compute trends and AI forecasting on this episode of the TDS podcast. *** Intro music : - Artist: Ron Gelinas - Track Title: Daybreak Chill Blend (original mix) - Link to Track: *** Chapters: 2:00 Trends in compute 4:30 Transformative AI 13:00 Industrial applications 19:00 GPT-3 and scaling 25:00 The two papers 33:00 Biological anchors 39:00 Timing of projects 43:00 The trade-off 47:45 Wrap-up

You can listen to 119. Jaime Sevilla - Projecting AI progress from compute trends online on Radio and Podcast. Open the player on this page to stream the available audio.

119. Jaime Sevilla - Projecting AI progress from compute trends is an episode from Towards Data Science by The TDS team .

This episode is 00:48:34 long.

This episode was published on Apr 13, 2022.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from Towards Data Science when more episodes are available from the podcast feed.