120. Liam Fedus and Barrett Zoph - AI scaling with mixture of expert models

Towards Data Science by The TDS team

Apr 20, 202200:40:47Technology

AI scaling has really taken off. Ever since GPT-3 came out, it’s become clear that one of the things we’ll need to do to move beyond narrow AI and towards more generally intelligent systems is going to be to massively sc...

Related Episodes

Continue listening to more episodes from Towards Data Science.

130. Edouard Harris - New Research: Advanced AI may tend to seek power by default

Oct 12, 2022 - 00:58:22

Progress in AI has been accelerating dramatically in recent years, and even months. It seems like every o...

129. Amber Teng - Building apps with a new generation of language models

Oct 5, 2022 - 00:51:21

It’s no secret that a new generation of powerful and highly scaled language models is taking the world by...

128. David Hirko - AI observability and data as a cybersecurity weakness

Sep 28, 2022 - 00:49:02

Imagine you’re a big hedge fund, and you want to go out and buy yourself some data. Data is really valuab...

127. Matthew Stewart - The emerging world of ML sensors

Sep 21, 2022 - 00:41:34

Today, we live in the era of AI scaling. It seems like everywhere you look people are pushing to make lar...

126. JR King - Does the brain run on deep learning?

Sep 14, 2022 - 00:55:43

Deep learning models — transformers in particular — are defining the cutting edge of AI today. They’re ba...

125. Ryan Fedasiuk - Can the U.S. and China collaborate on AI safety?

Sep 7, 2022 - 00:48:19

It’s no secret that the US and China are geopolitical rivals. And it’s also no secret that that rivalry e...

124. Alex Watson - Synthetic data could change everything

May 18, 2022 - 00:51:47

There’s a website called thispersondoesnotexist.com . When you visit it, you’re confronted by a high-reso...

123. Ala Shaabana and Jacob Steeves - AI on the blockchain (it actually might just make sense)

May 12, 2022 - 00:54:43

Two ML researchers with world-class pedigrees who decided to build a company that puts AI on the blockcha...