121. Alexei Baevski - data2vec and the future of multimodal learning

If the name data2vec sounds familiar, that’s probably because it made quite a splash on social and even traditional media when it came out, about two months ago. It’s an important entry in what is now a growing list of strategies that are focused on creating individual machine learning architectures that handle many different data types, like text, image and speech. Most self-supervised learning techniques involve getting a model to take some input data (say, an image or a piece of text) and mask out certain components of those inputs (say by blacking out pixels or words) in order to get the models to predict those masked out components. That “filling in the blanks” task is hard enough to force AIs to learn facts about their data that generalize well, but it also means training models to perform tasks that are very different depending on the input data type. Filling in blacked out pixels is quite different from filling in blanks in a sentence, for example. So what if there was a way to come up with one task that we could use to train machine learning models on any kind of data? That’s where data2vec comes in. For this episode of the podcast, I’m joined by Alexei Baevski, a researcher at Meta AI one of the creators of data2vec. In addition to data2vec, Alexei has been involved in quite a bit of pioneering work on text and speech models, including wav2vec , Facebook’s widely publicized unsupervised speech model. Alexei joined me to talk about how data2vec works and what’s next for that research direction, as well as the future of multi-modal learning. *** Intro music : - Artist: Ron Gelinas - Track Title: Daybreak Chill Blend (original mix) - Link to Track: *** Chapters: 2:00 Alexei’s background 10:00 Software engineering knowledge 14:10 Role of data2vec in progression 30:00 Delta between student and teacher 38:30 Losing interpreting ability 41:45 Influence of greater abilities 49:15 Wrap-up

You can listen to 121. Alexei Baevski - data2vec and the future of multimodal learning online on Radio and Podcast. Open the player on this page to stream the available audio.

121. Alexei Baevski - data2vec and the future of multimodal learning is an episode from Towards Data Science by The TDS team .

This episode is 00:49:31 long.

This episode was published on Apr 27, 2022.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from Towards Data Science when more episodes are available from the podcast feed.