#71: Alex O'Connor — Transformers, Generative AI, and the Deep Learning Revolution

Alex O’Connor—researcher and ML manager—on the latest trends of generative AI. Language and image models, prompt engineering, the latent space, fine-tuning, tokenization, textual inversion, adversarial attacks, and more. Alex O’Connor got his PhD in Computer Science from Trinity College, Dublin. He was a postdoctoral researcher and funded investigator for the ADAPT Centre for digital content, at both TCD and later DCU. In 2017, he joined Pivotus, a Fintech startup, as Director of Research. Alex has been Sr Manager for Data Science & Machine Learning at Autodesk for the past few years, leading a team that delivers machine learning for e-commerce, including personalization and natural language processing. Favorite quotes “None of these models can read.” “Art in the future may not be good, but it will be prompt.” Mastodon Books Machine Learning Systems Design by Chip Huyen Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurélien Géron Papers The Illustrated Transformer by Jay Alammar Attention Is All You Need by Google Brain Transformers: a Primer by Justin Seonyong Lee Links Alex in Mastodon ★ Training Dream Booth Multimodal Art on HuggingFace by @akhaliq NeurIPS arxiv.org : Where most papers get published Nono’s Discord Suggestive Drawing : Nono’s master’s thesis Crungus is a fictional character from Stable Diffusion’s latent space Machine learning models Stable Diffusion Arcane Style Stable Diffusion fine-tuned model ★ Imagen DALL-E CLIP GPT and ChatGPT BERT, ALBERT & RoBERTa Bloom word2vec Mupert.ai and Google’s MusicLM t-SNE and UMAP : Dimensionality reduction techniques char-rnn Sites TensorFlow Hub HuggingFace Spaces ★ DreamBooth Jasper AI Midjourney Distill.pub ★ Concepts High-performance computing (HPC) Transformers and Attention Sequence transformers Quadratic growth Super resolution Recurrent neural networks (RNNs) Long short-term memory networks (LSTMs) Gated recurrent units (GRUs) Bayesian classifiers Machine translation Encoder-decoder Gradio Tokenization ★ Embeddings ★ Latent space The distributional hypothesis Textual inversion ★ Pretrained models Zero-shot learning Mercator projection People mentioned Ted Underwood UIUC Chip Huyen Aurélien Géron Chapters 00:00 · Introduction 00:40 · Machine learning 02:36 · Spam and scams 15:57 · Adversarial attacks 20:50 · Deep learning revolution 23:06 · Transformers 31:23 · Language models 37:09 · Zero-shot learning 42:16 · Prompt engineering 43:45 · Training costs and hardware 47:56 · Open contributions 51:26 · BERT and Stable Diffusion 54:42 · Tokenization 59:36 · Latent space 01:05:33 · Ethics 01:10:39 · Fine-tuning and pretrained models 01:18:43 · Textual inversion 01:22:46 · Dimensionality reduction 01:25:21 · Mission 01:27:34 · Advice for beginners 01:30:15 · Books and papers 01:34:17 · The lab notebook 01:44:57 · Thanks I'd love to hear from you. Submit a question about this or any previous episodes. Join the Discord community . Meet other curious minds. If you enjoy the show, would you please consider leaving a short review on Apple Podcasts/iTunes? It takes less than 60 seconds and really helps. Show notes, transcripts, and past episodes at gettingsimple.com/podcast . Thanks to Andrea Villalón Paredes for editing this interview. Sleep and A Loop to Kill For songs by Steve Combs under CC BY 4.0. Follow Nono Twitter.com/nonoesp Instagram.com/nonoesp Facebook.com/nonomartinezalonso YouTube.com/nonomartinezalonso

You can listen to #71: Alex O'Connor — Transformers, Generative AI, and the Deep Learning Revolution online on Radio and Podcast. Open the player on this page to stream the available audio.

#71: Alex O'Connor — Transformers, Generative AI, and the Deep Learning Revolution is an episode from The Getting Simple Podcast by Nono Martínez Alonso.

This episode is 01:45:32 long.

This episode was published on Apr 26, 2023.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from The Getting Simple Podcast when more episodes are available from the podcast feed.