SE Radio 703: Sahaj Garg on Low Latency AI

In this episode, Sahaj Garg , CTO of wispr.ai, joins SE Radio host Robert Blumen to talk about the challenges of building low-latency AI applications. They discuss latency's effect on consumer behavior as well as interactive applications. The conversation explores how to measure latency and how scale impacts it. Then Sahaj and Robert shift to themes around AI, including whether "AI" means LLMs or something broader, as they look at latency requirements and challenges around subtypes of AI applications. The final part of the episode explores techniques for managing latency in AI: speed vs accuracy trade-offs; speed vs cost; latency vs cost; choosing the right model; reducing quantization; distillation; and guessing + validating. Brought to you by IEEE Computer Society and IEEE Software magazine .

You can listen to SE Radio 703: Sahaj Garg on Low Latency AI online on Radio and Podcast. Open the player on this page to stream the available audio.

SE Radio 703: Sahaj Garg on Low Latency AI is an episode from Software Engineering Radio - The Podcast for Professional Software Developers by SE-Radio Team.

This episode is 54:50 long.

This episode was published on Jan 14, 2026.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from Software Engineering Radio - The Podcast for Professional Software Developers when more episodes are available from the podcast feed.