Benchmarking Domain Intelligence Data Brew Episode 45

In this episode, Pallavi Koppol, Research Scientist at Databricks, explores the importance of domain-specific intelligence in large language models (LLMs). She discusses how enterprises need models tailored to their unique jargon, data, and tasks rather than relying solely on general benchmarks. Highlights include: - Why benchmarking LLMs for domain-specific tasks is critical for enterprise AI. - An introduction to the Databricks Intelligence Benchmarking Suite (DIBS). - Evaluating models on real-world applications like RAG, text-to-JSON, and function calling. - The evolving landscape of open-source vs. closed-source LLMs. - How industry and academia can collaborate to improve AI benchmarking.

You can listen to Benchmarking Domain Intelligence Data Brew Episode 45 online on Radio and Podcast. Open the player on this page to stream the available audio.

Benchmarking Domain Intelligence Data Brew Episode 45 is an episode from Data Brew by Databricks by Databricks.

This episode is 31:41 long.

This episode was published on Apr 24, 2025.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from Data Brew by Databricks when more episodes are available from the podcast feed.