Docker Model Runner

Docker launched "Docker Model Runner" to run LLMs through llama.cpp with a single "docker model" command. In this episode Bret details examples and some useful use cases for using this way to run LLMs. He breaks down the internals. How it works, when you should use it or not use it; and, how to get started using Open WebUI for a private ChatGPT-like experience. 🙌 The Agentic DevOps Guild has launched! It's a training + community + mentorship program for engineers wanting to learn the latest CI/CD automation and dive into Agentic DevOps. Meetups are happening now, with new course videos dropping every few weeks. Join the Guild and become your team's leader in AI for infrastructure automation 🍾 ★Topics★ Model Runner Docs Hub Models OCI Artifacts Open WebUI My Open WebUI Compose file Creators & Guests Cristi Cotovan - Editor Beth Fisher - Producer Bret Fisher - Host (00:00) - Intro (03:03) - Model Runner Elevator Pitch (03:45) - Enabling Docker Model Runner (06:45) - Self Promotion! Is that an ad? For me? (07:20) - Downloading Models (09:28) - Architectrure of Model Runner (13:06) - ORAS (13:26) - What's next for Model Runner? (14:30) - Troubleshooting You can also support this podcast by subscribing to my YouTube channel and my weekly newsletter at bret.news ! Grab the best coupons for my Docker and Kubernetes courses . Join my cloud native DevOps community on Discord . Grab some merch at Bret's Loot Box Homepage bretfisher.com

You can listen to Docker Model Runner online on Radio and Podcast. Open the player on this page to stream the available audio.

Docker Model Runner is an episode from DevOps and Docker Talk by Bret Fisher.

This episode is 15:24 long.

This episode was published on Apr 21, 2025.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from DevOps and Docker Talk when more episodes are available from the podcast feed.