Radio and PodcastRadio and PodcastLive Radio & Podcasts
Google crawlers behind the scenes artwork
Technology

Google crawlers behind the scenes

Search Off the Record by Google Developer Studio

Mar 12, 202625:07Technology

Developers often talk about Googlebot as if it were a single program you could just run as "googlebot.exe", but that is not how Google's crawling actually works. In this episode of Search Off the Record, Martin and Gary...

Related Episodes

Continue listening to more episodes from Search Off the Record.

Do sitemaps still matter?

Oct 1, 2026 - 26:26

Sitemaps have been a core element of web development and search engine optimization for two decades, yet...

How to read the Indexing Report

Jul 16, 2026 - 31:40

Should you panic when your Search Console indexing report is showing pages that aren't indexed? Is a 404...

Should I use markdown for my site?

Jun 15, 2026 - 26:12

Should you convert your website into Markdown to help Large Language Models (LLMs) understand your conten...