Reading view

Google explains how crawling works in 2026

Gary Illyes from Google shared some more details on Googlebot, Google’s crawling ecosystem, fetching and how it processes bytes.

The article is named Inside Googlebot: demystifying crawling, fetching, and the bytes we process.

Googlebot. Google has many more than one singular crawler, it has many crawlers for many purposes. So referencing Googlebot as a singular crawler, might not be super accurate anymore. Google documented many of its crawlers and user agents over here.

Limits. Recently, Google spoke about its crawling limits. Now, Gary Illyes dug into it more. He said:

  • Googlebot currently fetches up to 2MB for any individual URL (excluding PDFs).
  • This means it crawls only the first 2MB of a resource, including the HTTP header.
  • For PDF files, the limit is 64MB.
  • Image and video crawlers typically have a wide range of threshold values, and it largely depends on the product that they’re fetching for.
  • For any other crawlers that don’t specify a limit, the default is 15MB regardless of content type.

Then what happens when Google crawls?

  1. Partial fetching: If your HTML file is larger than 2MB, Googlebot doesn’t reject the page. Instead, it stops the fetch exactly at the 2MB cutoff. Note that the limit includes HTTP request headers.
  2. Processing the cutoff: That downloaded portion (the first 2MB of bytes) is passed along to our indexing systems and the Web Rendering Service (WRS) as if it were the complete file.
  3. The unseen bytes: Any bytes that exist after that 2MB threshold are entirely ignored. They aren’t fetched, they aren’t rendered, and they aren’t indexed.
  4. Bringing in resources: Every referenced resource in the HTML (excluding media, fonts, and a few exotic files) will be fetched by WRS with Googlebot like the parent HTML. They have their own, separate, per-URL byte counter and don’t count towards the size of the parent page.

How Google renders these bytes. When the crawler accesses these bytes, it then passes it over to WRS, the web rendering service. “The WRS processes JavaScript and executes client-side code similar to a modern browser to understand the final visual and textual state of the page. Rendering pulls in and executes JavaScript and CSS files, and processes XHR requests to better understand the page’s textual content and structure (it doesn’t request images or videos). For each requested resource, the 2MB limit also applies,” Google explained.

Best practices. Google listed these best practices:

  • Keep your HTML lean: Move heavy CSS and JavaScript to external files. While the initial HTML document is capped at 2MB, external scripts, and stylesheets are fetched separately (subject to their own limits).
  • Order matters: Place your most critical elements — like meta tags, <title> elements, <link> elements, canonicals, and essential structured data — higher up in the HTML document. This ensures they are unlikely to be found below the cutoff.
  • Monitor your server logs: Keep an eye on your server response times. If your server is struggling to serve bytes, our fetchers will automatically back off to avoid overloading your infrastructure, which will drop your crawl frequency.

Podcast. Google also had a podcast on the topic, here it is:

💾

Google went through crawling, fetching, and the bytes it processes.

ChatGPT enables location sharing for more precise local responses

OpenAI now allows users of ChatGPT to share their device location so that ChatGPT can know more precisely where the user is and serve better answers and results based on that location.

The feature is called location sharing, OpenAI wrote, “Sharing your device location is completely optional and off until you choose to enable it. You can update device location sharing in Settings > Data Controls at any time.”

What it does. If ChatGPT knows your location, it can return better local results. OpenAI wrote:

  • “Precise location means ChatGPT can use your device’s specific location, such as an exact address, to provide more tailored results.”
  • “For example, if you ask “what are the best coffee shops near me?”, ChatGPT can use your precise location to provide more relevant nearby results. On mobile devices, you can choose to toggle off precise location separately while keeping approximate device location sharing on for additional control.”

Privacy. OpenAI said “ChatGPT deletes precise location data after it’s used to provide a more relevant response.” Here is how ChatGPT uses that information:

  • “If ChatGPT’s response includes information related to your specific location, such as the names of nearby restaurants or maps, that information becomes part of your conversation like any other response and will remain in your chat history unless you delete the conversation.”

Does it work. Does this work? Well, maybe not as well as you’d expect. Here is an example from Glenn Gabe:

I shared about the "Near Me ChatGPT Update" the other day and just let ChatGPT use my device location. This is supposed to enhance results for local queries. I just asked for the "best steakhouses near me" and several of the restaurants are ~45 minutes away. Both restaurants… pic.twitter.com/gRkMeuzMQt

— Glenn Gabe (@glenngabe) March 30, 2026

Why we care. Making ChatGPT local results better is a bit deal in local search and local SEO. Knowing the users location and better yet, precise location, can result in better local results.

Hopefully this will result in ChatGPT responding with more useful local results for users.

Google March 2026 core update rolling out now

Google released the March 2026 core update today, the company announced.

This is Google’s first core update of 2026. It follows the quick March 2026 spam update from a couple of days ago and the February 2026 Discover update.

What Google is saying. Google updated its Search Status Dashboard to state:

  • “Released the March 2026 core update. The rollout may take up to 2 weeks to complete.”

Google added on LinkedIn:

  • “This is a regular update designed to better surface relevant, satisfying content for searchers from all types of sites. The rollout may take up to 2 weeks to complete.”

About core updates. Core updates roll out several times each year. They introduce broad, significant changes to Google’s search algorithms and systems, which is why Google announces them.

What to do if you are hit. Google didn’t share new guidance specific to the March 2026 core update. However, Google has previously offered advice on what to consider if a core update negatively impacts your site:

  • There aren’t specific actions you can take to recover. A negative rankings impact may not mean anything is wrong with your pages.
  • Google provided a list of questions to consider if your site is hit by a core update.
  • You may see some recovery between core updates, but the biggest changes tend to follow another core update.

In short: write helpful content for people, not for search engines.

  • “There’s nothing new or special that creators need to do for this update as long as they’ve been making satisfying content meant for people. For those that might not be ranking as well, we strongly encourage reading our creating helpful, reliable, people-first content help page,” Google said previously.

For more details on Google core updates, you can read Google’s documentation.

Previous core updates. Here’s a timeline and our coverage of recent core updates:

Why we care. With any core update, you often see significant volatility in Google search results and rankings. These updates may improve visibility for your site or your clients’ sites, but you may also see fluctuations or declines in rankings and organic traffic. We hope this update rewards your efforts and drives strong traffic and conversions.

Google Search Live expands globally where AI Mode is available

Today, Google released Google Search Live globally where AI Mode is available, for these languages and regions. This brings Search Live to more than 200 countries and territories.

Google credits its new audio and voice model, Gemini 3.1 Flash Live, which it says “delivers even more natural and intuitive conversations.” The “new model is also inherently multilingual, which means that people around the world can now speak with Search in their preferred language,” Google added.

How it works. To use Search Live, open the Google app on Android or iOS and tap the Live icon under the Search bar. From there, you can ask your question out loud to get a helpful audio response, then continue the conversation with follow-up questions or dive deeper with helpful web links. If you want to ask about something in front of you, like how to install a new shelving unit, you can enable your camera to add visual context. This way, Search can see what your camera sees and offer helpful suggestions, plus links to more information on the web.

You can also access Search Live if you’re already pointing your camera with Google Lens — just tap the Live option at the bottom of the screen to have a real-time, back-and-forth conversation about what you see in the real world.

More. Last September, Google made Search Live with video available in the U.S, prior to that, it was an opt in beta and before that it was talk and listen, without video.

Why we care. This is another way users can have conversations with Google’s AI instead of typing queries. Answers could increasingly bypass traditional clicks, and further erode traffic to websites. The inclusion of links (citations at the bottom) means publishers and brands could still see some benefits, but most searchers likely will have little need or desire to click on those links or dig deeper after getting their answer.

Google March 2026 spam update done rolling out

Google released the March 2026 spam update less than 24 hours ago and it is already done rolling out. The update finished today at 10:40 a.m. ET.

  • This update was released yesterday (March 24) at 3:20 p.m. It took 19 hours and 30 minutes to fully roll out, which is super fast.

Why we care. This is the second Google algorithm update announced in 2026. It’s unclear what spam it targeted, but if you see ranking or traffic changes in the next few days, the Google March 2026 spam update could be the cause.

More on spam update. Google’s documentation says:

“While Google’s automated systems to detect search spam are constantly operating, we occasionally make notable improvements to how they work. When we do, we refer to this as a spam update and share when they happen on our list of Google Search ranking updates.

For example, SpamBrain is our AI-based spam-prevention system. From time-to-time, we improve that system to make it better at spotting spam and to help ensure it catches new types of spam.

Sites that see a change after a spam update should review our spam policies to ensure they are complying with those. Sites that violate our policies may rank lower in results or not appear in results at all. Making changes may help a site improve if our automated systems learn over a period of months that the site complies with our spam policies.

In the case of a link spam update (an update that specifically deals with link spam), making changes might not generate an improvement. This is because when our systems remove the effects spammy links may have, any ranking benefit the links may have previously generated for your site is lost. Any potential ranking benefits generated by those links cannot be regained.”

Impact. This update should only impact sites spamming Google Search, so hopefully you didn’t see any major negative impact.

Google releases March 2026 spam update

Google released its March 2026 spam update today at 3:20 p.m. It’s the second announced Google algorithm update of 2026, following the February 2026 Discover core update.

  • This is the first spam update of 2026.
  • Google’s most recent spam update was in August 2025.

Timing. This update may only “take a few days to complete,” Google said. On LinkedIn, Google added:

  • “This is a normal spam update, and it will roll out for all languages and locations. The rollout may take a few days to complete.”

Why we care. This is the second announced Google algorithm update of 2026. It’s unclear what spam this update targets, but if you see ranking or traffic changes in the next few days, it could be due to it.

More on spam update. Google’s documentation says:

“While Google’s automated systems to detect search spam are constantly operating, we occasionally make notable improvements to how they work. When we do, we refer to this as a spam update and share when they happen on our list of Google Search ranking updates.

For example, SpamBrain is our AI-based spam-prevention system. From time-to-time, we improve that system to make it better at spotting spam and to help ensure it catches new types of spam.

Sites that see a change after a spam update should review our spam policies to ensure they are complying with those. Sites that violate our policies may rank lower in results or not appear in results at all. Making changes may help a site improve if our automated systems learn over a period of months that the site complies with our spam policies.

In the case of a link spam update (an update that specifically deals with link spam), making changes might not generate an improvement. This is because when our systems remove the effects spammy links may have, any ranking benefit the links may have previously generated for your site is lost. Any potential ranking benefits generated by those links cannot be regained.”

Update, March 25. The update completed in less than 24 hours. See: Google March 2026 spam update done rolling out

❌