❌

Normal view

Yesterday β€” 3 February 2026Main stream

Google: 75% of crawling issues come from two common URL mistakes

3 February 2026 at 17:30

Google discussed its 2025 year-end report on crawling and indexing challenges for Google Search. The biggest issues were faceted navigation and action parameters, which accounted for about 75% of the problems, according to Google’s Gary Illyes. He shared this on the latest Search Off the Record podcast, published this morning.

What is the issue. Crawling issues can slow your site to a crawl, overload your server, and make your website unusable or inaccessible. If a bot gets stuck in an infinite crawling loop, recovery can take time.

  • β€œOnce it discovers a set of URLs, it cannot make a decision about whether that URL space is good or not unless it crawled a large chunk of that URL space,” Illyes said. By then it is too late and your site has slowed to a halt.

The biggest crawling challenges. Based on the report, these are the main issues Google sees:

  • 50% come from faceted navigation. This is common on ecommerce sites, where endless filters for size, color, price, and similar options create near-infinite URL combinations.
  • 25% come from action parameters. These are URL parameters that trigger actions instead of meaningfully changing page content.
  • 10% come from irrelevant parameters. This includes session IDs, UTM tags, and other tracking parameters added to URLs.
  • 5% come from plugins or widgets. Some plugins and widgets generate problematic URLs that confuse crawlers.
  • 2% come from other β€œweird stuff.” This catch-all category includes issues such as double-encoded URLs and related edge cases.

Why we care. A clean URL structure without bot traps is essential to keep your server healthy, ensure fast page loads, and prevent search engines from getting confused about your canonical URLs.

The episode. Crawling Challenges: What the 2025 Year-End Report Tells Us.

πŸ’Ύ

Gary Illyes says faceted navigation and action parameters dominate Google’s crawl waste, trapping bots in infinite URLs and straining servers.

Microsoft rolls out multi-turn search in Bing

3 February 2026 at 17:00

Microsoft today rolled out multi-turn search globally in Bing. As you scroll down the search results page, a Copilot search box now dynamically appears at the bottom.

About multi-turn search. This type of search experience lets a user continue the conversation from the Bing search results page. Instead of starting over, the searcher types a follow-up question into the Copilot search box at the bottom of the results, allowing the search to build on the previous query. Here’s a screenshot of this feature:

Here’s a video of it in action:

What Microsoft said. Jordi Ribas, CVP, Head of Search at Microsoft, posted this news onΒ X:

  • β€œAfter shipping in the US last year, multi-turn search in Bing is now available worldwide.
  • β€œBing users don’t need to scroll up to do the next query, and the next turn will keep context when appropriate. We have seen gains in engagement and sessions per user in our online metrics, which reflect the positive user value of this approach.”

Why we care. Search engines like Google and Bing are pushing harder to move users into their AI experiences. Google is blending AI Overviews more deeply into AI Mode, even as many publishers object to how it handles their content. Bing has now followed suit, fully rolling out the Copilot search box at the bottom of search results after several months of testing.

πŸ’Ύ

Bing's new multi-turn search is now global. As you scroll, a Copilot box appears at the bottom so follow-ups build on the last query.
Before yesterdayMain stream

1/3rd of publishers say they will block Google Search AI-generative features like AI Overviews

29 January 2026 at 22:25

Google announced yesterday that it is exploring ways for sites to opt out of Google using their content for its AI-generative search features, such as AI Mode and AI Overviews. I asked the SEO community on X if they would opt out of these Google Search AI-generative features or not.

The results. Of the over 350 responses that took the poll yesterday, most said they would not opt out. However, about 1/3 of respondents said they would block or opt out of these features. Here is the breakdown:

Question: Would you block Google from using your content for AI Overviews and AI Mode?

  • 33.2% – Yes, I’d block Google
  • 41.9% – No, I wouldn’t block
  • 24.9% – I am not sure yet.

Here is the actual poll:

Would you block Google from using your content for AI Overviews and AI Mode – Google may be giving us more controls – take my poll below. https://t.co/60M3Vt0YlN

β€” Barry Schwartz (@rustybrick) January 28, 2026

How to opt out. We don’t know. Google only said it is β€œexploring” ways to handle this but has not provided any mechanism for this. So we don’t know how hard or easy it would be to opt out. The easier it is, the more likely sites will opt out; the harder, the less likely.

Why we care. The true number of sites that might opt out of AI Mode or AI Overviews won’t be known until the mechanism is out to handle this. And trust me, there will be many reports on how many sites are opting out.

Like recently, β€œSome 79% of almost 100 top news websites in the UK and US are blocking at least one crawler used for AI training out of OpenAI’s GPTBot, ClaudeBot, Anthropic-ai, CCBot, Applebot-Extended and Google-Extended,” reported The Press Gazette.

My recommendation; once it is out, it is something you will want to test and see the results of opting out or opting in.

❌
❌