Expect Googlebot Crawl Increase With Changes to CDNs
Whenever you make major changes to something about the website, it is natural you see significant changes in the amount of crawling the Google but does on the site. This is completely normal and expected.
The same is also true when you switch over your site to CDN. But if you remove the CDN or otherwise change it, you will also see the same kind of Google but collectivity on the site when that’s done.
John Mueller from Google confirmed this via twitter.
If you remove / change the CDN, it will affect crawling — that's expected.
— John ☆.o(≧▽≦)o.☆ (@JohnMu) August 22, 2018
The increase crawling activity is simply because Google is reindexing these pages with a CDN addition, removal or change. Site owners will also see the same kind of crawl activity when they do a site move or switch to HTTPS or make other major changes to the site.
That said, site owners should be concerned about an increase in crawling is if there is an issue with the increase crawling causing server issues, such as Google but crawling the site so hard it is taking down the server. While this is pretty unusual it can happen, especially if there’s something about the site that has created infinite URLs for example. So if you have made major changes to a site, expect increased crawl activity, but with no associated changes to a site, it can sometimes highlight issues Googlebot has come across, such as a million pages being crawled from a calendar on a site or parameters gone wild.
Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn’t sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.
Latest posts by Jennifer Slegg (see all)
