Google: Disallowed URLs Do Not Affect Crawl Budget via @MattGSouthern
Google’s Gary Illyes updated his original writeup on crawl budget with clarification about disallowed URLs.The document now includes the following information:“Q: Do URLs I disallowed through robots.txt affect my crawl budget in any way?A: No, disallowed URLs do not affect…
