Dec 15

You can now set your own Googlebot crawl speed

Feed the Spiders, Google, Heavy construction, Internet Marketing No Comments »

crawlspeed.jpg

You can now set your own crawl speed on Google Webmaster Tools. What does this mean?

Many things actually.

1st, this shows that Google acknowledges that its automated settings are sometimes crappy, and an algorithm cannot always predict the perfect crawl rate for a site. Google is seldomly known to give options to the webmaster that affect rankings.

2nd, this makes Google Webmaster Tools even better than they already were.

All of you newbies, try to resist the urge to set the crawl rate at max. You don’t need it. On 99% of sites, the automatic setting works fine. There are only a few cases where you might need a custom setting. For example I have a few sites with thousands of pages, but only 10% of them are indexed. I assume that setting a faster crawl rate on those will help index a few more thousand pages.

It will be rare for someone to want to reduce it, despite the fact that Googlebot is the most frequent visitor, the automated setting is usually optimal. Maybe on huge indexed database sites? Nah, not even then.

So, when should you increase it? If you have a very new site, setting the bar at max might increase the indexing time. No idea, I am just guessing here. Try it.

Also, you might want to increase it a bit if you have a frequently updating site and you don’t see your new posts on a search after a few hours. Although, as I said, the automated settings usually adapt to that scenario. But, if it does fail, here is your setting that will fix it.

Related Blogs

Popularity: 47% [?]

written by Glowleaf \\ tags: , ,