Monday, June 14, 2010

How Much Time Yahoo Search Engine Takes To Crawl Your Wepages??


How much time yahoo search engine takes to crawl your wepages?


I have always noticed that Yahoo crawls less often to any web content than other Search Engines. Googles crawls faster. This makes Google more special for a website or blog. So the question is does Yahoo sets a time period for visiting a webpage. And what I found was that yes Yahoo Bot crawls a bit slower but it crawls regularly. 
A solution for this might be that you should bookmark your site on yahoo. 
But for some sites yahoo does better than Google and MSN. Some factors that may affect the crawl rate of your website are:
1. Server Response Time.
The better is the Server response time of your website the faster the crawlers will love to visit your site. Otherwise the crawlers will leave your page to make sure it is not overloading the server. 
2. Page Load Times
The faster your individual pages load, the more the crawlers can crawl your website pages. You can monitor the page load times of your website pages in Google Webmaster Tools. 
3. Your Content
If you have wonderful unique content in your website and you keep updating your site regularly then the crawlers will visit your website more frequently. 
4. URLs, Redirecting and Mising Pages
If you have issues related to URLs, Redirecting and Mising Pages then the crawlers will get stuck into it and will not love to crawl your pages unless you have sorted out those problem. You can see if there is any crawler issue with your site in Google Webmaster Tool. 
5. Bot Efficiency
You can adjust the crawl times of both Bing and Yahoo!'s bots by using a crawl-delay setting in your robots.txt file. Another good way of being able to tell if the other bots are indexing too slowly is by checking on Google's own crawl speed - it may be a good indicator as well.
Hope this information will also be helpful in letting you know why Yahoo crawler or other are slow in crawling your web pages. 

1 comment:

  1. It actually depends. But if you really want to have pages crawl fast, you might want to use ping services.

    ReplyDelete