Yahoo! Launches New Slurp
Date Published 02/08/2006 - Click here for more recent news
In the Yahoo! search blog on July 31, 2006 - they've announced the launch of a new crawler, Slurp advertising that, "The new-Slurp is faster and more efficient."
They go on to write that, "In addition to crawling the Internet faster, our new crawler is more efficient at visiting websites. As a result, site owners will notice as much as a 25% reduction in the number of requests and bandwidth consumed by the crawler. While transitioning to the new crawler over the past few weeks, we had been running both crawlers in tandem. In some cases, this increased the frequency of Yahoo! Search requests to websites. Now, with the new crawler in full production, we have turned off the old crawler and site owners should see a much lower crawl load without a loss in content coverage."
Well that's nice to know. Like most other followers of search engine behaviours and sometime misbehaviours - here at EASIserv.com we've been a bit puzzled for a couple of months as to why we were getting such a battering from the Slurpmeister. Now we have an answer - Yahoo! were doubling-up on their bots as they road-tested new-Slurp whilst running old-Slurp in parallel. Sensible enough practice we think.
However, during the period when we were being puzzled by this overwhelming Yahoo! interest - we'd actually carried out a comparison head-count of the combined Google, Yahoo! and MSN bot requests to our website over a 24 hour period on 9th July 2006. On that day Slurpers (inktomisearch.com IDs) accounted for 81% of all bot activity. After reading this news release from Yahoo! that we could anticipate "a 25% reduction in the number of requests" - we did a follow-up count over 24 hours on 1st August 2006 just to check that claim out? The result was that when comparing the two 24 hour periods - we identified that Slurp requests to our website had in fact reduced by a massive 70%? Can't be bad.
PS: The 'greater than forecast' % reduction in Slurp requests that we noted is probably explained by the fact that, although we didn't know it on 9th July 2006 - Yahoo! was already girding its loins for its own major search index update four days later - on 13th July 2006. So presumably Yahoo! were hammering everybody around that time and our 9th July 2006 base figure may have been aberrantly high. Statistics, eh - doncha just luv 'em?
Featured Links
Accreditations
Partners
