今天在跑主機報表時發現多了一個異常的來源 202.160.xxx.xxx 全都是大陸的 IP，反查一下，發現到全是 Yahoo Slurp ?!
奇怪了，何時 Yahoo 開始不從美國總部來抓資料了呢 ? 而改用大陸來源 ?? 從 Yahoo 的 Search Blog 中我看到了一則公告，內容就有提到了搬家的事宜，怪怪~~ 大陸可是在網路上有 100% 操控權的耶! 會不會從年底開始就搜尋不到台灣相關的資料了呢? 尤其是最近火熱的 UN For Taiwan ..^^||||
March 28, 2007
Yahoo! Search Crawler, Slurp, is moving
Anyone who has seen their web server logs, has seen the Yahoo! crawler come by and leave its mark. It identifies itself as ‘Yahoo! Slurp’ and its domain is inktomisearch.com. For those of you who follow the search space, you can easily guess that this is one of the last historic remnants from our acquisition of Inktomi a few years ago. Well, the crawler finally decided to move and find a new home. We are moving our crawler from inktomisearch.com to crawl.yahoo.net.
The user-agent will continue to be Yahoo! Slurp, hence you do not need to make any changes to your robots.txt file. Nor are we changing the actual IP addresses of the crawler infrastructure during this transition. However, if you do any reverse DNS checks for the crawler identity or have any network access rules to allow inktomisearch.com, please also update them to allow for crawl.yahoo.net.
The crawlers will be switched over in phases starting immediately. You’ll see an increasing number of Yahoo! Slurp visits from crawl.yahoo.net and fewer and fewer from inktomisearch.com during the transition, which will take a few weeks.
Keep an eye on this space and the Yahoo! Search Blog for more details on when the migration is complete and details on bot verification with the new domain name.
Product Manager, Yahoo! Search