jump to navigation

MSNbot useragent verification December 1, 2006

MSNBbot now provides a way to verify if the bot crawling your site is really from microsoft, you can check out micorosft blog on search engines in disguise on the detailed process on how they implemented it. Basically the same process as what Google have implemented few months ago which involves two steps, a reverse DNS lookup followed by a forward DNS lookup.

Could Yahoo implement the same strategy for the web?