September 1st, 2006 by metapilot

Following Google’s lead, Yahoo has came out with their own version of a sitemaps tool and rolled into the the Yahoo Site Explorer Beta tool. There is a lot of debate over the value of Google’s sitemap tool and Yahoo doesn’t really make any advances over it. One nice thing about it, though, is that you can now easily get the last-crawled date for any page that you have listed in “My Sites” (you have to have a free yahoo acount in order to access My Sites information).
In order to set up a new site in Yahoo Site Explorer, enter the site’s URL and click “Add My Site”, afterwhich you’re presented with links to Manage and Authenticate your new site. The coolest thing about the tool is that you can communicate to Yahoo that you want it to visit your site and it will go there and grab your feed in “real time”. Your feed can be RSS, Atom, a txt file or a compressed text file (.gz only) and by real time, I mean that you might have to wait a few minutes for it to refresh your screen and show you that it’s verified your feed exists.

The feed is the conduit through/by which you are telling Yahoo about URLs you want it to crawl. Google Sitemaps adds an additional conduit, or feed choice– a .xml file with which you can make your list of URLs dynamic by running a python script on your web server and I expect to see something of that nature coming from Yahoo in the near future. Very basically, though, all you need for Yahoo Site Explorer is a .txt file with a list all the URLs on your site that you wnat crawled (one URL per line) named urllist.txt.. Upload it to your root directory and after you click on “Manage Site” in Yahoo Site Explorer, type “urllist.txt” into the field and off goes the bot to check it out.
Before you can get to the “good” information about your site, you have to authenticate your site. This lets Yahoo know that you currently have access to the site’s rood directory, which means you’re likely to worthy of knowing the any little insights Yahoo Site Explorer might provide you. Whey you click on the “Authenticate” link, you can choose to download an authentication file (which you can save directly to your root directory, if you want) or make your own authentication file with the file name and contents presented. Once the file is placed in your root directory, click “Authenticate” and your site gets put into a pending authenication que until Yahoo crawls the feed. Within 24 hours, I could see that my status was no longer “Pending” but rather, I was now a “processed” site.

On to the real business, Yahoo has racheted up the number of indexed pages to 12. It’s good to see things filling in there. Over on the marginally more useful Google Webmaster Tools, I can see that the index is ranking the old site for some odd keywords, however zero traffic comes from any of his old link partners or search engine listings–at least not from anyone directly clicking on a link.