September 4th, 2006 by metapilot

You may have read recently (August 2006) that AOL inadvertently released information about searches done through the AOL site. SEO Scoop discovered a forum post detailing the click through rate for ranking #1 vs. #2-#10. While AOL users, as a group, do tend to be less web savvy than the average web user, the sample size is large enough that we can now make more educated guesses about the way people use search engines.
Results from:
Total Searches:9,038,794
Total Clicks: 4,926,623

In the above sample of 4,926,623 clicks,
2,075,765 of the clicks (42.1%) were on the #1 ranked result.;
586,100 of the clicks (11.9%) were on the #2 ranked result (3.5 x fewer clicks than the #1 position);
418,643 of the clicks (8.5%) were on the #3 ranked result (1.4 x fewer clicks than the #2 position):
298,532 of the clicks (6.1%) were on the #4 ranked result (1.4 x fewer clicks than the #3 position)
242,169 of the clicks (4.9%) were on the #5 ranked result (1.2 x fewer clicks than the #4 position)
199,541 of the clicks (4.1%) were on the #6 ranked result (1.2 x fewer clicks than the #5 position)
168,080 of the clicks (3.4%) were on the #7 ranked result (1.2 x fewer clicks than the #6 position)
148,489 of the clicks (3.0%) were on the #8 ranked result (1.1 x fewer clicks than the #7 position)
140,356 of the clicks (2.8%) were on the #9 ranked result (1.05 x fewer clicks than the #8 position)
147,551 of the clicks (3.0%) were on the #10 ranked result (1.05 x more clicks than the #9 position)

September 2nd, 2006 by metapilot

With Google steady at the 9000 to 12000 (old-site, supplimental) pages showing for the site:metapilot.com search and up until today showing only two pages from my site (at the very bottom of the all the other supplimental pages), and Yahoo up the page count of the new site by page or two a week, I getting figitty.Google had shown one of the blog pages in that search, but today that is gone and only the home page, indexed Aug. 21, is showing. I’m not at all sure how google found that blog page since it wasn’t until at least a week later that I put any sort of public link to the blog. The only thing I can think of is that in messing around during the installation, and working on making a static home page (coming) I had an index.php page in the root along with the index.html page and somehow that got picked up? Thing is it wasn’t even the index.php page that was indexed, it was an archive page. Anyhow, It’s gone at least for the moment.

Since I am getting figitty, I re-crawled the site with my trusty dusty sitemap tool (I use the one over at auditmypc.com) so that my sitemaps included all the pages currently in the blog. I have two sitemaps, urllist.txt and sitemap.txt because urllist.txt used to be the only filename that Yahoo look when you submitted a sitemap as a feed–that was before Yahoo Site Explorer, which doesn’t clearly define a specific file name. Sitemap.txt is the file name Google suggests if you are using a text file as the sitemap you submit to Google Webmaster Tools and rather than telling Google to look for a file created spefically for Yahoo Sitemapts, I make one with that name too. For the time being it is faster and easier to have these two sitemaps than to figure out the ideal way to have a single one. (Did you get all that?)

So the new sitemaps were submitted to Google and Yahoo yesterday and the domain submitted to MSN, as well. I’m hoping this jogs some changes in the index in the next week or so. Of course, I won’t be able to be sure that this is what caused them but at least, it’s gives the felling of doing something to help push the process along.

September 1st, 2006 by metapilot

Following Google’s lead, Yahoo has came out with their own version of a sitemaps tool and rolled into the the Yahoo Site Explorer Beta tool. There is a lot of debate over the value of Google’s sitemap tool and Yahoo doesn’t really make any advances over it. One nice thing about it, though, is that you can now easily get the last-crawled date for any page that you have listed in “My Sites” (you have to have a free yahoo acount in order to access My Sites information).
In order to set up a new site in Yahoo Site Explorer, enter the site’s URL and click “Add My Site”, afterwhich you’re presented with links to Manage and Authenticate your new site. The coolest thing about the tool is that you can communicate to Yahoo that you want it to visit your site and it will go there and grab your feed in “real time”. Your feed can be RSS, Atom, a txt file or a compressed text file (.gz only) and by real time, I mean that you might have to wait a few minutes for it to refresh your screen and show you that it’s verified your feed exists.

The feed is the conduit through/by which you are telling Yahoo about URLs you want it to crawl. Google Sitemaps adds an additional conduit, or feed choice– a .xml file with which you can make your list of URLs dynamic by running a python script on your web server and I expect to see something of that nature coming from Yahoo in the near future. Very basically, though, all you need for Yahoo Site Explorer is a .txt file with a list all the URLs on your site that you wnat crawled (one URL per line) named urllist.txt.. Upload it to your root directory and after you click on “Manage Site” in Yahoo Site Explorer, type “urllist.txt” into the field and off goes the bot to check it out.
Before you can get to the “good” information about your site, you have to authenticate your site. This lets Yahoo know that you currently have access to the site’s rood directory, which means you’re likely to worthy of knowing the any little insights Yahoo Site Explorer might provide you. Whey you click on the “Authenticate” link, you can choose to download an authentication file (which you can save directly to your root directory, if you want) or make your own authentication file with the file name and contents presented. Once the file is placed in your root directory, click “Authenticate” and your site gets put into a pending authenication que until Yahoo crawls the feed. Within 24 hours, I could see that my status was no longer “Pending” but rather, I was now a “processed” site.

On to the real business, Yahoo has racheted up the number of indexed pages to 12. It’s good to see things filling in there. Over on the marginally more useful Google Webmaster Tools, I can see that the index is ranking the old site for some odd keywords, however zero traffic comes from any of his old link partners or search engine listings–at least not from anyone directly clicking on a link.