In my day to day SEO for my clients and for myself, I’ve come across a number of tools of the trade, some really good, some, well, not so good. Here are a few items in my geek toobox that I use daily and highly recommend.
1. Keyword Tool - I have tried them all and this is the one I always go back to for my keyword research. Not only does it give you variations on the keyword phrase you are searching for, but also provides the WordTracker count and daily estimated searches on Google, Yahoo and MSN along with shortcuts to various tools like Google Trends, Keyword Discovery and several other online tools. And, you can export the thing as a CSV file. Way to go Aaron!
2. Check Server Headers Tool – Quick and easy way to check on whether your URL is being seen and followed properly by the spiders. For instance, I recently installed a WordPress plugin which appeared to work fine in a browser, but when I checked the page URLs that it produced here, I found that those pages were producing 404 errors, meaning the web surfer could see the pages, but the spiders couldn’t. Naturally, I ditched the plugin. The site also includes a batch URL processing capability (up to 25 URLs at once).
3. Web Page Analyzer – This online tool checks the speed of your site and lets you know what the download time would be at various connection speeds. Granted, most folks have broadband these days but you still don’t want a page to take several minutes to load on a 56k dial up connection. The test gives you suggestions on ways to speed up your site for visitors and spiders. Both will go away if your site is too slow.
4. Yahoo Site Explorer -Yes, Google gives you some information on sites that link to you, but not like Yahoo’s Site Explorer, which is easy to use and just requires a Yahoo login. You can filter inbound links to see internal or external linking, number of pages Yahoo sees and more.
5. Spider Simulator – Just one of many free online tools offered by this site, I jump here when I need a quick look at what the spiders are seeing. A more comprehensive spider simulator report is available in the iBusinessPromoter client software on my PC, but this online utility serves my purpose most of the time.
6. Tweetscan - These days keeping up with what is said about you and your clients is a must. I use Tweetscan to search for references to me or my clients in Twitter for reputation management, goodwill and networking opportunities.
7. SearchStatus - This is a Firefox plugin that, among other things, allows you to highlight and see nofollow links. This comes in real handy when checking backlinks or sculpting the links on your own site. The plugin includes utilities to check backlinks, Alexa rankings and so forth, but I primarily use the nofollow highlight feature.
8. MyBlogLog - Although the community aspects of the social site are free, I do use one paid service that this Yahoo owned site offers – statistics. For about $25 per year, I can get almost real time traffic stats coming off of web sites. I can see my site traffic nearly as it happens, where surfers are coming from and where they are going. From this, I can see if there is a trend or if something is wrong on a site now, not tomorrow when my Google Analytics stats are refreshed. I mentioned this service in my post on Web Analytics. This is the only non-free tool I mention in this list, but it’s such a bargain, I had to include it.
9. Google Chrome - Though not technically a tool, Google’s first attempt at a web browser has one feature that keeps it open on one of my monitors all day – the ability to log into different Google accounts in different tabs. I keep my domain e-mail, which is hosted through Google Apps, in one tab and Google Webmaster Tools and Google Analytics for my work accounts in other tabs. Now, if Chrome would just pick up some cool plugins!
10. Google Webmaster Tools - For something that I paid little attention to when first released, Google Webmaster Tools is now also open on one of my monitors all day. It just keeps getting better. From tracking down dead URLs on my sites to testing a robots.txt file, I can locate site issues that I wouldn’t otherwise know about. Though far from perfect, it’s just about the most valuable online tool I use these days.
I suspect most of us set up our robots.txt file as basically a one-size-fits-all for the spiders. We’ll instruct all spiders to crawl or not to crawl the same files. For instance, a simple robots.txt file covering all spiders would look something like this:
