If you’ve ever wanted to do a little competitive analysis, you’ve probably had the misfortune of having to view the page source of many pages on a particular website. Searching meta tags isn’t the best way to find the right terms to target for your own website, but it’s a good indicator of what other people are targeting and it’s a perfect start for your keyword research.
You’ve probably done this before, and realized how tedious and boring checking pages of one site can be, so after searching the web for something more automated and not finding anything, a coworker mentioned a program called GSiteCrawler. It’s works fairly well actually.
Note: If you are using Microsoft Vista, you have to right click the program file and choose “run as administrator” or the program won’t work properly.
Here’s how to set it up:
a.) Set up a new project by clicking the “Add” button on the bottom left corner of the screen. Use the New Site Wizard and hit Next on the next screen.
b.) Enter your website URL and give your project a name and click Next.
c.) On this screen, you can leave everything as is, or you can choose the check box next to “URLs are case-sensitive” if you’re on a Linux server. Change anything else if you wish, but I left the rest the same. Click Next.
d,) This program was created to be sitemap generator so the next step offers to upload the sitemap file to your website via FTP. Just skip this and click next.
e.) You can read through all of the options on the next page, but I just checked them all. Click the finish button and the program will begin to scan your website.
There are lot of cool things this program can show you, but for the purpose of this tutorial, we’re only going to look at the Titles and the Meta tags of the website.
Once the scanning is complete, you can navigate to the “URL list” tab on the main screen and then click the “Refresh List” button to show all of the URLs that the program found.
You can click on any of the URLs and it’ll show you the URL, Page Title, Description, and Keywords for that page. The one downfall is if the Title, Description, or Keywords fields are too long, it’ll just cut it off, so you won’t be able to see the whole thing on some sites, but it’s still a good tool for some quick analysis.