People who are serious about search engine optimization and online marketing realize the importance of having a timely, relevant and precise data that allows them to respond to search engine behaviour immediately. Having direct access to website traffic statistics allows for “on the fly” page optimisation to maximize current positive search engine behaviour.
There are numerous free and commercially available website statistics packages. They are good in providing historical data, but often lack functionality to provide real time data. In other words, while you can see your website traffic statistics for yesterday, you have no clue on who is visiting the site right now.
Raw Log Files
When choosing a web hosting provider for your blog or company web site it is absolutely critical that the package include direct access to web site access and error log files. Normally these are stored inside your FTP home directory or can be accessed from within a control panel. It should be possible to open these log files in a text editor and see web site statistics from the start of the log (normally log files are rotated nightly) to the current time – right to the moment when the file is opened.
What To Look For
While error logs will show all errors that appeared on your site, such as missing files, broken images or generic web server errors, access log files will show all the details of your web site visitors. These include the IP address, browser type, server response code (e.g., 404 for missing pages or 200 for successful access etc.), and visit referrer details, the last one being the most important piece of information, showing where the visit came from, what keywords were used and so on. To make the job easier, you could search for text string “search” or “search?” and find all leads that came from organic search engine search results.
What To Do With It
While there will be key phrases that you would normally expect to lead to your site. These would be the search terms that your site is optimised for. You can leave these alone – don’t touch something that already works well. What you need to examine, though, is the new key phrases that you didn’t expect or didn’t predict to hit your web site.
Copy the referrer search string back into a web browser and see for yourself what position your web site is on the result page and who your competitors are. Click on the link for your site and see what page it lands (It is important to note here importance of not using the browser’s back button to go back to the search engine. It can be interpreted in a way that the search term was irrelevant to your page and you could be penalised by pushing your site down the search result list).
Next, examine your page how it ended up getting that organic lead. Take advantage of this key phrase, create other pages using the search term as an anchor text and link it back to this page. While you have the lead – use it to your advantage.
Use raw log files information to your benefit. Optimize your web site now, while it’s hot. Then use conventional web statistics package to see a global picture on what happened in retrospective.