Shostack + Friends Blog Archive

What I Want From A Log Analyzer

I’m becoming less and less satisfied with AWStats as a log analyzer. There are some things that it does reasonably well. But I’d really like a lot more. I’d like to be able to see how things have changed day to day (for example, how many new unique visitors did I get today?) I’d like to see that in a variety of contexts. (Percentage increase, comparison to other days, ‘this is your best day since…”)

I’d like to be able to spin the data based on a selection. For example, I’d like to select a page, and see how many hits it’s received, how many that is as a percentage of my daily or monthly hits, what search terms and pages have linked to it. Is its hit count rising or falling? What Google searches is it doing well in? (For example, Daniel Cuthbert’s Chewbacca Defense has fallen from about #6 in Google’s searches to #13. Perhaps optimization is in order (like a fresh link)?

There’s more after the break.

Similarly, I’d like to be able to look at a search term (say, security) and see what pages people have arrived at using that term.

I was hoping maybe MeasureMap would be better, but I haven’t gotten an invite yet. Although, “Measure Map Preview” says:

Measure Map also shows me incoming links to my blog, and hopefully, since it relies on JavaScript, I won’t be seeing those referrer spam links I always get with other tracking software, the downside of this is, that if a visitor has JavaScript turned off, the visit won’t be logged at all, but I can live with that.

I don’t think I can. I encourage people to turn off Javascript. The web is far safer without it, not to mention generally more pleasant to use. Using PithHelmet, you can turn it on for the few web sites that really need it. [Update: As Tim Bray points out in “You’re Being Watched,” the set of Javascript solutions don’t track people reading via RSS/ATOM/LiveJournal/Bloglines/radio userland. That’s only about 85% of the readership.]

So, what I’d like:

• A database driven analyzer, with the ability to understand and enhance the queries that it uses.
• Locally hosted. My logs are none of your business. (Although they’re probably exposed, I could fix that.)
• Log driven, without changes to my site. I don’t want to insert your code onto my pages, or rely on the client to do something to understand their behavior. (I’m willing to hear arguments that this is wrong-headed of me, but it seems that what I want should be possible with logs, and that logs are the right approach.)
• Unix server and web or Mac GUI based.
• Inexpensive. I’m willing to pay a little bit for the right tool, but this isn’t a commercial venture.

So what should I be looking at that can do better than AWStats for me?

Originally published by adam on 16 Nov 2005
Last modified on 16 Nov 2005
Categories:   blogging   Uncategorized   Usability