A small case for qualitative analytics.

Handaxe extremely archaic from the Quaternary ...
Image via Wikipedia

A lot of the work I do is focused on web analytics. When making things for online consumption, one of the few ways to know whether anything is working as intended is via web analytics. It’s our only way to observe how humans are really interacting with the things we make.

And while Google Analytics and other tools like it are great at getting the quantitative story–how many people saw what and did something–it isn’t very good at getting the qualitative story. The qualitative story would let us know how much people liked (or didn’t like) something we make. It might let us know more about the intent of audience when they found the things we made. Why did they use it? Did it work? Why so, why not? That kind of thing.

I use iPerceptions 4QSurvey tool. And I set the rate fairly high–if you’re reading this on Thoughtfaucet blog, you’ve probably been asked to fill out the response at some point. 4Q has an open-text-box as part of it’s data gathering survey. It isn’t uncommon to see people leave comments about the survey itself. It isn’t uncommon for those who leave comments about the survey to be unhappy about something (usually that it asks about whether you want to take the survey when you first arrive on the site, then gives you the survey to take when you leave the site).

Sorry unhappy survey-takers, the value of the information I get about what tasks people were trying to accomplish on my site and whether they were able to accomplish it or not is worth a couple unhappy survey-takers. It’s often easy to get worked up when the unhappy survey-takers unleash their unhappiness into the open-text-box. It’s easy to second-guess whether you should be getting that qualitative data when it might mean someone is upset. I tend to take the view that someone is upset before they take the survey.

That, and then there’s this: Thoughtfaucet works every day making things that can be measured–things where the success is important to a specific business metric. It’s important to walk the walk and so gathering data about how this site is doing makes sense. That’s the story from Thoughtfaucet’s point of view: we’re gathering useful observations about how well (or not well) we’re doing.

From the audience perspective it might look like what someone recently typed into the open-ended text-box in 4Q yesterday:

I love that you care enough to ask these questions.

That’s a response from someone who trusts that Thoughtfaucet is using that data to improve and make things people like. You can be doing the same thing for your online presence.


  1. Posted December 15, 2009 at 1:36 pm | Permalink

    Great write up! Finally someone’s thinking on lines of qualitative measurements & VOC. How do people draw insights without qualitative data? I like Survey Monkey too..very simple and helps me to drill down a lot. I am a die hard Google fan but not when it comes to 2 things 1) qualitative data 2) site overlay..
    They simply stink.

    Good Luck with all your venture.
    Bibi Mukherjee
    twitter @wookiesam

    • Posted December 15, 2009 at 2:56 pm | Permalink

      Thanks for stopping by Bibi. Great meeting you at #BTVSMB today. Also, I’m glad someone asked the presenter to clarify re: unique visitors vs visits vs branded visits.

      For site overlay I’m a big fan of CrazyEgg. GA’s site-overlay is disastrously useless–maybe even misleading in that one might mistake it for showing where on the page people click.


  2. Shannon Davidson
    Posted January 5, 2010 at 6:07 pm | Permalink

    Interesting post. Now I really want to take your survey! When do you think there will be a qualitative analysis product that will be a little more integrated with a website — so it doesn’t feel like you’re taking time to do a survey (even if you are) — and what do you imagine an integrated qualitative analysis product to look like?


  3. Posted January 7, 2010 at 4:58 pm | Permalink

    Hi Shannon thanks for stopping by and hope you’re doing well! Great question. Probably worth a whole post on it’s own. Let me mull it over. My guess is it would look a lot like social crm mashed up with web analytics.

Post a Comment

Your email is never published nor shared. Required fields are marked *


You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>