The Blog

Usability Tests vs Focus Groups

applesoranges1

It’s amazing how often people confuse Focus Groups with Usability Testing. Let’s set the record straight. What’s the difference?

Focus Groups

Focus groups are group discussions. You recruit several targeted users, get them in a room together with a moderator, and ask them questions.

Focus groups are wonderful for:

  • Learning how customers feel about your products, including their past experiences with them.
  • Generating ideas for new products or features (or getting reactions to your ideas).

 
For example, let’s say Facebook wanted to identify new features their users might like. Holding a series of focus groups would be a great way not only to get feedback on ideas you’re thinking of implementing, but in generating ideas you’d never even thought of.

Focus Groups are best used early in the process, i.e. before development (or re-development).

Pitfalls of focus groups include:

  • You only get subjects’ stated opinions. Sometimes, what people say is very different from what they’ll do. (Famously, the Edsel tested very well in focus groups.)
  • Participants may influence each other. Too often, one or two opinionated individuals will dominate the group. Other participants – who may have better insights – fear to differ.

 
It might sound like I’m dissing focus groups, but that isn’t my intent. Focus groups can lead to valuable insights. But they’re often used inappropriately. They are not, for example, the best way to find usability issues on a website.

Usability Tests

Usability tests are totally different. First, they aren’t groups at all; they’re conducted one-on-one.

And you don’t simply ask user’s opinions on your site. Rather, you to observe how people actually use and react to your site.

In a typical test session, a moderator sits beside the user and assigns a series of realistic tasks. The subject is asked to “think out loud” as he completes the tasks. The moderator carefully observes what the user is doing, and can ask follow-up questions, etc. (This is critical, as it helps uncover why users are stumbling, how they feel about the site, etc.)

Obviously, you don’t just do this once. You’d typically run 5 – 10 such tests, and you’d have several rounds at different stages of development.

Advantages of Usability Testing Include:

  • It can (and should) be done at any stage of development: from wireframes through to live sites.
  • It is the perfect complement to analytics: Analytics tells you what is happening on a website; user testing tells you why.
  • Recorded sessions are great for convincing doubtful or reluctant members of the development team that the site has problems that need to be fixed.

 
Sometimes other methods are more appropriate: Surveys, interviews, card sorts, eye tracking studies, remote testing, automated online tools… But one-on-one usability testing will always be the usability practitioner’s most powerful tool. There’s simply nothing better than actually watching someone use your website, and being able to ask live follow-up questions.

So Why the Confusion?

It’s really funny how often people mix the two things up… even though they are so dramatically different:

  • Focus Group:  Get a bunch of people together and ask their opinions on stuff.
  • Usability Test:  Give a user something to do and watch him do it.

 
The confusion is puzzling. But it is so widespread, Steve Krug produced a wonderfully funny little video about it. Check it out:

  • Allen

    Great post Michael, I appreciate that you clearly identify the advantages and disadvantages of both types of research (because ultimately, that is what both focus groups and usability testing is all about, right?).  You suggest that usability testing should be done throughout the development process — do you have any specific recommendations about continuing usability testing in production?  In what ways do the goals of usability testing differ between testing during development and testing on a live site?

    • Michael Straker

      Hi Allen,

       

      Thanks for the excellent questions.

       

      My #1 recommendation: Get Steve Krug’s excellent book, “Rocket
      Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems”.
      Here are some others (and they come mainly from Steve’s book):

      TESTING DURING DEVELOPMENT VS TESTING LIVE SITES

       

      Early in development, it’s easy to make big changes. So you
      can focus on the big stuff: the site’s information architecture, task flows,
      etc. The closer the site is to completion, the harder it is to make changes.
      (This is precisely why you should start testing as early as possible, on
      wireframes or even scribbles on napkins. As Steve Krug writes, “Start earlier
      than you think makes sense.”)

       

      When you’re testing a LIVE site, you need to be realistic
      and focus only on the most important issues, and those that can realistically
      be fixed within a reasonable timeframe. Some issues may have to wait until the
      next website iteration.

       

      RECOMMENDATIONS ABOUT CONTINUING USABILITY TESTING

       

      The trick it to make usability testing a standard part of
      your practice, a habit. Specifically:

       

      1.  Do it regularly
      (Steve recommends one morning a month).

       

      2.  Make it fun, get
      the whole team involved.

       

      3.  Keep it simple. (You
      don’t need a fancy lab, video editing software, or even users that precisely
      match your ideal target group. And most of all, you don’t need to write a huge
      report. )

       

      4.  Focus on fixing
      the most serious problems first. Don’t get bogged down trying for perfection.

      I hope this helps. But whatever else you do, get Steve’s book. 

      • Allen

        Hi Michael,

        Thanks for the quick response and great recommendations.  I will be sure to check out Steve’s new book.  I certainly learned a lot from “Don’t Make Me Think”, and it sounds like “Rocket Surgery Made Easy” will be a resource of equal or greater value.  

  • Allen

    Great post Michael, I appreciate that you clearly identify the advantages and disadvantages of both types of research (because ultimately, that is what both focus groups and usability testing is all about, right?).  You suggest that usability testing should be done throughout the development process — do you have any specific recommendations about continuing usability testing in production?  In what ways do the goals of usability testing differ between testing during development and testing on a live site?

    • Michael Straker

      Hi Allen,

      Thanks for the excellent questions.

      My #1 recommendation: Get Steve Krug’s excellent book, “Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems”. Here are some others (and they come mainly from Steve’s book):

      TESTING DURING DEVELOPMENT VS TESTING LIVE SITES

      Early in development, it’s easy to make big changes. So you can focus on the big stuff: the site’s information architecture, task flows, etc. The closer the site is to completion, the harder it is to make changes. (This is precisely why you should start testing as early as possible, on wireframes or even scribbles on napkins. As Steve Krug writes, “Start earlier than you think makes sense.”)

      When you’re testing a LIVE site, you need to be realistic and focus only on the most important issues, and those that can realistically be fixed within a reasonable timeframe. Some issues may have to wait until the next website iteration.

      RECOMMENDATIONS ABOUT CONTINUING USABILITY TESTING

      The trick it to make usability testing a standard part of your practice, a habit. Specifically:

      1.  Do it regularly (Steve recommends one morning a month).

      2.  Make it fun, get the whole team involved.

      3.  Keep it simple. (You don’t need a fancy lab, video editing software, or even users that precisely match your ideal target group. And most of all, you don’t need to write a huge report. )

      4.  Focus on fixing the most serious problems first. Don’t get bogged down trying for perfection.

      I hope this helps. But whatever else you do, get Steve’s book. 

      • Allen

        Hi Michael,

        Thanks for the quick response and great recommendations.  I will be sure to check out Steve’s new book.  I certainly learned a lot from “Don’t Make Me Think”, and it sounds like “Rocket Surgery Made Easy” will be a resource of equal or greater value.  

Copyright © 2014, All Rights Reserved. Privacy and Copyright Policies.