What focus groups don't tell you

Neo Insight’s e-newsletter
on Usability topics and techniques.
We invite you to subscribe
to our monthly e-newsletter


January 19, 2006 Usability
challenges of new Web technologies
– Neo Insight Workshop
January 26, 2006 Navigating
User Interfaces
– CapCHI presentation by Dr. Norm Vinson, NRC
February 16, 2006 Designing
Usable Online Services
– Neo Insight Workshop
April 22-27, 2006 CHI
2006, Montreal
QC – Annual conference of the Special Interest
Group on Computer-Human Interaction

focus groups don’t tell you

Web sites provide a combination
of information, interaction, and rich media. Organizations are still
learning how to employ the right mix of techniques to combine these
elements to meet user needs. Focus groups are one useful technique,
although, they need to be used at the right time to capture the right
kind of information.

Focus groups are most effective early in the requirements
process, to gather users’ stories and experiences, their opinions, and
their attitudes. Attitude and opinion help set early design direction.
However, attitude and opinion do not always accurately reflect real
life behaviour. As an example, Eysenbach
& K hler’s 2002 research paper
showed – amongst other things
– that people’s statements in focus groups differed from their actual
behaviour when assessing the credibility of Web sites providing health

This disconnect is for various psychological and
social reasons. In a design process, the problem is that people cannot
access or explain their detailed behaviour (sensory-motor and cognitive)
when performing a task – think of trying to describe the details of
driving from A to B, for example. People will, however, generate post-hoc
explanations, and this is another of the dangers of techniques which
rely on self-reporting.

To get the task-level data necessary for detailed
design decisions requires use of techniques in which user behaviour
is observed. Despite common misconceptions, the most cost-effective
technique is often Usability Testing. In this technique, users carry
out specific tasks with prototype or “live” versions of applications
or Web sites. Measures are taken of task performance, error types and
rates, amongst others. These quantitative measures can be used to set
and monitor performance targets for product acceptance. In our Usability
Testing, we also capture high-definition video of the screen, highlighting
cursor actions, as well as video of the users. These help communicate
the full emotional impact of the user experience to managers or to the
development team!

Usability testing also includes collection of qualitative
data, such as users’ opinions and explanations of their behaviour. Quite
often, one day’s-worth of Usability Testing (about 6 to 8 users) is
enough to identify significant usability issues, and to produce many
design change recommendations.

Related resources:

This entry was posted in User requirements. Bookmark the permalink.

Comments are closed.


  • +1 613 271 3001 Main
  • +1 866 322 8522 Toll-free
  • +1 866 232 6968 Fax

Follow Us    Twitter   LinkedIn

© 2012-7 Neo Insight. All Rights Reserved