About face in usability testing: 3 reasons not to turn the camera on

Posted on November 28, 2014 by Lisa

Are you doing usability testing that includes a video image of the face of the participant? You should stop – here’s why.

I was at two conferences this month where the presenter showed a video clip from a usability test. In both, the participant’s face was recorded with their screen – so the video clips included that image. It made me squirm. We’d just been talking about ethics, and bias, and about ensuring the participant was comfortable during the testing. And here was this image of a participant’s face up on the stage. After years of remote testing without video of faces, well, it just doesn’t sit right with me anymore, and here’s why.

1. You are biased, and so is the client and design team.  There are many aspects of faces that trigger unconscious and conscious bias – you can see the person’s race, gender, attractiveness, clothing style etc.. All of those things can trigger bias in anyone watching the video, including the experimenter!  It can inhibit empathy for the participant, and prevent the developers, design team members, executives, basically everyone, from accepting the participant’s behaviour as, well, normal behaviour. So it defeats the purpose of usability testing.

Don’t think you are biased? Get over it, there’s abundant research on unconscious bias and the many effects on behaviour and response to the person.

2. Faces are recognizable.  Our clients are not supposed to know the identity of our research participants. That’s a fundamental code of conduct for experimental research anywhere. Yet faces are inherently recognizable! Heck, the human brain is far better at recognizing faces than names. To have a privacy policy about protecting the participant names and then showing their faces in a video to the client and to others just seems bizarre.

Even worse, at one of the conferences, the presenter admitted they didn’t have permission from the participant to show their video at a conference – the same conference where a presenter absolutely flamed the UX Community about not following ethical practices. Justifiably in this case!

I too showed a video clip of some usability test participants at a conference. I had digitally disguised the participant’s voices by changing the pitch and speed and of course, there was no camera view of their face.  Since they weren’t recognizable, I was comfortable sharing that video with the audience. But after the flaming ethics talk, I was concerned enough to check in with the Chair of an Ethics Committee about it, and he agreed that since the person wasn’t recognizable, likely even to themselves, it’s ok.

3. Performance is measurable in usability testing,  emotion is not.  Usability testing is about measuring the performance of your web site or application, and identifying usability issues. The performance of YOUR site is the target here, not emotional reactions – keep the focus on actionable results.

When I voice these concerns about conducting testing in person, or turning on the camera during remote testing, researchers often wail “But faces tell us SO much!” Sorry, but they don’t tell you about whether the person can or cannot succeed at this task. They can’t tell you exactly which words or part of the design is causing the problems.

Instead of inferring their emotions from their face and body language, learn to watch the cursor on the screen instead – you’ll learn so much more about the design issues. You can hear their confusion, anger, frustration, happiness etc. in their voice – put it together with what their cursor is doing on the screen, and what they’re thinking aloud, and you’ve got a direct channel into usability issues and evidence-based decision making.

If you are planning some usability testing in the future, I hope you’ll think through these issues and consider NOT recording or showing the participant’s face.  And by the way, it will make it easier to recruit too – there are lots of people out there who are quite comfortable with participating in a session, but are really uncomfortable and balk at having a camera record their face and all that’s going on around them during an hour long session in their home or office environment.


URLs in this post:

[1] The Science of Your Racist Brain: http://www.motherjones.com/environment/2014/05/inquiring-minds-david-amodio-your-brain-on-racism

[2] Why Do We Remember Faces but Not Names?: http://www.sciencefriday.com/blogs/06/10/2013/why-do-we-remember-faces-but-not-names.html?series=28

[3] Marketing Research and Intelligence Association’s Code of Conduct and Good Practice: http://mria-arim.ca/sites/default/uploads/files/MRIAConduct-Dec2007REV2010.pdf

[4] Evidence-based decision making at Cisco suppor: http://www.gerrymcgovern.com/new-thinking/evidence-based-decision-making-cisco-support

Copyright © 2015 Articles. All rights reserved.