Lab based sessions are the realm of the big budget client. The facilities are comfortable and like a scene out of Mad Men the client can often observe participants through a one way window. Things can also get impressively scientific. In fact, in most cases way beyond what’s actually needed for the research objective.
In reality many clients are still reluctant to include user research at all in their already tight budgets. The benefits and comparative value of remote, moderated research for both empathy and task based sessions for new and existing products are huge and it’s an argument I push on a weekly basis.
The following (which I’ll unashamedly be pointing clients too in the future) is my reasoning for supporting the remote, moderated approach:
How it works
Subjects are tested in an external location to the facilitator on a one to one basis, on their own devices and in a location of their own choice. The key requirements for the session are an audio connection, video connection and realistically the ability to record both. My personal preference for such a tool is to use screen sharing via Go-To-Meeting but there’s plenty of alternatives out there.
Establish that connection and you’re ready, fire away with any instructions, questions and let your participant complete the tasks you require.
A facilitator should be experienced in “Think Aloud” testing, were the participant is asked to “commentate” on their thoughts and experience as they complete tasks.
In the past I’ve found lab based testing a little uncomfortable as a concept. I spend so much time ensuring solutions are designed so it’s a natural, intuitive experience for the user yet when I’d come to test them I’d watch as your average Joe was hooked up to the Matrix before being asked to add some products to a basket.
Whilst that’s an obvious exaggeration you can see what I’m getting at. To test a user experience we want the subject in their natural habitat. Whether that’s feet up with an iPad or in the office on a lunchbreak.
Remote testing gives us that freedom and gives us a true reflection of user behaviour whilst also potentially identifying issues that would get missed in a high-tech laboratory environment.
Greater Reach & Ease of Recruitment
Remote testing opens up a Worldwide audience. Even with a narrow target audience there is an obvious increase in reach through remote testing.
Add to this the inconvenience of a lab based session for the subject. Persuading a subject to take time out of their busy schedule and get themselves to a usability lab tends to mean some form of compensation. With a bit of flexibility on the side of the facilitator remote tests can be carried out at the convenience of the subject, greatly increasing the likelihood of participation.
The reality is outside of big budget organisations lab testing simply isn’t viable. Despite the protests of the UX resource both agency and freelance experience suggests user testing is still one of the first project tasks to be axed when costs need to be cut.
The results of such projects tend to end up back with the said resource later being employed to perform a “Usability Review” tasks. Often prior to another expensive design and development process aimed at fixing the mistakes made through lack of testing the first time round.
For clients who are willing to invest in user testing but do have a limited budget a remote, moderated approach is beneficial in terms of saving on lab costs and the reduced cost in compensation for participants (a subject is much more likely to participate if they can do so lounging on their couch rather than having to trek to a testing lab)
And the Weaknesses…..
Connectivity & Technology
A fair few years ago online meeting solutions were busy claiming the physical meeting was about to become a thing of the past. Whilst that’s not strictly true remote teams successfully converse throughout the working day across the World. There’s still invariably the odd connectivity issues though and throw into the mix testing with users who aren’t comfortable with the sometimes complex process of launching online meeting applications and a testing session can become a frustrating experience.
Furthermore, mobile and tablet testing presents it’s own set of testing challenges that even UX laboratories are only just coming to terms with. For the time being the tools available to facilitators are inappropriate for a remote session on mobile.
The Human Touch
You’ll probably never meet your test subject in person. Does this matter? In terms of facilitation it shouldn’t. Our focus should be on observing the participant’s actions and how they perform any tasks set. A facilitator who is keen to impose their personality on a participant is missing the point of the exercise. If body language is important to a test it’s entirely possible to video the participant as they are taking a test but again in my experience participants are more than willing to voice their issues without prompting.
Is Remote, Moderated Testing the right approach for you?
If you’re carrying out empathy interviews to understand user behaviour or testing the high level usability of a website, moderated testing is a valuable and cost effective approach.
You’ll find it much easier to reach a sample of testers across your market and they’ll be carrying out tests in their chosen environment.
More importantly the gap between what’s possible in a laboratory and what can be achieved remotely is getting smaller all the time as dedicated tools and techniques are created.