How to Do User Testing Remotely

How to Do User Testing Remotely

FROM Digital’s Director of User Experience Dan Willig details how to get the insights you need during the pandemic—at a lower cost as well

Whenever you want to test an idea, whether it’s a website, a mobile application, a kiosk at a museum, or any new product that you’re developing, user testing is one of the best ways to get feedback.

However, it can be expensive and logistically challenging to get users into your facility, and now during COVID that becomes almost insurmountable. 

Remote user testing is a way to still get the insight you need but faster, at lower cost, and with no risk of COVID transmission. 

You just need to know some important things about when and how to use this capability to get maximum results and avoid pitfalls.

In the old days, pretty much all user research was done by bringing somebody into our offices or going to another city to do it at a market research site or maybe a meeting room at a hotel where we would have people signed up to come in. 

But now, more and more companies are doing this via Zoom or WebEx instead because it just adds a lot of cost when you have to fly to 10 different cities to talk to users there. 

It can also be easier to recruit people because they don't have to drive somewhere. They can just do it from their homes.

Although remote user testing is clearly more convenient, there are researchers who feel it isn't quite as good as sitting with people when you can see them up close as they get to those areas that get them really frustrated. 

But in fact, there are remote user testing tools and techniques that you can use to effectively find the points where users are experiencing friction.

When doing remote user testing, just as you would when doing any kind of customer research, you can split your subjects into two groups: moderated and unmoderated.

MODERATED TESTS

For moderated groups, you can just get on the phone with someone and share links with them. 

Have them walk through things, share their screen and talk about their experience. 

It has a lot of the benefits that you get from in-person user testing in the sense that it's conversational.

If someone is doing something, you can ask them what they're thinking at that point and how they felt after they completed a task or weren't able to.

You can also ask them a little bit about what their expectations are, if they are to go to this other area of the experience that you're testing a prototype for but you haven't fully fleshed out yet. 

It’s a great way to see the physical cues and the body language when you're able to do screen sharing.

And these sessions are really easy to schedule. You can talk to someone in California and then 10 minutes later you can be on the phone with someone in Michigan. 

Also, there's that consistency of the moderator that you don’t get when you have to break up the team and different people will be moderating different sessions. 

You get great results still, but without that consistency, it would seem like you're getting a little bit of hearsay. 

When you start to look at the data, you wonder what exactly happened in that conversation. 

Was the user asked this follow-up question? There may be information that you think could be useful but wasn't captured.

UNMODERATED TESTS

However, there are times when you need to talk to a lot of people and that kind of setup is just not feasible anymore. 

There may also be a specific task that you're trying to get specific measurements on. 

While you can do that through remote moderated testing, it’s going to take some extra footwork to be done.

That's where unmoderated platforms come in. 

Essentially, what you do is create a script for what it is you're trying to test and set all the parameters up. 

Of course, everything has to be tied back to your business goals for your product, whether it’s an app or a website on desktop or tablet or mobile.

Introduce the variants that people are going to be having. 

For example, your script could start with: “Thanks for taking your time out of your schedule. We're going to be asking you a few questions to help us improve the experience of this website or app.”

And then, give people the exact instructions for the series of tasks that you're going to give them. 

It could be: “Go purchase these tires. And when you're just about to enter your credit card data, stop.” 

That could be one task. 

Oftentimes, you’ll also need to include this reminder in your script: “We'll be capturing your audio and video as you're going through this process. So, please remember to share your thought process and your frame of mind as you're going through the steps.”

The big disadvantage of the unmoderated test is you're not there to probe and ask those questions. 

But it's helpful to make sure that you're prompting people of several points throughout the script—tell them to share their thoughts—because people forget. 

Maybe things are working very well and they're excited to just finish that they're forgetting to tell you what they're thinking about all of this. 

You won't know that until you get the final video.

Most of the available platforms for unmoderated tests have user panels that you can recruit from. 

This can be international and you can focus on a particular continent or region if you want.

What you can do is set up demographic controls. 

For example, maybe you want to look for men and women in the Northeast who own a car and have owned it for the past two years. 

You can set all of that up and then the platform will look out at its user panel and only recruit those people who fit those criteria. 

They will be the ones who get the link to go to your test. Once they click the link, they can proceed to take the test.

One of the big advantages of using this method is it makes your analysis much easier.