Covid-friendly usability testing

Onboard your participants

Although remote usability testing isn’t a new technology, many of us usability professionals haven’t used it before and some guidance on how to get set up is useful. You really don’t want to use up any valuable time at the start of a session, having to explain how to install any software or waiting while your participant goes to hunt out a pair of headphones.

We’ve put together a short guide that we send out a few days beforehand that gives our users all of the information they need to be ready to start. It depends on what tool you use to run your testing as to what needs to be included in these instructions but the main points to communicate to your participants are:

• Make sure they have a quiet space where they won’t be disturbed for an hour

• Test their internet speed beforehand – we usually say that 5Mbps is the minimum to be able to video call and screen share with good enough quality

• Download the necessary software in advance – we include step by step instructions on how to get set up alongside this to account for anyone who isn’t very tech savvy

When you come to recruit participants, you might be slightly constrained by those who have access to good enough internet or a device suitable for your usability testing but by communicating exactly what is expected of them upfront you can still reach a wide audience.

The world is your oyster

Although there may be technical restrictions on who we can recruit, it’s not all bad news because remote testing removes other restrictions!

We have a custom-built UX space named ‘The Curious Lab’ set up in our new office in Salford Quays, kitted out especially for usability testing and designed to be welcoming and comfortable for our participants. However, this meant that we were limited by testing with people who could get to our offices and give us an hour of their time in the middle of a weekday. This ruled out anyone who worked a job that couldn’t get time off in the week, anyone who wasn’t local to the Manchester area, and anyone who might have any accessibility requirements that made it more difficult for them to physically get to our office.

Testing remotely means that we can expand our reach, it’s more likely that people will be able to spare an hour out of their day as they no longer have to travel to and from the usability test and it can easily be scheduled in around other commitments.

Work harder to build a rapport

As anyone who has spent the last 6 months on video calls can attest to, they’re no replacement for real-life human interaction! An important part of facilitating sessions is putting your participants at ease and creating a good relationship so they feel more open about sharing their thoughts with you.

This can be really difficult when you’re doing it through a screen, especially as usually they won’t be able to see your face when they start the task portion of the session.

We’d usually dedicate about 10 minutes at the beginning to introductions and warm up questions but there’s always an extra bit of conversation that happens even before that, when the participant is entering the room, taking off their coat and being offered a drink.

When testing remotely, there isn’t the same opportunity for that type of informal conversation but don’t be tempted to jump straight into your script from the second that the participant joins the call. Even just asking them how their day is going and if they managed to get into the session okay will help them feel a bit more comfortable.

Another simple trick for building a connection with users is to use their first name often when asking questions. It creates a more personal experience and helps reinforce that we’re giving them the full focus of our attention.

Home comforts can work to your advantage

In-person usability testing has the potential to be very daunting to members of the public. Having an appointment they need to be on time for, in an unfamiliar place and have a stranger probing them with questions is enough to put anyone on edge. We go to great lengths to try and make our participants as comfortable as possible but there’s no denying that it can be intimidating.

When we test remotely, participants are usually in their own home and using their own device. Although it is more of a challenge as a facilitator to build that rapport with the participants, we do benefit from them being in a more comfortable environment. Yes, we now have to contend with the possibility of a doorbell going off (on either side!) or noisy traffic outside a window, but we’ve definitely noticed our participants being more relaxed than they would be normally.

Adapt to relying more on verbal cues

Body language is next to impossible to gauge over a video call and that is a problem for both parties when testing remotely; the facilitator can’t as effectively see how a participant is feeling and the participant misses out on non-verbal reassurances from the facilitator. We have to work a little harder to make up for this and in turn, ensure we minimise the impact on our studies.

Some tricks that facilitators rely on to help users feel that we’re paying attention to them such as smiling, nodding and maintaining eye contact while they talk are suddenly taken away from us if they’re looking at a full screen prototype and not focussing on a video call. This means that we have to be a bit more vocal, acknowledge their responses with non-biased statements such as “I see” or by echoing back what they’ve said.

Another challenge to overcome here is for the note takers and observers. It’s a well known fact that what users tell us and what they’re doing often don’t match up. They could be using very positive language but saying something else with their body language or we could notice from watching their hands that they’re not actually completing the tasks very successfully, so we never take their words as evidence alone. When observing a session remotely, you have to be even more aware of the bigger picture and not just listen to what’s being said.

Debrief with the team

With remote usability testing it’s not just the participant and the facilitator that aren’t in the same room as each other, any members of the project team, stakeholders and note takers are also not together in an observation room anymore.

An important part of the process is comparing notes among the team after each participant. Normally, the facilitator would pop into the observation room in the breaks to discuss their view on the session as something might not have come across on camera and they’re the one seeing everything firsthand. Another reason this is useful is that people’s perception of the same test might vary, observers may have different opinions on if a participant was successful performing a task, for example.

These discussions usually happen organically with lab-based usability testing as observers and the facilitator are usually in the same building together. When we’re testing remotely, we have to be a bit more proactive about allowing these conversations to happen.

At Apadmi, we use a tool called Lookback which has a built in chat function where everyone who is observing can record their thoughts or ask questions amongst themselves – as they would do if they were all in a room together. It’s also a good idea to schedule in a quick video call afterwards for everyone to have an opportunity to compare notes and bounce ideas off each other.

More flexibility with the schedule

When we run usability testing sessions in person, we usually dedicate a whole day to them. A typical day of testing starts at 9 am and we run 5 sessions, each lasting an hour, with 15 minute breaks in between and a longer break for lunch.

Getting everything done in one go means that any stakeholder who wishes to observe can come to our office for the day, we can block book out the testing labs and make sure the relevant members of the team are all free. The down side of this is that it’s quite an intense day and can be especially taxing on the facilitator as they’ve got 5 hours worth of talking and concentrating to do!

When we’re testing remotely, we can be more flexible about the schedule. There’s no travel to factor in, no room bookings to work out and we can work to timings that are the most convenient for our participants. The most recent round of remote testing we did ran over 4 days and there were never more than two sessions in a day. As a facilitator, I really noticed a difference in my energy levels and less like I needed a lie down in a quiet room after I was finished!

It might be a while until we’re all back in our offices again, let alone inviting members of the public in to test with us. Luckily, with all the great technology available, we’re able to carry on as normally as possible. Although there are a few challenges we’ve encountered when testing remotely, it does offer a lot of flexibility for recruiting and scheduling participants and it might be something that we continue to do even when the world gets back to normal again. Whenever that may be…

If you’d like to find out more about how we integrate usability testing into our mobile-first approach, check out another of Lizzie’s insight pieces here or head over to our blog to catch-up with Apadmi life.