What is so important about User Testing?
Why do companies spend time and money to run seemingly obvious tests? It is common to hear of them running A/B tests on simple navigational issues or conducting an experiment to determine which call-to-action button works best. Then, there are the big-ticket items like feature and product testing. From a casual observer, these tests may appear pointless. After all, the resources spent in User Testing can accumulate rather quickly and for what? To identify a possibly obvious solution?
To understand why User Testing is a significant factor in the birth and life cycle of any product or service is to understand that on a salient level, it deals with human behaviour. And if there is one thing predictable about human behaviour, it is that humans tend to behave unpredictably. User Testing helps the UI/UX designer and their team to reveal the unpredictable and plan accordingly instead of acting on blind impulse or assumptions.
But it is entirely possible to mess up the user testing flow. So, the question on the collective business consciousness shouldn’t be, is user testing important, instead, how do we ensure that the time and money spent in testing will yield valuable results?
In this article, we will look the different aspects of user testing to understand what it is and what it is for toward the goal of delivering a product that your customers will love and reward you for.
First, what IS User Testing?
User Testing, as its name suggests, is a user-centered technique to evaluate a product by testing it directly on users. And if you’ve heard of the term User Testing, you probably would have heard of the term Usability Testing as well. These two terms are often used interchangeably and as a result, people tend to veer towards the conclusion that they are the same thing.
But are they?
The short answer is no. While user testing and usability testing can sometimes have similar techniques and approaches, they address a very different question.
User testing focuses on the product or service’s market need/demand, in which your user asks themselves, “Do I need this product or service?”
Whereas, Usability Testing focuses on the product or service’s platform usability, in which your user asks themselves, “Do I know how to use this product properly?”
For example, if we take a food delivery app, User Testing will find out if the people need or want a delivery service in the first place. If it turns out that most have trust issues when it comes to a stranger delivering food to them, the product might not be successful, despite having excellent usability design.
On the other hand, Usability testing will uncover if the users are able to operate the food delivery app properly to get the results and benefits that were promised without platform frustration.
Even if your product or service is highly regarded, bad usability will frustrate the users and prompt them to look elsewhere for the same service, giving your competitors the opportunity to steal your customers by using the same product idea, but with better usability.
For example, the Grab app versus Gojek. Broken down into its simplest form, Gojek’s entry into the market relied not just on its ability to provide competitive pricing and drivers for the passengers, but also how functional, efficient, and user-friendly its app is. It had to compete, arguably on a product level with the incumbent Grab app. Its success in the market hinged on its usability because user testing previously (done by Grab and to an extent, Uber), already showed that there’s a demand for these types of services.
While it’s not important to fixate on semantics, it is important to be clear on the testing objective(s). Knowing the difference between testing a market need or a platform usability will help focus your testing journey, and ultimately, your whole product development plan.
Now that we know what user testing is and for, let’s dive deep into the four testing approaches.
User testing approaches
1. Qualitative vs Quantitative research
Let’s look at the main differences as well as the common techniques and thought process below:
QUALITATIVE RESEARCH
What is it:
Makes use of numerical data to track and measure, with metrics often shown in graphs or charts. They usually deliver conclusive, measurable results, but on the downside, are not so explorative.
How to conduct:
- Surveys
- Questionnaires
When to conduct:
- When formulating your ideas and assumptions
- When you need quick, easy answers to general questions
QUANTITATIVE RESEARCH
What is it:
- Interviews
- Observational research
When to conduct:
- When validating your ideas and assumptions
- When you want to look beyond the issue at hand
2. Moderated vs Unmoderated Testing
In a nutshell, Moderated testing is more observational, hence it is used to understand the reasons behind the users’ behaviours. Unmoderated Testing IS used to test the user’s reactions towards the product.
Let's break it down a little more:
MODERATED TESTING
What is it:
Testing conducted by moderator/ facilitator on site to guide the user through the tests
Pros:
- Easier to test on more complex products
- Produces more product insights (follow up questions, observations
- Can offer technical support for participants if needed
Cons:
- Time-consuming and costly (especially in person)
- Require the users to travel unless it is a remote testing
UNMODERATED TESTING
What is it:
Testing without any external involvement, often with the help of automated tools
Pros:
- Able to get quick feedback on simpler products
- Users feel more autonomy and feel more at ease, producing more genuine results
- Cost & time friendly
Cons:
- Same questions / scenarios for all the users that might introduce some bias
- No user support provided if needed
3. In-person vs. remote testing
In-person testing is usually always the preferred method but sometimes, you might be involved with testing someone who isn't in the same country or who lack the time to travel down to the office for an in-person session. With Covid-19 and most people having to work from home as well, remote testing becomes the default method. Let's look at what this entails:
IN-PERSON TESTING
What is it:
Participants & moderator in the same location, in a dedicated space (research lab, office, home etc)
Pros:
- More user observations
- More support for both ends as both are in a dedicated space
- Can be adapted to each user's profile
Cons:
- Need to manage logistics & preparations
- Limited participants for the locations
- Higher cost and time consuming
REMOTE TESTING
What is it:
Participants & moderator not in the same place, performed over a conference call with screen recording / sharing
Pros:
- Cost & time friendly
- Users observed in their natural environment, less pressure
- Faster results
- Flexible organisation
Cons:
- Hard to fix technical issues
- Technical dependencies & limitations
- Task limitations
- Less behavioural insights, leading to more guessing
4. Explorative vs. Assessment vs. Comparative Testing
These are a series of tests used for different contexts and different objectives:
Explorative Testing
Often used during the discovery phase to collect new insights about a product – participants are asked to brainstorm, give their opinions and their feedback about new ideas and concepts.
Assessment Testing
Used after the development of a product to assess the user satisfaction about a product after using it.
Comparative Testing
Used during any phase to evaluate the preference of the users on different products.
Which User Testing approach is right for you?
As you can tell, there are many testing formats. How do we determine, then, which one is best suited for the project?
Apart from your testing objectives, there are a several factors to consider when choosing the right approach. These are budget, time, user context (how well do you know the users) and user autonomy (how much guidance will users need during the tests).
Once you measure these four factors, you can then decide with the help of the testing approach matrix guide below:
With that, we've come to the end of the User Testing approach. In our next article, we'll dive deeper into the 8 User Testing Methods & Techniques that work!.
If you prefer to watch the entire presentation on User Testing, you can check out our recorded webinar!
