Remote User Testing experiment

Tomáš Veselý - podpořen AI
1 day ago
3 min read

We're building a comprehensive knowledge library about product development as part of our mission. The library is for anyone looking to make better decisions — primarily decisions about product development. Whether you're an inventor, a product manager, or a Chief Product Officer, using the right research methods and experiments increases your chances of building the right things for the right audience. Today we'll introduce the Remote User Testing validation method.

When to Use This Experiment?

Remote User Testing can be deployed at any point in a product's lifecycle when you need to verify whether real users can complete key tasks without assistance. It's especially useful when:

feedback is needed quickly and at scale — dozens to hundreds of tests can run in parallel.
the target group is geographically dispersed and meeting in person would be costly or slow.
neither budget nor time allows for live moderated testing.
the goal is to validate usability and feasibility, not desirability or willingness to pay.
you want to observe how users behave in their natural environment rather than under a researcher's watch.
the product already has traction and users.

Basic Experiment Principles

The core of the experiment is letting respondents work through pre-prepared tasks on a product or prototype on their own, while their screen, voice, and reactions are recorded. The test is unmoderated — no one guides the respondent, and the only support is an opening brief.

Define the goal and hypothesis. Clarify exactly what is being tested and which assumption is being validated.
Prepare the tasks and brief. Formulate specific tasks, supporting questions, and a brief introduction to the context. The instructions must be understandable on their own, because the respondent will have no one to ask.
Choose respondents and a tool. Find participants from the target audience in relevant communities and forums, or through one of the remote testing platforms, and pick a tool that records the testing session. The most widely used include UserTesting, Maze, Lyssna (formerly UsabilityHub), Hotjar for session recordings, and Preely for testing early prototypes.
Deliver the test. Send the instructions to respondents, who complete the tasks in their own environment and time while the session is recorded.
Collect data. Track task success rate, time on task, error frequency, and qualitative friction signals — especially confusion, hesitation, and repeated clicking out of frustration (rage clicks). The decision signal: if most respondents complete the key tasks without help and the same obstacles don't keep recurring, usability is confirmed; if, on the other hand, many people fail at the same step, the design needs to be reworked and tested again.
Identify risks. The unmoderated format doesn't allow you to ask follow-up questions or clarify instructions, so any ambiguities go unresolved. Respondents may abandon the test early and need their own motivation to finish it. Artificially assigned tasks may not match real usage. Small samples yield directional rather than robust findings, and hired testers don't always precisely match the target audience.

Real-World Experiment Example

Link to research: KeyBank + UserTesting

The American bank KeyBank used a software upgrade to add touchscreens to its ATMs, but the now-defunct physical buttons remained along the sides of the display. Customers kept pressing them, didn't understand the change, and satisfaction measured by NPS (Net Promoter Score) dropped.

Instead of a costly hardware replacement, the in-house Key Design Studio used the UserTesting platform to run research directly at branches, where real customers worked through tasks on the new front panels and denomination-selection screens, and the team iteratively refined the design based on the feedback. The bank rolled the improvement out across its fleet of 40,000 ATMs in under two months and raised NPS by 44%.

What Can Be Tested With This Experiment?

The method's main strength is revealing where real users hit obstacles in a product — quickly, cheaply, and on a larger sample than moderated testing allows. Specifically, it's well suited to validate:

Usability of key flows: whether users complete a core task (registration, purchase, form submission); the signal is a high completion rate without assistance.
Friction points (trigger points): where users get stuck, hesitate, or click furiously; the signal is the same obstacle recurring across respondents.
Clarity of the interface and copy: whether people understand labels, buttons, and navigation terms; the signal is correct interpretation without having to ask.
Information architecture and navigation: whether users find a feature where they look for it; the signal is finding the path without wandering.
Prototype feasibility before development: whether the concept holds up in real use before developer time is invested in it.
Comparison of design variants: which version of a flow or screen leads to higher success; the signal is a difference in completion between variants.