Cognitive Task Analysis experiment

Tomáš Veselý - podpořen AI
1 day ago
4 min read

We're building a comprehensive knowledge library about product development as part of our mission. The library is for anyone looking to make better decisions — primarily decisions about product development. Whether you're an inventor, a product manager, or a Chief Product Officer, using the right research methods and experiments increases your chances of building the right things for the right audience. Today we'll introduce the Cognitive Task Analysis validation method.

When to Use This Experiment?

Cognitive Task Analysis is suited to situations where success depends on how a user thinks, decides, and assesses a situation, not just on the physical steps they perform. The method works well for tasks with a high degree of uncertainty, complexity, or expertise.

The product addresses a complex task where what an experienced user bases their decisions on isn't visible on the surface.
There's a large performance gap between novices and experts, and you need to understand what drives it.
Users work under pressure, with incomplete information, or in high-risk environments.
The existing interface leads to errors or unnecessary cognitive load whose cause isn't obvious.

Basic Experiment Principles

The essence of the method is to make the invisible visible, to uncover the thought processes, cues, and heuristics an experienced user relies on when completing a task, and then use them to build a value proposition. It works as follows:

Define the domain and select experts. Map the task and find domain experts (subject matter experts) — people who perform the task at a high level.
Prepare realistic scenarios. Put together specific situations to walk the expert through during the interview and use to examine their reasoning. It's best to draw on real or plausible situations in which key decisions play out.
Conduct interviews. Using structured interviews and observation, walk through specific situations and examine the decision points, the cues noticed, the mental models used, and the alternatives considered. Questions like "What were you looking for at that moment?", "What made the decision harder?", or "How did you know something was wrong?" help.
Build cognitive maps. Qualitative outputs are sorted into coding schemes and visual maps (decision ladders, cognitive flowcharts). It's worth tracking cognitive load, task-completion efficiency, and alignment with the user's mental model. A signal to build a product is a recurring decision point where experts hesitate or where novices systematically go off course. That's exactly where you can create a value proposition that helps with the decision.
Translate the findings into product design. The findings feed into product requirements, interface changes, or support aids that match the real cognitive demands of the task.
Identify risks. The method is demanding in both time and expertise, and it lives or dies by the quality of the experts chosen — a small or unrepresentative sample produces skewed conclusions. Some expert knowledge is tacit and hard to verbalize, so part of the decision-making logic may go undetected. The volume of qualitative data is large and its interpretation is subjective. On top of that, the results describe how the task is done today, not necessarily what the ideal future design should look like.

Real-World Experiment Example

Link to research: Lessons Learned from Customizing and Applying ACTA to Design a Novel Device for Emergency Medical Care

As part of an industrial (commercial) project, a new combined device for pre-hospital care was being developed to merge a vital-signs monitor, a ventilator, and a resuscitation unit into one. The goal was to reduce the physical and mental load on emergency teams, who today have to carry and operate several heavy devices, each with its own screen, during a callout. The key question was how to design an interface that matches how a team under pressure actually thinks and decides.

Over the course of a year, the team applied a fully digital version of Applied Cognitive Task Analysis (ACTA). They built a detailed hierarchical task diagram for a large-scale rescue scenario, ran eleven interviews with subject matter experts (experienced paramedics), and held two design workshops. The whole process was supported by cross-disciplinary teams that combined expertise in pre-hospital care, requirements engineering, and medical product development.

The result was 34 sketches and three mockups of the combined device's user interface, built on the cognitive demands uncovered what the team needs to see at a given moment and where the load is highest. In this way, CTA translated the experts' tacit knowledge into concrete product design requirements.

What Can Be Tested With This Experiment?

The method's main value is that it reveals the hidden cognitive demands of a task, the things that determine success but never surface in ordinary observation or a standard user test. Specifically, it's useful for validating:

Alignment with the mental model: whether the product's structure matches how the user pictures the task in their own head; the signal is that experts describe the process the same way the interface guides them through it.
Cognitive load: which steps overload working memory or force the user to hold too much information at once. These are the ideal candidates for automation.
Decision points and cues: which cues an experienced user relies on at critical moments, and which of them the product fails to support.
Tacit knowledge and heuristics: the shortcuts and rules experts apply automatically and that are missing from official procedures and from novice training. These shortcuts and rules are an ideal source of inspiration for new features.
Error proneness: where a novice's reasoning systematically diverges from an expert's, and where failure is therefore likely. These points in the product must be tested specifically to guarantee their value for both novices and experienced experts.
Information architecture: what data must be available at a given moment so the user can decide correctly.