The Perils Of Measurement

Although timing is a valuable tool for some types of testing, it is not the full story. Ultimately, it is why users slow down that is more important than how much time they actually spend.

The Perils Of Measurement

Most 'usability' practitioners quite rightly put great emphasis on the ease with which sample users can complete pre-selected tasks. After all, it makes sense to ensure that common tasks, identified after research into user needs, should be completed as efficiently as possible. It is when this 'efficiency' is measured that problems can begin.

Counting Clicks

It used to be standard practise to recommend that such tasks could be completed within a certain maximum number of 'clicks'. The fewer pages a user was required to navigate through, the more efficiently a given task could be performed. But a moment's reflection tells us that judging efficiency by number of clicks is a recipe for broad, 'horizontal' navigation structures that offer the user hundreds of options at the home page. In fact the logical conclusion of the 'click measurement' approach is a simple list of every page on the site - all a single click away.

Experience tells us that such a system is not necessarily the most effective design strategy. Unsurprisingly, broad navigation schemes, with many options, tend to leave users spending more time choosing which link to follow. Large lists of links are also more likely to lead to the sacrificing of accompanying text that can help guide users through a site and actually speed up task completion or information retrieval.

Timing Tasks

If judging a site's usability by counting clicks isn't such a good idea, what options remain? The obvious solution would be to use the time taken to complete a task as a useful indicator. This would certainly be preferable to clicks, but there are problems here too.

An individual's perception of the passage of time is extremely subjective. What might take 30 seconds can feel like a lot longer to a frustrated user, and unless your customers are stopwatches it is the perception of time that really counts.

Experimental evidence from a number of disciplines repeatedly illustrates that individuals who feel in control of an activity that is leading to a successful conclusion will always underestimate how long the task took. By contrast, those who flounder or wait without feedback, before stumbling accidentally upon the solution, will tend to feel the process took much longer than it did.

A New Approach

Aside from this difficulty, it should also be obvious that time itself is not a particularly accurate means of measuring the quality of an interface. Some tasks may be completed quickly in a way that leaves the user unsatisfied or unsure. Conversely, a satisfied user will probably be happy to spend time on something in the knowledge that it is being done right. Add to this the fact that real users, in the real world, tend to be interrupted or deal with several tasks simultaneously, reducing their awareness of how quickly a job is completed.

Although timing is a valuable tool for some types of testing, it is not the full story. What is far more important is measuring satisfaction and identifying problems that frustrate users. This can only truly be achieved by careful observation of the user during the test programme, and discussion of problems, delays and mistakes as they occur. Ultimately, it is why users slow down that is more important than how much time they actually spend.