Beyond Usability Testing

Traditional usability testing and even detailed log analysis will miss some of the user experience feedback required to make a site great. This article looks at why.

Usability testing is a powerful tool in identifying problems and issues that users may have with a website or software application. But for all its benefits, traditional testing does not necessarily give a complete picture at how effective a site or application is in terms of meeting business goals.

Much of this is due to the nature of traditional testing techniques. These tend to use test scripts to model certain 'common tasks', and ask participants to carry out these tasks whilst an observer notes any difficulties that are encountered. Obviously this method can identify a lot of problems and provide invaluable feedback to development and design teams. But at the same time, because the user is 'guided' through the site it can be less effective at answering other questions, such as:

• What messages is the user picking up from the site?
• Which areas of the site attract users who are acting of their own accord rather than following instructions?
• How long will a typical user remain at a site outside the context of a test procedure?

Any Web manager would certainly want an answer to these questions. Traditionally you might get some idea from accurate server logs, which record the total number of visitors to various pages on the site. Depending on the product used to interpret them, they can also provide an indication of common routes through the site and time spent on each page.

However, although this sort of analysis can provide plenty of statistics and identify trends, it cannot answer the most important questions; why users behave in the way they do, and what impressions they pick up from the site during their visit.

This is where alternative user-testing techniques come in. By broadening the base of enquiry and moving beyond the performance of pre-defined 'common tasks', tests can deliver essential information on how users experience the site as a whole. This does not replace traditional user testing - it should be seen as an additional technique that builds on the results of standard user testing methods.

Of course user experience is not something that can easily be given a value or score, but that doesn't mean it should be ignored or sidelined. Some simple techniques can answer the questions mentioned above in a test procedure. They include:

Allowing browsing to be as free as possible.

Ask test participants open-ended questions such as "what do you think this company does?" and give them time to find an answer. Questions like these are an excellent way of gauging how effectively a company communicates online. In some tests it may even be appropriate to simply ask users to browse the site in any way they wish and then discuss the company afterwards. This can reveal which areas of the site tend to attract casual browsers - and if they are regarded as the 'wrong' areas, something can be done about it.

Enabling users to leave if necessary.

If a certain task involves making a purchase, for example, enable users to use alternative sites if they become frustrated. This models real-life experience more closely and can help to predict whether a site will be competitive in the real world. Of course it is also necessary to 'force' users through the purchasing process in order to test usability, but this can be done during another part of the test.

Encouraging general feedback on attitudes and experiences

Don't just ask your users to talk about their usability issues, actively encourage them to express their feelings about the content they encounter, and their impressions of the company or organisation. Whilst testers must be careful not to read too much into subjective opinion (and should increase the size of the sample group when using this method), it can perform a similar function to the focus group without many of the disadvantages.