Beyond Usability Testing
17th March 2006
A Mercurytide white paper discussing the importance of User Centred Design.
For a web site to be successful its intended audience must find it easy to use. As this maxim finds increasing recognition organisations are asking: “What can we do to create a site that’s easy to use?” The answer for most, it seems, is usability testing.
But usability testing is only one part of producing highly-usable sites: the most effective and reliable way to ensure fundamental usability is to follow a user-centred design process. And the success of this approach is largely dependent on the wider organisational attitude towards usability.
In this paper we discuss what lies beyond usability testing: using ISO international standards as a basis we detail the range of activities that make up user-centred design and introduce formal levels of organisational maturity regarding usability. We also show how organisations can use this knowledge to optimise their design process.
One of the biggest obstacles in the way of a usable web site is an organisation refusing to acknowledge they have usability problems. Even once some people are in agreement, organisational inertia can make it a lengthier process than it needs to be. And then there is the question of what the usability problems are: how might they be discovered, and put right? The answer is often seen as usability testing.
Usability testing offers immediate results on a rough-and-ready level with the organisers needing little experience. However, usability testing is often carried out late in development when there is little scope for change (typically because of limited time or resources). As usability consultants we often find companies come to us in this position hoping their problems can be righted in a very short space of time — or even to just tick the box indicating everything is fine! Unfortunately, in this type of situation it seldom is.
One of our biggest challenges is convincing organisations that site usability does not stop at usability testing and expert evaluations, but is a product of following a user-centred design (UCD) approach. This approach was standardised in 1999 as ISO 13407: Human-centred design processes for interactive systems and was instrumental in recognising UCD as being equivalent to other key business processes.
Primarily written as guidance for project managers, ISO 13407 identifies five main principles:
- users must be involved early and consistently;
- there must be a clear understanding of user and task requirements;
- there must be an appropriate allocation of function between users and technology;
- design solutions should be iterative; and
- designs must be multi-disciplinary.
In addition to these principles, there are four key UCD activities:
- explicitly understanding and specifying the context of use;
- specifying user and organisational requirements;
- producing design solutions; and
- evaluating designs against requirements with representative users.
These processes are outlined in the diagram above right.
Notice, most importantly, no specific method of user-centred design is described; it is left to the project team and usability specialist to identify the appropriate method given the requirements for a project. Also of note is that usability testing would only take place in the final step, “evaluate designs against user requirements”. We’ll now summarise the five steps.
The ISO processes
Plan the UCD process
Usability is as important as any other project activity and should be treated as such. For this stage you should identify the goals and constraints of the system, the key functionality, appropriate usability methods for the project, and plan the required resources.
Specify the context of use
The usability of a product is greatly affected by the context in which it is used. The users, their tasks, physical and technical environments; these all have an impact. While most web-based projects will have fairly constant technical and physical environments (although connection speed, screen resolution, and browser need to be considered), users and tasks need much consideration. Tasks should be expanded into scenarios and personas should be developed from user profiles; this will allow the design team to have a clear idea for whom they’re creating designs and how and why somebody would use such a system.
Specify user and organisational requirements
Here you should specify the requirements based on the context of use, and identify objectives and metrics that you can test against and decide target levels. For example, an e-commerce project may decide the target is to have a certain percentage of visitors who can complete a purchase without error and within a certain time-period.
Legal requirements, such as the level of accessibility, should also be identified at this stage. Of key importance is an evaluation of the existing design in order to identify usability problems and create a benchmark for development.
Produce design solution
At this stage you create and test the site’s information architecture so you can identify and resolve any problems with the structure, grouping, and labelling of the information on the site. Create early design solutions on paper; this allows identification of usability problems without the considerable expense of coding and development. A style guide may also be developed to keep the design consistent and ensure the established design guidelines are adhered to. The development of design solutions should always be driven by the established usability requirements.
Evaluate designs against user requirements
Designs can be evaluated either through expert evaluation or by testing with representative users; often a combination of both is most effective. The findings are the impetus behind the redesign; once redesigned you should re-evaluate and re-test, and then redesign again. Continue this iterative process. When do you stop testing? Ideally, when the requirements are met; but more often it’s after a pre-determined number of iterations are carried out — one or two expert evaluations and a round of usability testing, for example.
After the design is launched you should gather feedback for use in any further development. This might take the form of interviews, online questionnaires, or usability testing.
On completion you can trust in something that is easier to use than if you had you followed only usability testing — or had nothing at all. With planning, requirements, and an iterative design you can be sure of something that is far less subjective; something you can present with confidence.
Key to establishing user-centred design is the internal attitude and culture of an organisation. In fact a successful move towards user-centred design should be based on a detailed understanding of current attitudes, skills, abilities, and practices within an organisation. Without any attempt at understanding, progress may well be costly and ineffective. Commitment from the top of the organisation is the best way to see user-centred design performed throughout each development project.
So you’re alone in realising your organisation’s web site has problems with usability; how do you go about improving the situation? The answer is a usability maturity model, which you can use as a basis for assessing usability capabilities. Of particular interest here is a standard linked to ISO 13407, the snappily-titled ISO 18529 Human-centred lifecycle process descriptions.
Related documents (see references) define a scale of organisational maturity that can be used to identify attitudes towards usability based on assessment of management practices. The table below illustrates the levels in the maturity model, along with typical indicators for each. (Note that the levels are optimistic enough not to include ‘We don’t care about usability’ this is a brick wall many find impossible to break down.)
|X||Unrecognised||We don’t have a problem with usability.|
|A||Recognised||Realisation that there are usability problems, but unsure of what to do; user requirements collected; user feedback obtained.|
|B||Considered||Some awareness training; realisation that usability can be improved by taking account of end-user requirements.|
|C||Implemented||End-users involved in end-to-end user-centred design process.|
|D||Integrated||UCD activities are fundamental to development; iteration used to achieve required level of usability.|
|E||Institutionalised||UCD embedded in organisational development strategy to ensure usability and the consistent achievement of related business objectives.|
Use this maturity model to identify the level of usability maturity and then establish the organisational activities by priority so skills, support, awareness, processes, and integration consistently ensure the usability of web-development projects. Business objectives are regularly achieved, and so web-development becomes as cost-effective as possible.
Usability testing has limited capabilities and should not be the sole indicator of a site’s usability — this is particularly true when carried out late in development with limited time and resources.
Usability should instead be seen as the product of an end-to-end user-centred design process. The success of this approach is highly-dependent on the organisational attitude, skills, and support for the process, and the extent to its integration with other development activities and business processes. The capability of an organisation to carry out development in this manner can be assessed using a usability maturity model, with the level of organisational maturity defined on a related scale.
This approach is essential to characterise current capabilities, identify the important internal initiatives, ensure that usability is fundamental to the organisation’s web-based projects, and to achieve goals in a consistent, controlled, and repeatable manner.
Following this approach allows for the most cost-effective development process and consistent achievement of business objectives.Written by Ross Phillip and Chris Rourke
- ISO 13407:1999 Human-centred design processes for interactive systems , April 2004.
- ISO/TR 18529:2000 Human-centred lifecycle process descriptions , April 2004.
- TRUMP usability maturity model: processes, London: Lloyd’s Register of Shipping.
- Usability maturity model: human-centredness scale, London: Lloyd’s Register of Shipping.
What Can you do next?
- Read some more usability and accessibility articles.
- Improve your site and benefit from a competitive advantage by conducting usability testing.
- Attend one of our usability training courses and learn the tricks of the trade for yourself.
Want this article on your website?
If you liked this article, feel free to republish it on your own website. All that we ask is that you include the citation below, including links, at the end of the article.
This article was written by Chris Rourke & Ross Phillip. Chris is the Managing Director and Ross is a Senior User Experience Consultant at User Vision, a usability and accessibility consultancy that helps clients gain a competitive advantage through improved ease of use.