Interview with Jared Spool
10th August 2002
Jared Spool is the Founding Principal of User Interface Engineering (UIE), a leading usability research firm based near Boston, USA. His recent talks in London and Edinburgh drew large audiences including web designers, usability professionals and others interested in what his findings mean to usability in e-commerce. Chris Rourke of User Vision met with him to discuss his views.
CR: How did you get into usability in the first place & what attracted you to it?
JS: I got involved in usability over 20 years ago because I was writing software for other people who had come up with the design. I couldn't understand how anyone was going to use this. On one hand it was supposed to be an application senior execs at large companies would use but it had an interface that only a computer programmer would appreciate. I just couldn't imagine it working and started asking questions – and the next thing I know I'm a usability guy!
CR: You just saw a natural need for it then?
JS: Yes, I didn't have a word at that time, human factors wasn't a term bandied about per se. We didn't have a language, we didn't have jobs we didn't have techniques… we didn't have any of that stuff.
CR: What bits of research are you working on now and which ones are most interesting?
JS: We're focussing on understanding e-commerce, something which we refer to as the convenience store experiment. Imagine you have this device that instantly tells you that someone within a 2-mile area has run out of milk. You rush to their house, and take them to the nearest store and, just to make sure they purchase, you give them the cash. E-commerce is always measured on this notion of a conversion rate and you'd think in that particular context you'd have 100% conversion rate – it's very unlikely this transaction will not succeed. Well we've done just this on the web. We've taken people who needed a product to websites that had that product and given them the cash to buy it. The first time we did that we got a 30% conversion rate! We've studied the simplest e-commerce or web scenario – there are very few things simpler than this. We've studied this process and tried to figure out why we got such low conversion rates. For the first time we've been able to tie design elements to bottom line revenue. I can tell you how much money your e-commerce site is leaving on the table, and how much is because people can't find what they're looking for, or search engine issues. I can tie specific design elements down to a dollar – and what the ROI is to improving it. The 'customer sieve' is part of this. You end up with fewer customers at the bottom than at the top. If we can see where they're being weeded out of the process we can look at getting more of them out of the bottom. It's the site design that's weeding them out, not the product as we've already established this is the product they want to buy.
CR: You & UIE seem driven by such empirical evidence. Do you find this helps to put your point across to people you're trying to convince.
JS: We focus a lot on actual evidence. We take a lot of the things people expound and create hypotheses of them. If the user hates scrolling – one of those sweeping statements everyone makes – people go to great lengths to design websites to avoid it, JC Penny's for instance. Watching people do it we find they grab the scroll bar and try and move it – People are desperately looking for things. Although the design doesn't prevent them from scrolling, they actually scroll more on that design than other designs that don't try and prevent them from scrolling. So they don't purchase as much or find their products as often.
CR: Have their been any findings that you wouldn't have discovered without research
JS: Yes, well scrolling is one. Finding the more you decrease white space, increasing the density of text on the page, the better the user buys. Actual download time has no effect on the usability of the site. Many commonly held beliefs turn out to not at all to be true.
CR: At times you've gone against the grain of conventional usability wisdom on topics such as usefulness of search engines, importance of download times, even minimum number of users required for usability tests. Has coming to those conclusions been gratifying for you?
JS: It convinces me that things I take for granted I shouldn't and that everything should be tested. When I was a kid I grew up not enjoying seafood at all. I moved to Boston with this perception. I tasted it first when I was 19 and realised that I'd spent 19 years of life with this misconception. I decided to throw out all my preconceptions of what I liked and didn't like and to try everything. To some extent that's the way I attack this. I'm testing all these conceptions as hypotheses now. To be fair there are many things we've found that are true – they just don't get headlines!
CR: What about search engines? Your recent findings surprised a few people.
JS: That's a very controversial thing we've done. Search engines seem to be a good idea, something you have real assurance from and makes sense. People would say "when I know exactly what I want I'll use the search engine, when I don't I'll browse". When we watched users we didn't find that was the case. Users who knew exactly what they want didn't seem any more likely to use the search engine than those who didn't. One of the theories floated about is this notion of search dominance – this says some percentage of the population will naturally use search first. Again we didn't see that – in none of our tests would any of our users use search first. There were sites where every user used search first and sites where every user didn't use search first. When you compare the design of the site you can see why that happened. The design of the site causes people to use search first, rather than an inclination for search. When people use search you'll find they are less successful in finding what they want than when they don't use search. When you combine these two facts you wonder why are we bothering with this technology? People are working so hard to make search work well, when if they put that effort into the underlying design of the site they could eliminate search. One of the most telling factors is in a recent study we did of 13 apparel & home good sites – 2 of the 3 best performing sites had no search engine, and not a single user noticed.
CR: So what makes up for the fact of lack of search engines then? How come they didn't notice that?
JS: Because the site worked. In the apparel sites we noticed that people don't use the search until they've made on average 4 clicks. So they're going into the site looking for what they want, it's only when they can't find it on the site architecture they use the search. The Gap was our best performing site. Users found what they wanted on average within 12 clicks, whereas on our worst performing site, one of which was Macy's, it took 51 clicks.
CR: Another area where you've run against popular wisdom is your views on Flash. Any thoughts?
JS: The notion that Flash is 99% bad I've always felt is a bizarre way to go about it. This is very much a "blame the tool" mentality. Flash is a tool that does some very cool things, but Flash makes bad design really easy. People who just want to play with the tool – and then take what they created and force it onto the public – are going to get their comeuppance. We took the idea that Flash is 99% bad and researched this hypothesis. We found that there are all sorts of really good applications with Flash – when we compared the good ones with the bad ones we started to see some patterns. The good applications all utilised certain aspects of Flash where HTML really suffered, e.g. representing something that happens over time, such as a demo how to put together a bookcase. When Ikea wanted to do this using HTML it was very clumsy, whereas a little Flash movie showing the five steps it takes to put it together will convince people to purchase. So Flash is ideal for that. If you're representing time or space, Flash is very good. In terms of applications, it turns out it's a great application development tool, it makes up for a lot of the failings that we see in HTML. You can also do really stupid things with it, there's nothing in the system that will tell you what you've done. It's just a matter of educating the design teams.
CR: What would you say are the most important aspects of good web design?
JS: Having a feedback route. In the IT world we get so involved in what we have to do that we take an application, and features end up not getting in due to being over committed, under resourced. Because of this you now have to start after launch to get the rest of it in. The best teams break the cycle, they stop to create a feedback route and ask "did we achieve the goals we've set"? They get to learn what worked and what didn't so they can go forward with these learning's. You can learn a lot just by watching.
CR: Does that sum up your view on usability testing? Some people go in for the big fancy usability labs. What's your view on usability testing for getting evidence in software development?
JS: I think usability testing is an unfortunate phrase – I've always thought it should be usability watching. You go out and find users in the first place and you just sit and watch them, see what noises they make. Just doing that will often yield so much value you don't need much more than a chair. All of the fancy equipment is just a distraction from the activity. It's critical that organisations just spend the time watching.
CR: Have you found that collecting such data increases the acceptance of your recommendations? Do you have more credence when you can show them evidence, numbers and maybe a snippet of videotape?
JS: We don't use videotape anymore. The most credence comes when people see things for themselves. We found when working with clients the most effective thing was to bring them into the test and have them sit in the chair themselves. Particularly if you get them to predict upfront what they think will happen and then it turns out to be completely wrong. We have developers show us the task that users should do, then we'll have users do it and we find the users do it a completely different way. But they can see why the users have done it that way.
CR: Some people are often critical of usability, saying it stifles creativity. What's your response to that?
JS: This comes from a contention between the world of the usability evangelist and the designers. They say the designers have failed us because they were trying to be artistic. The issue is who is the sponsor? Do the designers serve their sponsor or do they serve their own artistic presence. Artists believe art should be art for arts sake. Whereas design is all about matching form and function. This goes against the training of the artist. I think there are designs out there for purely aesthetic reasons, rather than taking into account of function. But who is the sponsor – is the design taking account of the sponsors need? So a web site designed to promote a movie for teenagers needs a very different aesthetic sense practically from a sponsor's perspective than a web site that helps you do your taxes if you're 40 or over. You never want to mix the design aesthetic for either of them.
CR: What do you think of the promotion of certain usability professionals as "gurus"?
JS: I cringe every time anyone refers to me as a guru. I don't know what that means. I don't know anything about web design – and our research is proving that on a daily basis. What I thought I knew it turns out I don't. So for me to consider myself a guru, well gurus are all knowing and wise and I find myself not knowing a lot & not very good at giving people advice about it. I think its premature having gurus in our field. So far those people who seem to revel in the guru terms aren't any better at telling people what to do than anything else I've seen.
CR: From your knowledge of user centred design practices in the UK and the USA, are there any different practices etc?
JS: I don't know any differences. I don't think in terms of nationality. I think the community has grown together so much you can't see in terms of geographic lines as before. I just see a blending, even as far as the Japanese, there's a blending in what's happening in the usability space.
CR: Many thanks for you time and thoughts, and best of luck in the future