Rocket Surgery Made Easy The Do-It-Yourself Usability Testing

Rocket Surgery Made Easy: The Do-It-Yourself Usability Testing Workshop Boston April 18, 2013

Who is this guy, anyway? < Steve Krug (steev kroog) (noun) 1. Son, husband, father 2. Resident of Brookline 3. Usability consultant < Advanced Common Sense 4 Me and a few well-placed mirrors 4 Corporate motto: “It’s not rocket surgery™” < Nice clients 4 Lexus. com 4 Bloomberg. com 4 Technology Review © 2001 Steve Krug

My intention for today < Give you some practice so you’re comfortable testing <

This morning < Why do usability testing? < Steve does a demo test <

This afternoon < You do your first practice tests < You do another practice

Ground rules < Tell the driver to speak up, if necessary < Interrupt ANYTIME

Anybody here from out of town? < Graphic designers < Information architects < Developers/programmers < “Marketing” < Usability ______ < Project managers < Writers/editors < Check signers < Other? < Left-handed? © 2001 Steve Krug

Anybody here from out of town? < Your experience with usability testing 4 Have conducted tests (facilitator)? 4 Have observed tests? 4 Have read usability test reports? < Your organization’s use of testing 4 Never? 4 Right before (or right after) product ships? 4 Routine (several times during development)? 4 Farmed out? 4 Have your own lab (and white coats)? © 2001 Steve Krug

One more question < What are the biggest obstacles to your doing testing? ©

Who’s read the book(s)? < DMMT? < Rocket Surgery? < Watched the video? <

So… < Why usability testing? < Ten years ago, I realized something… © 2001

“My ideal home page, ” as told by… © 2001 Steve Krug

And now, a live demonstration © 2001 Steve Krug

A brave volunteer? < We’ll try an actual test 4 It’s painless 4 It’s brief 4 You’ll get a round of applause when we’re done < Qualifying criteria: 4 Have used a Web browser 4 English-speaking adult 4 Doesn’t work at Fresh Tilled Soil 4 During the test 4 You are observers 4 Jot down top 1 or 2 problems you observed © 2001 Steve Krug

Debriefing < What were the most serious problems? 4 Observed problems © 2001 Steve

DIY usability testing (nutshell version) < Three users 4 You’ll find more than you can fix < No lab or mirrors 4 Set up a monitor in another room so the development team can watch < Record with Camtasia 4 or Morae (Techsmith) or various Mac products < No stats, no exit questions, no faux validity < No big honkin’ report 4 Debrief over lunch © 2001 Steve Krug

The maxims < Six of them < I could (and have) talk about them

A morning a month, that’s all we ask. © 2001 Steve Krug

When this happens: You’re not absolutely sure you know what the user is thinking (see below). Something happens that seems to surprise them. For instance, they click on a link and go “Oh” when the new page appears. Say this: “What are you thinking? ” “What are you looking at? ” (for variety) “What are you doing now? ” (e. g. , if you think they’re being silent because they’re reading) “Is that what you expected to happen? ” They’re trying to get you to give them a clue. “What would you do if you were at home? ” (“Should I use the ___? ”) “What would you do if I wasn't here? ” The participant makes a comment, and you’re not sure what triggered it. The participant suggests concern that he’s not giving you what you need. “Was there something in particular that made you think that? ” “No, this is very helpful. ” The participant asks you to explain how something is supposed to work. (“Do these support requests get answered right away? ”) “I can’t answer that right now, because we need to know what you would do when you don’t have somebody around to answer questions for you. But if you still want to know when we’re done, I’ll be glad to answer it then. ” The participant seems to have wandered away from the task. “What are you trying to do now? ” “This is exactly what we need. ” © 2001 Steve Krug

Start earlier than you think makes sense. © 2001 Steve Krug

Recruit loosely and grade on a curve. © 2001 Steve Krug

Naturally, we need to test people who are just like our target audience. Representative

Make it a spectator sport. © 2001 Steve Krug

Focus ruthlessly on a small number of the most important problems. © 2001 Steve

What’s funny about this? < Show of hands: Have you ever gone to a Web site and run into a serious usability problem? < Did you find yourself thinking “How can they not have noticed this? And fixed it? ” < Did you go back months later and it was still there? © 2001 Steve Krug

The problem is, testing works < If you’ve done any testing, you know it works < Uncovers lots of problems quickly < But I’ve finally realized this is part of the problem < You can find more problems in a day than you can fix in a month © 2001 Steve Krug

Problems you can find with just a few test participants Problems you have the

Things I have learned < It’s easy to get seduced into fixing the easier

When fixing problems, always do the least you can do™. © 2001 Steve Krug

Your motto < When fixing usability problems, your motto should be: 4 What’s the

Nothing New Under the Sun Department © 2001 Steve Krug

Choosing tasks to test < What do you have to show? 4 Try not to let this limit your thinking, though < You can get a long way with 4 A sketch or a few “comps” 4 Linked wireframes to test navigation 4 HTML of a few pages that let you complete a task as long as you don’t stray 4 See Carolyn Snyder’s Paper Prototyping © 2001 Steve Krug

Choosing tasks to test < What do you wake up thinking about in the middle of the night? < What tasks are people likely to do? < What tasks are crucial? 4 …to the user and your business model < Whenever possible, keep it real 4 Free-range browsing tasks are a good thing 4 Bad: “Buy a gift cupholder for under $35. ” 4 Better: “Order a book you’d like to have” © 2001 Steve Krug

Tasks vs. Scenarios < Task: 4 “Apply for a doctoral program at HBS” < Scenario: 4 “You’ve got an MBA, and after a lot of research you’ve decided to enter the doctoral program at HBS in Science, Technology & Management. 4 Apply for admission to the program. ” < A scenario. . . 4 Provides some context (“You are. . . ”, “You need to. . . ”) 4 Just enough; DON’T get carried away 4 Supplies specific information the user would actually have © 2001 Steve Krug

Writing the scenarios < Don’t telegraph it 4 Avoid using words that will appear on-screen 4 Bad: “Customize your LAUNCHcast station. ” 4 Better: “Choose the kind of music you want to listen to. ” 4 Can be the hardest part < Make it unambiguous 4 Misunderstandings waste time and don’t [usually] produce useful insights < Keep it short 4 Trim any detail that doesn’t contribute < Piloting the test will help © 2001 Steve Krug

Exercise: Preparing tasks < Jot down 3 -7 of the most important tasks people need to do on your site (4 min. ) 4 E. g. , “Find directions to your nearest Bank of America branch”, “Apply for a doctoral program” < Choose one to work on < Write a scenario for this task (10 min. ) 4 A short paragraph 4 “Don’t use Search” is OK 4 Write it out so you can read it verbatim © 2001 Steve Krug

Now send me your scenario(s) < Go to bit. ly/BOSTONSK < Click the orange “Post a new message” button < Use your name to sign in (so we can identify them) < Type the URL you’re testing < Type your scenario © 2001 Steve Krug

Exercise: Practice test < Pair up with someone who has a laptop < Take

Exercise: Practice test < Read the introduction from script (2 min. ) 4 Sign recording consent form 4 Start Camtasia (or equivalent) if you have it < Ask them the five questions (2 min. ) < Have them look at Home page (<2 min. ) < Have them do your task (8 min. ) 4 Give them the task 4 Keep them thinking aloud < Ask probing questions (2 min. ) < Thank them (30 sec. ) 4 Save your Camtasia file © 2001 Steve Krug

The debriefing < Over lunch (or dinner, or breakfast) < Right after the three test sessions < Objective: Deciding what you’re going to commit to fixing before the next round of testing © 2001 Steve Krug

The debriefing < Go around the room 4 Everyone contributes from their list of nine problems 4 Write on easel pad 4 Leave some space for improvements/amendments 4 People can say “Me too!” 4 Treat all contributions with respect 4 Not discussing yet 4 Stick to observed problems! © 2001 Steve Krug

The debriefing < Decide which are most serious 4 Some magic happens here 4 Voting/Dictatorship 4 Not usually as hard as it seems BECAUSE THEY ALL SAW THE SAME BEHAVIOR 4 Number them < Copy the numbered list 4 Ten is probably enough 4 Leave space in between © 2001 Steve Krug

The debriefing < Start at the top < Work down the list < Come up with rough idea of how you’ll fix them 4 who will do it 4 the resources required < When you’ve allocated the resources you can commit in next month, STOP! 4 Tear off the rest of the list 4 Crumple it up 4 Throw it away 8 Thanks to Susan Weinschenck © 2001 Steve Krug

Mobile testing < Testing things on mobile devices 4 Apps and mobile sites < Head for Slideshare 4 Belen Barros Pena & Bernard Tyers 4 http: //www. slideshare. net/beleniq/do-it-yourselfmobile-usabilitytesting-at-euroia-2010 4 “The Mc. Gyver’s of Mobile Usability Testing” © 2001 Steve Krug

What the difference? < Basically the same < Except for a ton of issues 4 How does the facilitator see the screen? 4 How do the observers see the screen? 4 Fingers/No fingers? (Camera view or screen mirroring) 4 If camera, attached to device or strap the user to a chair? 4 Screen recording? 4 What devices? User’s own or supplied by you? 4 Test in context? If so, are there observers? © 2001 Steve Krug

My take on it < Two useful tools 4 Airplay (mirroring i. OS devices to PC or Mac via Wi. Fi) 4 Screen recorders for i. OS 8 UX Recorder and Magitest apps 8 Many limitations (browser only, performance issues), but valuable if “walking around” testing is needed 4 See my recent blog post 8 www. someslightlyirregular. com © 2001 Steve Krug

My take on it < Observers are important < Fingers are important for observers 4 Faces, not so much < My ideal finger-view camera 4 Featherweight, unobtrusive, stable, reliable 4 Manual focus 4 Inexpensive (<$50) © 2001 Steve Krug

+ Lightweight Clamp and Gooseneck © 2001 Steve Krug

My take on it < Optimal setup 4 Webcam > facilitator’s laptop via USB 4 Facilitator watches webcam on laptop (using Am. Cap or similar program) 4 Laptop mirrored to observers via screen sharing (Go. To. Meeting, Webex, etc. ) 4 Recording is made on observation room laptop (Camtasia, Morae, etc. ) © 2001 Steve Krug