Case Study: A Discount Approach to Comparing Multiple Design Alternatives

Slide 1: Case Study: A Discount Approach to Comparing Multiple Design Alternatives User Jared M. Spool Interface Founding Principal Engineering ©2008, User Interface Engineering. All rights reserved

Slide 2: The Problem

Slide 3: The Problem Home page redesign for a big site 4 million visitors a month

Slide 4: The Problem Home page redesign for a big site 4 million visitors a month Never done a usability test before

Slide 5: The Problem Home page redesign for a big site 4 million visitors a month Never done a usability test before Only allotted 2 days of testing

Slide 6: The Problem Home page redesign for a big site 4 million visitors a month Never done a usability test before Only allotted 2 days of testing Budget: less than $10,000

Slide 7: The Problem Home page redesign for a big site 4 million visitors a month Never done a usability test before Only allotted 2 days of testing Budget: less than $10,000 5 design alternatives By an outside design firm

Slide 8: Current Design

Slide 9: #1 #2 Alternatives #3 Current Design #4 #5

Slide 10: #1 Evaluate in 2 days #2 Alternatives #3 Current Design #4 #5

Slide 11: #1 Evaluate in 2 days #2 Alternatives #3 Current Design #4 Under $10k #5

Slide 12: Approach #1: “Within Subjects” with Multiple Designs Learning effects Too Many Combinations

Slide 13: Approach #2: “Between Subjects” with One Design Not enough users

Slide 14: but

Slide 15: but

Slide 16: Assumption Bias

Slide 17: Looking at the Problem Differently Based on Inherent Value Testing 10 to 12 participants Compares the existing design to the best alternative Six-Step Process

Slide 18: Step 1: Build a Weighted Differences Matrix

Slide 21: Current Design Design Design Design Design Weight Design Alt #1 Alt #2 Alt #3 Alt #4 Alt #5 Difference

Slide 22: Current Design Design Design Design Design Weight Design Alt #1 Alt #2 Alt #3 Alt #4 Alt #5 Difference Better search access 8 2 3 1 5 3 5 More descriptive links 6 3 4 2 2 3 1 Floating navigation 5 2 5 0 5 3 0 On-screen login 4 5 5 3 3 3 1 Visible top-rated items 6 1 3 5 3 5 0

Slide 23: Current Design Design Design Design Design Weight Design Alt #1 Alt #2 Alt #3 Alt #4 Alt #5 Difference Better search access 8 2 3 1 5 3 5 More descriptive links 6 3 4 2 2 3 1 Floating navigation 5 2 5 0 5 3 0 On-screen login 4 5 5 3 3 3 1 Visible top-rated items 6 1 3 5 3 5 0 Total Scores 70 111 62 107 99 59

Slide 24: Step 2: Recruit from Two User Groups

Slide 25: Step 3: Use an Inherent Value Testing Protocol

Slide 26: Step 4: Add in “Best” Alternative

Slide 27: Step 5: Add in Competitor’s Design (when possible)

Slide 28: Step 6: Evaluate Design Alternatives against Differences Matrix

Slide 29: Current Design Design Design Design Design Weight Design Alt #1 Alt #2 Alt #3 Alt #4 Alt #5 Difference Better search access 8 2 3 1 5 3 5 More descriptive links 6 3 4 2 2 3 1 Floating navigation 5 2 5 0 5 3 0 On-screen login 4 5 5 3 3 3 1 Visible top-rated items 6 1 3 5 3 5 0 Total Scores 70 111 62 107 99 59

Slide 30: Current Design Design Design Design Design Weight Design Alt #1 Alt #2 Alt #3 Alt #4 Alt #5 Difference Clearer trigger words 9 0 1 1 3 3 1 Interactive drill-down 10 0 2 2 1 4 5 Remember Previous Picks 10 0 0 0 5 0 0 Better search access 8 2 3 1 5 3 5 More descriptive links 6 3 4 2 2 3 1 Floating navigation 5 2 5 0 5 3 0 On-screen login 4 5 5 3 3 3 1 Visible top-rated items 6 1 3 5 3 5 0 Total Scores 70 140 91 194 166 118

Slide 31: Comparing Multiple Design Alternatives Understand the differences between alternatives Account for assumption bias Focus on collecting data about the current design Learn the problems that users are currently facing Use the user data to make informed design decisions

Slide 32: More Info from User Interface Engineering Newsletter: UIEtips (FREE) UIE Virtual Seminars Reports User Interface Conference: www.uiconf.com Blog: www.uie.com/brainsparks Site: www.uie.com