A Comparative Usability Evaluation of User Interfaces for

A Comparative Usability Evaluation of User Interfaces for Online Product Catalogs by Ewa Callahan & Jurgen Koenemann presented by Andrew Trieu ICS 280 - Information Visualization Department ICS at UCI June 6, 2002

Introduction A comparative study of the usability of hierarchically structured and zoomable Electronic Product Catalogs (EPC), available online, VERSUS the Info. Zoom, a novel interface utilizing a compression technique, direct manipulation, and zooming.

Introduction (con’t) The authors had designed a test search interface and used it for comparison between EPC and Info. Zoom. Performance on both types of interfaces was evaluated in terms of search speed, accuracy, efficiency, and user satisfaction.

How EPC comes about Due to the increased in the WWW, many companies have used it as a medium to promote and sell their products and services. The increase in electronic commerce creates a need for Electronic Product Catalogs.

Customers The customer enjoys a higher level of interactivity such as a variety of searching possibilities, comparison features to get access to more information about the different product, etc. However, navigation problems are the second most cited reason for not shopping online.

Introduction (con’t) The most common catalog interfaces on the Web are hierarchically organized catalogs, where the consumer sees the list of the products and can follow the links to their more detailed description.

Problems & Solution Problems of the hierarchically organized catalogs are: z. Confusing, z. Items in dropdown menus are not visible, z. Lengthy result pages, and z. Lack of filtering options of results. The solutions are: z. Recommendation approach z. Interaction approach

Recommendation Approach The first approach includes the recommendation systems which suggest to the user what the user should buy (recommendation approach). This will require consumers to enter personal data which many consumers will not do.

Interaction Approach The second approach concentrates on improving and facilitating the consumers’ interaction with the interface, allowing the user to select products more easily. The authors were interested in this direction and wanted to investigate whether or not highly interactive interfaces that support the user in the selection of products can improve the finding of products.

Evaluation of Existing Online Catalogs For the evaluation, the authors had selected 14 German and English automobile EPCs available on the Web in the Fall of 1999.

A Summary of the Results

Analysis of the Result z. Database size z. Searching mode z. Attributes z. Specifying Ranges z. Search History z. Result Pages z. Sorting Search Results z. Item Comparison

Info. Zoom z. Based on Ahlberg and Schneiderman’s dynamic queries and Furnas’ fisheye techniques z. Displays database relations as tables with attributes as rows and objects as columns z. Overview mode and Compressed mode z. Offers many different ways of searching z. Can easily compare, sorting, and re-sorting the results.

Overview Mode

Compressed Mode

Usability Study z. Based on the feature comparison of Info. Zoom and the hierarchical interfaces, Info. Zoom should be a more efficient tool that facilitates searches for products. z. The authors conducted a comparative usability study to confirm their analysis. Also, they performed a controlled experiment that resembled online counterpart.

Materials z. The database used for this study (Auto. DB) contains 1690 cars with over 20 attributes. z. The hierarchical interface was implement with forms in MS Access in a way to resemble the “Look & Feel” of the typical online EPC catalog features.

Hierarchical Interface (1)

Hierarchical Interface (2)

Hierarchical Interface (3)

Participants z 26 participants (randomly assigned to either Info. Zoom or hierarchical interface) z 15 males and 11 females z 13 ages between 21 -30, 9 ages between 31 -40, one teenager, and one over age 51 z 16 use computer above 10 hr/wk, 6 use computer between 3 -9 hr/wk, and 4 use computer less than 2 hr/wk.

Tasks z. Find Attribute value for given Object T 1: Does the Jaguar XJR have side airbags? z. Find Database Subset Attribute Value T 2: How many different models does the Peugeot class 106 have? T 4: How many cars have the following characteristics: convertible, 100 -150 HP, front drive, and cost 40. 000 DM or less?

Tasks (con’t) z. For given Attribute(s) value(s) Find Object(s) T 3: Which middle class car is the cheapest one in the database? T 6: Which Funcar has the quickest acceleration? z. Compare Attribute Values for 2 Objects T 5: Which car has better fuel consumption: the Audi A 6 1. 8 or the Volvo S 70 2. 0?

Tasks (con’t) z. Find Database Attribute Value T 7: How many cars are in this catalog? z. Compare Attribute Values for Subset T 8: What is the price range (from the cheapest to the most expensive model) for the Alfa Romeo 145 series? T 9: Is the acceleration of the Mercedes-Benz S 430 Automatic slow, normal, or quick for a car of its class?

Procedure z. Participants were briefly introduced to the purpose of the study and upcoming tasks. z. Info. Zoom participants got a 5 minutes demo about Info. Zoom. z. Participants were introduces to 9 tasks. After each task, they answer 3 questions about satisfaction, perception of the time needed, & perceived efficiency. z. After last task, summarize their experience

Results were analyzed for each category of data: z. Accuracy - equal between both interfaces z. Completion time AVGIZ = 19. 4 min. ; AVGHI = 27. 7 min. z. Navigation (out of 7 scale) Ease of use: IZ = 5. 5; HI = 4. 9 Efficiency: IZ = 5. 0; HI = 4. 8 Overall, both were valued higher for IZ.

Results (con’t) Results were analyzed for each category of data: z. Satisfaction (7 -point Likert scale) Comfortable: IZ = 5. 15; HI = 5. 23 Fun: IZ = 6. 3; HI = 5. 0 Overall satisfaction with both system was high

Completion rate & accuracy

Hierarchical Interface

Info. Zoom

Observations The authors observed the participants throughout the experiment. Throughout the experiment, the authors noted: z. Users’ strategies z. Critical incidents - breakdowns and things that went especially well z. Users’ comments

Hierarchical Interface z. Dislike the ways of looking for the desired information from the main menu. z. Seem lost on pages that looked alike z. Annoyed by the long pull-down menus and lengthy result pages without sorting z. Confused about default values z. Criticized at the result pages z. Stopped the search after getting message “no car matched” the second time

Info. Zoom Interface z. Feel overload of information upon their first glance at overview screen. z. Difficulty in choosing the appropriate search strategy at the beginning of each task. z. Participants who preferred selecting objects on the screen have difficulty to double-click on small sectors. z. Difficulty with tasks that required comparison or gathering more than one different piece of information.

Conclusion z. EPC’s interfaces were rated high, however Info. Zoom was rated higher in overall performance. z. Info. Zoom supports Choice by Attribute searching strategy better than hierarchical interface. z. Info. Zoom interface is easy to learn. z. Info. Zoom is a promising solution for an interactive online catalog interface used in e-commerce.