Usability testing for library catalogs October 25 2001

Usability testing for library catalogs October 25, 2001 Nicole Hennig, Web Manager libraries. mit. edu/barton

Thank you Tracy Gabridge Librarian for Civil & Environmental Engineering • led the HTML customization team

Details available. . . http: //macfadden. mit. edu: 9500/ webgroup/usability 2001/barton/test 1/ overview. html

Outline 1. background 2. the tests 3. problems & solutions 4. future directions

1. Background

6 month process January - June 2001 • old system: GEAC Advance • new system: Ex. Libris ALEPH

Web OPAC project teams • web OPAC team - public service librarians - circulation staff - processing staff - cataloger - web manager

Web OPAC project teams • HTML customization team same as previous, plus - systems office staff - programmer

Bibliography • on handout • includes background on display and interface design of library catalogs

Background research • a lot of research on OPAC design available • but not based on observing users or usability testing • library system vendors are not following basic good design principles

Who makes design decisions? • we have more control now that we can customize HTML screens • the vendors need to practice good design in building the system

A work in progress • libraries. mit. edu/barton • more rounds of testing and improvements are coming later in the spring

Usable design goals • every page is self-explanatory • “self-teaching” interfaces

Will it apply? • some things are specific to Ex. Libris systems • many things are general - could apply to any OPAC

General principles • success summary http: //macfadden. mit. edu: 9500/ webgroup/usability 2001/barton/ test 2/success. html

2. The tests

The test • we had already done extensive usability testing while redesigning our web site

Latest thinking has changed • 1999: Large test, 30 users, timed people - quantitative • 2001: - More frequent, smaller tests, 5 -6 people at a time - qualitative

The test • 1/2 hour long • 10 questions • think out loud

The test • observer takes detailed notes • train observers to not answer how it was supposed to work until end of the test • each observer tests 2 people (2 week time frame)

Designing questions • easy, basic tasks that a firsttime user should be able to accomplish • real-world tasks (give them a real article citation)

Designing the questions • no need to obsess over perfect, “scientific” questions • you will learn plenty from watching people use the catalog

The questions • 1 - 5: known items • 6 - 10: general research • complete list: http: //macfadden. mit. edu: 9500/webgroup/ usability 2001/barton/test 1/questions. html

The questions • test the questions • get the bugs out • print out the questions in large type

Who we learned from • Washington State University Janet Chisman, et al. “Usability Testing: A Case Study” College & Research Libraries Nov. 1999

What we learned • multi-part questions - if user can’t complete first part, observer does it so they can try second part

What we looked for • features that were confusing or unclear • aspects of the system that worked well

The tests test 1 test 2 Who 7 students 3 library staff 3 students 4 library staff 4 disabled Catalogs our old web catalog: Barton (6) Mc. Gill: MUSE (2) Boston College: QUEST(2) 1 st draft of new Barton screens Dates Jan. 22 - Feb. 1, 2001 May 21 - June 1 Successes 4 of 10 tasks 7 of 10 tasks

3. Problems & solutions

Problem 1 • people usually picked the default choices or the first choices without thinking much about it (not always the best strategy for their search)

Example people used first box, ignored second

Solution Default choice is keyword. This casts a broad net for those who forget to make a choice.

Problem 2 • Difference between browse & keyword search not clear

Example ? ?

Solution No need to know difference between keyword and browse search. Combined in one menu.

Problem 3 • it wasn’t clear how to input a search string (people used initial articles, author’s first name first, thought they had to type the entire title)

Example • carefully typed complete title, with article: The Journal of the American Chemical Society

Examples far away

Solution • include examples and instructions of how to input data near the search box and in the search menu

Examples for each type Example changes when menu changes.

Grouping { Group different title searches, author searches, and subject searches together.

Problem 4 • very busy screens with many buttons were overwhelming for people

Example

Solution • Present choices only where needed • Group navigation links in ways that make sense

Problem 5 • it was difficult to find clickable URLs for electronic titles

No URL on brief results

Better: URLs showing

Problem 6 • not enough information on brief results screen to choose the most relevant titles (especially for subject searching) • [see David Thomas article in bibliography]

keyword: women scientists

includes subject headings

Problem 7 • pop-up windows caused confusion

Solution • minimize number of pop-up windows • use only in cases where it’s handy to have the previous screen in the background

Problem 8 • title you input isn’t at the top

Solution • add marker (wish list)

Problem 9 • holdings info for serials was very confusing to everyone

Do we have Dec 13, 2000 issue? Example

Example

Example • people don’t understand open date range v. 1 (1879)-

Not easy to fix • NISO standard • way the data is input • limitations of the system

Example

A better holdings display

Problem 10 • back buttons or back links didn’t behave as expected (a problem with frames)

Coming soon • no frames version due from Ex. Libris soon • telnet version next year • minimal javascript

What worked well? - hyperlinked author names - hyperlinked subject headings people found and used these very successfully

Self-teaching interfaces • For difficult searches where you need to combine fields in a specific way: - design screen so user doesn’t need to know - it just does the right thing

Users with disabilities • 2 blind users: one used “Jaws”, one used “Window Eyes” • 1 user with dyslexia • 1 user without use of hands, used pencil in fist to type, and large trackball

Users with disabilities • these users had same problems and successes as everyone else (but the problems were magnified) • everything took longer

Users with disabilities • solutions that help everyone help disabled • solutions that help disabled help everyone

Categories of problems we can fix by: • changing the HTML • changing the tables in the database • adding custom programming • changing our indexing decisions • changing cataloging practice problems that only the vendor can fix

Other problems • many other problems not mentioned here today are described on our web site

4. Future directions

Latest usability research • most large web sites have hundreds of usability problems • continuous rounds of testing are necessary to find and fix all problems • better to begin with “usercentered design”

Latest usability research User Interface 6 East, Cambridge, MA Oct. 2001 Proceedings available http: //www. uie. com

Still to test Not so basic features, like: • email/save/print • your bookshelf • advanced searching • complex limiting • etc.

Cycles of testing • frequent small tests • test your solutions • informal tests with handful of people

Future directions • share information • compile guidelines • influence vendors

Future directions • are other libraries testing web OPACs? • contact me to share test results Nicole Hennig hennig@mit. edu