TM 8 th International Bielefeld Conference Academic Library

TM 8 th International Bielefeld Conference Academic Library and Information Services: New Paradigms for the Digital Age Bielefeld, Germany February 8, 2006 Presented by: Dr. Colleen Cook, Dean Texas A&M University & Dr. Fred Heath, Vice Provost of General Libraries University of Texas Project web site – www. arl. org/libqual/

Why Assess? “In an age of accountability, there is a pressing need for an effective…process to evaluate and compare research libraries. ” u 700 participants in Lib. QUAL+™ u 123 Association of Research Libraries (ARL) alone, over $3. 4 billion dollars were expended in 2003/2004 Note. M. Kyrillidou and M. Young. (2005). ARL Statistics 2003 -04. Washington, D. C. : ARL, p. 5.

Libraries Remain a Credible Resource in 21 st Century 98% agree with statement, “My … library contains information from credible and known sources. ” Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment.

Changing Behaviors Recent Survey: Only 15. 7% agreed with the statement “The Internet has not changed the way I use the library. ” Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment.

Faculty: Dependence on Electronic Resources Will Increase “I will become increasingly dependent on electronic research resources in the future. ” http: //www. arl. org/arl/proceedings/144/guthrie_files/guthrie. ppt

Research Behavior: Personal Control When searching for print journals for research: • Only 13. 9% ask a librarian for assistance • Only 3. 2% consider consulting a librarian a preferred way of identifying information Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment.

Total Circulation Note. M. Kyrillidou and M. Young. (2005). ARL Statistics 2003 -04. Washington, D. C. : ARL, p. 6.

Reference Transactions Note. M. Kyrillidou and M. Young. (2005). ARL Statistics 2003 -04. Washington, D. C. : ARL, p. 6.

Web Usage Total File Requests - UT Austin Libraries 2000 -2003

Enter Lib. QUAL+™ n The necessity of assessment n Rapid shifts in information-seeking behavior n The reallocation of resources from traditional services and functions

The Lib. QUAL+™ Premise PERCEPTIONS SERVICE “…. only customers judge quality; all other judgments are essentially irrelevant” Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.

13 Libraries English Lib. QUAL+™ Version 4000 Respondents Emergent 2000 QUAL PURPOSE Describe library environment; build theory of library service quality from user perspective Lib. QUAL+™ Project DATA Unstructured interviews at 8 ARL institutions ANALYSIS Content analysis: (cards & Atlas TI) PRODUCT/RESULT Case studies 1 Valid Lib. QUAL+™ protocol QUAN Test Lib. QUAL+™ instrument Web-delivered survey Reliability/validity analyses: Cronbachs Alpha, factor analysis, SEM, descriptive statistics Scalable process Enhanced understanding of user-centered views of service quality in the library environment 2 QUAL Refine theory of service quality Unstructured interviews at Health Sciences and the Content analysis Smithsonian libraries Cultural perspective 3 QUAL Refine Lib. QUAL+™ instrument E-mail to survey administrators Content analysis Refined survey delivery process and theory of service quality 4 QUAN Test Lib. QUAL+™ instrument Web-delivered survey Reliability/validity analyses including Cronbachs Alpha, factor analysis, SEM, descriptive statistics Refined Lib. QUAL+™ instrument 5 QUAL Refine theory Iterative 2005 700 Libraries English, Dutch, Swedish, German Lib. QUAL+™ Versions 160, 000 anticipated respondents Focus groups Content analysis Vignette Re-tooling Local contextual understanding of Lib. QUAL+™ survey responses 6

76 Interviews Conducted n n n York University of Arizona State University of Connecticut University of Houston University of Kansas n n n University of Minnesota University of Pennsylvania University of Washington Smithsonian Northwestern Medical

Loaded. PT: P 1: 01 xxxxxxx. txt, S: AdminColleenServ. Qual InterviewsTEXT Only1 xxxxx. txt

Dimensions of Library Service Quality Information Control Affect of Service Empathy Scope of Content Responsiveness Convenience Assurance Reliability Ease of Navigation Library as Place Utilitarian space Symbol Refuge Model 3 Timeliness Equipment Self-Reliance

Affect of Service “I want to be treated with respect. I want you to be courteous, to look like you know what you are doing and enjoy what you are doing. … Don’t get into personal conversations when I am at the desk. ” Faculty member

Library as Place “One of the cherished rituals is going up the steps and through the gorgeous doors of the library and heading up to the fifth floor to my study. … I have my books and I have six million volumes downstairs that are readily available to me in an open stack library. ” Faculty member

Information Control “…first of all, I would turn to the best search engines that are out there. That’s not a person so much as an entity. In this sense, librarians are search engines [ just ] with a different interface. ” Faculty member

Information Control “By habit, I usually try to be self-sufficient. And I’ve found that I am actually fairly proficient. I usually find what I’m looking for eventually. So I personally tend to ask a librarian only as a last resort. ” Graduate student

Multiple Methods of Listening to Customers n Transactional surveys* n n Mystery shopping New, declining, and lost-customer surveys Focus group interviews Customer advisory panels Service reviews Customer complaint, comment, and inquiry capture Total market surveys* Employee field reporting n Employee surveys n Service operating data capture n n n *A SERVQUAL-type instrument is most suitable for these methods Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000). Paper presented at ARL Symposium on Measuring Service Quality, Washington, D. C.

Lib. QUAL+™ Resources n An ARL/Texas A&M University joint developmental effort based on SERVQUAL. n Lib. QUAL+™ initially supported by a 3 -year grant from the U. S. Department of Education’s Fund for the Improvement of Post-Secondary Education (FIPSE) n Initial project established a expert team, re-grounded SERVQUAL concepts, and designed survey methodology n Survey conducted at over 700 libraries resulting in a data base of over half a million user responses n NSF funded project to refocus Lib. QUAL+™ on the National Science Digital Library (NSDL)

World Lib. QUAL+™ Survey 2005 Participating Libraries

Lib. QUAL+™ Languages

Rapid Growth in Other Areas n Languages q q q n American English British English French Dutch Swedish Types of Institutions q q q In development q q n q q Chinese Greek Spanish German q n Countries q n Consortia q Each may create 5 local questions to add to their survey Academic Health Sciences Academic Law Academic Military College or University Community College European Business Hospital Public State U. S. , U. K. , Canada, the Netherlands, South Africa, Sweden, France, Australia, New Zealand, Malaysia

“ 22 items” 2000 2001 2002 2003 41 -items 56 -items 25 -items 22 -items Affect of Service Affect Reliability Library as Place Reliability Personal Control Information Control Provision of Physical Collections Self-Reliance Information Access to Information

Survey Instrument

“And a Box” Why the Box is so Important q q q About 40% of participants provide openended comments, and these are linked to demographics and quantitative data. Users elaborate the details of their concerns. Users feel the need to be constructive in their criticisms, and offer specific suggestions for action.

Understanding Lib. QUAL+™ Results • Measures the distance between minimally acceptable and desired service quality ratings • Perception ratings ideally fall within the Zone of Tolerance

Key to Bar Charts

Lib. QUAL+™ 2004 Summary Colleges or Universities American English (n = 69, 449)

Adequacy Gap The difference between the minimum and perceived score Lib. QUAL+™

Score Norms n n Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample. Lib. QUAL+™ norms have been created at both the individual and institutional level

Lib. QUAL+™ Interactive Institution Statistics

In Closing Lib. QUAL+™ n Focuses on success from the users’ point of view (outcomes) n Demonstrates that a web-based survey can handle large numbers; users are willing to fill it out; and survey can be executed quickly with minimal expense Requires limited local survey expertise and resources Analysis available at local and inter-institutional levels Offers many opportunities for using demographics n n n

Lib. QUAL+™ Resources n Lib. QUAL+™ Website: http: //www. libqual. org n Publications: http: //www. libqual. org/publications n Events and Training: http: //www. libqual. org/events n Lib. QUAL+™ Bibliography: http: //www. coe. tamu. edu/~bthompson/servqbib n Lib. QUAL+™ Procedures Manual: http: //www. libqual. org/Information/Manual/index. cfm

Lib. QUAL+™ Contact Information n Amy Hoseth Ø Ø n Richard Groves Ø Ø n Statistics Research Assistant [email protected] org Ma. Shana Davis Ø Ø n Lib. QUAL+™ Communications Coordinator [email protected] org Junior Technical Applications Developer [email protected] org Martha Kyrillidou Ø Ø Director, ARL Statistics and Measurement Program [email protected] org

https: //webspace. utexas. edu/fh 355/www/

- 30 -