E-Use Measurement A Detour Around the Publishers ALCTS

E-Use Measurement: A Detour Around the Publishers ALCTS | Summer 2005 Joe Zucca Assessment, Planning and Publications Librarian University of Pennsylvania Library

ALCTS Summer 2005 Student Use of E-Resources: Three Profiles by Residence logins per capita, by hour. Feb ‘ 03 -May ’ 05 [49, 857 logins total] Harnwell N=757 Avr. Login per Resident Stouffer N=345 Kings Court N=358 Hour of Day

ALCTS Summer 2005 The Top 15 Titles by Use, Comparative Ranks: 3 Houses and All Users 3 HOUSES as a GROUP ALL DOMAINS Lexis. Nexis Academic 1 8 Factiva 2 4 Onefile 3 11 MEDLINE 4 1 JSTOR 5 12 Science. Direct Journals 6 2 Oxford English Dictionary 7 27 Pub. Med Plus 8 3 MLA International Bibliography 9 25 Psyc. INFO 10 10 CINAHL/ Nursing 11 17 ABI/Inform - Proquest 12 18 ISI Citation Indexes 13 5 Encyclopaedia Britannica 14 37 Oxford Reference 15 28 TITLES

ALCTS Summer 2005 Key Attributes of Metrics and Measurement Processes Metrics Comprehensive [Licensed Stuff, Local, Public] Consistent [Uniform Across Resources, Replicable] Context-Sensitive [Location, Time, Demographic, Cost, Resource Type & Class, Need] Processes-Data Granular, Normalized, Integrative, Anonymous Processes-Architecture Scaleable, Efficient (high output for low effort), Sustainable, Distributed

ALCTS Summer 2005 Schematic View of the Data Logging Process 1. User initiates a session using pages spawned by a database 4. Pass to vendor site 2. Db swaps handle for authentic url 3. Proxy scan 3. 1 Authentication challenge 5. Db logs event 5. 1 Proxy server logs the event ERMS 6. Data Farm

ALCTS Summer 2005 Data Farm Project: Events, Data, Plumbing, Information Service Events Borrow a Video Customers Staff Inform Access an E-journal Catalog a Book Staff Visit a Library Copy or Print Data Streams People and Network Data Clean | Anonymize | Integrate Data Farm Environment

ALCTS Summer 2005 Data Farm Oracle Space: 14+ gb, in 75 tables, tracking events and their properties E-Resource Use Circulation Reference Contact Web Analytics Acquisitiions Funds Resolve Resource Holdings Resolve People Tech Processing Workflow Resolve Places Image Collection Use Digital Library (erms) Print Library (Voyager) Building Use Copier | Printer Use Staff Census Reference|Instruction (dynamic) Metadata Gate Swipes Administration LDAP

ALCTS Summer 2005 So, what do we do with the data? Provide a distributed and multi-tiered environment for interacting with raw data and generating quantitative information Dashboard Reports (static) Report Builders (dynamic) Data Bureau (ad hoc reports and fishing expeditions) Lower barriers between management information and people with management responsibility (or any other staff) Help to scale and institutionalize assessment

ALCTS Summer 2005 INTERACTIONS with DATA Dashboards: Vital signs at a glance

ALCTS Summer 2005 INTERACTIONS with DATA Report Builders

ALCTS Summer 2005 Attempts at ranking for selection based on use and other factors

ALCTS Summer 2005 A Hypothetical Scheme for Journal Selection Ranking Qualitative Scores Librarian rank Impact Title 1 1 2. 60 Title 2 1 Title 3 3 Title n Quantitative Scores Penn Artcls. Factor Rank Use Metric Cost Effc 1 3 4 -1. 54 1 3. 87 6 4 54 -1. 72 2 22. 4 9 9 8 -1. 89 3

ALCTS Summer 2005 INTERACTIONS with DATA Distribution of E-resource Use Across the Libraries FY 04 Ad Hoc | Mediated Logins Pct. Cml Pct. 174, 584 55. 2% Biomedical (public areas) 41, 577 13. 2% 68. 4% Staff (Van Pelt-Dietrich and Biomedical) 41, 356 13. 1% 81. 5% Electronic Classroom 14, 535 04. 6% 86. 1% Fine Arts 13, 671 04. 3% 90. 4% Lippincott 10, 293 03. 3% 93. 6% Veterinary 5, 708 01. 8% 95. 4% Dental 4, 304 01. 4% 96. 8% Museum 3, 598 01. 1% 97. 9% Chemistry 1, 950 00. 6% 98. 6% Math 1, 901 00. 6% 99. 2% Undergraduate Study Ctr 1, 502 00. 5% 99. 6% Rare Book and Manuscript 954 00. 3% 99. 9% High Density Storage 209 00. 1% 100. % Van Pelt (public areas) Total Libraries 316, 140 100%

ALCTS | Summer 2005 Ad Hoc Interactions with the Database Items Charged and E-resource Log-ins Fall 2004

ALCTS | Summer 2005 Relationship Between E-Resource Use and Item Circulation, Fall 2004 72 observations

ALCTS Summer 2005 Practical Applications of Data Farm Resources Budget defense (slow but steady rates of increase, curtailing the need to make major cuts in materials) Print collection management E-resource selection (driving trade-offs more typically than cuts) Staffing | Facilities Planning Developing and testing web tools Feeding the data-mavens and spreading best practices

ALCTS Summer 2005 Key Attributes of Metrics and Measurement Processes [REPRISED] Metrics Comprehensive [Licensed Stuff, Local, Public] Consistent [Uniform Across Resources, Replicable] Context-Sensitive [Location, Time, Demographic, Cost, Resource Type & Class, Need] Processes-Data Granular, Normalized, Integrative, Anonymous Processes-Architecture Scaleable, Efficient (high output for low effort), Sustainable, Distributed

E-Use Measurement: A Detour Around the Publishers Joe Zucca zucca@pobox. upenn. edu University of Pennsylvania Library

ALCTS | Summer 2005 Science. Direct Journals: Regression of View Counts (Elsevier) on Penn Login Counts. Feb-Nov 2003. [1, 270 cases]

ALCTS | Summer 2005 OCLC Databases: Regression of Vendor Login Counts on Penn Login Counts, jl 2001 -je 2002. [48 cases] 1800 2 R = 0. 9814 1600 OCLC Login Count 1400 1200 1000 800 600 400 200 0 0 500 1000 1500 Penn Login Count [count-use] 2000

ALCTS | Summer 2005 Three E-Use Profiles by Residence: Top Ten Titles Used Logins per Capita, Feb ‘ 03 -May ‘ 05 Harnwell Kings Court Stouffer N=757 N=358 N=345 Lexis. Nexis Academic 2. 7 1. 7 0. 6 Factiva 2. 2 1. 3 0. 5 Onefile 1. 4 0. 4 JSTOR 1. 2 0. 7 0. 5 MEDLINE (complete file) 1. 1 0. 8 0. 5 Oxford English Dictionary 0. 7 0. 8 0. 4 Pub. Med Plus 0. 8 0. 7 0. 2 MLA International Bibliography 0. 7 0. 8 0. 3 Psyc. INFO 1. 0 0. 3 0. 2 CINAHL 0. 7 0. 4 0. 5

ALCTS | Summer 2005 Three E-Use Profiles by Residence: Top 15 Titles Used (by login count) Harnwell Kings Court Stouffer Total Logins Lexis. Nexis Academic 2011 597 201 2809 Factiva 1672 454 164 2290 Onefile 1381 611 188 2180 MEDLINE 916 297 200 1413 JSTOR 887 261 158 1306 Science. Direct Journals 809 226 190 1225 Oxford English Dictionary 565 286 126 977 Pub. Med Plus 617 250 82 949 MLA Intrn’l Bibliography 511 301 104 916 Psyc. INFO 731 102 53 886 CINAHL/ Nursing 563 134 171 868 ABI/Inform 689 121 43 853 ISI Citation Indexes 392 204 110 706 Encyclopaedia Britannica 359 235 45 639 Oxford Reference 354 215 64 633 TITLES