Assessment Programs in Academic Libraries: Overview of an Emerging Field 高校图书馆的评估项目：新兴 领域的概述 Scott Walter, M. L. S. , Ph. D. Associate University Librarian for Services and Associate Dean of Libraries University of Illinois at Urbana-Champaign
What We Will Cover • Assessment Basics • The Assessment Movement in U. S. Libraries • Assessment in Academic Libraries • Assessment Tools • Professional Education for Assessment
讲座内容 • • • 评估基础 美国图书馆评估运动 高校图书馆的评估 评估 具 评估的职业教育
Assessment Basics: What is Assessment? Assessment “On-going efforts to gather, analyze, and interpret evidence which describes individual, programmatic, or institutional effectiveness and using that evidence to improve practice. ” Source: Love, P. G. , & Estanek, S. M. (2004). Rethinking student affairs practice. San Francisco: Jossey-Bass.
评估基础： 什么是评估？ 评估 “不断的努力去收集、 分析和解释描述了个 人、项目或者制度的 有效性的证据并使用 这个证据来改进实践。 ” 来源: Love, P. G. , & Estanek, S. M. (2004). Rethinking student affairs practice. San Francisco: Jossey-Bass.
Assessment Basics: What is the Purpose of Assessment? “The purpose of assessment is to answer questions about the library environment such as what does the library own, who are its users, are users satisfied, and if not, why not? Assessment is followed by analysis, deciding what the collected data mean and what decisions they support. ” Source: Chrzastowski, T. (2008). Assessment 101 for librarians: A guidebook. Science & Technology Libraries, 28 (1), 155 -176.
评估的基础： 什么是评估的目的？ “评估的目的是回答关于图 书馆环境的问题，比如图 书馆拥有什么，谁是它的 用户，用户是否满意，如 果不满意，原因何在？评 估紧接分析之后，决定收 集到的数据的意义以及这 些数据支持什么样的决定。 ” 来源: Chrzastowski, T. (2008). Assessment 101 for librarians: A guidebook. Science & Technology Libraries, 28 (1), 155 -176.
Assessment Basics: Terminology • Aims – the overarching purposes of the library or of the organization of which it is a part • Objectives – refinement of aims into specifically achievable terms (usually expressed as something that can be achieved within 1 -5 years • Impact – effect of the service Source: Markless, S. , & Streatfield, D. (2006). Evaluating the impact of your library. London: Facet Publishing.
评估基础： 术语 • 目标 – 图书馆或其所在 机构的的首要目标 • 目标 – 目标的细化，细 化成可实现的事项（通 常被表达为 1到 5年内可 实现的事项） • 影响 – 服务的效果 来源: Markless, S. , & Streatfield, D. (2006). Evaluating the impact of your library. London: Facet Publishing.
Assessment Basics: Where Can Libraries Make an Impact? • Public Libraries – Community Literacy – Lifelong Learning • Medical Libraries – Information Use in the Clinical Setting – Consumer Health • School Libraries – Promoting Inquiry-Based Learning
评估的基础： 图书馆如何产生影响力？ • 公共图书馆 – 社区文化素养 – 终生学习 • 医学图书馆 – 医院情境中的信息使用 – 消费者健康 • 学校图书馆 – 促进探究式学习
Assessment Basics: Defining Objectives • Objectives for Public Libraries – To build individuals’ capacity for lifelong learning • Objectives for School Libraries – To enable parity of access to resources across schools • Objectives for Special Libraries – To promote managers’ use of the library in decision-making • Objectives for Academic Libraries – To develop the skills of library staff delivering instruction Source: Markless, S. , & Streatfield, D. (2006). Evaluating the impact of your library. London: Facet Publishing.
评估基础：明确目标 • 公共图书馆的目标 – 培养个人终生学习的能 力 • 学校图书馆的目标 – 使平等获得学校资源成 为可能 • 专业图书馆的目标 – 推广管理者在决策中使 用图书馆 • 高校图书馆的目标 – 发展图书馆员 的授课 能力 来源: Markless, S. , & Streatfield, D. (2006). Evaluating the impact of your library. London: Facet Publishing.
The Assessment Movement in U. S. Libraries “Over the past ten years we have observed a veritable explosion in the growth of assessment within libraries, with. . . new tools and measures that not only assist in understanding user needs and improving library services but also are critical in redefining library purpose and roles. ” Source: Kyrillidou, M. , Hiller, S. , & Self, J. (2011). 2010 Library Assessment Conference: Reflections from the conference cochairs. The Library Quarterly, 81 (1), 3 -5.
美国图书馆的评估运动 “在过去的10年，我们观 察到图书馆内评估的剧烈 增长. . . 新 具和衡量方 法不仅帮助理解用户需求 和改进图书馆服务，而且 对于完善图书馆的目标和 作用至关重要。 ” 来源: Kyrillidou, M. , Hiller, S. , & Self, J. (2011). 2010 Library Assessment Conference: Reflections from the conference cochairs. The Library Quarterly, 81 (1), 3 -5.
The Assessment Movement in U. S. Libraries • Activity Measures – Gate Counts – Circulation Transactions – Reference Transactions • Efficiency Measures – Turn-Around Time – Discovery to Delivery • Satisfaction Measures – User Surveys • Impact Measures – Learning Outcomes – Return on Investment
美国图书馆的评估运动 • 活动测量 – 大门计数 – 流通记录 – 参考记录 • 效率测量 – 完成一个 作的时间 – 发现到递送 • 满意度测量 – 读者调查问卷 • 影响力测量 – 学习结果 – 投资回报
The Assessment Movement in U. S. Libraries: A Culture of Assessment “A Culture of Assessment is an organizational environment in which decisions are based on facts, research, and analysis, and where services are planned and delivered in ways that maximize positive outcomes and impacts for customers and stakeholders. A Culture of Assessment exists in organizations where staff care to know what results they produce and how those results relate to customers’ expectations. ” Source: Lakos, A. , & Phipps, S. (2004). Creating a culture of assessment: A catalyst for organizational change. portal: Libraries and the academy, 4 (3), 345 -361.
美国图书馆的评估运动： 评估的文化 “评估文化是一个决定的做出是 基于事实、研究和分析的机构 环境，并且其服务是以最大化 积极成果和对用户和关系人影 响的方式计划和执行的。评估 文化存在于员 在乎其 作成 果及这些成果如何与用户的期 望值相关联的机构中。” 来源: Lakos, A. , & Phipps, S. (2004). Creating a culture of assessment: A catalyst for organizational change. portal: Libraries and the academy, 4 (3), 345 -361.
The Assessment Movement in U. S. Libraries: A Culture of Assessment “Assessment cannot be seen as a separate ‘management activity’ but must be appreciated and valued by all members of the culture and assumed to be part of their regular work. ” Source: Lakos, A. , & Phipps, S. (2004). Creating a culture of assessment: A catalyst for organizational change. portal: Libraries and the academy, 4 (3), 345 -361.
美国图书馆的评估运动： 评估的文化 “评估不能被看成是一 个单独的 ‘管理行为’， 但必须被文化所有成 员重视和认可及被假 定为日常 作的一部 分。” 来源: Lakos, A. , & Phipps, S. (2004). Creating a culture of assessment: A catalyst for organizational change. portal: Libraries and the academy, 4 (3), 345 -361.
The Assessment Movement in U. S. Libraries: An Evolving Culture • The Culture of Assessment for Reporting • The Culture of Assessment for Improvement • The Culture of Assessment for Research “Librarianship is an "information poor" information profession. Decisions are rou-tinely not supported by the evidence of welldesigned investigations. Research in the field is poorly communicated, understood, and applied. It is imperative that academic librarians and higher education libraries develop and carry out systematic research and development programs. ” Source: Neal, J. G. (2006). The research and development imperative in the academic library: Path to the future. Portal: Libraries and the Academy, 6 (1), 1 -3.
美国图书馆的评估运动： 一个演变中文化 • 报告的评估文化 • 改进的评估文化 • 研究的评估文化 “图书馆事业是”信息贫穷“的信息 职业。决定不是由设计良好的调 查研究支持的。这领域的研究没 有得到很好的沟通、理解和应用。 高校图书馆员和高等教育图书馆 发展和实行系统研究和发展项目 势在必行。” 来源: Neal, J. G. (2006). The research and development imperative in the academic library: Path to the future. Portal: Libraries and the Academy, 6 (1), 1 -3.
The Assessment Movement in U. S. Libraries: Questions and the Assessment Cycle • What are we trying to do and why? • What is my program supposed to accomplish? • How well are we doing it? • How do we know? • How do we use the information to improve or celebrate successes? • Do the improvements we make work? Source: Bresciani, M. J. , Zelna, C. L. , & Anderson, J. A. (2004). Assessing student learning and development: A handbook for practitioners. Washington, DC: NASPA.
美国图书馆的评估运动： 问题和评估周期 • 我们试图做什么和为什 么？ • 我的项目应该完成什么 ？ • 我们做得好吗 • 我们从何而知？ • 我们如何使用信息来改 进或者庆祝成功？ • 我们做出的改进有效吗 ？ 来源: Bresciani, M. J. , Zelna, C. L. , & Anderson, J. A. (2004). Assessing student learning and development: A handbook for practitioners. Washington, DC: NASPA.
The Assessment Movement in U. S. Libraries: Evidence-Based Librarianship “Evidence-based librarianship (EBL) provides a process for integrating the best available scientificallygenerated evidence into making important decisions. EBL seeks to combine the use of the best available research evidence with a pragmatic perspective developed from working experiences in librarianship. ” Source: Eldredge, J. (2006). Evidencebased librarianship: The EBL process. Library Hi Tech, 24 (3), 341 -354.
美国图书馆的评估运动： 实证图书馆学 “实证图书馆学 (EBL)为将 最佳可用的科学证据整 合进重大决策中去提供 了一个过程。EBL试图 把最佳可用的研究证据 与图书馆事业 作经验 中发展出来的实用观点 结合起来。” 来源: Eldredge, J. (2006). Evidencebased librarianship: The EBL process. Library Hi Tech, 24 (3), 341 -354.
The Assessment Movement in U. S. Libraries: Evidence-Based Librarianship • Prediction Questions – Does information literacy instruction improve the information seeking skills of students? • Intervention Questions – Do students learn information literacy skills more effectively from librarians or teaching faculty? • Exploration Questions – Why do potential users, who are presently non-users, not use the library? Source: Booth, A. (2006). Clear and present questions: Formulating questions for evidence based practice. Library Hi Tech, 24 (3), 355 -368.
美国图书馆的评估运动： 实证图书馆学 • 预测问题 – 信息素质教育是否提高了 学生的信息搜寻技巧？ • 干涉问题 – 学生从图书馆员上学习信 息素质技能是否比从授课 老师那更有效呢？ • 探索问题 – 潜在用户为什么不使用图 书馆？谁现在不是图书馆 的用户？ 来源: Booth, A. (2006). Clear and present questions: Formulating questions for evidence based practice. Library Hi Tech, 24 (3), 355 -368.
Assessment in Academic Libraries: Assessment Across the University “Without assessment, student affairs is left only to logic, intuition, moral imperatives, goodwill, or serendipity in justifying its existence. ” Assessment Across the University • Strategic Planning • Budget and Resource Allocation • Accreditation • Public Accountability Source: Upcraft, M. L. , & Schuh, J. H. (1996). Assessment in student affairs: A guide for practitioners. San Francisco: Jossey-Bass.
高校图书馆的评估： 大学里的评估 “没有评估，学生事 务就全留给了使其 存在正当化的逻辑、 直觉、道德责任、 善意或者意外收获。 ” 大学里的评估 • • 战略规划 预算和资源分配 认证 公共责任 来源: Upcraft, M. L. , & Schuh, J. H. (1996). Assessment in student affairs: A guide for practitioners. San Francisco: Jossey-Bass.
Assessment in Academic Libraries: Demonstrating Value “We’re not just trying to prove that what libraries do now and have always done is great. Libraries have new opportunities brought about by changes in technology and scholarship. There are so many new things libraries can do. . Which of these new opportunities has the most value to our constituents? ” Lib-Value: Value, outcome, and return on investment of academic libraries. http: //libvalue. cci. utk. edu/ Source: Havens, A. , & Storey, T. (2010). ROI 2020. Next. Space, no. 17. Retrieved from http: //www. oclc. org/nextspace/017/1. htm
学校图书馆的评估： 展示价值 “我们不仅是试图证明图书 馆现在和一直做的事很伟 大。技术和学术研究的变 化给图书馆带来了新机会。 图书馆由非常多的新事情 可做 . . 这些新机会中 哪个对我们的构成有最重 要的价值呢？” 图书馆价值: 高校图书馆的价值、结果 及投资回报 http: //libvalue. cci. utk. edu/ 来源: Havens, A. , & Storey, T. (2010). ROI 2020. Next. Space, no. 17. Retrieved from http: //www. oclc. org/nextspace/017/1. htm
Assessment in Academic Libraries: Demonstrating Value • Benefits of Buying Power • Support for Accreditation • Contributions to Student Recruitment and Retention • Impact on Student Learning • Contributions to Faculty Recruitment • Impact on Faculty Productivity in Teaching and Research • Contributions to Strategic Priorities Oakleaf, M. (2010). The value of academic libraries: A comprehensive research review and report. Chicago: Association of College & Research Libraries.
学校图书馆的评估： 展示价值 • • • 购买力的好处 支持认证 对学生招募和保留的贡献 对学生学习的影响 对教师聘任的贡献 对教师教学和研究生产力的 影响 • 对战略优先性的贡献 Oakleaf, M. (2010). The value of academic libraries: A comprehensive research review and report. Chicago: Association of College & Research Libraries.
Assessment in Academic Libraries: Evolution of Traditional Services • Activity Assessment – Tick Marks • Performance Assessment – Unobtrustive Observation – Transcript Analysis (digital reference) • Satisfaction Assessment – WOREP – Lib. QUAL+ • Service Assessment – READ Scale http: //www. dom. edu/library/READ/ index. html
高校图书馆的评估： 传统服务的演变 • 活动评估 – 刻度标记 • 绩效评估 – 不被察觉的观察 – 记录分析 (数字参考咨询) • 满意度评估 – 威斯康星和俄亥俄州参考评 估项目 – 衡量图书馆服务质量项目 • 服务评估 – 参考服务评估数据测量数值 http: //www. dom. edu/library/READ/ index. html
Assessment in Academic Libraries: Impact on Student Learning • Processes used to determine an individual’s mastery of complex skills through performance • Use of standardized examinations to establish/meet benchmarks • Program evaluation aimed at gathering evidence to improve curriculum or pedagogy Source: Ewell, P. T. (2002). An emerging scholarship: A brief history of assessment. In T. W. Banta (Ed. ), Building a scholarship of assessment (pp. 3 -25). San Francisco: Jossey-Bass.
高校图书馆的评估： 对学生学习的影响 • 用于确定一个人表现 出的对复杂技能的掌 握的过程 • 用标准化的测试来建 立/满足基准 • 项目评估目标在于为 改进课程或教学方法 收集证据 来源: Ewell, P. T. (2002). An emerging scholarship: A brief history of assessment. In T. W. Banta (Ed. ), Building a scholarship of assessment (pp. 3 -25). San Francisco: Jossey-Bass.
Assessment in Academic Libraries: Impact on Student Learning Options for Assessment of Student Learning • • Performance Assessment Capstone Assessment Student Portfolios Course-Based Assessment Approaches Source: Palomba, C. A. (2002). Scholarly assessment of student learning in the major and general education. In T. W. Banta (Ed. ), Building a scholarship of assessment (pp. 201 -222). San Francisco: Jossey. Bass. Assessment of Student Learning in Academic Libraries • Information Literacy Competency Standards for Higher Education • VALUE Rubric for Information Literacy • Portfolio assessment, e. g. , as part of writing assessment • Course-integrated assessment
高校图书馆的评估： 对学生学习的影响 学生学习评估的选项 • • 学业评估 压轴课程评估 学生档案 课程为基础的评估方法 来源: Palomba, C. A. (2002). Scholarly assessment of student learning in the major and general education. In T. W. Banta (Ed. ), Building a scholarship of assessment (pp. 201 -222). San Francisco: Jossey-Bass. 高校图书馆学生学习评估 • 高等教育信息素质能力标准 • 信息素质评价标准 • 档案评估，例如，是写作评 估的一部分 • 综合课程评估
Assessment in Academic Libraries: Impact on Faculty Productivity “I would leave this university in a nanosecond if the library deteriorated to the point of making me uncompetitive for research and funding. ” Source: Kaufman, P. T. (2008). The library as strategic investment: Results of the Illinois return on investment study. Liber Quarterly, 18 (3/4). Retrieved from http: //liber. library. uu. nl/publish/articles/000269/arti cle. pdf
高校图书馆的评估： 对教师生产力的影响 “我将以闪电般的速 度离开这所大学， 如果图书馆退化到 使我在研究和筹措 资金上毫无竞争力 的地方的话。” 来源: Kaufman, P. T. (2008). The library as strategic investment: Results of the Illinois return on investment study. Liber Quarterly, 18 (3/4). Retrieved from http: //liber. library. uu. nl/publish/articles/000269/arti cle. pdf
Assessment Tools • ARL Statistics Program – – Annual Statistics Lib. QUAL+ MINES for Libraries Climate. QUAL • Information Literacy Rubrics • Balanced Scorecard • User Studies – User Surveys – Usability – Ethnography
评估 具 • 研究图书馆协会统计项目 – – 年度统计 衡量图书馆服务质量项目 测量网络化电子服务的影响 组织氛围和多样化评估 • 信息素质评分 • 平衡记分卡 • 用户研究 – 用户调查问卷 – 易用性 – 人种志
Assessment Tools • Access Services – Benchmarks and the Balanced Scorecard http: //www 2. lib. virginia. edu/bsc/metrics/all 0708. html • Collection Services – ARL E-Metrics http: //www. arl. org/stats/initiatives/emetrics/index. shtml • Information Services – Reference Effort Assessment Data http: //www. dom. edu/library/READ/index. html • Instructional Services – Project SAILS https: //www. projectsails. org/sails/about. SAILS. php? page=about. SAIL S Additional resources available at http: //www. library. illinois. edu/assessment/
评估 具 • 访问服务 – 基准和平衡记分卡 http: //www 2. lib. virginia. edu/bsc/metrics/all 0708. html • 馆藏服务 – 研究图书馆协会电子资源测量 http: //www. arl. org/stats/initiatives/emetrics/index. shtml • 信息服务 – 参考服务评估数据 http: //www. dom. edu/library/READ/index. html • 教学服务 – 信息素质技能标准化评估 https: //www. projectsails. org/sails/about. SAILS. php? page=about. SAIL S 其它资源可参考 http: //www. library. illinois. edu/assessment/
![Professional Education for Assessment “[The] ability to design an assessment program and to use](https://present5.com/presentation/aac687c263844fe3cddfaf7f6f15eaaa/image-48.jpg "Professional Education for Assessment “[The] ability to design an assessment program and to use")
Professional Education for Assessment “[The] ability to design an assessment program and to use its results in planning and decision making is a critical skill set for any information professional. Assessment is the new black – it goes with whatever job you have, and it is relevant to every library environment. ” Source: Walter, S. (2009, April 22). Assessment is the new black. Posted to ACRLog. Retrieved from http: //acrlog. org/2009/04/22/assessme nt-is-the-new-black/
评估的职业教育 “设计一个评估项目和将其 结果用于规划和决策的 能力是对于任何信息专 业人员而言非常重要的 能力。评估是新贵－不 管你从事什么职位都需 要并和每个图书馆环境 都相关。” 来源: Walter, S. (2009, April 22). Assessment is the new black. Posted to ACRLog. Retrieved from http: //acrlog. org/2009/04/22/assessme nt-is-the-new-black/
Professional Education for Assessment: What Do Librarians Need to Learn? • How to develop an assessment plan – Identifying the purpose of the assessment activity • Conceptualizing library impact – Articulating questions about library impact • Defining an action plan – Identifying outcomes – Determining assessment method – Gathering data • Implementing the assessment method – – Usability Survey Transaction Analysis Artifact Analysis • Communicating Library Impact – Identifying relevant results – Communicating results to secure resources Source: Oakleaf, M. (2011). Are they learning? Are we? Learning outcomes and the academic library. Library Quarterly, 81 (1), 61 -82.
评估的职业教育： 图书馆员们需要学习什么？ • 如何开展一个评估计划 – 确定评估活动的目标 • 具体化图书馆的影响 – 明确提出关于图书馆影响 的问题 • 确定一个行动方案 – 确定结果 – 确定评估方法 – 收集数据 • 执行评估方法 – – 易用性 调查问卷 记录分析 人为因素分析 • 传达图书馆的影响 – 确定相关的结果 – 将结果传达给可靠的资源 来源: Oakleaf, M. (2011). Are they learning? Are we? Learning outcomes and the academic library. Library Quarterly, 81 (1), 61 -82.
Professional Education for Assessment: Opportunities for Learning • Assessment in Higher Education • Assessment in Service Areas – Association of American Colleges and Universities http: //www. aacu. org/resources/ assessment/index. cfm – Assessment Institute http: //www. planning. iupui. edu/i nstitute • Assessment in Academic Libraries – Association of Research Libraries http: //www. arl. org/stats/ – Information Literacy Immersion http: //www. ala. org/ala/ mgrps/divs/acrl/issues/in folit/professactivity/iil/im mersion/assessment. cfm – Institute on Scholarly Communication http: //www. ala. org/ala/ mgrps/divs/acrl/issues/sc holcomm/scinstitute. cfm
评估的职业教育： 学习的机会 • 高等教育的评估 • 服务领域的评估 – 美国高校联合会 http: //www. aacu. org/resources/ assessment/index. cfm – 评估机构 http: //www. planning. iupui. edu/i nstitute • 高校图书馆的评估 – 研究图书馆协会 http: //www. arl. org/stats/ – 信息素质沉浸 http: //www. ala. org/ala/ mgrps/divs/acrl/issues/in folit/professactivity/iil/im mersion/assessment. cfm – 学术通讯研究所 http: //www. ala. org/ala/ mgrps/divs/acrl/issues/sc holcomm/scinstitute. cfm
• Texts Professional Education for Assessment: Opportunities for Learning • Blogs – Matthews, J. R. (2007). – libraryassessment. info Library assessment in higher http: //libraryassessment. inf education. Westport, CT: o/ Libraries Unlimited. • Discussion Lists • Journals – Canadian Library Association – Evidence-Based Library & Evidence Based Information Practice Librarianship Interest Group http: //ejournals. library. ualb http: //lists. mun. ca/archives/ erta. ca/index. php/EBLIP eblig. html – Performance Measurement – IFLA Statistics and Evaluation and Metrics Section http: //infoserv. inist. fr/wwsy mpa. fcgi/info/iflastat
评估的职业教育： 学习的机会 • 教科书 • 博客 – Matthews, J. R. (2007). – libraryassessment. info Library assessment in higher http: //libraryassessment. inf education. Westport, CT: o/ Libraries Unlimited. • 讨论区 – Canadian Library Association • 期刊 – Evidence-Based Library & Evidence Based Information Practice Librarianship Interest Group http: //ejournals. library. ualb http: //lists. mun. ca/archives/ erta. ca/index. php/EBLIP eblig. html – Performance Measurement – IFLA Statistics and Evaluation and Metrics Section http: //infoserv. inist. fr/wwsy mpa. fcgi/info/iflastat
Professional Education for Assessment: Opportunities for Learning • Association of Research Libraries – Library Assessment Conference http: //libraryassessment. org/ • Evidence Based Library and Information Practice Conference http: //www. eblip 6. salfor d. ac. uk/ • Northumbria International Conference on Performance Measurement in Libraries and Information Services http: //www. york. ac. uk/a bout/departments/suppo rt-and-admin/information directorate/events/perfor mance-conference-2011/
评估的职业教育： 学习的机会 • 研究图书馆协会– 图书 馆评估会议 http: //libraryassessment. org/ • 实证图书馆和信息实践 会议 http: //www. eblip 6. salfor d. ac. uk/ • 诺森比亚图书馆信息服 务绩效测量国际会议 http: //www. york. ac. uk/a bout/departments/suppo rt-and-admin/information directorate/events/perfor mance-conference-2011/
Questions 问题
Contact联系方式 Scott Walter Associate University Librarian for Services and Associate Dean of Libraries University of Illinois at Urbana-Champaign swalter@illinois. edu
Скачать презентацию Assessment Programs in Academic Libraries Overview of an
aac687c263844fe3cddfaf7f6f15eaaa.ppt