- Количество слайдов: 25
One Shot to Teach One Shot to Assess Outcome Assessment For One Shot Library Instruction Sessions
What do we mean by assessment? from the University of Oregon Teaching Effectiveness Program: “Assessment is the process of gathering and discussing information from multiple and diverse sources in order to develop a deep understanding of what students know, understand, and can do with their knowledge as a result of their educational experiences; the process culminates when assessment results are used to improve subsequent learning. ” http: //tep. uoregon. edu/workshops/teachertraining/learnercentered/assessing/definition. html
First Things First: Surveys (this is what we usually do) • Surveys typically gauge user satisfaction • Surveys are sometimes useful for self-evaluation • Surveys of students help us know what they like, not what they’ve learned • Surveys of faculty help us know what they like and may help us to evaluate what we can do better for their classes • Surveys are frequently used by libraries to satisfy accrediting organizations and campus administrators and, as such, may not really tell us what we need to know to improve our teaching
Surveys are not Assessments …not really • Surveys do not really help us estimate what learning has taken place • Surveys do not tell us what else we need to teach • Surveys do not tell us what our students don’t know • Surveys do not tell us if we have really helped our students find the information that they need to complete their research projects
So, why do we need assessment? • Internal Review – Anecdotal evidence is common, but how do we really know if we’re succeeding in our instructional programs? • Feedback to Instructional Faculty – What did we teach their students in the short time that we had them? • Program Improvement – Part of internal review. What can we do better to make our instructional program firstclass? • Accreditation – Accrediting agencies now focus on instructional outcomes. What have students gained from instruction? Have they learned?
The most important reason SACS aside, all other accrediting agencies aside, the most important reason to assess our one-shot sessions is to find out what our students are learning and what we can do to improve their learning experiences. Why try to teach, if we can’t determine that we are really teaching something?
The goal… back to Oregon “…the process culminates when assessment results are used to improve subsequent learning. ” The whole goal of teaching is to enrich the lives of those who are the recipients of teaching. Assessment is circular and should lead us back to our primary goal for teaching: ensuring learning.
How can we assess the 50 minute sessions? There are many ways to assess learning. If we establish clearning objectives and teach to the objectives, we should be able to assess student learning. Right?
If assessment is so straightforward, why doesn’t everyone do it? • Time – Most frequently cited reason for not assessing one-shot sessions • Nature of the one-shot instruction session – we see these students once; follow-up is difficult • One size does not fit all – library sessions will be as diverse as the curriculum: English, anthropology, computer science, biology, nursing, etc.
…and further • Student cooperation/participation – we can’t force them to complete an assessment • Faculty buy-in – ties in with student cooperation • Difficulty in fitting the library into the syllabus • Timing – when to schedule the assessment
So, what to do? A few ideas • Some libraries use the “one-minute paper” or “half sheet response” • Brief, but effective • Focuses on testing responses to a handful of questions that center on learning objectives for the session
Sample one-minute paper questions 1. What is the difference between an indexing/abstracting database like Basic Biosis (or other database) and a full-text database like Ebsco. Host? 2. Which kind of database was more useful to your research and why? 3. How can you tell if an article is from a scholarly source or from a popular magazine? 4. How do you know if a web site is suitable for academic work? Source: Choinski, E. and Emanuel, M. (2006) The one-minute paper and the one-hour class. Reference Services Review, 34(1): 148 -155.
Another idea: minute paper/muddiest point Reflective exercise that asks two questions: What was one new thing that you learned today? What questions do you still have about things covered in class today? Source: Radcliff, C. J. , Jensen, M. L. , Salem, J. A. , Burhanna, K. J. , & Gedeon, J. A. (2007) A practical guide to information literacy assessment for academic librarians. Westport, CT: Libraries Unlimited.
Minute paper/muddiest point Advantages (according to Radcliff, et al. ): Requires little time and money Can be administered in the classroom Little or no need for faculty collaboration Measures affective, behavioral, and cognitive domains
Minute paper/muddiest point Disadvantages (Alderman) Reviewing/evaluating answers will be time-consuming Probably not practical for every class Student participation voluntary Really should be coordinated before-hand to be most effective
Another idea: one sentence summary Asks the question: Who does what to whom, when, where, how, and why? Originally conceived by T. A. Angelo and P. Cross in their book Classroom Assessment Techniques (1993). (cited by Radcliff, et al. )
One sentence summary example WHO OR WHAT? The Boolean operator “AND” DOES WHAT? limits or narrows TO WHAT OR WHOM? search results WHEN? in a keyword search WHERE? in the online catalog HOW? by joining two different concepts WHY? in order to provide a list of books that are most appropriate for my topic Resulting sentence: The Boolean operator “AND” limits or narrows search results in a keyword search in the online catalog by joining two different concepts in order to provide a list of books that are most appropriate for my topic. Source: Radcliff, et al.
Advantages/disadvantages Pretty much the same as for the one minute paper: Easy to administer Doesn’t take long to complete Doesn’t require coordination with faculty BUT Lots of paperwork to read and assess Student participation is voluntary May not be practical for every class
Another idea: defining feature matrix Example: Source: Radcliff, et al.
And there are other assessment tools • Multiple choice instruments could be designed for specific classes to test stated objectives • Survey and assessment could be used with the same class to gauge satisfaction/instructor effectiveness and to assess learning outcomes and so on…
Ideally… • Teaching/instructional librarians would have no other job assignments but teaching and assessment • Library instruction would be integrated into ALL research classes • Teaching/instructional librarians would be able to follow their trainees throughout the semester and really be able to assess what they learned • Academic libraries would have adequate staff to do all of the above
Realistically… We’ll find a method that “satisfices. ” This may mean randomly selecting classes for evaluation This may mean having each teaching/instructional librarian run evaluation on one class each semester This may mean identifying willing subject faculty who can include a formal library component in their research courses
What we plan to do at the Carpenter Library • Design a brief web-based questionnaire • Provide library users who have attended instruction an opportunity to participate • Maybe even run a prize drawing to encourage participation • Use this as a first step in learning what our students are learning • Investigate formalizing a library component within some of the core research classes at UNF
“Had we but world enough, and time…” Andrew Marvel The above was lifted from an article by Donald Barclay in RQ. More pointedly, Barclay questions “How can the average teaching librarian conduct meaningful evaluation? ” His response “Set your sights lower and do the best evaluation you can with what you have. ” Source: Barclay, Donald. (1993). Evaluating library instruction: Doing the best you can with what you have. RQ, 33(2): 195 -204.
References Barclay, D. A. (1993). Evaluating library instruction: Doing the best you can with what you have. RQ, 33, 195 -202. Carter, E. W. (2002). "Doing the best you can with what you have: " lessons learned from outcomes assessment. The Journal of Academic Librarianship, 28(1/2), 36 -41. Choinski, E. , & Emanuel, M. (2006). The one-minute paper and the one-hour class: Outcomes assessment for one-shot library instruction. Reference Services Review, 34(1), 148 -155. Colborn, N. A. W. , & Cordell, R. M. (1998). Moving from subjective to objective assessments of your instruction program. Reference Services Review, 26(3 -4), 125 -137. Lindstrom, J. , & Shonrock, D. D. (2006). Faculty-librarian collaboration to achieve integration of information literacy. Reference & User Services Quarterly, 46(1), 18 -23. Mackey, T. P. , & Johnson, T. E. (2007). Developing an integrated strategy for information literacy assessment in general education. JGE: The Journal of General Education, 56(2), 93 -104. Neyer, L. (Fall 2003). Library instruction assessment in upper-level courses. Academic Exchange, 7(3), 85 -89. Rabine, J. L. , & Cardwell, C. (2000). Start making sense: Practical approaches to outcomes assessment for libraries. Research Strategies, 17(4), 319 -335. Radcliff, C. J. (2007). A practical guide to information literacy assessment for academic librarians. Libraries Unlimited. Saunders, L. (2007). Regional accreditation organizations' treatment of information literacy: Definitions, collaboration, and assessment. The Journal of Academic Librarianship, 33(3), 317 -326. Wong, G. , Chan, D. , & Chu, S. (2006). Assessing the enduring impact of library instruction programs. The Journal of Academic Librarianship, 32(4), 384 -395.