What Are Outcomes and Why Should I Want

What Are Outcomes and Why Should I Want to Assess Them? Dr. Jim Wilson, AVP Academic Affairs Wilmington University Dana Santoro, Data Analyst – Institutional Research Wilmington University 1

Class of 2011 n n n n What Berlin wall? They never “rolled down” a car window They have grown up with bottled water Wal-mart has always been a larger retailer than Sears and has always employed more workers than GM High definition televisions have always been available China has always been more interested in making money than in re-education They never saw Johnny Carson live on television Food packaging has always included nutritional labeling Courtesy of Beloit College Public Affairs 2

Things Have Changed! n Higher Education Research Institute (HERI) ¨ UCLA 40 yr Trends Presentation 3

Things Have Changed! Item 1970 1995 Today Postage Stamp $0. 06 $0. 32 $0. 39 Bread $0. 24 $1. 40 $1. 39 Milk $1. 32 $2. 59 $4. 29 Gas $0. 40 $1. 35 $2. 60 Car $2, 500. 00 $12, 800. 00 $27, 800. 00 Income $6, 933. 00 $19, 717. 00 $43, 200. 00 House $40, 000. 00 $111, 000. 00 $264, 000. 00 4

Education: Costs Wilmington University Tuition 30 credits, Undergraduate 1971 Full year tuition: $1, 500 Today Full year tuition: $8, 400 (New Castle, UG) 5

Education: Some Changes n Today ¨ Student-centered learning ¨ The “Net Generation” ¨ Distance Learning, Online databases n 1967 ¨ Teacher-centered learning ¨ Classroom, Books, Library card catalogs ¨ Al Gore was only 20 years old Courtesy of Beloit College Public Affairs 6

Education: Student Learning n A new “paradigm for higher education” (Suskie, 2006. ) ¨ Learning Centered ¨ Assessment feedback helps faculty understand What is and is not working effectively n How to improve the curriculum as well as teaching and learning strategies n 7

Wilmington University Mission 8

What is Assessment of Student Learning? n “Good assessment must begin in the classroom and end there. ” (Wolvord & Anderson, 1998, p. 149) 9

What is Assessment of Student Learning? n “Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations. ” (Suskie, 2006, p. 3. ) 10

What is Assessment of Student Learning? n “Assessment is first and foremost a tool for faculty members to use as they do their very best to teach their students well. ” (Middle States, 2003) 11

Assessment Is A Friend Who… Honestly tells us how we are doing n Helps to clarify what we are doing and why n Helps us to make informed decisions n Improves student learning n Helps us to improve n Brags about how well we are doing n 12

Assessment Could Be a Foe! Federal requirements n Regional Accreditation n Legislators n Parents and Community groups n Students n Professional organizations n 13

Summary What is Assessment of Student Learning? n Process of gathering & evaluating information ¨ Quantitative and qualitative ¨ From multiple sources n To enhance student learning 14

Student Outcomes Assessment Framework 15

Three Assessment Areas n Student Learning n Teaching Effectiveness n Student satisfaction with their academic experience 16

Wilmington University ACADEMIC AFFAIRS ASSESSMENT PLAN COLLEGE MISSION Assessment of Teaching Effectiveness Assessment of Student Learning Outcomes Assessment of Student Satisfaction Graduation Competencies Benchmarks Division / Program Mission Professional Standards ---- PROGRAM COMPETENCIES • IDEA Results • Faculty Evaluations • GPA Reports • Enrollment Data • ACT Survey Results Benchmarks License & Certification Exams College Outcomes Course Work Major Field Exams CECRAM Rubrics, Test Embedded Questions, Clinical Evaluations Data Collection, Analysis, Reporting, Decisions, Actions 17

Program Improvement Cycle 18

Language of Assessment n Direct evidence ¨ Direct evidence of student learning indicates whether or not a student has a command of specific content n Examples: Homework assignments, research papers, rubrics ¨ CECRAM (Course-Embedded Criterion. Referenced Assessment Measures) 19

Language of Assessment n Indirect Evidence ¨ Indirect evidence of student learning is correlational – data exists which indicates that students are probably learning. n Examples: Course grades, time spent on service learning or homework 20

Language of Assessment n Formative assessment ¨ Ongoing assessment that is intended to improve an individual student’s performance. ¨ Purpose: to improve course content, provide feedback to faculty. 21

Language of Assessment n Summative assessment ¨ Provides a true gauge of “outcomes” of student learning. ¨ Data is typically gathered near the end of program completion ¨ Examples: course-embedded project/test/portfolio, clinical evaluations, standardized exams 22

WU Language n “Good enough” ¨ The process is accurate and truthful (Suskie, 2004, p. 302) n “Closing the loop” ¨ Outcomes data has been analyzed and appropriate changes have been made. 23

Elements of Assessing Learning 1. 2. 3. 4. Articulated expectation of student learning at various levels. A plan that describes student learning assessment activities. Evidence that student learning assessment information is used. Documented use of student learning assessment as a part of institutional assessment. 24

Background Information n Academic Affairs Assessment Plan adopted in 2003 n Plan called for evaluation in 2006 25

MSCHE Outcomes Assessment Guidelines Assessment should be: ¨ Useful ¨ Cost-effective, simple ¨ Reasonably accurate and truthful ¨ Organized, systematized and sustained. Suskie, L. (2005) 26

Major Changes Related to Outcomes Assessment Five Year Data Collection and Implementation Cycle n Revision of Graduation Competencies n Representative Sampling n Formal Reporting Cycle n ¨ Five Year Program Review 27

How to maintain momentum? n A 5 year cycle will be utilized ¨ Year 1 – Implement findings from program review ¨ Year 2 – Data collection ¨ Year 3 – Reflection, take appropriate action and collect “other kinds” of data ¨ Year 4 – Data collection ¨ Year 5 – Program Review Report 28

What are Examples? (of closing the loop) Program: B. S. , Finance n n n Weakness: Undergraduate Finance majors were missing some important finance skills Source: ETS Major Field Exam Action ¨ Adjust the content of FIN 305 ¨ Add a new requirement, FIN 306 n Competency ¨ Disciplined Inquiry ¨ Finance program competencies 29

What are Examples? (of closing the loop) Program: All UG Education programs n n n Weakness: Students were unclear of required skills Source: Portfolio review Actions: ¨ Curriculum changes ¨ Provide candidates with exemplars ¨ Ensure understanding of required skills n Competency ¨ Disciplined inquiry 30

What are Examples? (of closing the loop) Program: M. S. , Community Counseling n Weakness: Appraisal Techniques n Source: CECRAM, CPCE*, NCE* & surveys n Response: Integration of assessment with diagnosis and treatment planning. ¨ Psychopathology course will precede the appraisal course *Standardized exams 31

What are Examples? (of closing the loop) Program: B. S. , Nursing n n n Weakness: Student teaching projects - measurable objectives were not reliably clear Source: CECRAM Actions: ¨ Add content and a teaching video to NUR 425 – Community Health Practitioner ¨ Added teaching content to NUR 330 ¨ Revised curriculum and developed a new course: NUR 323 - Nurse as Teacher 32

Conclusion What is outcomes assessment of student learning? Why should I support the assessment process? 33

References n n Caffo, Betty, (2003). Striving for Excellence. Faculty Senate Presentation, Wilmington College. Middle States Commission on Higher Education. (2006). Characteristics of excellence in higher education. Philadelphia: Author. Suskie, L. (2004). Assessing student learning: A common sense guide. Bolton, MA: Anker. Walvoord, B. E. , and Anderson, V. J. (1998). Effective grading: A tool for learning and assessment. San Francisco, CA: Jossey -Bass. 34