Encouraging Student Engagement and Educational Effectiveness at UW

Encouraging Student Engagement and Educational Effectiveness at UW Oshkosh Jillian Kinzie Associate Director NSSE Institute and Indiana University Center for Postsecondary Research

Agenda 1. Welcome and Introductions 2. Why student engagement matters 3. Some insights from NSSE 4. NSSE, FSSE and UW Oshkosh 5. Effective Educational Practice 6. Creating an Action Plan 7. Keeping the Engagement Agenda manageable 8. Wrap Up and Next Steps

The Challenge: “With respect to college, people have thought that there were two important issues: first, getting in & being able to afford college, and second, to finish and have a degree. But very few people have asked the question, What happens in the four or five years in between those two points? And we’re beginning to find out that what’s going on in that black box called college is less than we had hoped; that maybe the ‘higher’ in higher education is lower than we think. ” - RICHARD H. HERSH, Former President, Trinity College (CT)

The Challenge “We can tell people almost anything about education except how well students are learning. ” “Higher Education is stalled despite high school improvement. ” – Patrick M. Callan, President of the National Center for Public Policy and Higher Education

Student Engagement Quiz What percent of full-time seniors study, on average, more than 20 hours per week? (a) 14% (b) 23% (c) 32% (d) 41% (e) 50%

Time on Task – Average Hours Per Week First-Year Students Task Seniors Part-time Full-time Studying 9 13 10 14 Working on-campus 2 3 3 4 Working off-campus 18 5 20 10 Participating in cocurricular 1 5 2 5 Relaxing and socializing 10 12 10 11 Caring for dependents 13 2 12 4 Commuting to class 5 4 5 5

Student Engagement Quiz What percent of first-year students believe their institution provides the support they need to succeed academically? (a) 44% (b) 55% (c) 66% (d) 77% (e) 88%

Student Perceptions of Their Campus Environment Percent students say institution provides substantial* emphasis: First-Year Students Seniors Part-time Full-time Spending significant amounts of time studying 72% 81% 77% 81% Providing academic support 66% 78% 67% 73% Helping with non-academic responsibilities 27% 33% 22% 26% * substantial = ‘quite a bit’ or ‘very much’

The Challenge How might we more effectively use data about quality in undergraduate education to: üprovide evidence of student learning ümotivate and inspire institutional improvement üpromote student success?

Lessons from the Research • What matters most to desired outcomes is what students do, not who they are w A key factor is the quality of effort students devote to educationally purposeful activities w Educationally effective institutions channel student energy toward the right activities

Good Practices in Undergraduate Education (Chickering & Gamson, 1987; Pascarella & Terenzini, 2005) ü Student-faculty contact ü Active learning ü Prompt feedback ü Time on task ü High expectations ü Experiences with diversity ü Cooperation among students

Engagement Really Matters Because individual effort and involvement are the critical determinants of college impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement. Pascarella & Terenzini, How College Affects Students, 2005, p. 602

Foundations of Student Engagement Time on task (Tyler, 1930 s) Quality of effort (Pace, 1960 -70 s) Student involvement (Astin, 1984) Social, academic integration (Tinto, 1987, 1993) Good practices in undergraduate education (Chickering & Gamson, 1987) College impact (Pascarella, 1985) Student engagement (Kuh, 1991, 2005)

Defining Student Engagement Two components to student engagement w What students do —time and energy devoted to educationally purposeful activities w What institutions do —using effective educational practices to induce students to do the right things

Student Engagement is… “… the time and energy students devote to educationally sound activities inside and outside the classroom, and the policies and practices that institutions use to induce students to take part in these activities. ” (George Kuh, Change, March/April 2003)

Promise of Student Engagement “If faculty and administrators use principles of good practice to arrange the curriculum and other aspects of the college experience, students would… write more papers, read more books, meet with faculty and peers, and use information technology appropriately, all of which would result in greater gains in such areas as critical thinking, problem solving, effective communication, and responsible citizenship. ” Kuh, Kinzie, Schuh, Whitt & Associates, Student Success in College, 2005

Taking a look at NSSE: w Based on effective educational practices w Designed and tested for high validity and reliability w Relatively stable over time w High credibility of self- reported data w Students will participate w Actionable data w Psychometric properties document on website *What questions do you have about the items? ?

NSSE: The College Student Report Student Behaviors Institutional Actions & Requirements Reactions to College Student Background Information Student Learning & Development

Five Indicators of Effective Educational Practice Supportive Campus Environment Enriching Educational Experiences Level of Academic Challenge Student Faculty Interaction Active & Collaborative Learning

NSSE Project Scope § § § § Launched in 2000 First Years & Seniors Spring Administration Database =613, 500 students from 850+ different schools 50 states, PR, Canada 40+ consortia 41% response rate Oshkosh 2004 Stats: üSpring 2004, Web Admin ü 33% Response Rate (38% WI system) üSample error 3. 7% ü 302 FY / 302 Seniors

What have we learned from NSSE so far? w The single best predictor of student satisfaction with college is the degree to which they perceive the college environment to be supportive of their academic and social needs. w Effective educational practices measured by NSSE are independent of institutional selectivity. w Schools that have a lower student-faculty ratio, more full- time faculty, and more classes with fewer than 20 students generally score higher on all five NSSE benchmarks. w Grades, persistence, student satisfaction, and engagement go hand in hand.

Benchmark Scores for All Students by Undergraduate Enrollment

Academic Challenge at Two Public Universities Student engagement varies more within than between institutions.

Relationship between NSSE & Graduation Rates First-year students* Seniors* Academic Challenge . 60 . 46 Active & Collaborative Learning . 23 . 09 Student Faculty Interaction . 28 . 37 Enriching Educational Experiences . 53 . 48 Supportive Campus Environment . 38 . 26 *All correlations are significant at p<. 01

NSSE Promising Findings w More than 75% of “A” students say they are highly motivated to succeed compared with only half of the “C” students. w At institutions where faculty members use effective educational practices more frequently in their classes, students are more engaged over all and gain more from college. w A majority of students (54% of first-year students and 63% of seniors) says they often discuss ideas from readings or classes with others outside of class, and well over 90% do this at least sometimes.

Engagement Challenges & Responses CHALLENGES EFFECTIVE RESPONSES w Institutional size w University of Kansas w Varied levels of w engagement within institution w Research mission competes with undergraduate education Research Mission but focus on undergraduate teaching w Miami University w Student – faculty research emphasis

NSSE Disappointing Findings w Three of ten first-year students reported working just hard enough to get by. w Between 40% and 50% of first-year students never used career planning, financial advising, or academic tutoring services. w Almost half of all students (43% first-years, 48% seniors) spend no time on cocurricular activities.

Considering Student Engagement at UW Oshkosh Exercise #1: 1. Assess Oshkosh’s effectiveness on the five NSSE Clusters of Effective Educational Practice Ratings: + we do this very well we do this pretty well - we could improve in this area ? not sure

What do we know about Academic Challenge at UW Oshkosh? Students indicate how much (1 = “very little” to 4 = “very much”) coursework emphasized: 1. Analyzing basic elements 2. Synthesizing and of idea organizing ideas Students indicate extent institution emphasizes (1 = “very little” to 4 = “very much”) : 3. Spending significant amounts of time studying and on academic work Oshkosh – Seniors (% quite a bit, very much) 1. 85% (=) 2. 71% (=) 3. 71% (=) [= to UW system; - < UW system]

Student Engagement at Oshkosh True or False? 1. More Oshkosh seniors than first year students report that they worked harder than they thought they could to meet an instructors standards or expectations.

What do we know about Active & Collaborative learning at UW Oshkosh? Students indicate how often (1 = “never” to 4 = “very often”) they’ve done the following: Oshkosh – First-Years (% often, very often) 1. Asked questions in class or contributed to class discussions 1. 39% (-) 2. Made a class presentation 2. 25% (=) 3. Worked with other students on projects in class 3. 39% (=) 4. Worked with classmates outside of class to prepare assignments 4. 26% (-) Participated in a communitybased project (e. g. service learning) as part of a course 5. 5% (-) 5. [= to UW system; - < UW system]

Student Engagement at Oshkosh True or False? 2. More Oshkosh seniors than first year students report that they frequently* worked with classmates outside of class to prepare class assignments. (*% reporting “often” and “very often”)

Student Engagement at Oshkosh 3. What percent of Oshkosh seniors “never” participated in a community based project as part of a course? (a) 75% (b) 60% (c) 54% (d) 48% (e) 40%

What do we know about Student-Faculty Interaction and Enriching Educational Experiences at UW Oshkosh? Students indicate how often (1 = “never” to 4 = “very often”) they’ve done the following: 1. 2. Discussed ideas from your readings or classes with faculty members outside of class Oshkosh – FY and Seniors (% often, very often OR “Done”) 1. FY 8% (=/-) SR 17% (=) Received prompt feedback from 2. FY 43% (=) faculty on academic performance Seniors indicate if they have done the following: 3. Practicum, internship, co-op 4. Community service 5. Work on a research project with faculty member outside course or program requirements SR 61% (=) 3. SR 46% Done (=/-) 4. SR 58% Done (=) 5. SR 10% Done (-)

Student Engagement at Oshkosh 4. What percent of Oshkosh seniors “never” talked about career plans with a faculty member or advisor? (a) 50% (b) 43% (c) 30% (d) 27% (e) 21%

Student Engagement at Oshkosh 5. What percent of Oshkosh first-year students rate the quality of advising as “good” or “excellent”? (a) 60% (b) 68% (c) 74% (d) 80% (e) 85%

Some Findings for Oshkosh First. Years (FY) w Only 48% of FY “often-very often” prepared 2 or more drafts of a paper/assignment before turning it in w Few FY worked with classmates outside of class to prepare assignments w FY have limited interaction with faculty – they infrequently discuss grades or assignments, rarely email faculty, and don’t interact much with faculty in co-curricular settings w Oshkosh FY students are not engaged as much as their peers at other WI schools in active learning, and report limited use of practices associated with enriched learning (multiple drafts, talking with faculty, studying with peers, attending arts, theater events) w How can expectations for active learning be raised for first years?

Some Findings for Oshkosh Seniors w 58% of seniors report spending 0 hrs. a week in co-curricular activities (vs. 38% at UW syst) w 61% of seniors report little institutional emphasis on attending campus events (vs. 43% at UW syst) w Oshkosh seniors report limited engagement in activities associated with enriched learning for seniors (working with faculty on research, or in out-of-class activities, co-curricular involvements, attending cultural activities) w How can Seniors’ experience be enriched?

Considering Student Engagement at Oshkosh 1. 2. 3. 4. Do these data fit your perceptions? What surprised you? Where might you look to improve? Around which benchmarks and data points do you want more evidence?

Effective Educational Practice at UW Oshkosh Exercise #2: 1. Which activity listed in NSSE question #1 if increased would lead to greatest learning and development for first-year students? for seniors? 2. What could you do to influence this item? 3. What activities in NSSE question #1 are most interesting to UW Oshkosh? 4. What other NSSE items (#2 -30) are of interest to you?

Encouraging Educational Effectiveness: Lessons from Project DEEP: To discover, document, and describe what 20 high performing* institutions do to achieve their notable level of effectiveness. *better-than-predicted graduation rates and student engagement scores

DEEP - Six Shared Conditions 1. 2. 3. 4. 5. 6. “Living” Mission and “Lived” Educational Philosophy Unshakeable Focus on Student Learning Environments Adapted for Educational Enrichment Clearly Marked Pathways to Student Success Improvement-Oriented Ethos - “Positive Restlessness” Shared Responsibility for Educational Quality

Student-Faculty Interaction Student role in campus governance All University of Kansas committees are required to have 20% student representation, including search and screen committees. Therefore, new faculty recruits interact with students from the start.

Supportive Campus Environment Intentionally orchestrated, educationally purposeful peer interaction Longwood University values “students helping other students” as a catalyst to promote student achievement and learning and “to wake up students’ volunteerism and academic pursuits. ” Peer mentors in the Longwood Seminar, residence halls leadership roles, and the strong co-curricular program makes this possible.

Unshakeable Focus on Student Learning Timely and Apt Feedback GMU, Ursinus, CSUMB…students use electronic technology (email, Blackboard) to submit drafts of papers for feedback. “Emailing a professor is a much more efficient way to interact. . It reduces the wait between when I have a question and when I can get a response from my professor. ”

Environments Adapted for Educational Enrichment Physical space promotes collaboration Wofford’s Milliken Building -- its science center -- was intentionally designed with plenty of “fishbowls” and other areas for group work space. “Homework lounges, ” adjacent to faculty offices, also promote interactive learning.

Clearly Marked Pathways to Student Success Redundant early warning systems -“Tag Teaming” w Wheaton’s first-year student advising team includes faculty, student preceptors, librarians and administrative staff. w At Ursinus, Miami, and Wheaton representatives from both academic affairs and student affairs serve as academic advisors.

Engaging ALL Students in Learning: Lessons from DEEP w CSUMB anchored in an “assets philosophy” – students prior knowledge is used to foster learning w UTEP learning communities emphasize active and collaborative learning, structured group presentations, peer evaluation, and meetings outside of class w Sewanee emphasizes importance of prompt, detailed, meaningful feedback w George Mason, Wheaton focus on upper division experiential learning in the form of internships, field studies, service learning, capstone courses

DEEP issues to ponder: 1. To what degree are all students having a quality learning experience? 2. What might be done to improve the conditions for student success? 3. How might curricular and co-curricular opportunities be integrated to enrich student learning? 4. How do new students learn what it takes to succeed? How does this differ for seniors? 5. To what degree are faculty and staff encouraged to adopt new approaches to engaged learning?

National Survey of Student Engagement (pronounced “nessie” and “fessie”) College student survey that assesses the extent to which students engage in educational practices associated with high levels of learning and development Faculty Survey of Student Engagement Complements NSSE; To measure faculty expectations for student engagement in educational practices known to be empirically linked with high levels of learning and development

NSSE – FSSE Oshkosh Learning Activity Importance - Frequency Gap Oshkosh Upper Div. Faculty Importance ITEM Oshkosh SENIOR 89% Practicum, Internship 48% “Done” 55% Work on research project 10% “Done” 59% Work with classmates outside of class to prepare assignments 61% “Often or Very Often” (very important - imp) 66% Put together ideas or 65% “Often or Very concepts from Often” different courses

NSSE-FSSE Gap – UW Oshkosh Prompt Feedback Lower Division FACULTY gave prompt feedback often or very often 84% | 97% 1 st yr. Students STUDENTS received prompt feedback often or very often Upper Division Seniors 43% | 61%

NSSE-FSSE Educational Gains Gap [Oshkosh Seniors] Upper Div. Fac Structure Courses Quite a bit/Very much so students develop ITEM % SENIOR Frequency (very much/Quite a bit) 63% Write Clearly 70% (=) 49% Speak Clearly 66% (-) 57% Work with others 73% (-) 92% Think Critically & Analytically 83% (=) 67% Solve Complex Real Problems 49% (=) 77% Acquiring Workrelated Knowledge 72% (=)

NSSE-FSSE Gaps What to make of this? 1. When faculty members emphasize certain educational practices, students engage in them to a greater extent than their peers elsewhere. 2. Conversation starters about expectations for teaching and learning

Creating an Action Plan around Student Engagement and Educational Effectiveness

Using NSSE Results…Stimulating Conversation on Campus “NSSE is a great way to stimulate reflection and debate about what we do more and less well, and why. For us it’s proving an exciting and enlivening tool for self-reflection and self-improvement. ” --Michael Mc. Pherson, President of The Spencer Foundation (former President of Macalaster College)

Best Practices in Using NSSE Results: Institutional Examples w NSSE pivotal tool to document general characteristics undergrad ed of w Used by colleges (College of Arts and Sciences) w NSSE data used by different stakeholders w w FY data more useful for Student Affairs, SR data more useful as a proxy for student learning in Acad Affairs, w Departments use NSSE items as “big picture” indicators w Means of assessing strategy and priority for quasi-academic programs (ADP, service learning) w Fodder for department meetings w “Just another piece of the assessment puzzle” w Future ideas: Find ways to incorporate NSSE more directly into departmental assessment (primarily for Gen Ed classes); Continue to share NSSE widely; Many want data on seniors/graduates: Alumni, Endowment, Student Affairs; Use the freshmen data much more in Academic Affairs

Best Practices in Using NSSE Results: Institutional Examples w Used NSSE items in 11 a-p to assess institutional impact on college-level competencies (a. k. a. , indirect measures of student learning outcomes) w Undergraduate seniors 2005 NSSE results confirmed findings from 2004 w Most seniors (75%+) reported that KSU experience had “substantial impact” (VM+QAB) in 9 or 16 college-level competencies w KSU rank ordered competencies, showing connection to mission, and compared to other master’s instit where KSU was sig. higher, comparable, sig. lower on competencies

Best Practices in Using NSSE Results: Institutional Examples w NSSE results framed a “Sophomore Experience” w 2005 = Pace’s 5 th year of participation w Concern regarding SP- JR persistence; FY results offers context for understanding exp. as students enter SP year w Established “SP Experience Working Group” to investigate if FY exp. carried over in SP year. Focused on low NSSE score items, conducted focus groups, created sophomore survey. Led to pilot of “Pace Plan” (mentoring), includes Career Exploration Course, Sophomore Kick-Off Day w NSSE also used in strategic indicators, Accred, NCATE, AACSB, Faculty Development/Colloquia, items used by offices (Technology, Multicultural Affairs), studies performed by Enrollment Mngmt.

Best Practices in Using NSSE Results: Institutional Examples w Concerned about FY-SP retention. Used w w w NSSE to identify conflict between UC image and student experience. Shared with stakeholders to brainstorm around retention. Only 50% of FY students reported that they participated in a “learning community, ” though all were required. UC made LC goals, purpose more explicit, saw rise to 75% in next NSSE. UC mission focuses on writing, yet NSSE data did not stand out. Used data to propose new strategies around developmental writing. Saw 5% retention boost, over 2 years.

Best Practices in Using NSSE Results: Institutional Examples w Created engagement agenda on campus w NSSE introduced to New Faculty Teaching Scholars w Workshops held with academic leaders to link results with University’s strategic indicators w NSSE items in course evaluations w Open forum to get student input about ways to improve learning environment

LESSONS FOR MAXIMIZING USE OF NSSE DATA 1. Get the ideas right Focus on a real problem Ø Ø Ø Ø Persistence Under-engaged students Fragmented gen ed program Tired pedagogical practices Poor first-year experience Low academic challenge Connections to real world Capstone experiences

2. Get grass roots buy-in ü Confirm/corroborate results ü Drive data down to dept level ü Gain consensus on student engagement priorities

3. Fashion data-informed monitoring systems Ø Use multiple sources of data ü ü ü ü ACT/SAT score reports BCSSE NSSE FSSE CIRP/CSS Noel Levitz CLA ACT CAAP Ø Explain every number Ø Consider a systematic review of policies and practices (ISES)

NSSE: Only one step towards educational effectiveness Step #1: Survey Data Step #4: Follow-up • Use Results as Benchmarks to Monitor Progress • Faculty & Student Focus Groups • Survey Students • Review Results • Develop Preliminary List of Strengths and Opportunities for Improvement Step #3: Action Plan Institutional Effectiveness • Finalize Plan • Share Plan with Appropriate Groups • Link to Strategic Plan • Implement Action Step #2: Feedback • Share results with Faculty, Administrators & Students • Identify Themes & Priorities • Design Action Plan

Inventory for Student Engagement and Success A qualitative assessment of educational effectiveness based on findings from Project DEEP (Documenting Effective Educational Practice) regarding conditions that matter to student success

5. Stay the course Ø Emphasize quality Ø Front-load resources Ø If it works, consider requiring it Ø Scale up effective practices Ø Sunset ineffective programs Ø Beware the implementation dip

Creating an Action Plan: “Ready, fire, aim. ” Weick, 1995, p. 168

Keeping the Engagement Agenda Manageable… “The good-to-great transformations never happened in one fell swoop. There was no single defining action, no grand program, no one killer innovation, no solitary lucky break, no miracle moment. Sustainable transformations follow a predictable pattern of buildup and breakthrough…” (Collins, 2001, p. 186)

Exploring Next Steps at Oshkosh • What steps might you take to generate more interest in student engagement and NSSE data? • What is one thing you can commit to doing now? • What do you hope to do next? • How will you ensure the success of your 2006 NSSE administration? • Which results do you hope to monitor?

Discussion and Comments Jillian Kinzie, Ph. D. NSSE Institute – Indiana University Center for Postsecondary Research 1900 East 10 th Street Eigenmann Hall, Suite 419 Bloomington, IN 47406 Ph: 812 -856 -5824 Fax: 812 -856 -5150 nsse@indiana. edu www. iub. edu/~nsse