Why a measure of noncognitive abilities is essential to drive student success Innovations 2013 Dallas, Texas March 2013 Victoria Monaghan and Steve Robbins Educational Testing Service
We don’t need to remind you: More than 1. 7 million students begin community college each year. But many are unprepared • 50 percent of community college students need remedial – or below college level – courses • 20 percent of 4 -year college students also start in remediation • 40 percent of these students – frustrated by their low placement – don’t even bother to enroll in classes 2
And the path to college-level work has detours 4 our of every 10 students in remedial classes never complete their remedial sequence Placement tests alone do not yield strong predictions of how students will perform in college We believe there is a critical need to use a holistic, multiple measures approach to college course placements 3
What are Noncognitive Skills? Conscientiousness Social Support Teamwork Self Efficacy Worry Metacognition Study Skills Anything outside of “academic ability” or “academic intelligence” that contributes to or is part of student learning Institutional Commitment Motivation Goal Setting 4 Test Taking Strategies
“Genius is 1% inspiration and 99% perspiration. ” ~Thomas Edison The appeal of these skills • We like them • We (and others) value them • They do indeed work • We can change them 5
Our mission statements agree, identifying noncognitive skills frequently Taxonomy of Valued Academic Skills: Mission Statements from 35 schools (College Board/Michigan State University)1, 2 • • • 1 2 Knowledge, learning, mastery of general principles Continuous learning, intellectual interest and curiosity Artistic cultural appreciation and curiosity Multicultural tolerance and appreciation Leadership • • Social responsibility, citizenship and involve Physical and psychological health Career orientation Adaptability and life skills Perseverance Interpersonal skills Ethics and integrity Schmitt, Oswald, Kim, Imus, Drzakowski, Friede, & Shivpuri (2007). Oswald, Schmitt, Kim, Ramsay, & Gillespie (2004). 6
And employers care as well Conference Board/SHRM survey of > 400 employers from across the United States (Casner-Lotto & Barrington, 2006) Skill 88 26 Mathematics 64 18 Science 33 15 Teamwork/ Collaboration Noncognitive % Prepared English Language Cognitive % Important 94 25 Professionalism/ Work Ethic 94 17 Ethics/ Social Responsibility 86 18 7
Adding a noncognitive assessment allows for a holistic view of students • Respects the whole student (examines cognitive, non-cognitive, behavioral factors) • Focuses on the unique individuality of students • Focuses on factors educators can control and empower students to modify: such as motivation, social connectedness, as opposed to socio-economic, situational factors • A common language for early warning, advising, and acceleration decisions 8
Strategies for organizing work and time. Meeting Class Expectations Doing what’s expected to meet the requirements of your course including assignments and in-class behaviors. I attend almost all of my classes. I complete the reading that is assigned to me. Commitment to College Goals Perceived value and determination to succeed in and complete college. One of my life goals is to graduate college. The benefit of a college education outweighs the cost. Attachment to and positive evaluations of This is the right school for me. the school. I’m proud to say I attend this school. Tendency to feel frustrated, discouraged, I get stressed out easily when things don't or upset when under pressure or burdened go my way. by demands. I am easily frustrated. Academic Self-Efficacy Belief in one’s ability to perform and achieve in an academic setting. I'm confident that I will succeed in my courses this semester. I can do well in college if I apply myself. General reactions to test-taking experiences, including negative thoughts and feelings (e. g. , worry, dread). When taking a test, I think about what happens if I don't do well. Before a test, my stomach gets upset. Connectedness Self-Management Reactions to academic and daily stress Organization Test Anxiety Commitment Active pursuit toward an academic goal. Definition Sensitivity to Stress Academic Skills Tools and strategies for academic success Facet Institutional Commitment Domain A general sense of belonging and engagement. Social Support Institutional Support Connecting with people and students resources for success Barriers to Success Example Items I make a schedule for getting my school work done. I take due dates seriously. I feel connected to my peers. People understand me. If I don't understand something in class, Attitudes about and tendency to seek help I ask the instructor for help. from established resources. I know how to find out what's expected of me in classes. Family pressures make it hard for me to Financial pressures, family responsibilities, commit to school. conflicting work schedules, and limited People close to me support me going to institutional knowledge. college. 9
From the “so what” to the “now what”……. . Data Integration & Use Across Silos Holistic Assessment Student Commitment & Classroom Success How do we. . . • Identify these skills in an operational setting? • Best disseminate that information to students, faculty, staff, and institutions? • Develop and target sound and practical interventions? • Consider individual differences in background, skills, abilities, and experiences? 10
Early Intervention is key Three basic implications: • Educators have a profound influence on students – Faculty can motivate and inspire students – Educators can also shape and foster their intentionality and academic determination • Observable behaviors are indicative of internal states – E. g. , Motivation, engagement • Educators have the opportunity and capability to guide students towards desirable classroom/academic behaviors 11
Noncognitive Assessment Experience at Wright College semester Note. Results from Wright College (Spring 2009). n = 453; All paths are statistically significant. 12
Research shows that behavioral monitoring and intervention are key to student success Math Readiness Effort Level High Medium Low High . 96 . 91 -- Medium . 94 . 77 . 82 Low . 86 . 61 . 38 Note. N = 411. Success rates were not calculated for cells with insufficient sample sizes (denoted by a dash). 13
ETS/CCC Collaboration Objectives • Combine COMPASS and Psychosocial Measure to accelerate student progress and college completion – Students placed within the upper-band with high psychosocial scores can enroll in the next level course • Allow more individualized, actionable plans for students tied to curricular goals • Non-cognitive survey and in-class behavioral ratings will be used to trigger Early Alert and intervention strategies • Create supplementary services tied to Early Alert and to behavioral monitoring
Comprehensive Early Interventions Map support services to non-cognitive psychological constructs: Academic Skills, Self Management, Motivation, Social Support Non-Cognitive Construct Psycho-social Skills Existing natural fit support services at CCC Academic Skills Study skills workshop Time management skills Study skills workshop Emotional control Wellness Center Resilience Meeting with advisors, tutors and instructors Degree intention Transfer center, advising Class attendance Advisors, tutors, instructors Engagement Advisors, tutors, instructors Help seeking New student orientations, advisors, service referrals Peer support Student clubs and organizations Self Management Motivation Social support Gap analysis: Need additional intervention strategies at individual college?
Student Name: Student ID: Score Date: Juan Rodriguez 12 -34567 Sep 12, 2013 Matthew’s Background Matthew’ • Race: Latino • Gender: Male • Parental Education: Unknown (Mother), Unknown (Father) Academic Skills – Tools to succeed in the classroom Advisor Report COURSE ACCELERATION ACADEMIC SUCCESS SCORE* NOT RECOMMENDED LOW RETENTION SUCCESS SCORE* LOW SKILL REPORT Skills • Sometimes uses organizational tools (e. g. , checklist, planner) • Occasionally mismanages his schedule regarding assignments and exams Next Steps Refer Juan to the Tutoring Center for goal setting strategies and organizational tools. You can find additional ways of working with Matthew by clicking here. MODERATE Commitment – Active pursuit towards an academic goal Skills • Moderately committed to attaining an associate degree • Consistently attends class • Moderate interest in engaging with resources on campus Next Steps The Career Center can help Juan align his degree goals with his career goals. You can find additional ways of working with Matthew by clicking here. MODERATE Self-Management – Managing the daily expectations Skills • Usually calm during important tests and assignments • Occasionally gives up when frustrated with assignments Next Steps Refer Juan to the Wellness Center on campus to help him manage pressures and succeed. For effective ways of helping Matthew manage every day academic stress, click here. MODERATE Social Support – Networks to support academic success Skills • Rarely engages resources on and off campus • Parents show little support for his college attendance Next Steps The Office of Student Life can connect Juan with valuable student groups on campus. For helpful tools and tips about student connections, click here. LOW *Course Placement: Recommended academic placement action for student. * Academic Success Score: Weighted composite of student ’s Success. Navigator ™ profile and other academic indicators of student preparedness, such as self or institution reported high school GPA, SAT/ACT, etc. * Retention Success Score: Projected likelihood that student will return for a second year at the institution.
Juan Background: Latino Male Major: Business Status: Part-time (10 credits) Current GPA: 2. 29 Age: 18 Parental Education: Unknown (Mother), Unknown (Father) Misc: Undocumented ESL student who is in the National Guard to obtain citizenship. He works 40 hours/week at UPS to cover his tuition and help his family financially. Academic Preparation: • 2. 3 HS grade point average • Scored 900 on SAT • Placed in bottom third of placement tests • Taking remedial Algebra and English Success Navigator profile indicates: He has moderate Academic Skills, Commitment, and Self. Management, but low Social Support Notes: Overall Juan is struggling both academically and socially and is at-risk for dropping out. His financial / work responsibilities and full-time student status have limited his ability to focus on his academics and interact with other students. Since he has little support for attending college he would benefit from engaging with formal (e. g. tutoring) and informal resources on campus. Also, because he remains dedicated to earning his degree and is confident in his ability to succeed, he might benefit from reducing his course load even further which will help him balance all his responsibilities. Student Withdrew
Student Name: Student ID: Score Date: Sarah Smith 13 -75680 Sep 2, 2013 Sarah’s Background Sarah’ Advisor Report COURSE ACCELERATION • Race: Caucasian • Gender: Female • Parental Education: High School (Mother), Below High School (Father) Academic Skills – Tools to succeed in the classroom ACADEMIC SUCCESS SCORE* RECOMMENDED RETENTION SUCCESS SCORE* MODERATE HIGH SKILL REPORT Skills • Sometimes uses organizational tools (e. g. , checklist, planner) • Occasionally mismanages her schedule regarding assignments and exams Next Steps Refer Sarah to the Tutoring Center to provide goal setting strategies and organizational tools. You can find additional ways of working with Sarah by clicking here. MODERATE Commitment – Active pursuit towards an academic goal Skills • Strongly committed to attaining an associate degree • Consistently attends class • Strong interest in engaging with resources on campus Next Steps The Career Center can help Sarah align her degree goals with her career goals. You can find additional ways of working with Sarah by clicking here. HIGH Self-Management – Managing the daily expectations Skills • Usually calm during important tests and assignments • Sometimes gives up when frustrated with assignments Next Steps Refer Sarah to the Wellness Center on campus to help her manage pressures and succeed. For effective ways of helping Sarah manage every day academic stress, click here. MODERATE Social Support – Networks to support academic success Skills • Sometimes engages resources on and off campus • Parents show some support for her college attendance Next Steps The Office of Student Life can connect Sarah with valuable student groups on campus. For helpful tools and tips about student connections, click here. MODERATE *Course Placement: Recommended academic placement action for student. * Academic Success Score: Weighted composite of student ’s Success. Navigator ™ profile and other academic indicators of student preparedness, such as self or institution reported high school GPA, SAT/ACT, etc. * Retention Success Score: Projected likelihood that student will return for a second year at the institution.
Sarah Background: Caucasian Female Major: Liberal Arts Status: Full-time Current GPA: 3. 0 Age: 19 Parental Education: Below High School (Mother), High School (Father) Misc: She is the first member of her family to attend college. She was very involved in high school extra-curriculars. She was class secretary, earned a varsity letter in soccer, and was a member of the chorus. Academic Preparation: • 2. 8 HS grade point average • Scored below 900 on SAT • Placed in remedial Algebra Noncognitive profile indicates: She has strong Commitment and above average Academic Skills, Self Management, and Social Support Prime candidate for acceleration Notes: Although Sarah struggles academically, especially in math, she is a conscientious student who is in regular contact with her advisor and has used study groups to help her academically. Since she has very high commitment both to the institution and to the goal of earning a degree, she would benefit from continual encouragement to make use of the academic resources on campus such as the tutoring center. She should also be encouraged to maintain regular contact with her professors.
Student Name: Student ID: Score Date: Janet Jerzyk 18 -34287 Sep 25, 2013 Janet’s Background Janet’ • • Advisor Report COURSE ACCELERATION ACADEMIC SUCCESS SCORE* RECOMMENDED HIGH Major: Education Race: Multiracial/multiethnic Gender: Female Parental Education: Some College (Mother), High School (Father) Academic Skills – Tools to succeed in the classroom RETENTION SUCCESS SCORE* MODERATE SKILL REPORT Skills • Always uses organizational tools (e. g. , checklist, planner) • Successfully manages her schedule regarding assignments and exams Next Steps Refer Janet to the Tutoring Center for goal setting strategies and organizational tools. You can find additional ways of working with Janet by clicking here. HIGH Commitment – Active pursuit towards an academic goal Skills • Moderately committed to attaining an associate degree • Consistently attends class • Moderate interest in engaging with resources on campus Next Steps The Career Center can help Janet align her degree goals with her career goals. You can find additional ways of working with Janet by clicking here. MODERATE Self-Management – Managing the daily expectations Skills • Calm during important tests and assignments • Persists when frustrated with assignments Next Steps Refer Janet to the Wellness Center on campus to help her manage pressures and succeed. For effective ways of helping Janet manage every day academic stress, click here. HIGH Social Support – Networks to support academic success Skills • Sometimes engages resources on and off campus • Parents show some support for her college attendance Next Steps The Office of Student Life can connect Janet with valuable student groups on campus. For helpful tools and tips about student connections, click here. MODERATE *Course Placement: Recommended academic placement action for student. * Academic Success Score: Weighted composite of student ’s Success. Navigator ™ profile and other academic indicators of student preparedness, such as self or institution reported high school GPA, SAT/ACT, etc. * Retention Success Score: Projected likelihood that student will return for a second year at the institution.
Janet Background: Multiracial/Multi-ethnic Female Major: Education Status: Full-time Current GPA: 3. 25 Age: 19 Parental Education: Unknown (Mother), Unknown (Father) Misc: Janet has significant financial hardship and has received considerable financial aid. She also works 25 hours/week as a waitress. She picks up extra shifts whenever possible. Academic Preparation: • 3. 3 HS grade point average (top 20% of class) • Scored 1200 on SAT • Placed into college level courses Noncognitive profile indicates: She has strong Academic Skills, and Self-Management, and average Commitment and Social Support. Notes: Janet is a capable student with strong organizational skills with promising academic potential. She is confident in her ability to succeed and does not easily get distressed by her school work. Janet is not inclined to seek out help and should be encouraged to make use of campus resources. In particular, Janet has financial constraints and should be encouraged to reach out to her financial aid counselor to discuss the availability of scholarships Also, although Janet is committed to earning a degree, she has expressed interest in transferring soon and may not feel connected to institution. She may benefit from seeing a transfer counselor and engaging in extra-curricular activities.
Student Name: Student ID: Score Date: Julia Fuentes 13 -34588 Sep 15, 2013 Julia’s Background Julia’ Advisor Report COURSE ACCELERATION ACADEMIC SUCCESS SCORE* NOT RECOMMENDED HIGH • Race: Latina • Gender: Female • Parental Education: Some College (Mother), Some College (Father) Academic Skills – Tools to succeed in the classroom RETENTION SUCCESS SCORE* LOW SKILL REPORT Skills • Sometimes uses organizational tools (e. g. , checklist, planner) • Occasionally mismanages her schedule regarding assignments and exams Next Steps Refer Julia to the Tutoring Center to provide goal setting strategies and organizational tools. You can find additional ways of working with Julia by clicking here. MODERATE Commitment – Active pursuit towards an academic goal Skills • Strongly committed to attaining an associate degree • Consistently attends class • Strong interest in engaging with resources on campus Next Steps The Career Center can help Julia align her degree goals with her career goals. You can find additional ways of working with Julia by clicking here. HIGH Self-Management – Managing the daily expectations Skills • Usually anxious during important tests and assignments • Occasionally gives up when frustrated with assignments Next Steps Refer Julia to the Wellness Center on campus to help her manage pressures and succeed. For effective ways of helping Julia manage every day academic stress, click here. LOW Social Support – Networks to support academic success Skills • Rarely engages resources on and off campus • Parents show little support for her college attendance Next Steps The Office of Student Life can connect Julia with valuable student groups on campus. For helpful tools and tips about student connections, click here. LOW *Course Placement: Recommended academic placement action for student. * Academic Success Score: Weighted composite of student ’s Success. Navigator ™ profile and other academic indicators of student preparedness, such as self or institution reported high school GPA, SAT/ACT, etc. * Retention Success Score: Projected likelihood that student will return for a second year at the institution.
Julia Background: Latina Female Major: Liberal Arts, transfer to Psychology Status: Full-time Current GPA: 4. 0 Age: 19 Parental Education: Some College (Mother), Some College (Father) Misc: Is the first in her family to attend college. She has five younger siblings whom she is responsible for while her parents work. She was a top student in high school and a member of the National Honor Society. Academic Preparation: • 3. 9 HS grade point average (top 5% of class) • Scored 27 on ACT (over 1200 on SAT) • Placed into college level courses Noncognitive profile indicates: She has significantly above average Commitment, moderate Academic Skills and Self-Management, and low Social Support Notes: Julia is an academically able student who made the Dean’s List in her first semester despite having taken 17 credits. She is very committed to earning a degree and her long term educational aspirations of earning a Ph. D. Due to her desire to do well academically, she is easily stressed and gets very nervous prior to tests. In addition, because of her heavy course load and family responsibilities she has not made many friends on campus. Julia would benefit from seeing a counselor to help her overcome her anxiety and should be encouraged to engage in extracurricular activities.
Questions? Or ask us later ……. Steve Robbins, Director, Research Innovations, Academic and Workforce Readiness and Success, ETS. srobbins@ets. org Victoria Monaghan, Director of Strategic Initiatives in Higher Education, ETS. vmonaghan@ets. org 24
References Allen, J. , & Robbins, S. (2010). Effects of interest-major congruence, motivation, and academic performance on timely degree attainment. Journal of Counseling Psychology, 57, 23 -35. Allen, J. , Robbins, S. , Casillas, A. , & Oh, I. -S. (2008). Third-year college retention and transfer: effects of academic performance, motivation, and social connectedness. Research in Higher Education, 49(7), 647 -664. American Association of Community Colleges (AACC). (2009). Community college fast facts. Retrieved July 10, 2009 from http: //www 2. aacc. nche. edu/research/index. htm Belfield, C. R. , & Crosta, P. M. (2012). Predicting success in college: The importance of placement tests and High school transcripts. CCRC Working Paper No. 42. New York: Community College Research Center, Teachers College, Columbia University. Berkner, L. , & Choy, S. (2008). Descriptive summary of 2003 -04 beginning postsecondary students: Three years later. Washington, DC: U. S. Department of Education, Institute of Education Sciences. Bragg, D. D. (2001). Community college access, mission and outcomes: Considering intriguing intersections and challenges. Peabody Journal of Education. 76(1), 93 -116. Community College Research Center. (2002 a). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008, from http: //ccrc. tc. columbia. edu/Seminar. asp? uid=8 Community College Research Center. (2002 b). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008, from http: //ccrc. tc. columbia. edu/Seminar. asp? uid=8 Habley, W. , Bloom, J, & Robbins, S. (2012). Increasing persistence: Research-based strategies for college student success. San Francisco: Jossey-Bass 25
References (cont. ) Habley, W. , & Mc. Clanahan, R. (2004). What works in student retention – All survey colleges. Iowa City, IA: ACT, Inc. Harris, S. (1991) Can’t you guys read? Cartoons on academia. New Brunswick, NJ: Rutgers University Press. Herzog, K. , & Miller, M. P. (Eds. ). (1985). Scholarship: More great cartoons from the Kappan. Bloomington, IN: Phi Beta Kappa International Inc. Le, H. , Casillas, A. , Robbins, S. , & Langley, R. (2005). Motivational and skills, social, and self-management predictors of college outcomes: Constructing the Student Readiness Inventory. Educational and Psychological Measurement, 65, 482 -508. Li, K. , & Cordes, C. (2010). Individualizing the Delivery of Math Developmental Education. Talk presented at the 34 th Annual National Association of Developmental Education Conference, Columbus, OH. Lotkowski, V. , Robbins, S. , & Noeth, R. (2004). The role of academic and non-academic factors in improving college retention. Retrieved from ACT, Inc. website: www. act. org/research/policy/index. html. Marke, R. , & Robbins, S. , (2013). Breaking the developmental education logjam: A holistic approach to acceleration. Princeton, NJ: Educational Testing Service. Marke, R. , & Robbins, S. , (2013). A holistic view of course placement fecisions – Avoiding the HS GPA trap. Princeton, NJ: Educational Testing Service. Mc. Cabe, R. H. (2000). No one to waste. Washington, DC: Community College Press. NCES (2004). Remedial education at degree-granting postsecondary institutions in fall 2000 (NCES 2004 -010). NISOD (2012, March). Assessing college-readiness assessments. Webinar delivered at the annual NISOD Conference. Pascarella, E. T. , & Terenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco: Jossey-Bass. 26
References (cont. ) Peterson, C. H. , Casillas, A. , & Robbins, S. B. (2006). The Student Readiness Inventory and the Big Five: Examining social desirability and college academic performance. Personality and Individual Difference, 41, 663 -673. Porchea, S. , Allen, J. , Robbins, S. , & Phelps, R. (in press). Predictors of long-term enrollment and degree outcomes for community college students: Integrating academic, psychosocial, socio-demographic, and situational factors. Journal of Higher Education. President’s Commission on Higher Education. (1947). Organizing higher education. Washington, DC: Author. Robbins, S. B. , Allen, J. Casillas, A. , Akamigbo, A. , Saltonstall, M. , Campbell, R. , Mahoney, E. , & Gore, P. A. (2009). Associations of resource and service utilization, risk level, and college outcomes. Research in Higher Education, 50(1), 101 -118. Robbins, S. , Allen, J. , Casillas, A. , Peterson, C. , & Le, H. (2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, 598 -616. Robbins, S. B. , Lauver, K. , Le, H. , David, D. , Langley, R. , & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261 -288. Robbins, S. , Oh, I. , Le, H. , & Button, C. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: Integrated meta-analytic path analyses. Journal of Applied Psychology, 94, 1163 -1184. Scott-Clayton, J. (2012). Do High-Stakes Placement Exams Predict College Success? Predicting Success in College: The Importance of Placement Tests and High School Transcripts. CCRC Working Paper No. 41. New York: Community College Research Center, Teachers College, Columbia University. Swail, W. S. (2004, January 23). Legislation to improve graduation rates could have the opposite effect. The Chronicle of Higher Education, B 16. 27
Скачать презентацию Why a measure of noncognitive abilities is essential
78a6b68e95e8b896ce4f49b6799f8500.ppt