A Sample of Best Practices in University Lower

A Sample of Best Practices in University Lower Division Science Education CSE Brownbag 18 October Dan Bernstein, University of Kansas djb@ku. edu

Overview of session • Uses of class time • Capturing out of class time • Alternative course designs • Discussion of implementation • costs and benefits • cultural context • recommendations • Disclaimer

Using class time • Pause for problems / interaction • Mazur is the poster child • Survey of KU clicker users – Attendance and pop quizzes – Check for understanding 3 rd – no plans for how to proceed • Pollock argued that main benefit is collaboration during breaks – “Stop learning and listen to me” • Could be routine feature of classes

Group work tricky in large classes • • Managing groups requires planning One KU professor takes on Budig 120 Tim Shaftel (Business) inserts group days Random assignment to fours w/ warmup Works a problem on the big screen Breaks out for comments and suggestions Roams the room for solutions Consider the video

Tutorials - U of Washington PEG • Lillian Mc. Dermott and colleagues • Crafted generative problem tutorials • Intended to replace lecture/problem sessions [TA doing the problem] • Active engagement in figuring conceptual features of physics • Consider these examples

Collisions in 2 -D

Progressive questions per set up Focus on explanations

More challenging particulars

Magnetism -- with magnets

Progressively complex

Steve Pollock, Physics University of Colorado • Replaced typical discussion for Intro to Physics with Tutorials. • Result: very high learning gains, by national standards. (“The final score matches what our junior physics majors get on this hard exam!”)

Colorado -- BEMA pretest BEMA = “Brief E&M Assessment”, validated research-based survey of Conceptual elements of E&M. Blue data above is F 04 (N=319) Pretest ave 26%

BEMA post -- Comparable to Grad Students F 04 (N=319) 26% -> 59%, S 05 (N=232) 27% -> 59% “Posttest results yield an impressive replication for two semesters High by nat’l standards (typical trad courses, post score = 30 -40% !)”

Pre/post FMCE (Sp 04)

This is their research area

Inquiry laboratories • Related to the tutorials -constructivist model of understanding • Taken to full hands on laboratory • Joe Heppert, Jim Ellis, Jan Robinson • Engage students in process • Embedded, inductive, open-ended

High End -- Studio Physics • • • Hands on discovery in place of lecture Reorganize even very large classes Two hour blocks of time Measure conventional and conceptual skills Taking inquiry lab, constructivist model to the whole experience • Robert Beichner, NC State, one example

Teaching space very different

Conventional exam questions

MC items - Studio v. three lecturers

Studio => comparable problem sets

Failure ratio: Lecture/Studio

FCI gain - Highly replicable

Semester gain by class rank

Outside of Class Time • Some use technology – Center for Academic Transformation • Others based on peers – Community building – Meta cognitive coaching

Carol Twigg invested Pew funds • Re-gifted the money for course redesign • Focused on technology as tool • Emphasized saving money through efficient non-human or lower cost human delivery • Committed to evaluation by learning and completion rates • Increased success and/or difficulty of course • Tracked learning downstream in curriculum • Decreased rates of D, F, and Withdrawal

Carnegie Mellon -- Statistics • Created Stat. Tutor program • Open-ended intelligent tutoring software – Gives feedback on individual paths – Focuses on decision making en route • Aimed for high levels of skill not previously attainable • 22% increase in scores • Critical skill is selecting appropriate statistics to use

High rates of success • Replicated in two course offerings (N>400) • Selection error rates dropped from ~9 to <1 • Two skills not attempted before reached 70% correct

Ohio State University - Statistics • Buffet of options for >3000 students / year • Discovery laboratories, small groups, small lectures, training modules, video reviews • All take common examinations • Learning was greater than comparable daytime lecture based course • Greatly enhance retention of students • Fewer W’s, F’s, and I’s • Modular credit (1 -5), reducing full retakes

Tutorial out performed day class • Large class equaled smaller night class • Fewest failures • Maintained large enrollment

Penn State - Statistics • Reduced lectures from 3 to 1 per week • Replaced with computer lab time – Computer mediated workshops – Extended practice in computerized testing • • • Lecture: Exam pre-post was 50% => 60% Redesign: Exam pre-post was 50% => 68% Selection of correct tool: 78% => 87% DFW rate: 12% => 10% 2200 students per year

University of Iowa - Chemistry • 1300 students / year • Pressure from Engineering and Pharmacy • Fewer lectures, modular content, active participation, computer simulations • Inquiry based laboratories • No difference on common exam items • Am Chem Soc exams: 58 => 65, 52 => 61 • DFW: 24 -30% => 13% • DF: 16% => 9%; W: 9% => 4%

U Mass - General Biology • • • 700 students / semester Lectures: 3 => 2, add review session Inquiry lab already in place Interactive class technology, online quizzes Peer tutoring and supplemental instruction Use Class. Talk network for students Exams: 61% => 73% correct Questions: 23% => 67% required reasoning DF: 37% => 32%

Peer led workshop groups • • • Northwestern University Biology course Based on legendary work of Uri Treisman Peers prepared as facilitators at UT Austin Led group problem solving 2 hr / week Majority students outperformed controls – Steady improvement across exams • Minority students outperformed controls – Improvement noted on last exam – Historic controls show decline over course – 3 rd exam exceeds majority controls

Wendi Born @ CTE 7 November

Supplemental Instruction • Peer led sessions with trained facilitators • Part content and part meta cognition – Study skills – Learning about learning • Designed at UMKC by Deanna Martin – Address high failure rate by minorities in professional programs – Identifies at risk courses, not at risk students • Lani Guinier on the canary

Key Characteristics • All students invited, not targeting weakest • Always with faculty cooperation • Sessions begin right away – Not associated with having problems • Minority students: – SI participants have 2. 02 GPA in courses – Non SI participants have 1. 55 in same courses • DFW rate: – Non SI at 43%, SI at 36%

Huge international following

Nebraska - Intro to Chemistry Non SI had more than double the failure rate 83% passed with SI, 57% without

“Universal” aid, like Studio Physics

[Universal] Design for Success • Presume students can learn • Discount need to sort or differentiate • Maximize overall course performance

Benjamin Bloom promoted mastery • Based on practice and feedback • Divide course into many smaller units • Take examinations and get results • Require taking exam again until high score • IFF 95% correct => study next unit

Fred Keller promoted mastery • IFF 95% correct => study next unit • Course grade is number of units passed • No penalty for repeating and learning • All who pass 12 units => grade of A • Do A work on 10 units => grade of C

Also taught conventional lecture • Mastery Class • 95% contingency • No penalty for learning • Immediate feedback • No lecture required • Lecture class • Same exam questions • Two attempts per test • In class feedback • No contingency

Total amount learned • Nearly twice as many at the high end of learning • Virtually no one failed to learn • Maximized learning for many students

Showed in amount and accuracy • Many more questions answered • Took 12 15 -item tests • Lecture was three tests of 20 items each • Certify more learning • Overall percent correct also higher

• • • No magic -- students studied better They put in more time to their learning There was more work asked for by the course Note that they report doing the reading more Preparation for class is key issue (later also) Guideline in US -- 2 hours outside for every 1 hour in class

Major meta analysis of 100 studies • • • Kulik, & Bangert-Drowns Consistently more learning More time on task Greater retention over time Lower completion rates when used without deadlines and incentives

Placement and Prerequisites • Variation on the same theme • Languages require competence • Tracking skill downstream in the curriculum • Using mastery criteria for foundation courses • Requires some coordination within and between units • Could benefit from tutorials and SI

Marginal gains not clear • Are these effects additive? • Maybe they all help the same students in the same way

Ernest Boyer: The work of the scholar remains incomplete until it is understood and used by others.

Challenges on teaching science • Do we really want success? Grade inflation? • How do we handle the coverage/depth issue? • What about the resources? – Space, funds, faculty time • Students should also be responsible • Are these technologies transferable/robust? • What about bureaucratic hurdles? – Remedial courses/tutorials, Undergrad TAs – Semester based credit and tuition

Your Insights? http: //www. cte. ku. edu

Three functions of grading • • • Certification of learning Motivation for learning Differentiation among learners Each has a legitimate purpose No one system does all well

Variability in conventional course • • Students learn at different rates When course ends, fast learners get best results Very good at identifying fast learners (differentiate) Less good at motivating for more work

Variability in a mastery course • • Everyone learns until material is mastered Reward is for work; subjective probability of success Very good at certification of learning Provides incentive for studying, no penalty if slow

When is mastery the right approach? • Foundation courses -- want knowledge • Programs in which rate of learning is not a criterion for success • Situations in which performance will not be timed • Professions in which high skill is expected • Why tolerate ineffective teaching? • If we don’t care or think it can not be learned by all

How much can a student learn? • • Boundaries are time, effort, and capacity Time and capacity are fixed Your leverage into learning is effort Organize a system that allows extra work Honor that work when it succeeds May lose some identification of capacity Greatly expand the amount of learning

Scholarship Assessed (1997) • All forms of scholarship include: – Clear goals – Adequate preparation – Appropriate methods – Significant results – Reflective critique – Effective presentation Glassick, Huber, & Maeroff

Communities of inquiry on learning • Being very public with teaching in same sense of a center for research • Faculty need another lens to complement student voice, converging measures • Have an external community that values this work • Stresses our existing skills at intellectual inquiry as basis of exploration

Building a community to discuss ideas about teaching • Workshops and seminars for faculty members • Straightforward process of peer interaction • Exchange ideas around three themes • Provide resources for exploration • Written product of thinking and planning

Collaborative Working Seminars • Discussion of shared issues with colleagues • Time for reading and searching • Targets for writing and sharing • Intentional plan is the product