Learning How Pupils Make Sense of Algebra Word

Learning How Pupils Make Sense of Algebra Word Problems: Tool Development • Robert E. Floden & Algebra Teaching Study (ATS) Team • Michigan State University & UC-Berkeley

• This material is based upon work supported by the National Science Foundation under Collaborative Grant Nos. NSF collaborative grant DRL-0909815 & DRL 090985. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation Support from NSF

• MSU: Robert Floden, Rachel Ayieko, Sihua (Adrienne) Hu, Jerilynn Lepak Jamie Wernet • Berkeley: Alan H. Schoenfeld, Evie Baldinger, Danielle Champney, Fady El Chidiac, Denny Gillingham, Hee. Jeong Kim, Jerilynn Lepak, Nicole Louie, Sarah Nix, Daniel Reinholz, Alyse Schneider, Mallika Scott, Kim Seashore, Niral Shah, and alumni ATS Team

• • • Take situations described in words Build mental models Represent using algebra symbolism Operate on symbols Check results Classroom learning to make sense of algebra word problems

• Link to CCSSM: • Standards for Mathematical Practice • Domains • Expressions & Equations • Functions • Mathematics researchers & educators • Recommendations for instructional practices • Little empirical work, especially at scale Why important?

Research on the Linkage ? Classroom Practices Student Learning

• Complexities of teaching and learning • Conceptualize key dimensions of student learning • Measuring learning along these dimensions • Conceptualizing key dimensions of classroom practice • Recording those dimensions • Keeping mathematics prominent Challenges for research

• Robust algebraic understanding • • • Interpret text; understand context Identify salient quantities & relationships Algebraic representations of relationships Execute calculations and procedures Explain and justify Conceptualizing focus for student learning

• Draw inferences about competence from performance on complex tasks • MARS tasks (built from Balanced Assessment) • MARS scoring is global, with no detail about dimensions for robust understanding • Modest attention to asking for explanations Measuring student learning

Illustrative excerpt

• Selection of tasks, largely from MARS • Cognitive interviews to compare yield from written response to yield from interview • Rubrics generate • Overall score • Score for each dimension of robust understanding (robustness criteria or RC) • Multiple 45 -minute forms Iterative development of scoring rubric

• Validity studies • Agreement with holistic scoring • Agreement of correctness score with MARS • Comparisons with MARS population • Reliability studies • Compare to “expert” • Random assignment to scorers • Almost all inter-rater reliability >. 75 • Will be higher for classroom means Reliability and validity

Sample Rubric Hexagons – Part 2 Task (points) Associated RCs Find 10 th term in pattern – perimeter of a row of 10 hexagons (1) - Possible Strategies Associated RCs 3 a, 4 b Generates pairs of numbers (e. g. shows recursive thinking not in a table) RC 5 3 a, 4 a Generates representation – diagram of 10 tiles Explain how you know your answer is correct (1) Generates representation – table, graph, or equation 4 a Articulates (in words) relationship between size number and number of people 2 b Draws on representation 3 b Draws on context (arrangement of hexagons) 4 b

• Robustness Criteria As A Framework To Capture Students’ Algebraic Understanding Through Contextual Algebraic Tasks • Sihua Hu, Rachel Ayieko, Jerilynn Lepak, Jamie Wernet Scoring rubrics poster session at PME-NA

Developing a classroom observation scheme

• CLASS (UVA) • IQA (Pittsburgh) • Plato (Stanford) • MQI (Michigan) Review existing schemes

• IQA comes closest: • Cognitive demand • Accountable talk • One thing missing is attention to classroom dimensions that capture tight plausible link to developing robust understanding of algebra What is missing?

• Few publicly available recordings of entire lessons (about to change with Gates MET) • ATS data collection Michigan and California • Evolving agreement on promising dimensions • Teaching for Robust Understanding of Mathematics (TRU MATH). Begin development by watching algebra classes

• Mathematical Focus, Coherence & Accuracy • Cognitive Demand • Access • Agency: Authority & Accountability • Uses of assessment Broad dimensions

• Robust algebraic understanding • • • Interpret text; understand context Identify salient quantities & relationships Algebraic representations of relationships Execute calculations and procedures Explain and justify Specific to robust understanding

RC 3 Rubrics from scheme Level 1 Level 2 Level 3 Generating Purposefully Level 2 and representations generated with attention to why • 3 a and 3 b Generated by way explicit attention to the representation RC 3 a of relationships of practice between the relationship is a good choice for quantities between variables the given situation Representations are Global features of Interpreting and Level 2 and interpreted locally representations are making connections among or in part. No explicated to RC 3 b connections multiple connections highlight the between representations are between multiple covariation between representations explored representations. quantities

• Follow teacher, including during group work • Divide lesson into episodes, with new episode when: • Move to different mode of instruction (e. g. , whole class to group work) • Move to new mathematical topic • When five minutes have gone by since beginning of episode • Code five general dimensions for every episode • Code for robust understanding when activity warrants When and what to record

• Teaching for Robust Understanding • Hee-jeong Kim, Kimberly Seashore, Daniel Reinholz Scheme poster session at PME-NA

Illustrative preliminary analyses

• Data are episodes, length of time in episode, rubric score for episode for each dimension • Student learning related to amount of time spent in activity • Many possible ways to aggregate, e. g. , • Average rubric score, weighted by time • Time spent at particular level of rubric Aggregating Dimension Scores

• Data are episodes, length of time in episode, whether RC-activity happens during episode, rubric score for RC when it happens • Student learning related to amount of time spent in activity • Many possible ways to aggregate, e. g. , • Total time in episodes with RC activity • Time spent at particular level of RC rubric Aggregating RCs Scores

Weighted Average for Dimensions 1 -5 3. 00 2. 80 2. 60 2. 40 2. 20 IM 2. 00 CG A 1. 80 AAA 1. 60 UA 1. 40 1. 20 1. 00 Tch 1 Obs A Tch 1 Obs B Tch 1 Obs C Thc 2 Obs A Thc 2 Obs B Tch 2 Obs C Tch 3 Obs A Tch 3 Obs B Tch 3 Obs C Tch 4 Obs A Tch 4 Obs B 4 Teachers-3 Lessons Tch 4 Obs C

Between-teacher difference

Time at Level 3 1. 20 Percent of Coded Time 1. 00 0. 80 IM 0. 60 CG A 0. 40 AAA UA 0. 20 0. 00 Tch 1 Obs A Tch 1 Obs B Tch 1 Obs C Thc 2 Obs A Thc 2 Obs B Tch 2 Obs C Tch 3 Obs A Tch 3 Obs B Tch 3 Obs C Tch 4 Obs A Tch 4 Obs B 4 Teachers-3 Lessons Tch 4 Obs C

Between-teacher difference

Average gain in score (%) Student Gains in Target Classrooms

• Observation scheme • Can capture differences among teachers in general dimensions • Still working on criteria for recording events tied to robust understanding • Student assessment • Reliable scoring • Challenges with student motivation Initial thoughts

• Transitioning From Executing Procedures To Robust Understanding Of Algebra • Rachel Ayieko and Jamie Wernet, with Sihua Hu and Daniel Reinholz Paper session at PME-NA

• Working version of rubric for scoring student work, with all scoring completed for last year’s data • Close to working version of observation scoring system, with initial scoring for 3 observations in each of 4 classes • Exploring approaches to analysis • Many conference presentations; one publication • Seeking funding to validate on Gates MET videos • Exploring use for capturing classroom processes in Shell Center formative assessment lessons • Projected paper making comparisons to other observation schemes Where we are

• ATS Web site: http: //ats. berkeley. edu • NSF REESE projects: https: //arc. uchicago. edu For more information