Making Thinking Visible Assessing students knowledge and reasoning

Making Thinking Visible: Assessing students’ knowledge and reasoning in Plate Tectonics. Janice D. Gobert, Ph. D. The Concord Consortium jgobert@concord. org For more on The Concord Consortium, visit www. concord. org This talk is based on two forthcoming papers: Gobert, (in press), JGE (on SERC website) Gobert, J. (in press). Leveraging technology and cognitive theory on visualization to promote students’ science learning and literacy. To appear in Visualization in Science Education, J. Gilbert (Ed. ), Springer-Verlag Publishers, Dordrecht, The Netherlands. Making Thinking Visible was funded by the National Science Foundation, REC #998060. Any opinions, findings, and conclusions expressed are those of the presenter and do not necessarily reflect the views of the National Science Foundation.

Cognitive Processing of Representations & Model-based learning • Active and constructive process within the human mind. • Information-processing determines memory representation for model and utility for further reasoning. • In Science education, this is called model-based reasoning. • MBR is similar to hypothesis generation in experts (Clement, 1993).

Not mutually exclusive Visualization processes act on setting up mental model and reasoning with it Internal Model, i. e. , Mental model used in reasoning External reps, i. e. topographical maps, etc.

Literature on Student Difficulty in Learning from Models Previously, it was thought that simply adding a diagram or a model to text would facilitate deep learning in science but it was found that… • • increases cognitive load on learners (Sweller, et al, 1990). students lack necessary domain knowledge to guide their search processes through diagrams/models (Lowe, 1989; Head, 1984; Gobert, 1994). Thus, students need scaffolding (i. e. , guidance) in order learn from models: to guide their search processes, to support perceptual cues afforded by models, and to support inference-making from these perceptual cues (Larkin & Simon, 1987).

A few projects…. • Students’ Models in Plate Tectonics • Making Thinking visible • Modeling Across the Curriculum

Project 1: Learning Plate Tectonics It is difficult to learn because: 1) the earth’s internal layers are outside our direct experience, 2) the size scale and the unobserved processes, e. g. , convection, are difficult to understand (Ault, 1984; Gobert & Clement, 1994; 1999), 3) the time scale of geological processes is difficult for people to conceptualize since it surpasses our reference of a human lifetime (Jacobi et al. , 1996), and

Student learning difficulties in PT as reflected by their models…. (1) problems with setting up a correct static model of the layers, i. e, students lack basic knowledge of the spatial structure of the earth. (2) difficulty understanding causal and dynamic information, e. g. , heat as causal in forming convection currents, or currents causing plate movement-inherently difficult concepts. (3) difficulties with the integration of several different types of knowledge including causal and dynamic knowledge into a causal chain in order to build an integrated mental model of the system. Each type of difficulty has different ramifications on model construction and revision processes, as well as the transfer and inference-making afforded on the basis of the model (for more detail, see Gobert, 2000).

Typical models of structure of earth (Gobert, 2000) Type 0= 10. 6%, Type 1=89. 4%

Typical models of volcanic eruption; 4. 25%, 61. 6%, 29. 8%, 4. 25% respectively (Gobert, 2000)

Visualizations CAN permit perceptual affordances and inferences Student 1 Student 2 E: “Why is the magma hot? ” S: “. . . the core is really hot and the mantle is right near the core. ” From Gobert, J. (2000). A typology of models for plate tectonics: Inferential power and barriers to understanding. International Journal of Science Education, 22(9), 937 -977. (on website)

Coding students’ understanding from their models HOW? • See Gobert & Clement, 1999 and Gobert, 2000. • In brief, “expert” model from a think aloud protocol (Ericsson & Simon, 1980)can be used to develop coding scheme. • Curricular materials (text describing PT) used to derive propositions of semantic info in text and used in turn to code students’ diagrams.

Project 2: Scaffolding students’ learning with visualizations ? Making Thinking Visible (mtv. concord. org) - 2 week curriculum: What’s on your plate? -2000+ middle and high school students from CA and MA collaborated on-line about plate tectonic activity in their respective location in a 2 -week curriculum. - driving question: causes for diffs in PT in each coast. - Implemented in WISE (Web-based Science Inquiry Environment; Linn, 1999), based in 15 years of research.

Model-based activities and scaffolding for unit: What’s on your plate? • Draw, in WISE, their models (& explanations) of plate tectonics phenomena. • on-line “field trip” to explore differences between the East and West coast in terms of earthquakes, volcanoes & mountains. • “Visit” the location of earth’s plates and view dynamic models about transform, divergent, collisional, and convergent boundaries. • View dynamic models about causal mechanisms in plate tectonics, i. e. , convection & subduction.

Model-based activities and scaffolding (cont’d) • Critically evaluate their peers’ models (helps them think critically about their own models). • Model revision based on their peers’ critique of their model and what they have learned. • Reflect on how their model was changed and what it now helps explain, e. g. , : Ø Ø “I changed my original model of. . because it did not explain or include. . ” “My model is now more useful for someone to learn from because it now includes…. ” • Transfer what they have learned in the unit to answer intriguing points: Ø Ø Why are there mountains on the East coast when there is no plate boundary there? How will the coast of California look in the future?

Hypotheses & Findings In doing this unit, it was hypothesized that students develop… • • • Content knowledge of the spatial, causal, dynamic, and temporal features underlying plate tectonics. -- Confirmed (Gobert, J. et al, 2004). Epistemological understanding of the nature of scientific models. -- Confirmed (Gobert, J. & Pallant, A. , 2004) Inquiry skills for model-building and visualization. -- under revision for Journal of Learning Sciences.

• Original model- focus on crustal layer, no causal mechanisms for what causes mountain formation. • W. coast partners’ critique requested labels. • Revised model-includes labels, and a cut away view of the interior of the earth which includes convection in the mantle.

• Original model- cross section, no causal mechanisms for what causes mountain formation. • W. coast partners’ critique requested information about direction of plate movement. • Revised model-includes a cross section with plate movement, added the mantle as an interior layer.

For more on the MTV project • See mtv. concord. org • Papers available as well as e-mail me for papers not yet on website.

Project 3. Modeling Across the Curriculum Principal & Co-Principal Investigators Paul Horwitz, Concord Consortium, Principal Investigator Janice Gobert, Concord Consortium, Co-PI & Research Director Bob Tinker, Concord Consortium, Co-PI Uri Wilensky, Northwestern University, Co-PI mac. concord. org; IERI #0115699 www. concord. org http: //ccl. northwestern. edu

Technology provides a promising approach for assessment… • Computers are becoming more ubiquitous, thus problems of access are lessening. • Powerful computation medium--> can run complex simulations beyond textbook diagrams. • WWW allows students access to authentic, real-time data and visualizations. • Learning environments and technology infrastructures are becoming widely available to support students in their science inquiry.

Overview & Technology Content Models • Our models are hypermodels- models that incorporate core science content that students learn through exploration and scaffolded inquiry. • Four content areas: Genetics (Bio. Logica), Newtonian Mechanics (Dynamica), Gas Laws (Connected Chemistry) & Atomic Structure (Chemica). Our Technological Infrastructure • We use Pedagogica, a powerful engine which~ Ø Ø drives all software tools controls all aspects of the learners’ interactions with the software tools provides scaffolding to guide learners in model-based manner provides assessments to be used as formative and summative assessment (for teachers) and performance assessments (log data for researchers).

MAC Enabling Architecture Scripting editor Script MAC activities Student computers Pedagogica™ Data Bio. Logica engine Dynamica engine Connected Chem engine Chemica engine

Instructional Design of Activities and Pedagogy • MAC activities are based on… Ø Model-based learning (Gobert & Buckley, 2000) and Ø progressive model-building (White & Frederiksen, 1990; Raghavan & Glaser, 1995). Ø students’ difficulties in learning with models (Sweller, et al, 1990; Gobert, 1994; Lowe, 1989; Head, 1984). • Thus, scaffolding … Ø Ø guides students’ search processes, supports perceptual cues allowed by models, supports inference-making from these perceptual cues (Larkin & Simon, 1987). activates prior knowledge, supports integration with new knowledge, and supports reflection & reification of this knowledge.

Two balls, different masses

Research & Assessment Log files on interactions with models-Time using interactive models & tries to success Interactions with models (systematicity) What info or help they seek Hypermodels Index of modeling skills-- tracking students’ interactions, strategies, and development within and across domains. Explicit assessments-Formative assessments and Pre/post assessments for teachers’ and students’ use Performance assessments (log files) for researchers’ use Interacts with students’ knowledge & epistemology

Technology & Benefits for Research & Assessment TECHNOLOGY Pedagogica generates logs for every student interaction capturing students’… – Data on duration and sequence – Actions and choices with models – Responses to questions Embedded & Performance Assessments with models & questions… – Generate profile for students – “Pivotal” points in curriculum – Responses to questions BENEFITS “Hard” data -- used for implementation variables: which activities were used, pattern of use (consecutive or intermittent days) at classroom level & student level. Finer-grained data can be used for – Measure of students’ systematicity – Duration as covariate for level of treatment. These data will be used to derive student reports…. ¯ Formative assessments ¯ Summative assessments

Scalability~Schools & Levels of Partnerships First 2 years of project… • • 3 Partner Schools 11 Member Schools added in 2002 – • large urban, suburban, small urban, very mixed SES As of January, we had in addition to 3 Partner schools & 11 Member schools … ˜ 2 Lab Schools ˜ 200+Contributing schools in 18 Countries ~Australia, Germany, Mexico, New Zealand, ˜ 1040 Demo “schools” Singapore, Switzerland, Canada, Scotland, England, Israel, Italy, the Netherlands, Ecuador, Columbia, Guadeloupe, Taiwan and the Philippines

Summary In summary we are leveraging technology … As a bird’s eye view into the “black box”. • Using students’ log files of their interactions with models to develop detailed understanding of students’ model-based learning ---> important for the Learning Sciences. For formative and summative assessment for teachers’ and students’ use • In the future, to provide individualized, technology-scaffolded learning that can be faded as student needs less scaffolding ---> important for Science Education & practice. Lastly, as part of the IERI initiative and of NCLB • To scale our tools and curricula to many schools worldwide--->important for students’ learning, scalability, & sustainability.