A Standard Survey for Undergraduate Non-Science Majors The

A Standard Survey for Undergraduate Non-Science Majors: The Astronomy Diagnostic Test Beth Hufnagel University of Maryland, USA hufnagel@astro. umd. edu 7 August 1999 AAPT Summer Meeting San Antonio, Texas 1

• Faculty E. F. (Joe) Redish David Hammer John Layman (emeritus) • Grad Students Aziza Baccouche Lei Bao Hadley Lawler Rebecca Lippmann Mel Sabella • Postdocs § Richard Steinberg Michael Wittmann Andy Elby* Apriel Hodari* Beth Hufnagel* (§ NAE Spencer Fellow *PFSMETE Fellow) 2

Overview Astronomy Education Research Students knowledge and beliefs about … The Universe Knowing “Conceptions” “Epistemology” Themselves “Affect” Development and assessment of curricula and teaching techniques 3

Measuring Concept Learning: Why A Standardized Survey? • Understand student preparedness. • Guide selection of curriculum/teaching approach. • Measure the effectiveness of different teaching approaches on learning. • Document the effectiveness of teaching. 4

Collaboration for Astronomy Education Research (CAER) Montana State Univ. of Maryland Jeff Adams Christine Brick Tim Slater Grace Deming Beth Hufnagel Univ. of Nebraska Univ. of New Mexico Rebecca Lindell Adrian Michael Zeilik 5

What is the Astronomy Diagnostic Test? • 33 multiple-choice questions – 21 concept/knowledge questions – 12 student background questions • Validated and Reliable – A wrong answer means the student doesn’t understand. – A right answer usually means the student understands. • Diverse Comparison Database – 22 classes ranging from 6 -201 students – 17 institutions of 5 types 6

Pre-Course Results Spring 1999 Sample ADT Question 8. Where does the Sun’s energy come from? A. The combining of light elements into heavier elements. B. The breaking apart of heavy elements into lighter ones. C. The glow from molten rocks. D. Heat left over from the Big Bang. Did not attempt to answer. Total No. 420 % 27 870 56 142 9 122 8 3 0 1557 100

1998/1999 Development of ADT version 2. 0 • • The choice of ADT topics was based on two prior diagnostics by Phil Sadler and Mike Zeilik. The ADT version 1. 0 was re-written using standard psychometric rules. 30 student written responses to open-ended questions. 2500+ students in 26 institutions of 6 types – • Statistics, e. g. , Spring 1999 Discrimination range 0. 32– 0. 67 (out of – 1. 00 to +1. 00). Validation based on ~60 interviews, – – open-ended in Fall 1998, with distractors in Spring 1999. 8

Research Questions Do the results depend on … • • • the type of institution community college, liberal arts, state university, or technical? Yes the class size? No gender? Yes – Why? 9

Results from Spring 1999 Pre-Course Scores by Gender Female Male N 825 683 Mean 28% 38% Std. Error 0. 4% 0. 6% Gender matters. 10

Summary • The ADT is now available in PDF format from http: //solar. physics. montana. edu/aae/adt/. • Encourage conceptual understanding as a goal for astronomy courses. • More quantitative measurement of the effectiveness of curricula/learning tools. • CAER will develop a web-based, ADT database with independent concept clusters. 11

Favorite Quote from Student Interview “I seem to know, or think I know, a lot of things. I just don’t know … the insides of them. ” - Debbie (Undergraduate astronomy student during the last week of the course. ) 12

Thanks to these generous participants: 13

• • • • • Phil Appleton, Iowa State University Chris Depree, Agnes Scott College Van Dixon, Vista Community College Andrea Dobson, Whitman College Jack Dostal, Hawkeye Community College Doug Hamilton, University of Maryland at College Park Mary Kay Hemenway, University of Texas at Austin Thomas J. Kassebaum, Denison University Janet Landato, W. R. Harper College Bernie Mc. Namara, New Mexico State University Heather Morrison, Case Western Reserve University Jeff Saul, North Carolina State University Steve Shawl, University of Kansas Harry Shipman, University of Delaware Patrick Shopbell, Prince George’s Community College Greg Stacy, Southern University 14 Bill Warren and Rolfe Chandler, Lord Fairfax Community College

Results from Spring 1999 Pre-Course Scores by Class Size The average pre-course score does not depend on class size. 15

Results Spring 1999 Pre-course Scores by Institution/Gender #Classes #Students Women% Men% State Universities 11 Liberal Arts 5 Community Coll. 4 Technical 1 Women’s 1 635/499 110/124 52/43 6/17 22/0 28(1. 0) 26(1. 1) 28(1. 7) 29(14) 30(8) 37(1. 5) 38(5. 1) 39(2. 6) 59(21) n/a (standard error in parentheses) In all 22 classes, the average pre-course score for men is higher than the average score for women. 16

Implications for Instruction: Explicitly Evoke Useful Ideas Students come to an introductory astronomy class with a great deal of knowledge. A professor can help her students use their existing correct and relevant ideas to construct theories closer to those held by astronomers. For example, all of the students knew that the Sun is hot, and that fluids (like coffee) will cool if no heat source is present. Use a demonstration to encourage them to apply this knowledge to stars so that they will conclude stars need an internal heat source. 17

San Antonio AAPT Summer Meeting 18