Classes in Bloom Jeff Nevid St John s

Classes in Bloom: Jeff Nevid St. John’s University To contact: jeffnevid@gmail. com © 2013 Jeffrey S. Nevid All Rights Reserved Applying Bloom’s Taxonomy in Teaching Psychology Presented at 2013 STP Best Practices Conference: Research-Based Approaches for Teaching Psychology, Atlanta, GA

Famous Blooms

Allan Bloom

Leo Bloom

Michael Bloom(berg)

Benjamin Bloom (1913 -1999)

Benjamin S. Bloom: Brief Bio Born 1913, died 1999 B. A. , M. A. , Pennsylvania State University, 1935 Ph. D. in Education, University of Chicago, 1942. Staff member of the Board of Examinations at the University of Chicago, 1940 -1943 University examiner, University of Chicago, 1943 -1959 Appointed instructor in the Department of Education at the University of Chicago, 1944 Appointed Charles H. Swift Distinguished Service Professor, 1970. Served as educational adviser to the governments of Israel, India and numerous other nations. • President, American Educational Research Association, 1965 (Preceded by Lee Cronbach and succeeded by Julian Stanley) • • Source: Eisner, E. (2000). Benjamin Bloom: 1913 -1999. In Prospects: The Quarterly Review of Comparative Educaiton, Vol. XXX, No. . 3, Paris, FR: UNESCO International Bureau of Education.

8 Major Concepts • Specifying and Organizing Educational Objectives • Led to development of “Taxonomy of Educational Objectives: Handbook 1, The Cognitive Domain” (Bloom et al. , 1956) • Has helped educators throughout the work organize educational objectives • Hierarchical Model • Objectives organized in a taxonomy predicated on belief that attainment of each subsequent level depends upon ability to perform at preceding levels in the hierarchy • Emphasis on Mastery Learning • How do we help students acquire skills needed to acquire basic information, understand that information, analyze and synthesize that information, and then evaluate that information?

Mastery Learning [Bloom]. . . recognized that was important in education was not that students should be compared, but that they should be helped to achieve the goals of the curriculum they were studying. Goal attainment rather than student comparison was what was important. Eliot Eisner, 2004

Major book titles • 1956. Bloom, B. S. , et al. Taxonomy of educational objectives: The classification of educational goals. Handbook I: The cognitive domain. New York: David Mc. Kay. 1980. • 1958. Problem-solving processes of college students. Chicago: University of Chicago Press. (with L. Broder) • • 1964 b. . New York, David Mc. Kay & Co. Taxonomy of educational obectives: Volume II, The affective domain (with B. Masia and D. Krathwohl) • 1966. International study of achievement in mathematics: A comparison of twelve countries. Vols. I & II. New York, John Wiley & Sons. (T. Husén, Editor; B. Bloom, Associate Editor. ) • 1971. Handbook on formative and summative evaluation of student learning. New York, Mc. Graw- Hill. (with J. T. Hastings, G. F. Madaus and others. ) • 1976. Human characteristics and school learning. New York, Mc. Graw-Hill. • 1980. All our children learning: a primer for parents, teachers, and other educators. New York, Mc. Graw-Hill. • 1981. Evaluation to improve learning. New York, Mc. Graw-Hill. (with G. F. Madaus and J. T. Hastings. ) • 1985. Developing talent in young people. New York, Ballantine. (with L. A. Sosniak et al. )

Bloom’s Taxonomy (Cognitive domain: original and revised) Source: Edtechvision. org (left), http: //newadventuresatwilkes. blogspot. com/2010/05/digitallearners-real-learning. html (right)

Bloom’s Taxonomy: Original vs. Revised

Bloom’s Taxonomy: Original and Revised Source: Anderson & Krathwohl, 2001

• Source: Royal Fireworks Press

• Source: Center for Teaching and Learning, Brigham Young University

360 Action Learning Verbs: Find the Verbs that Match Your Learning Goals

17 Old Version vs. New Version

18 Bloom’s Taxonomy in the Digital Age

19 http: //www. dr 4 ward. com/dr 4 ward/2012/03/blooms-taxonomy-triangle-great-resource-for-teachers-higheredinfographic. html http: //www. dr 4 ward. com/dr 4 ward/2012/03/blooms-taxonom. Soury-triangle-great-resource-for-teachers-highered-

The Challenges What do we want our students to learn? How do we know what they are learning? 3 / 20

1. Remembering (Knowledge) q How do we help students acquire basic knowledge? Objective: Demonstrate the ability to remember (recall or recognize) acquired knowledge q How do we help them recall what they have learned? Instructional techniques: • Textbook reading and use of in-text summaries and quizzes • Lecturing and reviewing • Linking information to existing knowledge structures • Practice quizzing (online, in-text, inclass) • Journaling (writing about concepts) • Study sheets (tied to learning objectives and key concepts) To KNOW Origin: before 900; Middle English knowen, knawen, Old English gecnāwan; cognate with Old High German cnāhan, Old Norse knā to know how, be able to; akin to Latin ( g ) nōvī, Greek gignṓskein. See gnostic

22 Have students acquired the basic information we want them to learn? Examples: ACTION VERBS • RECALL • DEFINE • IDENTIFY • RECALL • DEFINE • IDENTIFY • Define the terms conditioned response. unconditioned response. . . conditioned stimulus? • List six criteria for determining whether behavior is abnormal. • List three ways you can increase the strength of a conditioned response. • TELL • Describe the apparatus Skinner used to study operant conditioning. • LIST • Define the term encoding. • LABEL • NAME • Define the term sensory receptor. • List the levels in Maslow’s hierarchy of needs from the bottom to the top of the hierarchy

2. Understanding (Comprehension) Objective: Demonstrate ability to understand the meaning of the material students have learned q How do we help student grasp the meaning and importance of psychological concepts? q How do we instill the “now I get it” effect? Instructional techniques: • Use of anecdotes, personal stories, case examples, etc. , to bring concepts to life • Use of video clips, with guided questions to illustrate key psychological concepts • Practice in retelling, reciting, or rehearsing acquired knowledge within SQ 3 R framework • Set up pairs of students and have members of each pair explain concepts to each other

24 Do students understand what they have learned? ACTION VERBS • RECITE • PARAPHRASE • REWRITE • DESCRIBE Examples: • In your own words, describe the following terms: conditioned response, conditioned stimulus, . . . • Write a journal entry describing what you have learned about the types of long-term memory. • Recite the steps involved in training an animal to acquire a conditioned response. • In your own words, describe the Big Five model of personality. • Paraphrase Watson’s behaviorist challenge (“Give me twelve healthy infants. . “)

3. Applying (Application) q. How do we help students apply psychological concepts to real-life examples? q What is the cash value of psychological knowledge? Objective: Demonstrate ability to implement or put into use knowledge in a given situation Instructional techniques: • Link psychological knowledge to real-life examples (developing healthy eating and sleeping habits, becoming an authoritative parent, combating prejudice, resisting persuasive sales techniques, etc. ) • Use self-assessment questionnaires to encourage students to apply concepts discussed in class to themselves. • Use personal vignettes, stories, and film vignettes to link concepts to real-life examples

Can students apply or use knowledge they have acquired? ACTION VERBS • APPLY • DEMONSTRATE • PREDICT • SHOW • USE • SOLVE 26 Examples: • Show classical and operant conditioning come into play in daily life. • Apply principles of classical conditioning to explain the development of taste aversions and drug cravings. • Apply Weber’s law to determine the amount of weight needed to be added to detect a difference based on a starting weight of 100 pounds. • Apply Piagetian concepts to examples from daily life. • Apply the representativeness heuristic to explain how students may end up making the wrong decision about which college to attend based on a college tour.

4. Analyzing (Analysis) q How do we help students break down complex systems and conceptual frameworks into their component parts? q. How do we help them recognize the relational connections between parts of a complex system? Objective: Demonstrate ability to determine the essential parts of a system and how they are related to each other and to its overall structure Instructional techniques: • Use visual-spatial diagrams to illustrate relational connections between concepts • Use concept mapping to build knowledge structures linking related concepts. • Apply DSM diagnoses to classifying types of abnormal behavior • Pose mind puzzles that require analysis of complex systems, such as relationships among brain structures (Why is it impossible to tickle yourself? )

Do students understand how the parts fit together? 28 Examples: ACTION VERBS • ANALYZE • EXPLAIN • DIFFERENTIATE • SHOW • DIAGRAM • BREAK DOWN • RELATE • Diagram the processes and stages of memory. • Explain how an action potential is generated. • Differentiate the functions of the sympathetic and parasympathetic nervous systems. • Explain how the three mental structures in Freud’s theory of personality relate to each other? • Analyze how the availability and representativeness heuristics might lead investors to make dumb investment decisions. • Analyze a child’s temper tantrum based on principles of positive and negative reinforcement • Analyze how cocaine affects the brain at the neurotransmitter level.

5. Evaluating (Evaluation) q. How do we help students evaluate the worth or value of psychological concepts and theories? q. How do we help them become critical thinkers? Definition: Demonstrate ability to judge or assess the worth of something in relation to a standard or criterion; to appraise worth or value Instructional techniques: • Use critical thinking exercises that challenge students to weigh evidence and construct rival hypotheses to claims on TV or other media (e. g. , perform a rigged ESP demonstration and ask students to generate alternative explanations) • Hold a You. Tube day in class (debunking claims in online video clips about ESP, trance hypnosis, etc. ) • Weigh the significance of psychological research: Why does it matter in our daily lives? • Use assignments in which students compile and critically review information about topics of interest, using Wikis, video projects, and personal journals or blogs.

Can students evaluate the value or worth of psychological theories and principles? 30 Examples: ACTION VERBS • EVALUATE • Write a critique of Freud’s model of personality, citing strengths and weaknesses. • JUDGE • Evaluate the effectiveness of psychotherapy in the light of evidence. • APPRAISE • Evaluate the value of the DSM system. • COMPARE AND CONTRAST • CRITIQUE • DEBATE • EXPLAIN • Compare and contrast the James-Lange, Cannon-Bard, two-factor model, and dual pathway models of emotions. • Evaluate Piaget’s legacy. • Appraise the value of antidepressants in treating depression in light of evidence.

6. Creating (Synthesis) q How do we help students apply psychological knowledge in creative ways? q. How do we help them take multiple perspectives into account to explain complex phenomena? Objectives • Creating: Demonstrate the ability to bring something unique into being • Synthesis: Demonstrate the ability to combine elements into a single whole or unified structure (opposed to analysis). Instructional techniques: • Set up groups of students to propose ways of applying psychological theories and concepts to address problems such as reducing prejudice, combating substance abuse, living healthier lives, etc. • Explore overriding perspectives in psychology, such as using the biopsychosocial model to explain psychological disorders based on multiple factors and their interactions (e. g. , genetic-environmental interactions)

Can students pull information together to create novel applications or solutions to a problem? 32 Examples: ACTION VERBS • EXPLAIN • CREATE • IMAGINE • PROPOSE • INVENT • MODIFY • REORGANIZE • SYNTHESIZE • FORMULATE • Set up a class exercise in which groups of students compete with each other to propose ways of applying the contact hypothesis to improve inter-group relations on campus. • Assign a writing project in which students write a research proposal based on testing a theoretical position. • Have students propose a new psychological test or measure of a psychological construct (e. g. , anxiety) and how they would validate it. • Were Freud alive today, imagine what he might say about cognitive behavior therapy. • Set up groups of students to formulate hypotheses incorporating risk factors and moderators and mediators to explain negative outcomes such as divorce, obesity, and drug abuse.

Bloom Instructional Module: Research Methods 33 Objective Example Remembering Define the terms independent variable, and operational definition. Understanding Describe the basic features of the case study method. Applying Apply the survey method to study the sleeping habits of students on your campus. Analyzing Show you would use the experimental method to control for positive expectancies of research participants. Evaluating Evaluate the strengths and weaknesses of the case study method, the survey method, the correlational method, and the experimental method. Creating/Synthesizing Formulate a research design to test the hypothesis that “opposites attract” in romantic relationships.

Bloom Instructional Module: Classical Conditioning 34 Objective Example Remembering Define the terms conditioned stimulus and conditioned response Understanding Describe classical conditioning in your own words Applying Give examples of classical conditioning in daily life Analyzing Explain what is likely to happen to the strength of the conditioned response if (a) you lengthen the interval between CS and US, or (b) you reverse the order of CS and US. Evaluating Explain why it is important to study classical conditioning. Creating/Synthesizing Propose an alternative explanation of classical conditioning based on a cognitivist perspective. Propose a research study to test ways of strengthening or weakening conditioned responses.

Using Psychology to Teach Psychology: The Four E’s of Effective Learning 1. Engage student interest 2. Encode important information 3. Elaborate meaning 4. Evaluate progress 3 / 35

What’s Wrong with This Picture? 3 / 36

Why We Became Teachers

It certainly wasn’t for the money. . . Source: college_prof_football_coach_salaries. gif Graph comparing college professors' and college football coaches' average. . . joeydevilla. com

AND it certainly wasn’t for this. . .

Which can lead to this. . .

Or maybe this (well, maybe not). But grading papers does feel like a prison at times.

There’s got to be an easier way to make course evaluation yield more gain and less pain. • Source: Black Hills State University

Why measure learning outcomes? • To determine what our students are learning. Are our students acquiring the knowledge and skills we expect them to learn? • To improve learning. Knowledge from course assessments can inform how we teach and how we help our students acquire knowledge we expect them to acquire. • To meet accreditation standards. Course assessments provide data we can provide to accrediting bodies, such as state education departments, regional accrediting agencies, and professional organizations such as APA. • We live in an age of accountability in which regular assessment of learning outcomes is an important tool in the accreditation process.

Background Reading

Integrated Course Assessment Guided by three key principles. . . • Make it seamless • Make it sensible • Make it work (without requiring a lot of extra work)

APA Guidelines for the Undergraduate Psychology Major

Undergraduate Psychology Learning Goals and Outcomes: Approved by APA Council, August 2006 • The Guidelines for the Undergraduate Psychology Major provides details for 10 suggested goals and related learning outcomes for the psychology major, grouped into two major categories: • I. Knowledge, Skills, and Values Consistent with the Science and Application of Psychology • II. Knowledge, Skills, and Values Consistent With Liberal Education That Are Further Developed in Psychology • Each of these categories contains five goals:

APA Learning Goals 1 -5 • I. Knowledge, Skills, and Values Consistent with the Science and Application of Psychology • Goal 1: Knowledge Base of Psychology • Students will demonstrate familiarity with the major concepts, theoretical perspectives, • • empirical findings, and historical trends in psychology. Goal 2: Research Methods in Psychology Students will understand apply basic research methods in psychology, including research design, data analysis, and interpretation. Goal 3: Critical Thinking Skills in Psychology Students will respect and use critical and creative thinking, skeptical inquiry, and, when possible, the scientific approach to solve problems related to behavior and mental processes. Goal 4: Application of Psychology Students will understand apply psychological principles to personal, social, and organizational issues. Goal 5: Values in Psychology

APA Learning Goals 6 -10 • II. Knowledge, Skills, and Values Consistent With Liberal Education That Are Further Developed in Psychology • Goal 6: Information and Technological Literacy • Students will demonstrate information competence and the ability to use computers and • • other technology for many purposes. Goal 7: Communication Skills Students will be able to communicate effectively in a variety of formats. Goal 8: Sociocultural and International Awareness Students will recognize, understand, and respect the complexity of sociocultural and international diversity. Goal 9: Personal Development Students will develop insight into their own and others’ behavior and mental processes and apply effective strategies for self‐management and self‐improvement. Goal 10: Career Planning and Development

The IDEA MODEL. . . A Simpler Model Based on Four Key Skills: Identify. . . Key figures in psychology and parts of the body Describe or Define. . Key terms and concepts Evaluate or Explain. . . Underlying processes and mechanisms Apply. . . Concepts to examples

What’s the NEW IDEA in Course Assessment?

The IDEA Model of Course Assessment™ I D E A Bloom’s Taxonomy

Course Evaluation Matrix Knowledge Base of Psychology: Memory Sub. Goals 1. Demonstrate knowledge of processes and stages of memory Learning Objectives Describe the basic processes and stages of memory. Describe the different types of long-term memory. Explain the roles of the semantic network model and levelsof-processing theory in memory. Explain the difference between maintenance rehearsal and elaborative rehearsal. Apply constructionist theory to explain memory distortions. Evaluate the reliability of eyewitness testimony. Explain why the concept of recovered memory is controversial. 2. Demonstrate knowledge of theories of forgetting, methods of measuring memory, and types of amnesia Describe the major theories and factors in forgetting. Identify different methods of measuring memory. Describe the features of two major types of amnesia. Keyed Test Items [write test questions to measure the specific learning objectives]

Examples of Coded Test Items: Foundations of Modern Psychology

Need to Ground Pedagogy in Psychological Science l. Traditionally, pedagogical features in college textbooks development was driven by marketing concerns, not scientific research l. To date, little research has been conducted to evaluate effectiveness of pedagogical techniques used in class and psychology textbooks l. Textbook developers and instructor need to draw upon knowledge base in cognitive psychology

57 Sample of Prior Research Project Design Key Finding(s) Textbook modularization study (comparing traditional text with modular format) A randomized, counterbalanced design in which students read two text passages, one presented in a modularized format and the other presented in a narrative format. Students who preferred the modular approach performed significantly better on the accompanying quiz when they had read the material in their preferred format. A randomized, counterbalanced design in which students read two text passages, one with concept signaling and the other without. Concept signaling improved student performance on test items measuring knowledge of key concepts. No differences were found for non-signaled (surrounding) material. Source: Nevid & Carmony, 2002, Teaching of Psychology Concept Signaling Study (highlighting key concepts in margins of text) Source: Nevid & Lampmann, 2003, Teaching of Psychology

58 Project Design Key Finding(s) Mastery Quizzing (pre-post quizzes in class tied to specific concepts discussed during class) Analyzed student performance on course exams, disaggregated by signaled concepts (concepts tested in mastery quizzes), related concepts (other concepts discussed on mastery quiz days), and non-signaled concepts (control concepts discussed on other days). Students showed significant improvement in knowledge of mastery quiz content as assessed by pre-post lecture comparisons. Source: Nevid & Mahon, 2009, Teaching of Psychology Mastery quizzing cues students to attend to important concepts discussed in class, and provides incentives for coming to class, coming on time, and paying attention.

Current Study: Item Analysis of IDEA Model STUDY DESIGN: • Participants: Two introductory psychology courses at St. John’s University, comprising 144 students, 62 males and 82 female, mostly freshmen • Method: Item analysis based on student performance on three non-cumulative multiple choice exams, with test items coded by type of acquired skill (Identify, Define or Describe, Apply, and Evaluate). Items drawn from textbook test-item file. • Outcome measure: Student performance on each item type aggregated across the three exams. • Interrater Reliability: 90% concordance in blind interrater agreement study. • Item Analysis: Computation of item type difficulty level and discriminability • Source: Nevid, J. S. , & Mc. Clelland, N. (in press). Using action verbs as learning outcomes: applying bloom’s taxonomy in measuring instructional objectives in introductory psychology. Journal of Education and Training Studies.

Results: Internal Consistency • Kuder-Richardson 20 coefficients: • Range from. 75 to. 86 for item types (I, D, E, and A type questions), demonstrating adequate internal consistency for all item types. Also, intercorrelations among item types were high (rs =. 69 to. 83), which is suggestive of an underlying factor of general cognitive ability.

Results: Item Difficulty • Consistent with the Bloom taxonomy, “E” questions (Evaluate or Explain) proved to be more difficult than “I “(Identify), “D” (Define or Describe) , and “A“ (Apply) questions. • Also consistent with Bloom’s taxonomy, “A” questions were more difficult than “D” questions. • “I” questions were more difficult than “D” questions. • Average difficulty levels for the four item types fell within a moderate range of difficulty (range =. 53 to. 64).

Item Difficulty

Results: Item Discrimination Index • The higher the discrimination index, the better the items are in discriminating between poorer and better students >. 20 for marginally good discriminability >. 30 for reasonably good discriminability • “Evaluate” and “explain” questions were not only the most difficult items, but were also the best discriminating items: • Highest item discrimination index overall (. 33) • Highest percentage of items (61%) reaching. 30 criterion of reasonably good discrimination

Item Discrimination Index

Item Difficulty and Item Discrimination by Item Type in IDEA Model

Summary • The action verbs used comprised skills relating to identifying, defining or describing, evaluating or explaining, and applying knowledge of basic concepts in psychology. • Exam performance demonstrated higher level cognitive skills represented by action verbs evaluating and explaining were the most difficult for students to acquire, but also were the best discriminating items.

• Take-Away Message: • The IDEA model allows instructors to integrate APA learning goals with course examinations by coding test items that measure acquired skills consistent with learning objectives. • The IDEA Model provides a heuristic framework for evaluating areas of relative strength and weakness in acquired skills in college coursework organized within a hierarchical model of learning goals (Bloom’s taxonomy). • By identifying skills deficiencies, instructors can tailor teaching strategies to strengthen these types of learning outcomes.

Before we end. . . Four Key Steps for Mapping Active Learning Verbs to Learning Outcomes

Step 1. . . Determine the skill set you would like your students to acquire in a particular course • Specify learning objectives for each text chapter or instructional unit

Step 2. . . Select learning verbs to measure skills you want to assess • Use action verbs to frame learning objectives

Step 3 . . . Key course exams to learning objectives and action verbs • • Write items for each type of action verb Use textbook test-item file (you may need to retrofit key items to particular learning verbs)

Step 4. . . Measure student performance on exams to evaluate student competencies • • • Use machine-scored (Scantron) forms to break down performance on each acquired skill (e. g. , Identify. . . Define or Describe . . . Evaluate or Explain . . Apply) Basic Scantron system can be set up to provide four subtest scores (I, D, E, A), two on front and two on back of standard Scantron form Advanced Scantron system connected to computer can generate any combination of subtest scores

Sample References from the St. John’s University Pedagogy Research Program • Nevid, J. S. , & Carmony, T. M. (2002). Traditional versus modular format in presenting textual material in introductory psychology. Teaching of Psychology, 29, 237 – 238. • Nevid, J. S. (2003, September). Helping students get the point: Concept signaling as a pedagogical aid. Paper presented at the conference, Taking Off: Best Practices in Teaching Introductory Psychology, Atlanta, GA. • Nevid, J. S. , & Lampmann, J. L. (2003). Effects on content acquisition of signaling key concepts in text material Teaching of Psychology, 30, 227 -229 • Nevid, J. S. (2004, January). Graphing psychology: The effective use of graphs and figures in teaching introductory psychology. Invited address at the presented at the 26 th Annual National Institute on the Teaching of Psychology, St. Petersburg, FL. • Nevid, J. S. (2004, February). Evidence-based pedagogy: Using research to find new ways to help students learn. Invited closing address presented at the 11 th Midwest Institute for Students and Teachers of Psychology (MISTOP), Glen Ellyn, IL. 3 / 73

Sample References (contd. ) • Nevid, J. S. , & Forlenza, N. (2005). Graphing psychology: An analysis of the most commonly used graphs in introductory textbooks. Teaching of Psychology, 32, 253 -256. • Nevid, J. S. (2006, February). In pursuit of the “perfect lecture. ” APS Observer, Teaching Tips, Vol. 19(2). • Nevid, J. S. , & Blitzer, J. R. (2006, August). Educational benefits of mastery quizzes as signaling devices. Poster presented at the annual meeting of the American Psychological Association, New Orleans, LA. • Nevid, J. S. , & Mahon, K. (2009). Mastery quizzing as a signaling device to cue attention to lecture material. Teaching of Psychology, 36, 29 -32. • Nevid, J. S. (2009/2010, Winter). Reaching and teaching millennial students. Psychology Teacher Network, 19(4) pp. 1, 3, 4. • Nevid, J. S. (2011). Teaching the millennials. APS Observer, Teaching Tips, 24(5), in press. • Nevid, J. S. , Mc. Clelland, N. , & Pastava, A. (2011, August). Using action verbs as learning outcomes in introductory psychology. Poster presented at the meeting of the American Psychological Association, Washington, DC. • Nevid, J. S. , & Mc. Clelland, N. (2011, October). A new IDEA in course assessment: Linking learning goals to skills assessed in introductory psychology. Paper presented at the Best Practices in Teaching Introductory Psychology Conference, Atlanta, GA. • Nevid, J. S. , Pastva, A. , & Mc. Clelland, N. (2012). Writing-to-learn assignments in introductory psychology: Is there a learning benefit? Teaching of Psychology, 39, 272 -275. • Nevid, J. S. , & Mc. Clelland, N. (in press). Using action verbs as learning outcomes: applying bloom’s taxonomy in measuring instructional objectives in introductory psychology. Journal of Education and Training Studies.

Thank you! Please share with me your ideas about teaching psychology: jeffnevid@gmail. com 3 / 75