Chapter 10 Measuring Complex Achievement Essay Questions BY

Chapter 10 Measuring Complex Achievement: Essay Questions BY AIZADA IMANDOS AND BRENDA TEMPLE

We will be covering: What is an essay question? Types and uses of essay questions Examples of each type of essay question Advantages Disadvantages Constructing essay questions Types of rubrics Examples of each type of rubric Bluffing Constructing the rubric

What is an essay question? The measurement of complex achievement The higher order thinking skills being employed by the student A performance-based assessment The student must demonstrate their ability to communicate their complete understanding of a topic as well as how to apply, analyze, synthesize, and evaluate the information. The emphasis is on doing, not merely knowing: on process as well as product. The freedom of response provided by essay questions is not an all-or-nothing affair, but rather a matter of degree to which the answer is correct.

Types and Uses of Essay Questions There are two types of essay questions. Restricted-Response Essay Questions Extended-Response Essay Questions

Objective Interpretive Exercises Not An Type of Essay Question Objective Interpretive exercises differ from essay questions due to the fact they are used to Select the best answer. These would include: Multiple Choice True-False Matching Fill in the Blank Key-Type

Restricted-Response Essay Questions An essay question with limitations to both the content and response but provide the measure of student ability to: Explain cause-and-effect relationships Describe applications of principles Present relevant arguments Formulate probable hypotheses State necessary assumptions Describe the limitations of data Explain methods and procedures

Example of Restricted-Response Essay Questions In a brief paragraph, explain why the barometer is one of the most useful instruments forecasting the weather. Describe two ways in which you could handle an unfair situation between classmates. Explain how you would handle being in the same situation. Compare and contrast the northern states and southern states during the period before the civil war.

Extended-Response Essay Questions Grants the student the freedom to: choose what information they believe is important demonstrate their ability to analyze problems organize their ideas describe in their own words develop their own coherent argument Integrate learnings in different areas Construct creative stories Persuade a reader Write an essay on a given topic

Examples of Extended-Response Essay Questions Describe the influence of Mendel’s laws of heredity on the development of biology as a science. You and a friend have found an lost dog. Describe what procedures you might follow to help find the owners. Imagine that you have repeated the circumstances of Robinson Crusoe. Tell the story of how you got home.

Advantages to Essay Questions Measure complex learning outcomes that cannot be otherwise measured Emphasis on integration and application of thinking and problem solving skills (Extended-Response) Offer a direct evaluation of writing skills Ease of construction and measurement of specific data good for the busy teacher Contribution to student learning through the preparation of a response being used as a learning exercise. Excellent for comprehension, application, and analysis level questions.

Disadvantages to Essay Questions Unreliability of the scoring Time consuming to score Limitations to sampling of content Number of questions Limitations in response in Restricted-Response Bluffing – Discussed later

Constructing Essay Questions Steps to create essay questions: Restrict the use of essay questions to those learning outcomes that cannot be measured satisfactorily by objective items. Construct questions that will call forth the skills specified in the learning standards. Phrase the question so that the student’s task is clearly indicated. Indicate an approximate time limit for each question. Avoid the use of optional questions.

Types of Rubrics A rubric is a means for communicating expectations for an assignment, providing focused feedback on works in progress, and grading final products. Scoring rubrics for restricted-response essay questions Analytic scoring rubrics for extended-response essay questions Holistic scoring for extended-response essays

Rubrics for Restricted-Response Made more quickly and easily if the teacher makes a list of acceptable answers and give the student a particular point value for each item they provide that is on the list. Lists of expectations for when the student is asked to write a paragraph The list of expectations should also provide information to the degree of explanation that would warrant partial credit.

Example: Rubric for Restricted-Response – Scenario Standard: RL. 5. 2. : Determine a theme of a story, drama, or poem from details in the text, including how characters in a story or drama respond to challenges or how the speaker in a poem reflects upon a topic; summarize the text. Instructional Goal: Understands theme Learning Outcome: distinguishes theme of the story Question: We have been reading “The Boy in the Stripped Pajamas”. In a paragraph explain theme of the story giving five examples of how the characters respond to the challenges they face.

Example: Rubric for Restricted-Response – Rubric Possible Themes Kindness/compassion Child-like innocence Humility Honesty Equality Character Responses Bruno is adventurous despite being warned not to – he explores without permission Bruno was lonely – He went out without his parents’ permission and found a friend Shmuel cannot find his father – Bruno enters the camp to help find Shmuel’f father. (The list would continue but for the sake of time, we will stop here. )

Analytic Rubrics for Extended-Response Allows the teacher to focus on one characteristic of the response at a time. Separating mechanics characteristics from quality material helps provide better feedback for the student to better understand their strengths and weaknesses of their response. May consist of two or more categories – more are useful for fine distinctions in the writing Good for expository essays and descriptive summaries

Example: Analytic Rubric for Extended-Response – Scenario Standard: RL. 5. 3: Compare and contrast two or more characters, settings, or events in a story or drama, drawing on specific details in the text (e. g. , how characters interact). Instructional Goal: Judges the value written works by means of internal criteria Learning Outcome: Compare two or more settings in a given text. Question: In class, we have been talking about The Wizard of Oz. There are four major settings in this story. Describe each and the effect it had on the characters.

Example: Analytic Rubric for Extended-Response – Rubric Unacceptable 0 Poor 1 Setting: Does not describe the settings at all. Describes one setting. Effect on Character Does not describe the effect on the characters at all. Grammar Incomplete sentences with several grammar mistakes Acceptable 2 Outstanding 4 Describes all two settings. three settings with detail. Provides only Provides two one effects on on the one or two characters. Several grammatical mistakes but complete sentences Good 3 Only a few minor grammar mistakes Provides several effects on one character. Provides several effects on all characters involved. Only one or No two grammatical errors! mistakes

Holistic Rubrics for Extended Response Holistic - characterized by understanding of the parts of something as directly related and clearly expressed only by reference to the whole. Deals with one overall score with the response as a whole in mind Usually quick and easy to create and can measure multiple responses more rapidly than Analytic scoring rubrics. Do NOT provide students with feedback on specific aspects of their response. This can be corrected with marginal notes.

Example: Holistic Rubric for Extended-Response – Scenario Standard: RL. 5. 4: Determine the meaning of words and phrases as they are used in a text, including figurative language such as metaphors and similes. Instructional Goals: interpret the meaning of words using context clues Learning Outcome: Explains the meaning of words in the text Directions: For each underlined word in the sentences below, explain the meaning of the word. Questions: It was an idyllic day; sunny, warm, and perfect for a walk. Marty is gregarious, not like his brother who is shy and quiet. My greatest expedition ever was my trip to Africa.

Example: Holistic Rubric for Extended-Response – Rubric Score Description 1 The response is incorrect. 2 The response uses the word to define the word. 3 The response provides a little clear information. 4 The response provides a clear understanding of the word’s meaning.

Bluffing and the Essay Question A special kind of problem Bluffing on an essay – student trying to get higher score by means of clever use of “tricks-of-the-trade” Writing something for every question, even if it is just a restatement of the question. Stressing the importance of the topic covered by the questions, especially when short on facts. Agreeing with the teacher’s views whenever it seems appropriate Being a name-dropper – The reader assumes the student knows about what they mention Writing on a related topic and fitting it to the question. Writing in general terms that can fit many situations Let’s face it, most of us have done this at some point. Am I wrong?

Constructing the Rubric Prepare an outline of the expected answer in advance. Use the scoring rubric that is most appropriate. Decide how to handle factors that are irrelevant to the learning outcomes being measured. Evaluate all responses to one question before going on to the next one. When possible, evaluate the answers without looking at the student’s name. If especially important decisions are to be based on the results, obtain two or more independent ratings.