ASSESSING READING PURPOSES AND TECHNIQUES Prepared by Marina

ASSESSING READING: PURPOSES AND TECHNIQUES Prepared by Marina Solnyshkina, Olga Safonkina and Yulia Polshina

Students can, with difficulty, escape from the effects of poor teaching. They cannot (by definition if they want to graduate) escape the effects of poor assessment. (Boud, 1995)

Outline v Reading as activity v Levels of reading skills v Rating reading passages v Designing a reading test procedure v Tips for assessing reading v Reading tasks: pros and cons

READING AS ACTIVITY

Types Levels Careful Global Expeditious Local

Expeditious reading v. Skimming – reading for the gist v. Search reading – locating information on predetermined topics v. Scanning – reading selectively, to achieve very specific reading goals (Weir, 2005)

Careful reading v. Favored by most educationalists v. Not selective v. Submissive role of a reader v. Macrostructure on the basis of majority of information v. Involves both bottom-up and topdown approaches (Weir, 2005)

Global Careful reading Local Establishing accurate comprehension of explicitly stated main ideas and supporting details Making propositional inferences Identifying lexis Understanding syntax Expeditious Skimming quickly to establish: Scanning to locate discourse topic and main specific points of reading ideas, or structure of text, or relevance to needs. Search reading to locate quickly and understand information relevant to predetermined needs. information

LEVELS OF READING SKILLS

Levels of reading skills Common European Framework of Reference (CEFR): http: //www. coe. int/t/dg 4/linguistic/cadre 1_en. asp Interagency Language Roundtable (ILR): http: //www. govtilr. org/Skills/ILRscale 1. htm

C 2 Can understand interpret critically virtually all forms of the written language including abstract, structurally complex, or highly colloquial literary and non-literary writings. C 2 Can understand a wide range of long and complex texts, appreciating subtle distinctions of style and implicit as well as explicit meaning. C 1 Can understand in detail lengthy, complex texts, whether or not they relate to his/her own area of speciality, provided he/she can reread difﬁcult sections. B 2 Can read with a large degree of independence, adapting style and speed of reading to different texts and purposes, and using appropriate reference sources selectively. Has a broad active reading vocabulary, but may experience some difﬁculty with low frequency idioms. B 1 Can read straightforward factual texts on subjects related to his/her ﬁeld and interest with a satisfactory level of comprehension. A 2 Can understand short, simple texts on familiar matters of a concrete type which consist of high frequency everyday or job-related language. Can understand short, simple texts containing the highest frequency vocabulary, including a proportion of shared international vocabulary items. A 1 Can understand very short, simple texts a single phrase at a time, picking up familiar names, words and basic phrases and rereading as required.

RATING READING PASSAGES

Passage Rating Triangle Mode Text Skill Descriptions Language Features

Modes v Orientation v Instructive v Evaluative v Projective (Jim Child, LARC, 2014)

Language Features v. Vocabulary v. Target Language Linguistic Features v. Author-Intended Inferences v. Cultural References (LARC, 2014)

Modes/Levels 0/0+ 1/1+ 2/2+ 3/3+ PROJECTIVE MODE X EVALUATIVE MODE X X INSTRUCTIVE MODE ORIENTATION MODE 4/4+ X X X

STEPS IN DESIGNING READING TEST

Planning Analyzing Writing Scoring Piloting Administering

Specifications v. Content v. Conditions v. Grading criteria (Coombe, et al. , 2007)

TIPS FOR ASSESSING READING

Input text should: v allow you to write (at least) the number of questions you need – long enough, including ideas, facts, opinions v be at the right level of difficulty for the test takers v be of a type that will be familiar and equally accessible to every test taker v not be taken from text books or other educational materials that any of the test takers may have seen before (Hughes, 1993)

Changes to source texts might include: v changing the title and subtitle or opening lines to clarify the content v cutting down the length and editing to maintain coherence v editing to avoid word matching (with questions and options) v editing potentially offensive, overly technical or culturally inaccessible content v rewording or explaining technical or culture-specific terms v clarifying ambiguous wording or phrasing (Hughes, 1993)

Questions should: vbe meaningful vgenerally follow the order of the information in the text vinclude all the information needed to find the answer vbe positively worded make it clear whether they relate to facts or opinions, to explicitly stated or implicit information vemploy simpler language that the text

Answers should: vbe unambiguous vnot be recoverable from the text by matching vnot be answerable without the text (Hughes, 1993)

READING TASKS: PROS AND CONS

Reading task formats v. Selected response v. Deletion-based tasks v. Limited production v. Extended production v. Identifying referents v. Guessing the meaning

Selected response tasks v. Multiple-choice v. True-false v. Matching v. Ordering

Multiple-choice

Multiple-choice v Practical, valid and reliable v Seem easy to write v Extremely difficult to write good ones without training and experience v Guessing v Harmful washback v Cheating

True-false v Add 3 rd distractor v Have students correct false statements v Scoring procedures

Matching

Matching v Relatively large number of distractors v Easy to write v Easy to go wrong v Include more alternatives than questions

Deletion-based tasks v Cloze v Rational-deletion cloze v C-test

Rational-deletion cloze v Deletions should not be clustered too closely v Set a target number of deletions v Start from the end of the passage v Leave one or two sentences at the beginning intact

Rational-deletion cloze v Exact word scoring v Acceptable response scoring

Limited production v Short-answer questions v Information transfer v Completing incomplete outline

Extended production v Recall tasks v Written summaries

Reading task formats v. Selected response v. Deletion-based tasks v. Limited production v. Extended production v. Identifying referents v. Guessing the meaning

Translation v. Limited or extended production v. Primarily assesses ability to translate v. Difficult to score

Conclusion It is important to understand that there is no one ‘best method’ for testing reading. No single test method can fulfill all the varied purposes for which we might test. (Alderson, 2005)