Lecture READING ASSESSMENT AND TECHNIQUES WHY DO WE

Lecture READING: ASSESSMENT AND TECHNIQUES WHY DO WE READ WHAT DOES READING INVOLVE TASK DEVELOPMENT ISSUES

READING is “…an enjoyable, intense, private activity, from which much pleasure can be derived, and in which one can become totally absorbed” (Alderson, 2000: 28). is “the activity of reconstructing the messages that reside in printed text” (Carroll 1964)

WHY DO WE READ? For survival : depends on the day-to-day needs of the reader and often involves an immediate response to a situation For learning : used in the classroom and is goal orientated. For pleasure : is something that does not have to be done For assessment : to check the validity of something

Why we read (and what we read) has an important influence on HOW we read: I. e. : Goal-driven reading, when the goal of reading affects the process of reading

THE CENTRAL IDEAS BEHIND READING: • The idea of meaning; • The transfer of meaning from one mind to another; • The transfer of a message from writer to reader; • How we get meaning by reading; • How the reader, the writer and the text all contribute to the process (Nuttall 1996).

WHAT DOES READING INVOLVE? • Eye movement – linear fashion vs. zigzagging through a text • Different levels of coverage – grazing vs. detailed • Fluent reading comprehension involves automatic word recognition • Fluent L 1 readers can recognize almost all the words they encounter (98 – 100%); 4 -5 words per seconds • The working memory keeps new information active 1 -2 seconds while it carries out the appropriate process • Automatic vs. controlled reading

WHAT DOES READING INVOLVE? The type of reading varies according to a number of factors: • Purpose for reading (Pleasure vs work vs survival) • Main ideas comprehension vs. Identification of specific information • Our reading ability (different approaches) • The type of text we are engaged with (narrative vs. argumentattive) (eye movement is different and processing is different) • The text topic (our familiarity with it regardless of yr ability ) • The difficulty level of grammar and syntax and vocabulary

WHAT DOES READING INVOLVE? Interactive processes: a mixture of the above two A fluent reader manages the following at the same time: ü Rapid recognition of words ü Analysis of the structure of sentences ü Builds up a main idea model of text comprehension ü Monitors what has been comprehended ü Call in appropriate schemata from long-term memory Recently, there has been a move towards a more interactive, hermeneutic approach, one that assumes a degree of bidirectionality in these processes (Hudson, 1998).

WHAT DOES READING INVOLVE? Recently, there has been a move towards a more interactive, hermeneutic approach, one that assumes a degree of bidirectionality in these processes (Hudson, 1998). Reading is an interactive process (a conversation between writer/reader, even though the writer is not present) and for it to occur two processes are necessary, top-down to predict the meaning and bottom-up to check it. The two are therefore complementary ways of processing a text.

WHAT DOES READING INVOLVE? Bottom-up processes: the reader is operating from the visual data in the text = “information processing” (with a focus on the processing of more micro-level constituents of texts – letter, words, phrases, sentences etc); Top-down processes : readers call upon their background knowledge and /or use contextual information = “analysis-bysynthesis” (with a focus more on macro-level constituents – genre, text structure, as well as the role of background schematic knowledge etc).

THE BOTTOM-UP SKILLS Comprehension through decoding (word-level) and focusing on vocabulary or grammar Some examples are: • Identifying the sound/symbol correspondence in English, and its varieties. • Recognizing “sight words” - common words that are read quickly and easily. • Recognizing what part of speech a word functions as and the systems of tense, agreement, and pluralization. • Recognizing cohesive devices and their function in signaling relationships in texts. • Think about some ways you know to assess these skills.

THE TOP-DOWN SKILLS Comprehension through using prior content knowledge / knowledge of the genre (textual schemata) / using knowledge of language Some examples are: • Recognizing rhetorical conventions of writing. • Recognizing the function of types of writing. • Inferring context and connections between ideas from writing. • Distinguishing between literal and implied meanings. • Recognizing and interpreting culturally specific references in texts. • Using reading skills (skimming, scanning, guessing meaning from context) to help in the interpretation of texts. • Think about some ways you know to assess these skills.

AUTOMATIC VS. CONTROLLED READING / Processing • When something is new, a learner pays it conscious attention, therefore processing is slow • As the input becomes more and more familiar, processing becomes faster and eventually automatic • Processing periodically breaks down causing the reader to have to re-read, employ guessing strategies etc. • Automatic processing requires little/no attention and as such is less likely to interfere with other processes at work • Controlled processes require attention and it is difficult to deal with more than one source of info at a time (Nagle, S. J. , & Sanders, S. L. (1986). )

READING SPEED According to Pressley (2006): • Readers who are engaged in R to learn at approximately 200 wpm • Reading for pleasure can btw 250 – 300 wpm According to Nuttal (Nuttall, C. (1996): • Secondary school 2 L students can read 120 - 150 wpm before training • University students in similar areas – about 200 wpm but can be as slow as 60 wpm • An L 1 speaker of English, of about average education and intelligence - 300 wpm (however rates of up to 800 wpm and down to 140 wpm are not uncommon)

WHAT DOES READING INVOLVE (C. Weir) TYPE OF READING Creating a text level structure Construct an organized representation of the text Careful reading Local Understand sentence Global Comprehend main ideas comprehend overall texts Building a mental model Integrating new information Enriching the proposition Expeditious reading Local Scan/search for specifics Global Skim for gist Search for main ideas and important details Inferencing Establishing propositional meaning At clause and sentence level General knowledge of the world Topic knowledge Parsing Lexical access Word recognition Visual input Syntactic knowledge Text structure knowledge Jenre Rhetorical tasks Lexicon: Lemma Meaning/word class Form Orthography/phonology/morphology

STEPS IN READING ASSESSMENT Choose a text • It can be any genre that is relevant to your students. • The length and difficulty will be determined by the level of the students’language proficiency. Choose a task that students can complete after reading the text.

TEXTS Three main criteria when choosing texts for tests to be used in the classroom: • suitability of content • exploitability • readability

SUITABILITY OF CONTENT • It is the most important feature • The reading material should interesting to the students and relevant to their needs • It should motivate as well as § Does the text interest the student? § Is it relevant to the student’s needs? § Does it represent the type of material that the student will use outside of the classroom?

EXPLOITABILITY How the text can be used to develop the students’ competence as readers (can be exploited for teaching purposes to be useful in the classroom) • Can the text be exploited for teaching purposes? • For what purpose should the text be exploited? • What skills/strategies can be developed by exploiting the text?

WHAT MAKES READING DIFFICULT? • • • Grammar and syntax Vocabulary Topic Background knowledge The length of text

READABILITY • It is used to describe the combination of structural and lexical difficulty of a text, as well as referring to the amount of new vocabulary and any new grammatical forms present. • It is important to assess the right level for the right students.

READABILITY Factors to measure readability : üspeed of perception üperceptibility at a distance üperceptibility in peripheral vision üvisibility üthe reflex blink technique ürate of work (e. g. , speed of reading) üeye movement üfatigue in reading http: //www. instantfundas. com/2011/09/test-your-document-readability-in. html

TEXT REQUIREMENTS • Authentic material – Texts written for native speakers and contain “real” language. “…real-life texts, not written for pedagogic purposes” (Wallace 1992: 145) “…materials that have been produced to fulfill some social purpose in the language community. ” (Peacock (1997), in contrast to non-authentic texts that are especially designed for language learning purposes. • Topics within Ss’ experience, but not too familiar. • Topics not biased for any Ss or upsetting • Readability suitable for level, Limit unknown words to 510%

Types of reading tested at levels A 2 to C 2 24

CEFR Levels: A 1 DIALANG What types of text Very short, simple texts, typically short, simple I descriptions, especially if they contain pictures. understand Short, simple written instructions e. g. short simple postcards, simple notices. What I understand Familiar names, words, basic phrases. Conditions and limitations Single phrase at a time, re-reading part of text. 25

CEFR Levels: B 1 DIALANG What types of text I understand Straightforward factual texts on subjects related to my field of interest. Everyday material, e. g. letters, brochures and short official documents. Straightforward newspaper articles on familiar subjects and descriptions of events. Clearly written argumentative texts. Personal letters expressing feelings and wishes. Clearly written, straightforward instructions for a piece of equipment. What I understand Understand straightforward factual language. Understand clearly written general argumentation (but not necessarily all details). Understand straightforward instructions. Find general information I need in everyday material. Locate specific information by searching one long or several different texts. Conditions and Ability to identify main conclusions and follow argument restricted to limitations straightforward texts. 10 February 2018 26

CEFR Levels: C 1 DIALANG What types of text I understand Wide range of long, complex texts from social, professional or academic life, Complex instructions on a new unfamiliar machine or procedure outside my area. What I understand Identify fine points of detail including attitudes and opinions which are not explicitly stated. Understand in detail complex texts, including fine points of detail, attitudes and opinions. Conditions and limitations Understanding of details of complex texts usually only if difficult sections are re-read. Occasional use of dictionary. 10 February 2018 27

The cognitive demands imposed by relative text complexity at each stage 28

Text length 29

30

TEST CONSTRUCTION: WHAT DO WE WANT TO TEST? We can assess different types of reading • skimming, scanning, or intensive reading We can assess different types of meaning • grammatical (syntax), informational, discourse, or pragmatic (meaning from the writer’s tone) We can assess different types of comprehension • recognition, identification, analysis, interpretation, inference, synthesis, or evaluation This is the most common way to assess reading.

L. F. Bachman, Fundamental Considerations in Language Testing, OUP 1990, p. 87.

LEVEL OF ENGAGEMENT (It refers to how much of a text (or texts) a reader is required to engage with in the performing of a prescribed task) LOCAL Sections of a text (e. g. single sentences, or groups of sentences) GLOBAL Larger textual units (e. g. a series of paragraphs, or a whole text)

TYPE OF ENGAGEMENT CAREFUL Understanding the text - Explicitly stated main idea(s) and / or distinguishing that from supporting details - Locating, identifying, understanding and comparing facts, opinions, definitions (this includes search reading strategies) - Logical organisation of the text, e. g. understanding relationships among ideas in a text (problem – solution, cause – effect, temporal sequence, etc. ) -Cohesive relationships (reference, ellipsis, substitution, conjunction, lexical cohesion) - Making propositional inferences (deducing information that is not explicitly stated from information that is explicitly stated) EXPEDITIOUS Skimming / reading for gist - Identifying text type - Identifying text topic - Identifying text purpose Scanning / reading for detail - Finding specific details (e. g. names, figures, dates, any other surface-level information)

IMPORTANT TO REMEMBER As reading is a receptive skill, it is difficult to assess in isolation, without using other skills. It is important to recognize when other skills, such as writing, are also being assessed with reading. • For example, if a student reads a text and then writes a summary, the skill of reading and writing are utilized and therefore assessed. Scoring can either: • Focus only on the reading and only assess the content of the response, not the correctness of the writing, or • Focus on both reading and writing, calling this an integrated assessment.

WHAT SKILLS DO YOU WANT TO ASSESS? Macro skills • Reading quickly to skim for gist, scan for detail, • Establish general organization • Reading thoroughly for main ideas, supporting details, argument, purpose, relationship of paragraphs, fact vs. opinion, etc. • Information transfer from nonlinear texts Micro skills • Understanding at the sentence level - Syntax, vocabulary, cohesive markers • Understanding at inter-sentence level Reference, discourse markers • Understanding components of nonlinear texts - Labels, captions, symbols

ASSESSING THE MACROSKILLS Students can be assessed through display writing(writing to demonstrate ability) at the lower levels. Some examples of this type of intensive writing are: • Dicto-comp: Students listen to a paragraph or a story at normal speed two or three times and then rewrite it as best they can. (This also assesses students’ listening. ) • Picture-cues: Students see a set of pictures and write the sequence of events that are depicted. • Short answer or sentence-completion: Students read a question or a sentence starter (as in a dialogue) and write a response. (This also assess students’ reading. )

WHAT MACRO SKILLS DO YOU WANT TO ASSESS? HOW ARE YOU GOING TO DO IT? • Macro skills – Reading quickly to skim for gist, scan for detail, establish general organization – Reading carefully for main ideas, supporting details, argument, purpose, relationship of paragraphs, fact vs. opinion, etc. – Information transfer from nonlinear texts • Assessment strategies – Ask questions about main ideas, specific information, organization, purpose etc. – Use common formats such as MCQ, TFN, short answer – Create a chart for Ss to fill in with information

TASKS FOR READING ASSESSMENT Comprehension questions • Multiple-choice questions are the most practical but it is difficult to write these types of questions reliably. • Short answer questions are easier to write but also assess students’ writing ability and can be more difficult to score. Summary writing • Students can choose the best summary of the reading from 3 or 4 options. This only assesses their reading skill. • Students can write a summary, which also assesses their writing ability (and writing summaries is hard!).

TASKS FOR READING ASSESSMENT Metacognitive judgments • Students determine which retelling is appropriate for different audiences. • This helps assess their ability to read tone and register of a text. • This is a very difficult skill! Question selection or writing • Students choose questions that will help a peer understand the important ideas in the text. • Students write questions that will help a peer understand the important ideas in the text. This also assesses the students’ writing ability.

TASKS FOR READING ASSESSMENT True/False Statements - the number of the line giving supporting evidence Matching (words with their explanations, summarising sentences with passages)