1 ASSESSING SKILLS LISTENING purposes and techniques

1

ASSESSING SKILLS: LISTENING: purposes and techniques • the nature of listening • things that make listening difficult • understand the purpose of teaching and testing listening • test formats and types of listening tests

The Wise Owl There was an old owl who lived in an oak, The more he heard, the less he spoke; The less he spoke, the more he heard. Oh, if men were all like that wise bird! (Ogdan Nash)

WHAT IS LISTENING? § listening is a two-stage process involving, in the first instance, the extraction of basic information followed by the use of that language for a communicative purpose (Buck, 2001: 510).

GLOSSARY Bottom-up describes a process that builds smaller units into larger ones (syllables into words and words into phrases) Top-down Strictly speaking, describes a view of listening as a process that uses larger units in order to identify smaller ones (e. g. word-level information to recognize phonemes) Input Reduction Redundancy the speech that reaches a listener’s ear: sometimes assumed to be acoustic in form, sometimes assumed to have been analyzed by the listener into phonemes the way in which words may become reduced in length and reshaped if they do not have a prominent role or intonation group natural repetitiveness of speech, where speakers often reiterate or rephrase information they have already given

GLOSSARY Phoneme variation Chunks Schema Authentic recording Elision Interactive listening Working memory Extensive listening the way in which the sounds of a language vary according to the sounds that adjoin them, the type of speech and the speaker small groups of words that commonly occur together a complex knowledge structure in the mind which groups all that an individual knows about, or associates with, a particular concept recordings which were not made with language learning in mind missing out a sound in order to make a word or two adjoining words easier to produce situation where a listener has to respond to the speaker a component of memory which holds short-term information and works upon it. Limited in how much it can hold and therefore how much attention can be allocated to a task a preliminary hearing of a recording, to identify the main points

FEATURES OF LISTENING § Active: The listener has to remake the speaker’s intended meaning. § Time-constrained: The input is not under the control of the listener § Transitory: No text to refer back to. The listener has to carry forward a recall of what has been said in his/her mind. § Oral: The input contains features which reflect the way speech is produced ( from Dr. John Field presentation)

TOP-DOWN or BOTTOM-UP PROCESSING § speech perception § word recognition § sentence processing § construct the literal meaning of the sentence § hold the inforamtion in short-term memory § recognize cohesive devices in discourse § infer the implied meaning and intention § predict what is to be said decide how to respond

BOTTOM-UP APPROACH It sees comprehension as a matter of listeners first decoding the smallest elements of what they hear – the sounds. These sounds are then combined and the individual words are decoded. The words are then combined into sentences and the listener works out the meaning. To this will be added recognition of features such as intonation and so on, until we finally reach the non-linguistic content. Rumelhart and Ortony (1977)

BOTTOM-UP SKILLS • Distinguishing the sounds of English and identifying meaning from them. • Recognizing stress, rhythm, intonation, and how these produce meaning. • Processing speech at different speeds. • Recognizing grammatical word classes (such as verbs), systems (such as tense), and rules. • Recognizing cohesive devices in spoken discourse.

12 TOP-DOWN SKILLS • Recognizing the communicative functions of an utterance. • Inferring context and connections using real-world knowledge (schemata). • Understanding main ideas, supporting ideas, new information, and given information in a listening. • Developing listening strategies such as detecting key words, guessing meaning from context, asking for help, and indicating comprehension or lack thereof. Mc. Kenzy brought me another present today. It was too late to save it so I buried it in the garden. I think I’m going to have to put a bell round his neck.

TOP-DOWN or BOTTOM-UP PROCESSING Bottom-up processes help comprehension through: • decoding (word-level) • focusing on vocabulary or grammar Top-down processes help comprehension through: • using prior content knowledge • using knowledge of the genre (textual schemata) • using knowledge of language

LISTENING COMPREHENSION § is not either top-down or bottom-up processing but § an interactive, interpretive process § listeners use both prior knowledge and linguistic knowledge in understanding messages.

WHAT MAKES LISTENING DIFFICULT? § phonological modification § clustering § reduced forms/redundancy § colloquial speech and accents § prosodic features § speech rate § performance variables § discourse structure

PHONOLOGICAL MODIFICATION § Highly variable. § The forms taken by phonemes and words vary greatly from citation forms. § There is nothing like the standardized spelling system of reading. § They depend on gender, age, shape and size of vocal tract.

CLUSTERING § Spoken language is “chunked” into phrases and clauses.

REDUCED FORMS /REDUNDANCY § phonological § morphological (contractions like I‟ll); § syntactic (elliptical forms like When will you be back? Tomorrow, maybe); § pragmatic (Mom! Phone!) § § re-phrasings repetitions elaborations insertions

ACCENTS AND COLLOQUIAL SPEECH Ø standard GB; Ø standard US/Aus. NZ; Ø Non-standard; § L 2 accent

PROSODIC FEATURES § Prosody the rhythm, stress, and intonation of speech. § It reflects various features of the speaker or the utterance: the emotional state; the form of the utterance (statement, question, or command); the presence of irony or sarcasm; emphasis, contrast.

SPEECH RATE § In general older people have a slower speech § males tend to speak slightly faster than females § the effect of L 1 (native language) on L 2 § (second language) speaking rate is L 1 dependent.

PERFORMANCE VARIABLES § hesitations § false starts § pauses § corrections

TYPES OF CLASSROOM LISTENING SKILLS § Reactive (listen and repeat) § Intensive (listen for specific sounds, discourse markers, intonation patterns, etc. ) § Responsive (listen and respond –briefly) § Selective (listen for particular items in a longer stretch of discourse) § Extensive (listen for global comprehension) § Interactive (authentic communication; listening as part of discussion, conversation, debate, etc. )

INTENSIVE LISTENING SAMPLES 1. Recognizing phonological and morphological elements Example: Students hear: “I talked on the phone for hours. ” Students read: A: I talked on the phone for hours. B: I talk on the phone for hours. 2. Paraphrase recognition Example: Students hear: “I’m so busy. I have to study hard for my test. ” Students read: A: She is a worker. B: She is a student.

RESPONSIVE LISTENING SAMPLES 1. Question and answer Example: Students hear: “How long did it take you to get here? ” Students read: A: In about 30 minutes. B: Yes, I did. C: About 30 minutes. D: About $30. 00. 2. Open-ended responses Example: Students hear: “How long did it take you to get here? ” Students write or say the answer.

SELECTIVE LISTENING SAMPLES Listening cloze Example: Students hear: “Good morning, ladies and gentlemen. I regret to inform you that flight 928 to Baltimore, Maryland has been delayed for 54 minutes. The new departure time is 10: 17 a. m. Please remain close to gate 39 in case of further changes. ” Students read and complete: Good morning, ladies and gentlemen. I regret to inform you that flight _______ to Baltimore, Maryland has been delayed for _______ minutes. The new departure time is _____ a. m. Please remain close to gate ____ in case of further changes. Note: This is a fairly simple cloze because the students must only attend to the numbers and where they occur. Cloze exercises can be more difficult for more advanced learners.

SELECTIVE LISTENING SAMPLES Information transfer Example: Students hear: “Sally has a very busy day today. She has to wake up early, at 6: 00 a. m. , and be at school by 7: 30 a. m. She has a test in Biology class at 8: 00 a. m. , a presentation in Math class at 10: 15 a. m. , and a quiz in History class at 11: 45 a. m. After she eats lunch at noon, she has English class at 1: 00 p. m. School ends at 2: 30 p. m. and then she has dance practice until 5: 00. What a busy day!” Students complete the table

EXTENSIVE LISTENING SAMPLES 1. Dictation In a traditional dictation, the first reading is done at normal speed and students listen for the gist (the main ideas). The second reading is at a slowed pace with pauses between phrases and students write. The third reading is again at normal speed and students check their answers. 2. Communicative stimulus-response Students hear a longer conversation, story or lecture and then answer comprehension questions about it. 3. Authentic listening tasks - Students listen to an actual news story or lecture (which can be found online at sites such as http: //www. voanews. com/learningenglish/home/or at http: //www. apple. com/education/itunes-u/). - Students are asked to do one of the following tasks: take notes which are collected and scored for accuracy and completeness. identify the differences between the listening and a reading about the same story with some different details. retell the main points of the lecture or story.

PURPOSES OF ASSESSING LISTENING COMPREHENSION Ø LISTENING is found in most general proficiency tests, academic tests, tests of language for business purposes Ø In achievement tests it is used to evaluate the achievements and to encourage students to practice listening Ø Listening is widely used in placement and diagnostic testing

Which kind of language should learners be able to understand at different levels? OVERALL LISTENING COMPREHENSION § C 2 Has no difficulty in understanding any kind of spoken language, whether live or broadcast, delivered at fast native speed § C 1 Can understand enough to follow extended speech on abstract and complex topics beyond his/her own field, though he/she may need to confirm occasional details, especially if the accent is unfamiliar. § Can recognize a wide range of idiomatic expressions and colloquialisms, appreciating register shifts. Can follow extended speech even when it is not clearly structured and when relationships are only implied and not signaled explicitly. § Can understand the main points of clear standard speech on familiar matters regularly encountered in work, school, leisure etc. , including short narratives.

§ B 2 Can understandard spoken language, live or broadcast, on both familiar and unfamiliar topics normally encountered in personal, social, academic or vocational life. Only extreme background noise, inadequate discourse structure and/or idiomatic usage influences the ability to understand. § Can follow extended speech and complex lines of argument provided the topic is reasonably familiar, and the direction of the talk is sign-posted by explicit markers. . § B 1 Can understand straightforward factual information about common everyday or job related topics, identifying both general messages and specific details, provided speech is clearly articulated in a generally familiar accent.

§ A 2 Can understand enough to be able to meet needs of a concrete type provided speech is clearly and slowly articulated. § Can understand phrases and expressions related to areas of most immediate priority (e. g. very basic personal and family information, shopping, local geography, employment) provided speech is clearly and slowly articulated. § A 1 Can follow speech which is very slow and carefully articulated, with long pauses for him/her to assimilate meaning.

MEASUREMENT PRINCIPLES OF ASSESSING LISTENING § reliability § validity § fairness

COGNITIVE VALIDITY • Employ in real-world event • Predict behaviour in an academic or professional context or as an immigrant member of an L 1 speaking community. University of Bedfordshire 10/02/2018 34

35 GENERAL VS ACADEMIC LISTENING MICRO-SKILLS

INTEGRATIVE NATURE As listening is a receptive skill, it is difficult to assess in isolation, without using other skills. It is important to recognize when other skills, such as writing, are also being assessed with listening. For example, if a student listens to a story and then writes a summary, the skill of listening and writing are utilized and therefore assessed. Always identify exactly what you are assessing.

INTEGRATIVE NATURE You can either: • Minimize the use of another skill (through use of pictures, for example), or • Utilize integrated assessment so students understand they are being assessed on listening and another skill. Example If students have to write sentences in response to the listening, you can tell them they will be scored for content, grammatical accuracy and spelling. Or you can just score them on content. It is your choice, but you must be clear.

AUTHENTICITY § It is clearly good to use recordings that are as close as possible to natural everyday speech. But a test designer needs to think carefully about the criteria to be followed. § ‘Authentic’ is not necessarily equivalent to informal. There are some formal listening conditions which provide useful material: job interviews, discussions, lectures. An important consideration is: What kind of speech will the test takers be exposed to: informal everyday – broadcast – professional/academic? § Especially important in some tests is the naturalness of the delivery. Are there: natural planning pauses – hesitations – relatively short utterances - lexical chunks etc. ? University of Bedfordshire 10/02/2018 38

AUTHENTICITY 1. Task authenticity • simulated: modeled after a real-life; nonacademic task such as filling in a form • minimal/incidental: checks understanding, but in a way that isn’t usually done outside of the classroom; numbering pictures to show a sequence of events or identifying the way something is said are examples 2. Input authenticity • genuine: created only for the realm of real life, not for classroom, but used in language teaching • altered: no meaning change, but the original is no longer as it was (glossing, visual resetting, pictures or colors adapted) • adapted: created for real life (words and grammatical structures changed to simplify the text) • simulated: written by the author as if the material is genuine; many genuine characteristics • minimal/incidental: created for the classroom; no attempt to make the material seem genuine University of Bedfordshire 10/02/2018 39

TEXT TYPES (HUGHES A. ): § Monologue; § Dialogue; § Polilogue/Multiparticipation/Conversation.

TEXT FORMS: § Description; § Narration; § Explanation; § Exposition; § Argumentation; § News item; § Review; § Instruction.

Text Forms NARRATION DESCRIPTION EXPLANATION Purpose To amuse/entertain the readers and to tell a story Structural Features 1. Orientation 2. Complication 3. Resolution 4. Reorientation to describe a particular person, place 1. Identification or thing in detail. 2. Description To explain the processes involved in 1. General statement the formation or working of natural or 2. Explanation socio-cultural phenomena. 3. Closing ANALYTICAL EXPOSITION 1. Thesis To reveal the readers that something 2. Arguments is the important case 3. Reiteration/ Conclusion NEWS ITEM 1. Newsworthy event(s) To inform readers about events of the 2. Background event(s) day which are considered 3. Sources newsworthy or important REVIEW INSTRUCTIONS 1. Orientation 2. Evaluation To critique or evaluate an art work or 3. Interpretative Recount event for a public audience 4. Evaluation 5. Evaluative Summation To tell how to do or make something 1. goal 2. materials 3. method or steps 4. evaluation (optional) Language Features 1. Using Past Tense 2. Using action verb 3. Chronologically arranged 1. Using Simple Present Tense 2. Using action verb 3. Using adverb 4. Using special technical terms 1. Using Simple Present Tense 2. Using action verbs 3. Using passive voice 4. Using noun phrase 5. Using adverbial phrase 6. Using technical terms 7. Using general and abstract noun 8. Using conjunction of time and cause-effect. 1. Using modals 2. Using action verbs 3. Using thinking verbs 4. Using adverbs and adjectives 5. Using technical terms 6. Using general and abstract noun 7. Using connectives/transition 1. Short, telegraphic information about story captured in headline 2. Using action verbs 3. Using saying verbs 4. Using adverbs: time, place and manner. 1. Focus on specific participants 2. Using adjectives 3. Using long and complex clauses 4. Using metaphor 1. use of action verbs (turn, put) 2. linking words to do with time 3. tense is timeless 4. use of precise vocabulary

The Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Strasbourg. CEF, Chapter 4. 6. 3 Text-types include: • • • public announcements and instructions; public speeches, lectures, presentations, sermons; rituals (ceremonies, formal religious services); entertainment (drama, shows, readings, songs); sports commentaries (football, cricket, boxing, etc. ); news broadcasts; public debates and discussion; inter-personal dialogues and conversations; telephone conversations; job interviews; etc.

TASKS FORMAT § Multiple matching § Multiple choice § Dictation § Open ended comprehension questions § Gap filling § Note taking § Written response § True/false § Map labelling

TASKS FORMAT Matching responses + can easily test large numbers of items; - they may only test single utterances (not discourse level skills); Multiple choice questions + test takers don’t need to write; - test takers have to keep all three or four options in their heads while listening (Hughes, 2003: 165); Dictation + easy to construct, quick to administer and quick and easy to make; - require far more than listening comprehension (short-term memory, writing skills, etc. );

TASKS FORMAT Short answer questions + realistic activity in some cases (e. g. EAP lecture); - more than one answer may be possible so this must be taken into account and will require rater training); Information transfer + particularly good for testing understanding of sequence or relationships in texts; - doesn’t test inferred meaning comprehension or determining speaker’s attitudes; Gap filling + works well where a short answer question with a unique answer doesn’t work; - often they only test a single word / short phrase at best;

48 ANSWER THE QUESTIONS 1. What instructions are given to the test-taker? 2. What are test-takers required to do? 3. Do you think this is a suitable and appropriate task for the level of the students taking this exam? 4. What sklls does the task assess? 5. What can the test-taker find difficult in the task? 6. Does the test-taker need to have any background knowledge to complete the task?

Steps to ensure valid and reliable listening assessment § Consult specifications before starting tests development process § Use the variety of tasks with different listening purposes § Ensure that the tasks reflect real-life situations and contexts § Use authentic material for listening scripts § Check listening equipment and does the sound check before the tests § Pay attention to issues like background knowledge and skills contamination Ch. Coombe, 2010

50 Review of Mr. Knotte’s test What are the strengths? What are the weaknesses?

51 THAT YOU FOR YOUR ATTENTION