1 ASSESSINGLISTENING: purposesandtechniques Prepared by Svetlana Sannikova,

1 ASSESSINGLISTENING: purposesandtechniques Prepared by Svetlana Sannikova, Vera Novikova and Olga Mironova Based on material by John Field

Therewasanoldowlwholivedinanoak, Themoreheheard , thelesshespoke Thelesshespoke, themoreheheard Oh, ifmenwerealllikethatwisebird! (Ogden. Nash)

Listeningisoftenseenasatwostageprocess involving, inthefirstinstance, theextractionof basicinformationfollowedbytheuseofthat languageforacommunicativepurpose. (Buck, 2001 p. 51)

Active: Thelistenerhastoremakethespeaker’sintended meaning. Timeconstrained: Theinputisnotunderthecontrolofthe listener Transitory: Notexttoreferbackto. Thelistenerhasto carryforwardarecallofwhathasbeensaidinhis/her mind. Oral: Theinputcontainsfeatureswhichreflecttheway speechisproduced

Top Down. Bottom Up

Predictwhatistobesaid Inferimpliedmeaning s andintention s Recognizecohesivedevicesindiscourse Holdtheinform a tioninshorttermmemory Constructtheliteralmeaningofthesentence Sentenceprocessing Wordrecognition Speechperception Top Down. Bottom Up

Comprehensionasamatteroflisteners… 1. Decodingsmallestelementsofwhattheyhear–thesounds. 2. Soundscombinedandindividualwordsdecoded. 3. Wordscombinedintosentences: listenerworksoutmeaning. 4. Addrecognitionoffeaturessuchasintonationandsoon… 5. Finallyreachnonlinguisticcontent. ( Rumelhartand. Ortony, 1977)

Youhear: Mc. Kenzy brought me another present today. It was too late to save it so I buried it in the garden. I think I’m going to have to put a bell round his neck. . (fromeltnotebook. blogspot. co. uk) What is the relationship between the speaker and Mc. Kenzy?

I snot either topdown or bottomupprocessing, butaninteractive, interpretiveprocesswhere listenersusebothpriorknowledgeandlinguistic knowledgeinunderstandingmessages.

Phonologicalmodification Clustering Reducedforms Colloquialspeechandaccents Prosodicfeatures Speechrate Performancevariables

Spokensoundsvary. Alot. In writing we have standard spelling – no spoken equivalent. Soundsofwordschangeaccordingtocontext: Ime t her /t/ Ime t you /t /∫ Sounds of words depend on g ender, age , shape andsizeofvocaltractofspeaker.

Spokenlanguageis“chunked”intophrasesand clauses. Canbedifficulttoseparateoutindividualwords fromthesechunksorclusters.

More reduced forms in speech than in writing. I am going to win = I’m gonna win In speech a lot of information comes from context and is not put into words: Not that one. On top!

Localcolloquialexpressionsandculturallyspecific patterns Accents – Standard British/ American /Australian/ New Zealand etc. – Non-standard – L 2 accent

Rhythm, stress, andintonationofspeech. Prosodyvariesaccordingto… Emotionalstateofspeaker Formofutterance(statement, question, orcommand) Irony, sarcasm, emphasis, contrast Howmanydifferentmeaningscanyouexpresswithjustthe word‘No’? Canyoumakeaconversationwiththreeturns?

Fastspeechisharderforlearnerstofollow. Thosefamiliarwithtextbookspeechareoften unabletokeepupwithanaturalpaceofspeech.

Hesitations Falsestarts Pauses Corrections

Assessinglisteningissimilartoreading Botharereceptiveskills; Canusemanyofthesametypesofitemsoften withthesameissues; Needareasonforlistening/reading; Itemscanbemarkedobjectivelyorsubjectively; Discretepointorintegrative; Bothincludebottomup, topdownand interactiveprocessing;

Assessinglisteningisdifferentfrom reading Realtimeprocessing>requiresautomaticity; Specificationsneedtoincludeinformationaboutspeed ofdeliverywpmorsps(Hughes, 2003: 163) ‘ Mentalgymnastics’(backtracking)ismoredifficult; Environmentismoreimportant(acoustics, qualityof recording, quietnessoflocation)(Hughes, 2003: 163) >construct–irrelevantvariance(Messick, 1993) Livedelivery: musthaveequal input/speed/intonation/empgasisindelivery(between speakersandforthesamespeaker)

Reactive(listenandrepeat) Intensive(listenforspecificsounds, discoursemarkers, intonation patterns, etc. ) Responsive(listenandrespond– briefly) Selective(listenforparticularitemsinalongerstretchof discourse) Extensive(listenforglobalcomprehension) Interactive(authenticcommunicationlisteningaspartof discussion, conversation, debate, etc. )

Listening isfoundinmostgeneralproficiency tests, academictests, testsoflanguagefor businesspurposes Inachievementtestsitisusedtoassess achievementandtoencouragestudentsto practiselistening Listeningiswidelyusedinplacementtestingto identifythelevelofclassastudentshouldenter.

OVERALLLISTENINGCOMPREHENSION (Common. European. Frameworkof. Referencefor. Languages) C 2 Hasnodifficultyinunderstandinganykindofspokenlanguage, whether liveorbroadcast, deliveredatfastnativespeed C 1 Can understand enough to follow extended speech on abstract and complex topics beyond his/her own field, though he/she may need to confirm occasionaldetails, especiallyiftheaccentisunfamiliar. Can recognise a wide range of idiomatic expressions and colloquialisms, appreciating register shifts. Can follow extended speech even when it is not clearly structured and when relationships are only implied and not signalled explicitly. Can understand the main points of clear standard speech on familiar matters regularlyencounteredinwork, school, leisureetc. , includingshortnarratives.

B 2 Canunderstandardspokenlanguage, liveorbroadcast, onboth familiarandunfamiliartopicsnormallyencounteredinpersonal, social, academicorvocationallife. Onlyextremebackgroundnoise, inadequate discoursestructureand/oridiomaticusageinfluencestheabilityto understand. Canfollowextendedspeechandcomplexlinesofargumentprovided thetopicisreasonablyfamiliar, andthedirectionofthetalkis signpostedbyexplicitmarkers. . B 1 Canunderstandstraightforwardfactualinformationaboutcommon everydayorjobrelatedtopics, identifyingbothgeneralmessagesand specificdetails, providedspeechisclearlyarticulatedinagenerally familiaraccent.

A 2 Canunderstandenoughtobeabletomeetneedsofa concretetypeprovidedspeechisclearlyandslowly articulated. Canunderstandphrasesandexpressionsrelatedtoareas ofmostimmediatepriority(e. g. verybasicpersonaland familyinformation, shopping, localgeography, employment) providedspeechisclearlyandslowlyarticulated. A 1 Canfollowspeechwhichisveryslowandcarefully articulated, withlongpausesforhim/hertoassimilate meaning.

Reliability Validity Fairness

Dothecognitiveprocessesinwhichtesttakers engageduringatestoflanguageskills correspondtothosewhichexpertusersofthe L 2 wouldemployintherealworldeventsthe testisintendedtopredict?

Recordings should be as close as possible to natural everydayspeech. ‘ Authentic’ material not necessarily informal. Some formal listening conditions provide useful test material: job interviews, discussions, lectures. – Key consideration: What kind of speech will test takers be exposed to: informal everyday – broadcast – professional/academic? Especially important in some tests is naturalness of delivery. Are there: natural planning pauses – hesitations – relatively shortutteranceslexicalchunksetc. ? 14/12/16 Universityof. Bedfordshire

Numberofparticipants (Hughes. A. 2003) Monologue Dialogue Polylogue/Multiparticipant. Conversation.

Textpurposes (Hughes 2003) Description Narration Explanation Exposition Argumentation Newsitem Review Instruction

The. Common. European. Frameworkof. Referencefor Languages: Learning, Teaching, Assessment. (Councilof. Europe 2001) CEFR, Chapter 4. 6. 3 Text typesinclude : Publicannouncementsandinstructions Publicspeeches, lectures, presentations, sermons Rituals(ceremonies, formalreligiousservices) Entertainment(drama, shows, readings, songs) Sportscommentaries(football, cricket, boxing, etc. ) Newsbroadcasts Publicdebatesanddiscussion Interpersonaldialoguesandconversations Telephoneconversations Jobinterviews etc.

Taskf ormats (Buck 2001) Multiplematching Multiplechoice Dictation Openendedcomprehensionquestions Gapfilling Notetaking Writtenresponse True/false Maplabelling

Matching responses + easy to make large numbers of items — may only test single utterances (not discourse level skills) Multiple choice questions + no writing involved — test takers have to keep options in their heads while listening (Hughes, 2003: p. 165) Dictation + easy to make, quick to administer — require far more than listening comprehension (short-term memory, writing skills, etc. )Advantagesanddisadvantagesof differenttasktypes (Buck, 2001)

Short answer questions + realistic activity in some cases (e. g. lecture) — several answers may be possible & must be taken into account (needs rater training) Information transfer + good for testing understanding of sequencing or relationships in texts — doesn’t test comprehension of inferred meaning or speaker’s attitudes Gap filling + can work well where short answer question with unique answer doesn’t work — often only test single word / short phrase. Advantagesanddisadvantagesof differenttasktypes (Buck, 2001)

Formats Mostofthetraditionaltestformatsareopentoquestionbecauseof theextenttowhichtheyaredependentuponthereadingskill. Thisis especiallytrueofmultiplechoicequestions. Traditionalformatsrequiretesttakerstomapfromastatementon thepagetoonethattheyhear. Thismatchingandcheckingvery rarelyoccursinrealworldlistening. Presetquestionstelltesttakersagreatdealofwhattheyareabout tohear. Thisencouragestestwisestrategiesbasedupon anticipatingwhatistocome. Testtakersexploittheknowledgethatitemsfollowtheorderofthe recording. Theylistenoutforwordsthatappearintheitems. They tuneinandoutoftherecording, knowingthatitemswillbeevenly spaced.

Do. Don’t Choosetextswiththe specificationsinmind; Keepalternativesfor. MCQs shortandsimple; Makesurethequestionscan’t beansweredwithoutthetext; Writeclearinstructions, promptsandexamples; Mapthetextbeforewriting itemssoyouhaveallthemain importantinformation; Trytoensurethetasksreflect the. TLUdomain; Makesuretesttakershave plentyoftimetoanswerthe questions; Usetextswhicharewritten texts(i. e. notdesignedas listeningtexts)(Hughes, 2003: 164); Writeitemswhicharemore difficultthanthetesttaker’s languagelevel; Usenegatives/double negativesasthesearemuch moredifficulttoprocess (Brownand. Hudson, 2002: 61); Don’taskquestionswhich require“mentalgymnastics” (Hughes, 2003: 165) (From. Weir, 2005)