ENLTA European Network for Language Testing and Assessment

ENLTA European Network for Language Testing and Assessment

EALTA European Association for Language Testing and Assessment

ENLTA • Funded by European Commission to create EALTA - a network of individual language testers • Two years, December 2003 – December 2005 • 14 partner institutions • 8 activities in the Work Plan

Activities 1. 2. 3. 4. 5. 6. 7. 8. The organisation of EALTA Analysis of training needs 1 st Annual Conference Survey of assessment policy and practice Develop and pilot a training model Develop a code of good practice 2 nd Annual Conference Identify a basic training resource

Presenters • Activity 4: Survey of assessment policy and practice in Europe • Gudrun Erickson • Sauli Takala • Dianne Wall

Presenters • Activity 5: Develop and pilot a training model • Ülle Türk • Melanie Ellis

Presenters • Activity 8: Identify a basic training resource • Melanie Ellis

E N L T A – Activity 4 Survey of Policy and Practice • Gudrun Erickson, Jan-Eric Gustafsson & Cecilia Nihlén, Sweden • Sauli Takala, Finland • Barbara Czarnecka-Cicha, Poland • Tania Horak & Dianne Wall, UK

AIMS • Links to, and short comments on previous studies of assessment cultures and practices (EALTA website) • Links to, and short comments on official language testing sites in different European countries (EALTA website) • Reports on [some] European students’ and teachers’ thoughts about language assessment practices (EALTA website)

Review of literature The purpose is to provide a list of readings which deal with pupils’ views of assessment, tests and examinations. We are mainly interested in empirical studies but also reviews, essays and position statements are of interest. We have consulted Finnish (Linda) and Swedish (Libris) data bases as well as ERIC, Language and Linguistics Behavior Abstracts, and a couple of other sources.

• Kärkkäinen, K. & Takala, S. (1978) A feasibility study of incorporating a structures test in the matriculation examination. Reports from the Institute for Educational Research, University of Jyväskylä, 290/1978. • IEA: Language Education Study (Ari Huhta et al. ) • Westhpal, P. B. , Wacha, H. & Rhodes, P. (2002) Assessing oral performance in the secondary classroom. French Review, v. 75, n 3, 560 -569. • Kenyon, D. M. & Malabonga, V. (2001) Comparing examinee attitudes toward computer-assisted and other oral proficiency assessments. Language Learning and Technology, v 5, n 2, 60 -93.

• Barnes, A. , Hunt, M. & Powell, B. (1999) Dictionary use in the teaching and examining of MFLs at GCSE. Language Learning Journal, 19 -27. • Powell, B. , Barnes, A. & Graham, S. (1996) Teachers´ views of target language testing. Language Learning Journal, 14, 3 -9. • Alderson, J. C. & Clapham, C. (1995) Assessing student performance in the ESL classroom. TESOL Quarterly, 29, 1, 184 -187. • Sjöberg, A. (2002) Functionality of language skills in occupational English: the point of view of language users, language training and language testing. Ph. D thesis, University of Oulu.

In order to be able to cover all the relevant literature we ask for your assistance. Please send information about electronic data bases that we might search. References following the model provided above are most welcome. To make the material more easily accessible, please provide a translation into English of the title (when needed). Also, whenever possible, provide a brief abstract or summary in English. If the whole publication is available, it can be sent to the following address.

sjtakala@hotmail. com Address: Luokotie 2 40950 Muurame Finland We will provide a register of the references to the EALTA membership and search for ways to analyse them an publish the results.

Links to official language testing resources in Europe We have drawn up a list of sites we have found. We would be very grateful if EALTA members sent us details of any other sites they feel would be of interest and relevance. If your own organisation is missing from the list on the next pages, please send information to d. wall@lancaster. ac. uk

Links to official language testing resources in Europe International: ETS Europe International Baccalaureate Organisation Belgium Centre for Language and Migration University of Antwerp Language Test Centre

Links to official language testing resources in Europe Czech Republic CERMAT (School-leaving examinations) STANAG Examination Centre Language School State Exams Finland Finnish Matriculation Examination Board

Links to official language testing resources in Europe Franc-Parler Germany DAAD Test. Da. F Thuringian Ministry of Education Unicert

Links to official language testing resources in Europe Greece Greek State Certificate Hellenic American Union Hyphen

Links to official language testing resources in Europe Hungary Hungarian Accreditation Board for Foreign Language Examinations Hungarian Association of Language Examiners and Measurement Specialists Hungarian Exams Reform Teacher Support Project Institute of International Education, European Office + about 20 links to exams approved by OKI

Links to official language testing resources in Europe Italy Trinity College Italy University for Foreigners of Siena University of Rome “Roma Tre” Latvia Centre for Curriculum Development and Examinations

Links to official language testing resources in Europe Lithuania National Examinations Centre Public Service Language Centre Luxembourg Centre de Langues Luxembourg

Links to official language testing resources in Europe The Netherlands Bureau ICE CITO Norway AKSIS/ University of Bergen Directorate for Primary and Secondary Schools

Links to official language testing resources in Europe Poland Central Examinations Commission Russia Federal Institute for Educational Measurement Federal Testing Centre Unified State Examinations

Links to official language testing resources in Europe Slovenia Centre for Slovene as a Second/Foreign Language National Examinations Centre Spain Escuela Oficial de Idiomas de Avilés Instituto Cervantes

Links to official language testing resources in Europe Sweden Göteborg University, Dept of Education Turkey OSYM – Student Selection and Development Centre

Links to official language testing resources in Europe United Kingdom Cambridge ESOL City and Guilds Pitman Qualifications Df. ES – Information on ESOL qualifications Dave’s ESL Café Assessment Forum Federation of Awarding Bodies Institute of Linguists Learning and Skills Council

Links to official language testing resources in Europe United Kingdom (cont. ) QCA – Qualifications and Curriculum Authority Scottish Qualifications Authority Test of English for Educational Purposes Trinity College London

Survey: Student Questionnaire Piloted in 3 countries (Nov. 2004) Administered in Dec. 2004 – March 2005 Referring to 1 st Foreign Language If possible, answers in English

Open-ended questions What is good language assessment? What is bad language assessment? Why? Other comments?

Likert scales Attitudes to and feelings about testing, assessment and achievement Areas emphasized in testing and assessment Distinction made between exams/formal tests and assessment during lessons

Students 1, 373 students in ten European countries Bulgaria, Estonia, Finland, Hungary, Norway, Poland, Slovenia, Spain (Catalunya) , Sweden, the UK End of compulsory school

Survey: Teacher Questionnaire Comments on the whys, whats and hows of language testing and assessment + Likert scales 62 responses

Some preliminary results • Agreement between students and teachers about what is most frequently tested / assessed: – grammar – words & phrases – reading comprehension – written production • Agreement that assessment of oral proficiency is much less frequent, especially in formal testing situations.

Students’ perceptions Examples of positive features in tests • Variety • Communication • “Usefulness” • Pedagogical potential • Clarity • Fairness • Enough time

Students’ perceptions Examples of negative features • Too much focus on single aspects of language, in particular grammar • Too little focus on communication, especially speaking • Too little focus on active language use • Stress

Examples of students’ comments “A good language test is a test whiff a lot of diferent parts. Not just gramma, it is to much. The best test is an essaytest, becouse there you test a lot of different things at the same time. ” “A good test is the one which enables you to learn and you can see (find out) what you know (can). A bad test makes you nothing to learn. You do not learn anything by doing it. ”

Examples of students’ comments “In my opinion, good language assessment is assessing students for all the time - during classes, not only during tests. For me, writing texts, grammar or vocabulary is the same importaint as speaking, reading or listening. A good language test is checking all language abilities. There are many people who are very good at grammer, but they have some problems with speaking; there also people who are good at speaking but they are not good at grammer, so they can’t improve their english at such ”bad” tests. I think the best way to teach and to learn english is to encourage and to be encouraged, to interest and to be interested in language. Some tasks, projects and films(movies) make students encouraged to learn the language. ”

Examples of students’ comments “I like language tests and assessments because they show much I know about the language. – I think speaking tests are better than writing because if you listen to somebody speak you hear his/her pronounciation as well and also how he/she expresses himself/herself and of course how he/she knows grammar. ” “BAD: -fast, I mean not enough time, so generating stress-veeery narrow domain-only ONE answer is correct. ”

Examples of students’ comments “This ENLTA/student questionaire doesen’t like me either because probably it’s never read by someone other than me so basicly it wastes our lesson time and don’t do any good for me or my classmates” “Thank you for taking time to lisen one my toughts. ” “This was nice questionnaire!” “Good luck with the reasearch!”

ENLTA Activity 5 Survey of training needs: Develop and pilot a model for training events and modes of delivery

The team • • Györgyi Együd (Hungary)*¤# Melanie Ellis (Poland)*# Neus Figueras (Spain)+ Sara Gysen (Belgium) Angela Hasselgreen (Norway)* Günter Nold (Germany)* Ülle Türk (Estonia)*# Norman Verhelst (the Netherlands) (Classical Test Theory)# • • * prepared materials + reviewed materials ¤ organised the training event # delivered training

Tasks • Plan training events and modes of delivery • Design pilot training events and time schedule of delivery • Deliver pilot training event • Report to 2 nd conference • Two face-to-face meetings: – Kranjska Gora (May 2004): initial planning – Frankfurt (September 2004): detailed planning

Initial decisions • Two separate events: – Introduction to classical test theory – Norman Verhelst – Introduction to testing and assessment – the rest of the team • A three-day event to take place in Szeged, Hungary in March 2005 • The target group: foreign language teachers who have had no previous training in testing and assessment • Classroom assessment, self-assessment and ELP included

Day One • 09. 30– 11. 30 Introduction to testing and other forms of assessment • 12. 00– 13. 30 Introduction to assessing receptive skills • 14. 30– 16. 00 Assessing receptive skills: Test formats, task types, principles of scoring, issues related to scoring • 16. 30– 18. 00 Classroom assessment including ELP – focus on listening

Day Two • 09. 00– 10. 30 Introduction to testing productive skills • 11. 00– 12. 30 Testing writing: focus on tasks • 13. 30– 15. 00 Testing writing: focus on assessment • 15. 30– 17. 00 Classroom assessment of writing

Day Three • 09. 00– 10. 30 Testing speaking: what and how to test • 11. 00– 12. 45 Testing speaking: assessment • 13. 30– 15. 00 Classroom assessment of speaking

Timeline • • • First draft 30 November Feedback 10 January Second draft 25 January Feedback from Neus 10 February Final version 01 March Training event in Szeged 10– 12 March

Lessons learnt • • 22 hours too little: – More time for reading and listening needed – No time for classroom assessment – ELP – a separate issue Possible solutions: – Exclude classroom assessment – Increase the number of hours A five-day event more realistic – Delivered as one event – Two modules: (1)General principles and testing receptive skills (2)Testing productive skills Local needs – Core materials supplemented by local materials

Activity 8: Resources • Melanie Ellis – Foreign Language Teacher Training College, Zabrze, Poland • Laurence Kane – University of Dortmund, Institute for English and American Studies • Milena Ivanova Grigorova – New Bulgarian University

Aims of the evaluation • To evaluate the pilot materials in use • To evaluate the training event • To gather ideas for resources teachers need to support their learning about testing and assessment

How did we carry out the evaluation? • Observation and Feedback sheets – timing, order of activities, choice of interaction – content evaluation: comments, suggestions, questions – overall impressions: two perspectives- the observer and the trainer • Feedback sheets on each session for participants • End of course feedback sheet

Qualitative evaluation • Focus group discussions at end of each day • Introductory session task and final reflection task By the end of the course I’d like to know more about. . . have learnt. . . be able to. . .

End of course feedback • I enjoyed the course 5 (13) 4 ( 5) • I found the course useful 5 (10) 4 (8) • I learned new things 5 (6) 4 (7) 3 (5) • The classes were well-organised 5 (11) 4 (6) 3 (1)

ENLTA Activity 8 As part of Activity 8 we are preparing a bibliography for teachers who want to develop their knowledge of and skills in testing. The result will be available on the EALTA wesite. Please help us by listing any book, CD, tape, video, computer program or website you have used and would recommend. Send your response to melanie@ellis. pol. pl. and put 'Bibliography' in the Subject line Thanks