THE ASSESSMENT CYCLE ASSESSMENT DESIGN AND SPECIFICATIONS Prepared

THE ASSESSMENT CYCLE, ASSESSMENT DESIGN AND SPECIFICATIONS Prepared by Maria Verbitskaya, Angelika Kalinina, Elena Solovova PROSET - TEMPUS 1

What makes assessment necessary? • Teachers want to know what parts of the course cause difficulties and require special attention • Teachers want to know what progress students have made • Students want to know their strengths and weaknesses • Parents want to know how their children are doing PROSET - TEMPUS 2

What makes assessment necessary? • Universities want to choose the best applicants for enrollment • Teachers want to know whether the students are coping with the programme • Employers want to choose the best applicant for the job PROSET - TEMPUS 3

THE ASSESSMENT CYCLE

Designers Specifications Users specification Decision decision The Assessment Cycle Validators Scorers script Guidelines Administrators Assessees Report report Script Organizers form Form guidelines Score report score Producers

Components of the assessment cycle: • designing a test and developing test specifications; • producing and working out the form of the test; • organizing a test and working out the guidelines; • test administration and organising the procedure; • preparing assessees for tests; • scoring the results of the test; • setting pass marks; • reporting outcomes of tests to stakeholders; • validation

TYPES OF TEST AND ASSESSMENT

What forms of assessment do you use with your students? • Test • Portfolio • Project work • Observation • Case study • Interview, oral questions • Experimental work, etc PROSET - TEMPUS 8

Testing & assessment portfolios observations Assessment exercises class tests Testing projects PROSET - TEMPUS national exams 9

What is a test? Any procedure for measuring • ability • knowledge, • or performance. (Longman Dictionary of Language Teaching and Applied Linguistics) PROSET - TEMPUS 10

What is a test? A test is a measurement instrument designed to elicit a specific sample of individual’s behaviour. … a test necessarily quantifies characteristics of individuals according to explicit procedures. (L. F. Bachman. Fundamental Considerations in Language Testing, 1990) PROSET - TEMPUS 11

Test as an instrument of assessment (measurement) WHY WHAT HOW PROSET - TEMPUS 12

What types of tests do we use? Test Purposes • Proficiency tests • Achievement tests • Placement tests • Diagnostic tests PROSET - TEMPUS 13

What types of tests do we use? • Direct vs Indirect • Discrete Point vs Integrated • Objective vs Subjective • Low stakes vs High stakes • Norm-referenced vs Criterion-referenced PROSET - TEMPUS 14

Norm-referenced testing: the bell curve Number of students 0 25 50 75 100 Score (points) PROSET - TEMPUS 15

DESCRIBING TESTS AND ASSESSMENTS

Specifications Users Form Validators Scorers PROSET - TEMPUS Guidelines guidelines Organizers script Administrators Assessees Report report Script form Producers Score report where does a test begin? specification decision Designers score The Assessment Cycle: 17

What kinds of information should be available about a test? WHY WHAT HOW PROSET - TEMPUS 18

Test specification ‘the official statement about what the tests and how it tests it … to be followed by test and item writers’ Multilingual Glossary of Language Testing Terms. CUP, 1998 • Test design statement • Blueprint • Task and item specifications PROSET - TEMPUS 19

Test design statement • the purpose, the knowledge, skills or abilities it is intended to assess • the resources available • the uses of the results, the intended impact of its use PROSET - TEMPUS 20

Blueprint • content • methods • scoring PROSET - TEMPUS 21

Test tasks • Prompt (a reading/listening text, an essay question, a picture to describe) • Response (ticking a box, giving a short answer, describing a picture, etc) PROSET - TEMPUS 22

The building blocks of a test • • • Instructions Item Stem Options Distractors Key PROSET - TEMPUS 23

Response types (Brown & Hudson, 1998) • Selected response • Constructed response – Short response – Extended response • Personal response PROSET - TEMPUS 24

Responses: EGE • Selected response Выбор ответа (A 1 -A 7, A 8 -A 14, A 15 -A 23) • Short response Краткий ответ (B 1, B 2, B 3, B 4 -B 10, B 11 -B 16) • Extended response Свободно конструируемый ответ (C 1, C 2) PROSET - TEMPUS 25

Task & item specifications • How many items are included in each task? • What area of knowledge, skill or ability is assessed in each task? • What should the test-taker do (instructions)? • How are scores on each item and task determined? PROSET - TEMPUS 26

One and the same specification for all? • • Test-takers Item writers Test validators Users of test results PROSET - TEMPUS 27

The same type of specification for a classroom test and EGE? EGE specification: • 11 pages • 12 parts (sections) • 4 tables • An appendix PROSET - TEMPUS 28

Your classroom test specification/ plan • • • test purpose number of tasks and items what these items assess how they are scored time limit ? ? ? PROSET - TEMPUS 29