THE ASSESSMENT CYCLE, ASSESSMENT DESIGN AND SPECIFICATIONS PROSET

THE ASSESSMENT CYCLE, ASSESSMENT DESIGN AND SPECIFICATIONS PROSET - TEMPUS 1 Prepared by Maria Verbitskaya, Angelika Kalinina, Elena Solovova

PROSET - TEMPUS 2 Teachers want to know what parts of the course cause difficulties and require special attention Teachers want to know what progress students have made Students want to know their strengths and weaknesses Parents want to know how their children are doing What makes assessment necessary?

What makes assessment necessary? PROSET - TEMPUS 3 Universities want to choose the best applicants for enrollment Teachers want to know whether the students are coping with the programme Employers want to choose the best applicant for the job

THE ASSESSMENT CYCLE

The Assessment Cycle Designers Producers form Organizers guidelines Administrators Assessees Scorers Users Validators specification report script score decision Decision Script Score report Report Guidelines Form Specifications

Components of the assessment cycle: designing a test and developing test specifications; producing and working out the form of the test; organizing a test and working out the guidelines; test administration and organising the procedure; preparing assessees for tests; scoring the results of the test; setting pass marks; reporting outcomes of tests to stakeholders; validation

TYPES OF TEST AND ASSESSMENT

What forms of assessment do you use with your students? PROSET - TEMPUS 8 Test Portfolio Project work Observation Case study Interview, oral questions Experimental work, etc

Testing & assessment PROSET - TEMPUS 9 Assessment Testing portfolios observations projects exercises class tests national exams

What is a test? PROSET - TEMPUS 10 Any procedure for measuring ability knowledge, or performance. (Longman Dictionary of Language Teaching and Applied Linguistics)

What is a test? PROSET - TEMPUS 11 A test is a measurement instrument designed to elicit a specific sample of individual’s behaviour. … a test necessarily quantifies characteristics of individuals according to explicit procedures. (L.F. Bachman. Fundamental Considerations in Language Testing, 1990)

Test as an instrument of assessment (measurement) PROSET - TEMPUS 12 WHY WHAT HOW

What types of tests do we use? PROSET - TEMPUS 13 Test Purposes Proficiency tests Achievement tests Placement tests Diagnostic tests

What types of tests do we use? PROSET - TEMPUS 14 Direct vs Indirect Discrete Point vs Integrated Objective vs Subjective Low stakes vs High stakes Norm-referenced vs Criterion-referenced

Norm-referenced testing: the bell curve Number of students 0 25 50 75 100 Score (points) PROSET - TEMPUS 15

DESCRIBING TESTS AND ASSESSMENTS

The Assessment Cycle: where does a test begin? PROSET - TEMPUS 17 Designers Producers form Organizers guidelines Administrators Assessees Scorers Users Validators specification report script score decision Decision Script Score report Report Guidelines Form Specifications

What kinds of information should be available about a test? PROSET - TEMPUS 18 WHY WHAT HOW

Test specification PROSET - TEMPUS 19 ‘the official statement about what the test tests and how it tests it … to be followed by test and item writers’ Multilingual Glossary of Language Testing Terms. CUP, 1998 Test design statement Blueprint Task and item specifications

Test design statement PROSET - TEMPUS 20 the purpose, the knowledge, skills or abilities it is intended to assess the resources available the uses of the results, the intended impact of its use

Blueprint PROSET - TEMPUS 21 content methods scoring

Test tasks PROSET - TEMPUS 22 Prompt (a reading/listening text, an essay question, a picture to describe) Response (ticking a box, giving a short answer, describing a picture, etc)

The building blocks of a test PROSET - TEMPUS 23 Instructions Item Stem Options Distractors Key

Response types (Brown & Hudson, 1998) PROSET - TEMPUS 24 Selected response Constructed response Short response Extended response Personal response

Responses: EGE PROSET - TEMPUS 25 Selected response Выбор ответа (A1-A7, A8-A14, A15-A23) Short response Краткий ответ (B1, B2, B3, B4-B10, B11-B16) Extended response Свободно конструируемый ответ (C1, C2)

Task & item specifications PROSET - TEMPUS 26 How many items are included in each task? What area of knowledge, skill or ability is assessed in each task? What should the test-taker do (instructions)? How are scores on each item and task determined?

One and the same specification for all? PROSET - TEMPUS 27 Test-takers Item writers Test validators Users of test results

The same type of specification for a classroom test and EGE? PROSET - TEMPUS 28 EGE specification: 11 pages 12 parts (sections) 4 tables An appendix

Your classroom test specification/ plan PROSET - TEMPUS 29 test purpose number of tasks and items what these items assess how they are scored time limit ???