PLASC NPD User Group Mick Quinlan Mick Quinlan naa

PLASC/NPD User Group • Mick Quinlan (Mick. Quinlan@naa. org. uk) • • Change, or the lack of it The Players: suppliers and users NC assessment process A wander through the ‘book’… an A-Z of the collection of NC assessment data

Forward • I’d like to change the current process but: – there are many players in the game each with existing systems and many dependencies – we’re not good at managing change!

Drivers for Change • 1998 Service delivery failure of data collection exercise • 1999 Rose report – Weighing the Baby • 2004 Beasley report – service delivery failure of KS 3 English test results • 2005 Statistics Commission report #23 – measuring standards in English Primary schools • 2005 Statistics Commission report #26 – school education statistics: user perspectives • 2006 NAO 3 rd validation compendium report – quality of data systems underpinning PSA targets – Broadly appropriate but system needs strengthening

Metadata • Statistics Commission #23, Feb 2005, § 11 – We feel that public presentation of the KS scores in statistical releases should include a clear statement about the uses to which the data may be put, and the limitations of the data in respect of those uses • Statistics Commission #26, June 2005, §D – Full and informative statistical commentary should be provided alongside the figures to guide users

Making Good Progress

Changes for 2006 • KS 2/3 Electronic collection of TA, including pupil-level non-core results for KS 3, via S 2 S – rather than OMR forms – A decision has been taken by the Department’s Head of Profession for Statistics not to provide this data until the quality and robustness can be assured (SFR 34 -2006) • KS 3 English: Richard III in, Henry V out • KS 3 ICT onscreen test 2 nd pilot (≈170, 000 pupils)

The Players: Df. ES • Df. ES – Paymaster/customer (TRD) – School Census/NPD and FSP/KS 1 data collection – Edu. Base – School Self-Evaluation/Improvement – Autumn Package (benchmarks, progress, target setting) – PAT/RAISEonline) – School Accountability - Performance Tables – SERAP, FORVUS, FFT – National Statistics – Statistical First Releases – PSA Targets

The Players: NAA • A division of QCA (the Regulator) • Managers of the process: – – – – Test development (NFER, Edexcel, UCLES, RM) Pupil registration (Df. ES, schools) Test Printing/Distribution (COI/Granby) Test marking (Edexcel) Teacher Assessments (schools, LAs) Data capture (Pearson) Non-standard processes (in-house)

The Process Schools Borderlined scripts + borderlined marksheets + school report (inc. marks/levels) Marker Standardisation May-Jun Teacher Assessments Jul 7. 2 Jul/Aug 7. 15 7. 9 Request for Process review Oct 7. 13 Guidance Sample scripts Marked scripts Mark Scripts May/Jun Sample scripts 7. 1 May/Jun Borderlining Jun 7. 7 Mark and level Jun-Aug May Jul/Aug DC results database R 2: Individual Re-marking Malpractice Level- Pretest 2: Draft Level thresholds Oct/Nov May/Jul Special consideration May/Jun 7. 13 7. 14 Re-review 7. 5 setting Recom 7. 4 mend. Jun 7. 6 ation Investigate review 7. 8 Selected scripts Script scrutiny Upheld review R 1: Clerical check Level-setting data collection 7. 3 Mark and level Data entry Cut scores Returned scripts Script pool collection Jun Script + Revised mark and level Request for Review 7. 14 Jul/Aug 7. 10 R 3/GR: Group review Jul/Sep 7. 11 7. 12 Re-marked script

A is for Assessment • National Curriculum – key stages: Foundation, KS 1, KS 2, KS 3 – Subjects • core: English, maths & science • Attainment Targets • non-core: Art & Design, D&T, Geography, History, ICT, MFL, Music and PE – Test/Task & Teacher Assessment – Levels

Levels • Q: What’s the difference between Level 4 and Level 5? • A: 1 mark • Making Good Progress (KS 2 -KS 3, page 9) – This shows that 15[%] pupils did not progress by a full level since finishing KS 2 and three[%] have actually regressed

A • • • A = pupil Absent from the test Aegrotat Achievement & Attainment Tables Age Standardised Scores (ASS) Additional time Appeals (process reviews)

BCD • • • B = Working Below the level of the test Borderlining Braille Compensatory award Classes/Teaching Groups – collected in 1997 • D = pupil Disapplied from the curriculum • Date of Birth

E • • E = Exceptional Performance Extension Papers Early takers Errors

FGH • F = Future sitting (subjects early takers not taking) • Fine grade • Foot and mouth • Gender • Handwriting

IJKL • Independent schools • L = pupil Left school (KS 2/3) • L = below Level 2 in the reading comprehension test (KS 1)

MNO • • • M = Missing scripts Maladministration/malpractice Missing pages Mental Maths test Modified large print (MLP) N = No test level (too few marks gained) Names, of pupils Overseas/Offshore schools Optional tests (Y 3 -5 & Y 7/8)

PQR • P = Past sitting (subjects early takers took last year) • Progress tests (Year 7) • P Scales • Q = result annulled - malpractice (Df. ES) • Reading • Reviews • Re-sits

ST • Service Children’s Education (SCE) Schools • Special Consideration • Shakespeare • Sub-Levels • Spelling test • T = Pupil unable to access the tests • Tier

UVW • • • U = No TA subject Level calculated (KS 1) Unique Pupil Number (UPN) V = mixed tiers Versions W = Working towards Level 1 Writing

XYZ • X = Marks pending (KS 2/3) • X = pupil not entered for reading comprehension test (KS 1) • Year groups • Z = ineligible pupil