- Количество слайдов: 38
‘Streamlining’ assessment Ashley Havinden – Design for Chrysler Motors Ltd, 1929 (detail) Making assessment more efficient and effective Ray Land, Discovery Point, Dundee, January 13 th 2004
Ashley Havinden 1903 -1973 Design for Chrysler Motors Ltd, 1929 (detail) Scottish National Gallery of Modern Art Archive © The artist’s estate with thanks to Daimler. Chrysler UK Limited • Principles and purposes of assessment (effective for what? ) • Effects of massification • Control strategies • Independence strategies • Issues of risk and support –examples of adaptive initiatives
Construction of identities through assessment Paul Ramsden – ‘a serious and often tragic enterprise’ Sally Brown – ‘a nightmare’ David Boud - ‘more bad practice and ignorance of significant issues in the area of assessment than in any other aspect of higher education’
Principles of Assessment • Beneficial • Actively foster learning • Fair • Diverse and varied • Valid • Reliable • • Transparent Representative Effective Practicable/ cost effective • Right of redress • Secure
Need to bear in mind that principles of assessment often contradict. What is efficient may not be effective. What is effective may not be efficient.
What is assessment? • Sampling evidence of performance • Making inferences • Estimating worth • Representing that worth (symbolically)
I keep six honest serving-men (They taught me all I knew) Their names are What and Why and When and How and Where and Who (Rudyard Kipling – Just-So Stories)
• What kinds of knowledge, skills, abilities or values are we seeking evidence of? • Why are we seeking to gain evidence of achievement in the first place? For what purpose will this evidence be used? • When should assessment take place – at the end of the course? at periodic intervals throughout it? continuously? at the start of the course?
• How should we gather the evidence? What is the most appropriate strategy to use for our particular course? • Where should assessment take place – in the examination hall? during a group presentation? In the workplace? On a computer? • Who should undertake the assessment – the teacher? Anonymous examiners? A workplace/clinical supervisor? The student’s peers?
What’s assessment for? • • Diagnostic testing Motivating/regulating students (and staff) Developing student confidence Evaluation of teaching Certification /licensing Providing feedback to students Selection for subsequent courses
• Information for employment • Predicting future performance • Monitoring progress • Public accountability • Institutional marketing • Social stratification
Effects of massification • • • Over assessment Lack of advice on improvement Lack of knowledge of progress Slow return of coursework Same outcomes repeatedly assessed (redundancy) Credit / workload not calibrated ‘Convergence’ of assessments leading to stress points Lack of variety/ repetition of method across modules Consumerism / commodification ‘Performativity’
Over assessment an issue for both staff and students
Issues arising from widening participation • Academic literacies debate / acculturation • Traditional dominance of particular modes (eg essay) • Pacing of learning / ‘Slow’ learning • Assessment in the first term • Sensitivity in feedback
Choice of strategies Control Independence (Gibbs and Jenkins, 1997)
‘control’ strategies traditional methods retained but carried out with increasingly greater (industrial) efficiency cf ‘Performativity’ (Lyotard)
‘independence’ strategies shift of responsibility for (usually formative) assessment to students themselves, on the grounds that, apart from economies of effort, this is in itself an educational benefit that is likely to enhance student performance
Change issues - Risk and support • • High Risk High Support High Risk Low Support Low Risk High Support Low Risk Low Support
control strategies • • • Set fewer assignments Set shorter assignments Set fewer exam questions MCQs Use standardised feedback forms CAA – eg computerised objective tests, wordprocessed statement banks, optical mark readers • Set limits on assignment length and materials costs; penalise those who go beyond the limit
control strategies (cont’d) • Conduct more in-class assessments • On written assignments mark in pencil to make alterations without time or trouble • Sort scripts into piles according to approximate standard – helps maintain consistent standard • Scrap grades and percentages and use pass/fail instead. Scrap degree classifications. • Refuse to accept work that is difficult to read or late without reason. Stick to this policy • Use placement and clinical assessors – standardise criteria and provide training
‘independence’ strategies • ‘front-ending’ assessment • devise group tasks and assess the group – group reach consensus on scores for individual members • Self assessment - require students to mark their own work with comments, or submit a completed standardised feedback form with their work • Peer assessment – ask students to use a marking scheme to mark each other’s work
What staff want are low risk, low support initiatives
1. Introductory Chemistry ‘Course puts multiple choice questions on the course’s website each week for the students to do in their own time. There are no marks for these tests, and no record is kept of who has taken them, but the students do the questions because they know from the beginning of the course that the end of module exam will include a section of MCQs and that half of these questions will have been selected from those questions used in the weekly tests’ (Rust 2003)
2. Anatomy Students undertake regular MCQ tests but instead of simply marking their chosen response on the test sheet they also have to indicate a ‘confidence measure’ (CF), in order to minimise the element of guesswork common to many MCQs. If they indicate a high CF and get the correct answer they gain a high mark. If they indicate a high CF and get it wrong, they will be heavily penalised.
3. The Geography course • convert all examination questions from 30/40 minute to hour-long essays. • thus fewer questions, but not possible to get away with only recall. • immediate response that quality of answers improved and students did more reading. • number of essays also reduced throughout all modules, with each essay requiring more work on the part of the student, and contributing more to the final grade.
• Class examinations removed from most modules and the number of exemptions given to first year students increased. • Pass/fail examinations being used to a larger extent, and more planned. • Project work also increased and now accounts for up to 45% of the module grade. • prior discussion between staff and students as to what makes a good project, followed up with input in tutorials • several options now incorporate oral assessment and are peer assessed, with half of the grade being awarded by the students.
• In feedback, some of the discussions revolve around why peer and tutor marks differ. • Department encouraging the submission of all work in a word-processed format, though concerns have been voiced about the increasingly poor writing styles on exam scripts. • The dissertation deadline has been changed to later in the degree programme, and the greater experience gained from options and the like has resulted in higher quality dissertations, which have been winning national prizes. (ASSHE Project 1997)
4. An Education Course
Original Course Staff Time New Course Lectures (18 weeks x 2 hrs) Seminars (18 wks x 9 groups of 8) 36 hrs 162 hrs 36 hrs 36 hrs Workshops (18 weeks x 2 hrs) Autonomous seminars (18 wks x 2 tutors touring 6 groups of 12) Total teaching hours 198 hrs 72 hrs Essays (72 x 2 essays @ 20 mins) Examination (72 x 2 questions @ 10 mins) 48 hrs 24 hrs 6 hrs 18 hrs 36 hrs Total assessment hrs 72 hrs 60 hrs MCQ tests (6 x 1 hrs administration) Seminar presentations (peer assessed, 1 hr/week collation) Portfolio (72 @ 30 mins)
5. Using ‘Discourse’ in History A classroom communication system (CCS)
see What was one effect of World War II on America? Explain. Women worked in the factories
Discourse – Tutor’s list of responses This is a list with the student’s name beside it. It is possible for responses to be anonymous. The responses can be readily kept as a record.
Responses can be kept for the tutor’s view or can be shared with all in the group. Wu, Ken What was one effect of World War II on America? Explain. Instead of looking at domestic affairs, America realised that it needed to be on top of world happenings
Effective uses of CCS systems • • • diagnostic/ formative potential quick-fire tests of understanding possibilities for peer judgement promotion of discussion useful research tool to create an instant teaching resource • evaluation of content / delivery • makes lectures less boring!
Differences between ‘Discourse’ and previous generations • used between people that are linked • bi-directional • web-based / doesn’t need special equipment • questions can be asked ‘on the fly’ • open-ended questions can be asked
6. The Patchwork Text A patchwork is not just a ‘collection’ but a ‘pattern’: in the end it does have a unity, albeit made up of separate components. The unity of the Patchwork Text has two dimensions. To begin with it is defined by academic staff, as they carefully derive a sequence of tasks from the course material. And finally it is, as it were, re-defined by individual students, who review (and perhaps edit) their separate pieces of work in order to write their final section as an interpretation of what this course materials ‘means’, to them, now… (Winter 2003)
References Gibbs, G. & Jenkins, A. (1992 )Teaching Large Classes in Higher Education Routledge. Falmer, London. Lyotard, J. L. (1986) The Postmodern Condition: A Report on Knowledge, Manchester University Press, Manchester. Rust, C. (2002) ‘The impact of assessment on stuydent learning: how can the research literature practically help to inform the development of departmental assesswmtn strategies and learner-centred assessment practices? ’ Active Learning Vol 3, 2 145 -158 July Winter, R. (2003) ‘Contextualizing the Patchwork Text: Addressing Problems of Coursework Assessment in Higher Education’, Innovations In Education and Teaching International, Vol 40 No 112 -122
ray. [email protected] ac. uk