Peer reviewing of case study assignments by 2 nd year Ba students Course: Organization Theory and Dynamics Bas Koene (course coordinator) Dept 2 - Organization and HRM
Intr odu ctio n Org aniz atio n theo ry • Theory not difficult • But application: – Apply carefully and precisely – Deal with real-life ambiguity in assessment and analysis – Relevance clear from practical application • Course OTD: – 900 students, mandatory introductory course – Lectures: overview organization theory – Learning Communities: support case research and writing – 2 lecturers, 6 LC coaches, 2 stud assistants
Lea rnin g com mu niti es • 1200 students • 24 learning communities – 50 students per community • 10 teams of 5 students • 7 LC meetings: interaction between teams – Poster presentations, short plenary presentations, role plays – Feedback on own work, ideas from what other teams do, practice skills (presentation, feedback, dialogue) • Peer review process
Peer revi ew pro cess • Final case study assignment build up in three parts – General description of organization – Investigation culture, photo ethnography – Interviewing about change objectives and methods • Peer reviews: – Active engagement with work other teams (see different solutions) – Creativity and benchmarking own progress – Engage with theory and reports in different way
Peer revi ew syst em: Exp erti za • Automated (dealing with 900 plus students) • Ability to give qualitative feedback, comments for improvement (not just scoring) • Meta-feedback possible – Get feedback on your review – Peer control, signaling function for teacher • Aggregate scores (feedback and meta-feedback available for teacher)
Exp erti za Pilo t: who invo lved ? • Surf-net: comparison peer reviews systems • RISBO / American developers – Comparison systems (surf-net), selection of system – Setting it up for use / adaptation for us – User manuals students / teaching staff & stud assistants – Support in first use / specialist ‘helpdesk’ • BIT: technical implementation • Department – Academic staff: review questions – Student assistant(s!): maintain system, first line helpdesk, etc – Secretariat: back up…
Imp lem enta tion pro gra m • Jan 08 – surf-net seminar peer reviewing, some non-teaching examples • Feb 08 – talks with RISBO about possibilities • May 08 – tentative choice for Expertiza, RISBO contact with US • June 08 – test version at RSM, adaptations made for us • July 08 – testing of system • Aug 08 – preparation of system for use • Sep 08 – first use of system
Amb ition s/ex pect atio ns – expe rienc e (1) Ambitions / expectations: Initial experiences: • • Structured peer review in depth feedback Need to learn to work with giving feedback – – • Steer by structuring process, not outcomes Know how to work with theory in analysis Value of being critical in a constructive way • Structured feedback list works • But need for staff feedback (as example / to set standard) early on or need less ambitious feedback lists? • Feedback helpful for improving assignments • Students asked for peer reviews in second round • Three reviews will safeguard some good feedback • First round sep: 1 in 3 reviews really useful, but: good reviews take time! • Possibilities for grading? • Quality numerical extremely (!) varied.
s / e x p e ct at io n s – e x p e ri e n ce ( 2) Ambitions / expectations: • Use independently of Learning communities - Initial perceptions: • Need for face-to-face contact after review First presentations in LCs Afterwards peer reviewing with additional insight, better reviews? Anonymity needed? • Students too nice to each other? • Students not serious, just going through the motions – – Peer control, leverage reviews in dialogue (Not anonymous!!) Work with paper and review comments immediately after receiving them • Good students extremely clear about bad (light) feedback • Yes, how set right ambition level for papers without direct teacher feedback? Not reading assignments / tasks disrupts online interaction! (interdependence teams) • • Deadlines necessary? • Yes! Clear deadlines are necessary • Use Expertiza to monitor peer review process • Pilot: fine-tuning system more work, – – monitoring face-to-face in my own LCs Expertiza interface not efficient if teacher wants to monitor quality of feedback • • • Teaching staff final evaluations also through system • Make questions that lead to comprehensive feedback ‘stories’ Organize feedback interface differently (by question not by respondent Need to do all grading off-line first, need overview scores in grading process
Wha t do you need to mak e it wor k? Our constraints: • Expertiza Pilot: first year • Large course: (1200+ students) • • Need dedicated help-desk for questions and for dealing with technical issues and deadlines (Have enough manpower available. At least two student assistants!) Extremely busy around deadlines (do not set them at 17. 00!) Need way to deal with exceptions (what if teams are late uploading their paper / reviewing)? FAQ system would be useful. Course email addresses + BB email facility worked, discussion board becomes to messy Guard simplicity of tasks and requirements. Way to remind them… • Pilot: • • • – – – The proof of the pudding… Hotline with professional support very useful! (in our case BIT and RISBO) How handle unexpected: Students appreciate information and feedback in case of unexpected events
Скачать презентацию Peer reviewing of case study assignments by 2
7c6ba247a6600dba93d77e287cbbb724.ppt