Developing a quality culture The basic framework Mag

Developing a quality culture: The basic framework Mag. Oliver Vettori March 2008

Introduction ‘The quality of higher education has proven to be at the heart of a European Higher Education area’. (Berlin Communiqué) EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 2

Introduction ‘The concept of quality is not new: it has always been part of the academic tradition. It is the outside world that now emphasises the need for attention to quality…It is the relationship between higher education and society which has changed’. (Vroeijenstijn 1995) EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 3

Introduction § Bologna process - European Higher Education Area: degree structure, student centred learning, student mobility § Accountability demands: evaluations, controlling, performance based funding schemes, employability rates § Institutional profiles and international competition: rankings, ratings, benchmarking § Changing role of the university management entrepreneurial universities, increased autonomy Rising importance of quality assurance EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 4

Quality notions Graduates Management Government Quality excellence Society at large Evaluierung & Qualitätsmanagement in der Lehre m on ey n or atio ef … form for ess Fitn pose pur Corporations, employers r de n ol io h t ke fac a St atis s Va lu Tran s Students Teachers Administrators March 2008 EQM 5

Differing views on Quality Cultures control oriented quality as an objective that can be created and managed clear understanding of quality required top down implementation of quality standards & strategies quality improvement can be measured – with unintended consequences participative quality as an emerging result that changes in a dynamic way permanent negotiations of quality notions required participative development & implementation of quality st. quality development can be influenced – with uncertain results EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 6

Setting the frame… Quality as a dynamic, multidimensional and perspective-bound concept Culture as a complex, interaction-based, and continuously changing construct Quality Cultures as stakeholder-dependent, historically grown and learning-oriented social phenomena The way(s) quality and culture can be related to each other have important consequences for QM and QA strategies Evaluierung & Qualitätsmanagement in der Lehre March 2008 EQM 7

Principles of a quality culture Empowering the stakeholders to develop their own quality goals, initiatives and measures Guaranteeing transparency and common standards without succumbing to a purely formal quality approach Showing trust without disregarding the risks involved Strengthening reciprocal communication processes Balancing the delegation and acceptance of responsibility Evaluierung & Qualitätsmanagement in der Lehre EQM March 2008 8

Fitness for purpose approach Graduate surveys, round tables, feedback box, awards, student evaluation of teaching questionnaires, program monitoring, focus groups, advisory board, accreditation, process analysis, student support program, discussion groups, staff development, premiums, fast feedback, midterm review, external evaluation, tutoring, COMMUNICATION, peer review, implementation of standards, rules and procedures, online evaluation, reporting, accompanying research projects, drop out study, steering committee, interviews, data ware house, meta-evaluation, counselling etc. Evaluierung & Qualitätsmanagement in der Lehre March 2008 EQM 9

QA roles: the formal structure Vice-Rector f. Academic Programs a. Student Affairs lays foundation Reports to Teaching Evaluation & QM Office Evaluation Board of the Senate concepts organisation operation conciliates Evaluierung & Qualitätsmanagement in der Lehre EQM March 2008 10

QA roles in a quality culture perspective Teachers Students improvement measurement dialogue ……. Management Administrators Evaluierung & Qualitätsmanagement in der Lehre EQM March 2008 11

QA roles: A few examples at WU Wien Encouraging quality intiatives on departmental level Rewarding initiatives for innovative and excellent teaching Reinventing program directors and coordinators Collaborating with staff development and controlling units as well as teachers and researchers throughout the university EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 12

… and some final caveats Organisational beliefs and values are not homogeneous, not even within a stakeholder group Quality cannot be decreed Permanent control or standardisation are no guarantors for success and transparency External and internal requirements cannot be satisfied by one and the same strategy Evaluierung & Qualitätsmanagement in der Lehre EQM March 2008 13

Part II The students‘ role EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 14

Student Contributions take responsibility give feedback participate in surveys and selfevaluations The students‘ role tutors and mentors self-assessment and self-evaluation take part in decision-making and design processes Evaluierung & Qualitätsmanagement in der Lehre EQM March 2008 15

student contributions - examples feedback tutoring course evaluations, sounding boards, feedback boxes, fast feedback … introductory tutors, learning coaches, buddy networks, teaching assistants… evaluating members of self-assessment teams, members of external review teams, graduate surveys … designing & deciding curriculum development boards, advisory boards, teaching & learning forums etc. EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 16

Student self-assessment (1) What did I learn today? What were my contributions in class today? What do I need help with? What do I want to know more about? What would you do differently, if you took this class Again? EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 17

Student self-assessment • self-evaluation forms • learning inventories • reflection logs and learning diaries • expectation exercise • discussions • peer assessment EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 18

Part III The administration‘s role EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 19

Administrative contributions establishing supportive structures information and communication providing quality services The administration‘s role Engaging in self. Assessment and selfreflection avoiding quality bureaucracy supporting feedback loops Evaluierung & Qualitätsmanagement in der Lehre EQM March 2008 20

feedback systems develop suitable quantitative and qualitative quality indicators that hold relevance for all the actors involved maintain a reasonable balance between the input (time, efforts) and outcome of evaluations support the other actors to make sense of their findings integrate evaluations into a broader system of quality assurance and development (follow ups) keep the feedback processes as transparent as possible Evaluierung & Qualitätsmanagement in der Lehre March 2008 EQM 21

necessary information • student progression and success rates • employability of graduates • students’ satisfaction with their programmes • effectiveness of teachers • profile of the student population • learning resources and their costs • the institution’s own key performance indicators ENQA Standards & Guidelines Evaluierung & Qualitätsmanagement in der Lehre March 2008 EQM 22

Criteria for self-evaluation (cost) effectiveness efficiency information quality celerity transparency service-orientation learning-orientation … EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 23

Part IV The teachers‘ role EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 24

Administrative contributions take the role of learners learning coaches creating suitable learning environments The teacher‘s role engaging in self. Assessment and selfreflection implementing a feedback culture peer teaching & peer reviews Evaluierung & Qualitätsmanagement in der Lehre EQM March 2008 25

Goals of evaluations Feedback improvement development assessment control Quality of content and didactics Quality of teaching and learning processes Staff satisfaction/student satisfaction Evaluierung & Qualitätsmanagement in der Lehre EQM March 2008 26

Course Evaluations EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 27

Student questionnaires objective method tools assessing the consistency of teaching goals, contents and methods regular student feedback (questionnaires) multifaceted item pool, flexibilised questionnaires monitoring Constant documentation/observance of results and developments Evaluierung & Qualitätsmanagement in der Lehre EQM March 2008 28

Additional Fast Feedback objective early diagnosis of potential problems, assessing standards of knowledge/understanding etc. flexible and individualised use: method § dimensions/aspects § time § extent Support counselling, support EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 29

Fast Feedback: Beispiel 1 Course: + What I liked about the course. . ? What could be improved… My further comments EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 30

Fast Feedback: Beispiel 2 Course Date Minute Paper Please answer the two questions below: What is the most important thing you learned today? What were the “muddy points”? EQM Evaluierung & Qualitätsmanagement in der Lehre March 2008 31