AUN-QA Assessments Challenges and Lesson Learnt Dr

AUN-QA Assessments, Challenges and Lesson Learnt Dr. TAN Kay Chuan AUN-QA Chief Quality Officer and Lead Assessor Director, Office of Quality Management National University of Singapore

Outline Background Initiation Trial implementation & challenges Tools development Implementation Actual quality assessments Challenges Special projects Improvement Lesson learnt and Challenges

Background Bangkok Accord 2000 AUN-QA Guidelines 2004 AUN-QA Manual 2006 AUN-QA Assessments 2007 AUN-QA Revised Operating Manual Guidelines Present Initiation Implementation Improvement

Bangkok Accord Appoint Chief Quality Officer (CQOs) Formulate common AUN-QA policies, criteria and benchmarking procedures Identify and encourage the implementation of QA good practices Enhance mutual collaboration and information exchange Invite and facilitate auditing, assessment and review by other member universities as well as by external bodies Be responsible for the implementation of this Accord Seek further and deeper engagement regarding QA in higher education with ASEAN Dialogue Partners

AUN-QA Guidelines are benchmark standards towards quality and quality assurance Strengthen the position of higher education in ASEAN Promote the harmonisation of quality assurance in higher education within and outside ASEAN Endorsed by AUN Board of Trustees as well as Presidents, Rectors and Vice-Chancellors of AUN member universities

AUN-QA Guideline AUN-QA Criteria 1: QA System Criteria 2: Teaching and Learning Course Curriculum Academic Staff Student Assessment Learning Process Environmental Health and Safety Standards Learning Resources Criteria 3: Research Funding and Facilities Research Output Criteria 4: Services Criteria 5: Ethics Criteria 6: Human Resource Development (HRD)

AUN-QA Manual (2006) Facilitate implementation of AUN- QA Guidelines Improve clarity and interpretation of quality criteria and standards Manual for Self-assessment and Auditing Endorsed by AUN Board of Trustees as well as Presidents, Rectors and Vice-Chancellors of AUN member universities

AUN-QA Manual (2006) Strategic (QA at Institutional Level) Systemic (Internal QA System) Tactical (QA at Programme Level)

AUN-QA Revised Manual (2011) First revision of the AUN-QA Manual Criteria and assessment process of AUN Actual Quality Assessment at Programme Level Associated resources (templates and samples) Effective August 2011

AUN-QA Standards – Institutional QA Stakeholder Satisfaction Mission Policy Plan A C H I E V E M E N T S Educational Activities Management Research Goals Human Resources Aims Funding Community Service Quality Assurance and (Inter)national Benchmarking Source: AUN QA Network

AUN-QA Manual – IQA System Internal Quality Assurance Monitoring Instruments Student Progress Pass Rate Drop-out Rate Feedback from the Labour Market and Alumni Research Performance Evaluation Instruments Student Evaluation Course and Curriculum Evaluation Research Evaluation Service Evaluation Special QA Processes Assurance Student Assessments Assurance Quality Staff Quality Assurance Facilities Quality Assurance Student Support Specific QA Instruments SWOT Analysis Inter-Collegial Audits Information System Quality Handbook Follow up

AUN-QA Standards – Programme QA

Initiation

Trial Run Burapha University, May 2007 To put into practice the “Manual for the Implementing of the Guidelines” of AUN-QA models for IQA system, institutional level and programme level Challenges Clarification of assessment criteria and self-assessment report (SAR) Key documents and records in local language Lack of a structured assessment process Lack of assessment and reporting tools

Tool Development Formalisation of assessment process (PDCA)

Tools Development Desktop Assessment Template Standardisation of assessment and reporting tools

Example Desktop assessment

Tool Development Assessment Report Template

Example Assessment Report

Tool Development Assessment Feedback

Tool Development SAR Diagnosing

Requirement of Self-Assessment Report Part 1: Introduction Part 2: AUN-QA Criteria Requirements Write-up on how the university, faculty or department addresses the requirements of the AUN-QA criteria (use checklist as a reference) Part 3: Strengths and Weaknesses Analysis Content page Executive summary Organisation of the self-assessment Brief description of the university, faculty and department Summary of strengths Summary of Weaknesses Completed checklist Improvement plan Part 4: Appendices Glossary Supporting documents and evidences

Implementation

Actual Assessment (2007 Onwards) 4 Years 7 Universities 13 Assessments 32 Programmes

Actual Assessment No. Period University Programme 1 st Dec 2007 University of Malaya • Biomedical Engineering • Computer-Aided Design/Manufacture Engineering 2 nd Aug 2008 • Chemical Engineering • Applied Economics 3 rd Dec 2008 Universitas Indonesia • Civil Engineering • Mechanical Engineering 4 th Dec 2008 Institut Teknologi Bandung • Physics • Pharmacy 5 th Oct 2009 • Pharmaceutical Sciences • Chemistry • Medical Education 6 th Dec 2009 Vietnam National University • Information Technology - Hanoi De La Salle University Universitas Gadjah Mada

Actual Assessment No. Period University Programme 7 th Dec 2009 Vietnam National University – Ho Chi Minh City • Information Technology • Computer Science & Engineering • Electronics & Telecommunications 8 th Oct 2010 Universitas Indonesia • Architecture • Electrical Engineering • Chemistry • Metallurgy & Material Engineering 9 th Nov 2010 De La Salle University • Chemistry • Psychology • Literature 10 th Dec 2010 Vietnam National University - Hanoi • Economics 11 th Jun 2011 Universitas Gadjah Mada • Biology • Geology Engineering • Civil & Environmental Engineering 12 th Nov 2011 De La Salle University • Physics • Applied corporate Management • Software Technology 13 th Dec 2011 Vietnam National University – Ho Chi Minh City • Biotechnology • Manufacturing Engineering • Vietnamese Studies

Results Overview Mean Min Max 1. Expected Learning Outcomes 15. Stakeholders Satisfaction 7. 0 2. Programme Specification 6. 0 14. Output 3. Programme Structure and Content 5. 0 4. 0 13. Stakeholders Feedback 3. 0 4. Teaching and Learning Strategy 2. 0 12. Staff Development Activities 5. Student Assessment 11. QA of Teaching and Learning Process 10. Facilities and Infrastructure 9. Student Advice and Support 6. Academic Staff Quality 7. Support Staff Quality 8. Student Quality

Challenges in Actual Assessment Shortage of qualified assessors Different stages of QA development and maturity within ASEAN Improving assessment efficiency and effectiveness Recognition of AUN QA Assessment Harmonisation of AUN QA framework within and outside ASEAN Sharing the knowledge gained and best practices

Special Projects Initiative for ASEAN Integration (IAI-CLMV) Project ADB-CLM Project (Supported by Asia Development Bank) ASEAN-QA Project (Collaboration with DAAD, Germany) Training for Non-member Universities

ASEAN-QA: Joint QA Capacity Development in the ASEAN region Intro IQA Theory SAR Writing Finalizing SAR Intro EQA Theory Peer Training Preparing Site Visits EQA: Staff of QA Agencies Site Visits Sensitization: Delegation of Vice-Presidents IQA: University QA Officers

Training for Non-member Universities Sep 2011, 1 st Workshop Apr 2012, 2 nd Workshop Dec 2012, 3 rd Workshop More than 50 participants from the ASEAN Region

CLM Project Enhance and strengthen the knowledge on QA system implementation and management particularly based on AUN-QA system, Build up qualified University QA personnel team in CLM countries, Establish QA system to leverage the quality of education management in CLM countries, and Provide countries with options for harmonizing university QA system to ensure applicability across Southeast Asia.

Improve

Training of New Assessors 1 st AUN New Assessors Training Workshop (2008) 7 new assessors 2 nd AUN New Assessors Training Workshop (2009) 12 new assessors Growing of qualified assessors from 6 to 25 Development path: junior to senior assessors

Revision of AUN-QA Manual for Programme Level Original AUN-QA Model for Programme Level 18 criteria, 72 statements Revised AUN-QA Model for Programme Level 15 criteria, 67 statements

Guidelines for AUN Quality Assessment and Assessors Assessment Certification

Guidelines for AUN Quality Assessment and Assessors The establishment of an AUN-QA Council which wil govern and formulate policies for the operation of the AUN-QA Network Define roles, responsibilities and requirements of the applying university, assessors, observers and staff of AUN Secretariat Provide a common frame of reference and accountability to the AUN quality assessment process and stakeholders for the purpose of certification

Sharing the knowledge gained and best practices via proposed IT platform sions Submis • SAR esults ment R s • Assess Report king nchmar ties ogress • Pr es & Be ersi Practic est ed Univ • B Certifi cies of AUN • List QA agen other Links to • AUN Secretariat Applicant QA Agency Assessor AUN Member Public User 38

Lessons Learnt

Lessons Learnt Trial implementation – validation of QA manual Formulate tools and assessment process before actual implementation Start small to gain experience Training and development of assessors Review for continuous improvement Assist CLMV partners within ASEAN to catch-up Gain credibility before considering QA certification Reach out to QA organisations within and outside ASEAN Use of IT to hasten harmonisation and sharing best practices

Challenges Globalisation of education Rising demand for better quality university education ASEAN Charter - promote greater mobility of students, faculty members, talent and labour Harmonisation of AUN-QA and national/local quality assurance framework for higher education

Quality Assurance in ASEAN 11 Countries: Different culture, language, political system, economy and HEI system Co co mm to mm on QA itm en t Different approaches, but many similar criteria A e. Q hav st Mo ncies age ve ll ha ut A , b s IQA varie A EQ

Quality Assurance in ASEAN As of end 2011 QAA + NQF QAA without National Qualification Framework (NQF) Either No Quality Assurance Agency (QAA) or just being established • • Indonesia Malaysia Philippines Thailand • • • Brunei-Darussalam Singapore Cambodia • • Myanmar Lao PDR Vietnam Timor Leste SEAMEO RIHED: Quality Assurance Landscape in Southeast Asia

Comparison of Assessment/Accreditation Criteria AUN-QA 1. Expected Learning Outcomes Timor Leste √ Thailand Philippine Malaysia Indonesia Vietnam √ √ √ 2. Programme Specification √ √ √ √ 3. Programme Structure and Content √ √ √ 4. Teaching and Learning Strategy √ √ √ 5. Student Assessment √ √ 6. Academic Staff Quality √ √ √ 7. Support Staff Quality √ √ √ 8. Student Quality √ √ √ 9. Student Advice and Support √ √ √ 10. Facilities and Infrastructure √ √ √ 12. Staff Development Activities √ √ 13. Stakeholders Feedback √ 14. Output √ 11. Quality Assurance of Teaching and Learning Process 15. Stakeholders Satisfaction

Harmonisation of AUN QA framework within and outside ASEAN

Thank You nus. edu. sg