The Ohio State University College of Education School

The Ohio State University College of Education School of Educational Policy and Leadership Policy Implications of PISA, TIMSS, & World-Class University Rankings Chuing Prudence Chou (周祝瑛) Professor, Cheng-chi University, Taiwan Email: iaezcpc@nccu. edu. tw May 17 th, 2007

YOU ARE GOING TO SHARE ABOUT… What are the Policy Implications of PISA, TIMSS and the World-class University Rankings? n What lessons can we learn from these global rankings from a comparative perspective? n

ARE YOU IN OR NOT? PISA (OECD) n TIMSS (IEA) n World-class University Rankings n

IS THIS EVERYBODY’S BUSINESS? n n n n The Finnish education phenomenon The German PISA shock Mexican’s indifferent attitude (For PISA and TIMSS) Brain Korea 21 (BK 21) Japanese public university incorporation China’s 211 and 985 Project Taiwanese world-class university funding program, etc (For World-class University Rankings)

WHAT ARE THESE BENCHMARKS? PISA (Program for International Student Assessment) (OECD), first in 2000 and then held every 3 years. n Three domains: reading literacy, mathematical literacy, & scientific literacy. n

TIMSS (Trends in International Mathematics and Science Study) n n n IEA Focus on international mathematics and science achievement assessment. First data collected in 1995, and 1999, 2003 and 2007. The United States is collecting TIMSS data now.

PISA OECD IEA Expert-driven Sponsor TIMSS Mathematics curriculum-driven Bottom-up approach Classroom math knowledge Framework Top-down approach Real-world application Problem-solving Curriculum content

Target 　group 　 An age-based 　　　 　　sample 　15 -year- olds 　 A grade-based 　　sample 　 Grade 3 & 4, 7&8 + high school seniors

WHAT ARE THE TEST OUTCOMES? n n There is a greater difference between Asian countries/regions and other countries in TIMSS rather than in PISA. Most English-speaking countries are ahead of Eastern European counterparts in. PISA, but fall behind in TIMSS.

WHY SUCH DIFFERENCES EXIT? n n TIMSS items tend to be more curriculum-based (more universal), PISA items are more application-based (more real -life experience and cultural-bound)

TIMSS (1997) (13 -year-old's average score, Third International Math and Science Study) Countries: (sample) Singapore Japan South Korea Czech Republic England Thailand Germany France United States Global rank 1 2 3 4 18 20 22 23 24 Math Score 643 605 607 564 506 522 509 538 500 Rank 1 3 2 6 25 20 23 13 28 Source: 1997 TIMSS, in The Economist, March 29 th 1997. Science Score 607 571 565 574 552 525 531 498 534 Rank 1 3 4 2 10 21 19 28 17

TIMSS (2003) (13 -year-old's average score, Third International Math and Science Study) Countries: (sample) Global rank Math Score Rank Science Score Rank Singapore Taiwan 1 2 605 585 1 4 578 571 1 2 South Korea Hong Kong 3 4 589 586 2 3 558 556 3 4 Japan Netherlands 5 7 570 536 5 7 552 536 5 9 England United States 10 12 498 504 18 15 544 527 7 11 Malaysia Italy 18 23 508 484 10 22 510 491 21 22 Sources: TIMSS Math 2003 and TIMSS Science 2003

2000 2003 Japan 557 534 Korea 547 542 Netherland 537 538 Finland 536 544 Australia 533 524 Canada 533 532 Switzerland 529 527 UK 529 Belgium 520 France 517 OECD 500 Germany 490 Russia 478 New Zealand US 529 503 523 493 Trends in PISA math achievement

There were 38 participating countries in 2003 PISA US ranked： n 7 th in reading literacy (15 th in 2000 out of 32 countries) n 24 th in math literacy (19 th in 2000) n 9 th in science literacy (14 th in 2000) n 20 th in problem solving

2000 2003 Japan 548 550 Finland 548 538 Korea 538 552 Australia 525 528 Netherland 524 528 New Zealand 521 Canada 519 Switzerland 513 France 511 Belgium 509 Germany 502 529 Ireland 513 UK 532 Swiss 512 Austria US 519 499 Trends in PISA science achievement

2003 Finland 543 Korea 534 Canada 528 Australia 525 New Zealand 522 Ireland 515 Swiss 514 Netherland 513 Belgium 507 Norway 500 Switzerland 499 Japan 498 Poland 497 France 496 Trends in PISA reading literacy achievement 2003

There were 38 participating countries in 2003 PISA US ranked： n 7 th in reading literacy (15 th in 2000 out of 32 countries) n 24 th in math literacy (19 th in 2000) n 9 th in science literacy (14 th in 2000) n 20 th in problem solving

As the provision of higher educational 　opportunities becomes increasingly 　international, institutional comparison 　becomes more in demand. The two most 　frequently cited rankings： 　1. Shanghai Jiaotong University, Academic Ranking of World Universities 　2. Times Higher Education Supplement, London, Times Higher University World Rankings

WHO NEEDS WORLD-CLASS UNIVERSITIES? FROM WHOES PERSPECTIVES? n n Becoming international, becoming competition-driven More pressure from domestic and abroad Who can afford joining the "World-Class Club"? Who benefits from these global, world-class competition?

MYSTERIES UNRESOLVED? n n n What does that mean by the "World Class University"? How many world class universities do we need? Does every country need one? How could universities be evaluated and compared in an international scope? What measures should be taken in order to establish world class universities? Who share the cost and who benefit?

World’s top 15 universities as an average of THES & SJTU 2005 rankings Rank Av. Rank Institution Country Times Shanghai 1 1 Harvard U. S. 1 1 2 2. 5 Cambridge U. K. 3 2 3 3. 5 Massachusetts Inst. of Technology U. S. 2 5 4 4 Stanford U. S. 5 3 5 5 University of California, Berkeley U. S. 6 4 6 7 California Inst. of Technology U. S. 8 6 6 7 Oxford U. K. 4 10 8 8. 5 Princeton U. S. 9 8 9 9 Yale U. S. 7 11 10 13 Cornell U. S. 14 12 11 13. 5 Chicago U. S. 18 9 11 13. 5 Columbia U. S. 20 7 13 17. 5 University of California, S. F. U. S. 17 18 14 18 Tokyo Japan 16 20 14 18 Imperial College London U. K. 13 23 Sources: Shanghai Jiaotong Academic Ranking of World Universities – 2005; Times Higher Education Supplement World University Rankings – 2005.

Rank Av. Rank Institution Country Tim es Sha ngh ai 16 21. 5 Duke U. S. 11 32 17 23 Johns Hopkins U. S. 27 19 18 23. 5 University of Pennsylvania U. S. 32 15 19 24 Swiss Federal Institute of Technology, Zurich Switzerland 21 27 20 25. 5 University of California, L. A. U. S. 37 14 Sources: Shanghai Jiaotong Academic Ranking of World Universities – 2005; Times Higher Education Supplement World University Rankings – 2005.

WHAT CAUSES THE GLOBAL RANKINGS AND WHAT THE EFFECTS ARE? n n The "Impact of Globalization " Borrowing and lending reform strategies In comparative education, there is a great need to examine how local reactions are to these external global forces. In policy studies, a new paradigm for educational policy-making and school reform is expected to be under way.

n n What can be learned and imported from elsewhere? (borrowing) What can be taught and exported elsewhere? (lending), The phenomena of cross-national policy attraction and educational borrowing Global and international convergence in education at the expense local needs and cultural diversity

Three types of National Policy Responses According to Steiner-Khamsi (2007), three types of policy reactions after international comparisons: 1. Scandalization (to blame one's own weak educational system), 2. Glorification (to praise one's good educational system), 3. Indifference to ranking results.

Typology of Political Reactions to International Comparative Studies Scandalization Glorification Indifference TIMSS PISA USA Germany, Israel Korea, Japan, Taiwan Japan, Finland Mexico UK, Finland USA, UK Mexico World-Class University Rankings

IS IT A FAIR GMAE FOR EVERYBODY TO PLAY? n n What makes countries to react so differently? Has scandalization led to increased policy import from other educational systems? Has glorification led to increased policy export to other educational systems? Why some remain indifferent to these global comparisons?

DO POLICY MAKERS BENEFIT FROM THESE GLOBAL RANKINGS? 1. More evidence-based research in public policy studies. 2. More Outcome-based and standard-based criteria in monitoring school reforms 3. A "scientific rationality" in education to appeal to the tax payers 4. A legitimate “self-referential system “ acting as policy borrowing ( a lesson learned from elsewhere) and a form of externalization.

WHY GERMAN KIDS FELL BEHIND? n n In the PISA study released in 2001, German students scored below the OECD-average with regard to reading literacy. The release of the PISA study led to a major uproar in the media, and calls for policy change and school transformation.

n n n n The Finnish Education Phenomenon German PISA Shock Brain Korea 21 (BK 21) Japanese university incorporation China’s 211 and 985 Project Taiwanese fifty-trillion dollars within five year’s world-class university funding program, etc. India’s brain drain of the university faculty

DARE TO SAY NO? ! Problems with global rankings : 1. Credibility and reliability issues of PISA & TIMSS 2. Danger of the international convergence of educational institutions and policies 3. Near-sighted government reform policies & projects aiming at policy borrowing across national boundaries.

The assumption of “standardized and universal qualifications” within universities 5. Diagnosed rather than prescribed formula approach, not aiming for solving local educational problems. 6. Indifference to the ranking impact on students, employers, teachers, and school officials. 4.

7. Criticism about these global rankings as the “fast capitalism” which benefits ranking benchmarks 8. The advantage of English-speaking countries in the global university rankings 9. Only the richest and the best survive in the worldclass university rankings 10. The rich get richer, and the poor get poorer. 11. Increasing polarization of institutional resource generation, public funding, and educational quality nation-wide and world-wide.

12. The cumulative advantage vs. equal opportunities for all 13. Most university ranking criteria focus on academic research productivity rather than human aspect in cultivation and collegiality.

14. Higher education shifted from public good to private commodity. 15. Many courses and degree programs are packaged and marketed just like consumer goods, and students are seen as customers with a world of choice in front of them. 16. “Exporting” university services to less developed countries.

THE RANKING GAME IS NOT OVER YET, BUT THE LECTURE IS GETTING. . . n n n International rankings and comparisons will continue in spite of criticism and protest. Support from many governments, mass media, educational institutes and the general public. Benchmarks create huge benefits.

QUESTIONS REMAIN… n Why the US has not so satisfactory PISA and TIMSS results, but has established so many world-class universities in the countries? n Does this global ranking competition bring real change or improvement to education at home? n What is the politics of comparison in global ranking?

n Does Taiwan need to be part of the game? Why?

THANK YOU ( 謝謝） & GOD BLESS YOU!! Comments are welcome! Eamil: iaezcpc@nccu. edu. tw