The SAGE Handbook of Comparative Studies in Education
Publication Year: 2019
Educational practices have changed in the last few decades, and with the evolution of technology, the ways in which exchange of information and learning are delivered and processed are ever changing. Yet, while the field of international comparative studies has grown, there has not been an extensive study on the relationship between educational practices, students, and how practitioners are prepared and trained. The SAGE Handbook of Comparative Studies in Education explores international educational practices and behaviours through new research and review of existing research. The editors have gathered the best empirical researchers in the world to investigate how frameworks for teaching and learning have developed and evolved, how they have been adapted, evaluated, and adopted or rejected. The Handbook has been structured to guide the ...
- Front Matter
- Subject Index
Part I: The Status of Comparative Education Research
- Chapter 1: The Status of Comparative Education Research in the 21st Century: An Empiricist's View
- Chapter 2: Critical Challenges in Approaches and Experience in Comparative Education Research
- Chapter 3: Enduring Issues in Education: Comparative Perspectives
- Chapter 4: Riddled with Gaping Wounds: A Methodological Critique of Comparative and International Studies in Education: Views of a Professor
Part II: Measurement Methods in Comparative Education Research
- Chapter 5: Challenges in International Large-Scale Educational Surveys
- Chapter 6: Non-Cognitive Attributes: Measurement and Meaning
- Chapter 7: Methodological Challenges to Measuring Heterogeneous Populations Internationally
- Chapter 8: The Participation of Latin American Countries in International Assessments: Assessment Capacity, Validity, and Fairness
- Chapter 9: Validity Issues in Qualitative and Quantitative Research of Cross-National Studies
- Chapter 10: Mixed Methods in Education: Visualising the Quality of Quantitative Data
Part III: Research Practices in Comparative Studies of Education
- Chapter 11: Growth and Development of Large-Scale International Comparative Studies and their Influence on Comparative Education Thinking
- Chapter 12: The Meaning of Motivation to Learn in Cross-National Comparisons: A Review of Recent International Research on Ability, Self-Concept, and Interest
- Chapter 13: Examining Change over Time in International Large-Scale Assessments: Lessons Learned from PISA
- Chapter 14: Qualitative Comparative Education Research: Perennial Issues, New Approaches and Good Practice
- Chapter 15: Methodological Challenges in Conducting International Research on Teaching Quality Using Standardized Observations
- Chapter 16: The Measurement and Use of Socioeconomic Status in Educational Research
Part IV: Lessons from International Comparisons of Student Behavior
- Chapter 17: Early Childhood Care and Education in the Era of Sustainable Development: Balancing Local and Global Priorities
- Chapter 18: Equity of Access to Pre-Primary Education and Long-Term Benefits: A Cross-Country Analysis
- Chapter 19: Primary Education Curricula across the World: Qualitative and Quantitative Methodology in International Comparison
- Chapter 20: Outside-School-Time Activities and Shadow Education
- Chapter 21: Measuring Opportunity: Two Perspectives on the ‘Black Box’ of School Learning
- Chapter 22: What Can International Comparative Tests Tell Us about the Future Supply of Highly Skilled STEM Workers?
Part V: International Comparisons of Instruction
- Chapter 23: Comparative Research on Teacher Learning Communities in a Global Context
- Chapter 24: Teaching Instructional Practices: Play-Based Learning – Supporting the Transition from Early Years to Primary Education
- Chapter 25: Challenges in Practice: A Critical Examination of Efforts to Link Teacher Practices and Student Achievement
- Chapter 26: Global Higher Education Trends: Implications for Policy and Practice
- Chapter 27: Digital Technologies and Educational Transformation
- Chapter 28: International Large-Scale Student Assessments and their Impact on National School Reforms
Part VI: Influence of Large-Scale Assessments on Policy
- Chapter 29: Changes in the World-Wide Distribution of Large-Scale International Assessments
- Chapter 30: The Uneasy Relation between International Testing and Comparative Education Research
- Chapter 31: Global and Local Dissonance when Comparing Nation-States and Educational Performance
- Chapter 32: Building Learning Assessment Systems in Latin America and the Caribbean
SAGE Publications Ltd
1 Oliver’s Yard
55 City Road
London EC1Y 1SP
SAGE Publications Inc.
2455 Teller Road
Thousand Oaks, California 91320
SAGE Publications India Pvt Ltd
B 1/I 1 Mohan Cooperative Industrial Area
New Delhi 110 044
SAGE Publications Asia-Pacific Pte Ltd
3 Church Street
#10-04 Samsung Hub
Editor: Jude Bowen
Editorial Assistant: Orsod Malik
Production Editor: Manmeet Kaur Tura
Copyeditor: Sarah Bury
Proofreader: Derek Markham
Indexer: Cathryn Pritchard
Marketing Manager: Joanna McDowall
Cover Design: Bhairvi Gudka
Typeset by: Cenveo Publisher Services
Printed in the UK
At SAGE we take sustainability seriously. Most of our products are printed in the UK using FSC papers and boards. When we print overseas we ensure sustainable papers are used as measured by the PREPS grading system. We undertake an annual audit to monitor our sustainability.
Introduction & editorial arrangement © Larry E. Suter, Emma Smith and Brian D. Denman, 2019
Preface © Larry E. Suter
Chapter 1 © Larry E. Suter, 2019
Chapter 2 © Brian D. Denman, 2019
Chapter 3 © Wing On Lee, 2019
Chapter 4 Rui Yang, 2019
Chapter 5 © Fons J. R. van de Vijver, Nina Jude & Susanne Kuger, 2019
Chapter 6 © Mary Ainley & John Ainley, 2019
Chapter 7 © Leslie Rutkowski & David Rutkowski, 2019
Chapter 8 © Guillermo Solano-Flores, 2018
Chapter 9 © Jae Park, 2019
Chapter 10 © David A. Turner, 2019
Chapter 11 © Larry E. Suter, 2019
Chapter 12 © Ming-Te Wang, Jessica L. Degol & Jiesi Guo, 2018
Chapter 13 © Christine Sälzer & Manfred Prenzel, 2019
Chapter 14 © Michele Schweisfurth, 2019
Chapter 15 © Anna-Katharina Praetorius, Wida Rogh, Courtney Bell & Eckhard Klieme, 2019
Chapter 16 © J. Douglas Willms & Lucia Tramonte, 2019
Chapter 17 © Abbie Raikes, Dawn Davis & Anna Burton, 2019
Chapter 18 © Gerard Ferrer-Esteban, Larry E. Suter & Monica Mincu, 2019
Chapter 19 © Dominic Wyse & Jake Anders, 2019
Chapter 20 © Siyuan Feng & Mark Bray, 2019
Chapter 21 © Leland S. Cogan & William H. Schmidt, 2019
Chapter 22 © Larry E. Suter & Emma Smith, 2019
Chapter 23 © Motoko Akiba, Cassandra Howard & Guodong Liang, 2019
Chapter 24 © Tanya Hathaway, 2019
Chapter 25 © Laura M. O'Dwyer & Catherine Paolucci, 2019
Chapter 26 © Christopher C. Blakesley, Donna J. Menke & W. James Jacob, 2019
Chapter 27 © Nancy Law & Leming Liang, 2019
Chapter 28 © Gustavo E. Fischman, Pasi Sahlberg, Iveta Silova & Amelia Marcetti Topper, 2019
Chapter 29 © Larry E. Suter, 2019
Chapter 30 © Martin Carnoy, 2019
Chapter 31 © Colin Power, 2019
Chapter 32 © Adriana Viteri & Pablo Zoido, 2019
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act, 1988, this publication may be reproduced, stored or transmitted in any form, or by any means, only with the prior permission in writing of the publishers, or in the case of reprographic reproduction, in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers.
Library of Congress Control Number: 2018962496
British Library Cataloguing in Publication data
A catalogue record for this book is available from the British Library
List of Figures[Page ix]
- 1.1 Number of bibliographic references per year 8
- 2.1 Mårtensson et al.'s concept hierarchy of research quality 31
- 2.2 Paulston's conceptual map of perspectivism, constructivism, and rationalisms 32
- 2.3 Research ‘stream’ trajectories (characteristics) 33
- 2.4 Data of seminal scholars and their approaches/experiences 34
- 7.1 Difference in per capita GDP in 2015-adjusted US dollars, estimated as GDPOECD – GDPPartner (dollar amounts in thousands) 128
- 8.1 Process of an international assessment and the impact of the national context 146
- 8.2 An example of the rubric used to assess one of the 112 elements included in the PISA for Development capacity needs analytical framework: Guatemala 150
- 8.3 Percentage of elements rated as ‘Advanced’ within each of the three dimensions considered in the capacity needs analysis 151
- 8.4 A hypothetical item. Are students from all Latin American countries taught to represent fractions this way? 156
- 10.1 Education statistics for Australia 180
- 10.2 Education statistics for Kyrgyzstan 181
- 10.3 Rotating an image (Australia, Figure 10.1) to see trends and errors 182
- 10.4 Education statistics for Latvia 183
- 10.5 Education statistics for Malaysia 183
- 10.6 Education statistics for New Zealand 186
- 11.1 Number of individual large-scale international surveys from all sources by world region: 1980 to 2016 199
- 11.2 Number of comparative education publications per year by author type and whether large-scale or not: 1960 to 2017 213
- 11.3 Number of different authors for large-scale and small-scale researchers 214
- 16.1 Differential item function curves for two indicators of SES 294
- 16.2 Socioeconomic gradient for Peru 296
- 16.3 School profile for Peru 298
- 16.4 A model for assessing equality, equity and school effects 298
- 18.1 Probability of students attending pre-primary education (odds ratios) 334
- 18.2a and 18.2b Probability of students attending pre-primary education by immigrant background and SES: native versus second-generation students 335
- 18.3a and 18.3b Probability of students attending pre-primary education by immigrant background and SES: native versus first-generation students 335
- 18.4 Cross-country analysis: difference in students performance according to attendance in pre-primary education 336
- 18.5 Probability of students attending pre-primary education: average marginal effects 340[Page x]
- 20.1 Key parameters in the ecology of OST activities 364
- 20.2 Percentages of 15-year-old students receiving supplementary education, 2012 367
- 21.1 Conceptual model of school learning (student achievement) 375
- 21.2 Model of potential educational experiences 380
- 21.3 Carroll's model of school learning 381
- 22.1 Number of countries in PISA 2015 by GDP per capita in PISA 2015 399
- 22.2 Percentage of 15-year-olds who aspire to a STEM, health, or education profession for 13 world regions 399
- 22.3 Percentage of students aspiring to any STEM occupation by age 30 by log per-capita country income 400
- 22.4 Percentage of students aspiring to a career as a scientist by per-capita level of country 400
- 22.5 Number of PISA countries at intervals of achievement in mathematics and science: 2015 402
- 22.6 Mean science achievement by average GDP per capita: 68 PISA 2015 countries 404
- 22.7 Average science score by log of per capita GDP 404
- 22.8 Proportion of students aspiring to a STEM career by age 30 by country level average science achievement: PISA 2015 406
- 22.9 Maximum and minimum bilateral correlations between pairs of six PISA attitude scales for participating countries: PISA 2015 408
- 22.10 Minimum, maximum and average correlation between student choice of scientist or engineer by age 30 with six attitude scales for 68 countries 409
- 23.1 Conceptual framework: teacher learning communities in a global context 422
- 27.1 A diagrammatic representation of the interrelationship among the different levels of indicators 528
- 29.1 Year countries first entered large-scale survey by year and region (whole countries) 560
- 29.2 Number of participants in large-scale surveys by World Bank income level, 1990–2015 (countries counted for each study) 561
- 29.3 Number and percentage of countries participating in large-scale surveys by population size 562
- 29.4 Percentage of world countries participating in large-scale surveys by average income per person per country 563
List of Tables[Page xi]
- 1.1 Major journals in international comparative education 13
- 3.1 Progress evaluation of the attainment of the EFA goals 54
- 3.2 Gini index of high-performing education systems in PISA, 2007 59
- 7.1 Number of OECD and partner countries since 2000 with GDP in 2015 USD 127
- 7.2 PISA 2012 booklet design 134
- 8.1 Participation of Latin American countries in four international assessments (1995–2018) and per capita gross national income (2017) 147
- 9.1 Comparison of emic and etic approaches 167
- 11.1 Number of comparative education publications by academic researchers and large-scale research organizations by source of data: 1960 to 2017 212
- 14.1 Summary of the contrasts between traditional comparative approaches and contemporary revisions 262
- 15.1 Overview of major international observation-based studies on teaching quality 271
- 15.2 Overview of pivotal challenges in conducting international studies on teaching 275
- 18.1 Probability of Students attending pre-primary education: coefficients, odds ratios and marginal effects 333
- 18.2 Cross-country analysis: difference in students performance according to attendance in pre-primary education by sub-samples (low versus high SES students) 337
- 18.3 Probability of students attending pre-primary education: coefficients and odds ratios 341
- 19.1 Extract from PIRLS 2016 Reading Achievement Distribution 351
- 19.2 Percent of PIRLS 2006 and PIRLS 2011 countries reporting levels of emphasis on reading for pleasure 353
- 19.3 Transition matrix of PIRLS 2006 and PIRLS 2011 of levels of emphasis on reading for pleasure 353
- 19.4 Percentage of PIRLS 2006 and PIRLS 2011 countries where inspection is used to assess implementation of primary reading curriculum at each time point 354
- 19.5 Transition matrix of PIRLS 2006 and PIRLS 2011 of inspection use to assess implementation of primary reading curriculum 354
- 21.1 Sociological perspectives on education opportunity 378
- 21.2 Student-level correlations of mathematics achievement with SES and its components in three international mathematics studies: TIMSS 1995 (T95), TIMSS 2011 (T11), and PISA 2012 (P12) 378
- 21.3 Psychological perspectives on opportunity to learn (OTL) 381
- 21.4 Classroom/school-level correlations of OTL with mathematics performance and OTL with SES in three international mathematics studies 383[Page xii]
- 21.5 Three three-level models predicting PISA mathematics literacy 386
- 22.1 Occupation aspiration by age 30 (sample size: PISA 2015) 397
- 22.2 Number of 2015 PISA countries/economies by ISO region and sub-region 398
- 22.3 Correlation coefficients between average country science achievement and proportion of students seeking careers in 11 fields: 68 PISA countries 2015 403
- 22.4 Average correlation (across individual countries) between student career choice at age 30 and achievement in science 405
- 23.1a Comparison of teacher collaboration activities from TALIS 2013 teacher survey 429
- 23.1b Comparison of teacher collaboration activities from TALIS 2013 teacher survey (continued) 430
- 23.2 Comparison of teacher collaboration activities by regions 431
- 23.3 Relationship between national mean frequency of teacher collaboration activities 432
- 24.1 Comparison of early learning principles 452
- 24.2 Comparisons in the frameworks’ perspectives on children's learning and play 458
- 24.3 The goals and principles of the British Columbia primary curriculum 460
- 24.4 Teaching instructional practices of the British Columbia primary curriculum 461
- 24.5 Teaching instructional practices of the Singapore primary curriculum ‘Learning through Play' 463
- 28.1 Most common international large-scale student assessments 533
- 28.2 PISA-based education goals 540
- 28.3 The use of international large-scale student assessments in national education policy contexts 542
- 29.1 Number of countries that participated in a large-scale international assessment study (LSAS) by world region, participants weighted by population 554
- 29.2 Large-scale international assessment studies (LSAS) grouped by subject and region 557
- 29.3 Large-scale comparative studies, 1960–2015, by year first initiated 567
- 32.1 Learning assessment units in Latin America 604
- Annex 32.1 Regional and International Learning assessments studies in Latin America 614
- Annex 32.2 National Learning assessments studies across countries in Latin America 616
Notes on the Editors and Contributors[Page xiii]The Editors
Larry E. Suter is a visiting scholar at the University of Michigan and lives in Woodstock, Maryland, as a consultant for educational research. His expertise is in analyzing large-scale international surveys, informal education, measurement of student achievement, and measurement of ‘soft skills', such as motivation, identity, and career interest. He retired from the US National Science Foundation in 2011. He received his undergraduate education in sociology at the College of Idaho and advanced degrees (MA and PhD) in sociology at Duke University in 1968 and 1975. He was employed for 21 years as a statistician at the US Census Bureau as chief of the Education statistics branch and at the National Center for Education Statistics where he expanded the program of international education to include large-scale cross-national surveys of science, mathematics, and reading. His published works include topics of research methods, international comparative studies, informal learning, and indicators of science education. He has been an adjunct professor at the University of Maryland, Georgetown University, and a visiting scholar at Stanford University.
Emma Smith is a Professor of Education and Director of the Centre for Education Studies at the University of Warwick, UK. Her teaching and research are mainly in the area of educational inequality and she is interested in the role that education policy can play in improving social justice and in making lives fairer. She has also recently worked with Sage in co-editing The BERA/SAGE Handbook of Educational Research and on the second edition of her book Key Issues in Education and Social Justice.
Brian D. Denman holds the title Senior Lecturer in Higher Education, Comparative and International Education, and Training and Development at the University of New England, where he also serves as Deputy Head of School. He has also worked as faculty director of an overseas university branch campus in China, director of international development for the first Prime Minister library in Australia (The John Curtin Centre), and study abroad coordinator, overseas opportunities coordinator, and principal/head teacher in the United States. His research has been published widely and supported by a number of international organizations, multilateral agencies and associations. Currently, he serves as Secretary-General of GlobalCIE, Secretary-General of the World Council of Comparative Education Societies (WCCES), UNESCO Fellow, UNE Council member (Board of Trustees), President of the Australian and New Zealand and Comparative and International Education Society (ANZCIES), and Editor-in-Chief of the International Education Journal: Comparative Perspectives.[Page xiv]The Contributors
John Ainley is a Principal Research Fellow (and formerly Deputy CEO) at the Australian Council for Educational Research (ACER). He has contributed to the IEA Civic and Citizenship Education Study (ICCS), the IEA Computer and Information Literacy Study and the OECD Teaching and Learning International Study 2018 (TALIS 2018). He also contributed to the Australian National Assessment Program sample studies of Civics and Citizenship and ICT Literacy. As director of the national and international surveys research program at ACER he directed many policy-oriented research and evaluation studies for national and state education authorities. Dr Ainley was a member of the Consortium Advisory Group for the Longitudinal Study of Australian Children from its beginning in 2003 until 2016. He is a member of the Publication and Editorial Committee of the International Association for the Evaluation of Educational Achievement (IEA).
Mary Ainley is an Honorary Fellow with Melbourne School of Psychological Sciences (The University of Melbourne). Her research interests and experience are in the areas of developmental and educational psychology. A major strand of her research has been around understanding the nature and development of interest, curiosity and information processing as they support student engagement with learning. This research has been published in a number of respected journals, such as Learning and Instruction, Journal of Educational Psychology and Contemporary Educational Psychology. Dr Ainley has also served on the Editorial Boards of these journals. Most recently, in collaboration with Dr John Ainley, she has investigated motivational constructs, most notably interest in science, from a cross-national perspective through secondary analyses of PISA data. This has been published in the International Journal of Science Education and Contemporary Educational Psychology.
Motoko Akiba is a Professor and the Chair of the Department of Educational Leadership and Policy Studies at Florida State University. She received her B.A. from the University of Tsukuba, Japan, and a dual-title Ph.D. in Educational Theory & Policy and Comparative & International Education from Pennsylvania State University-University Park. Dr. Akiba's areas of research expertise are teacher policy and reform, teacher professional development, and comparative education policy. Her publications include International handbook of teacher quality and policy (Routledge/Taylor & Francis, 2018), Teacher reforms around the world: Implementations and outcomes (Emerald Publishing, 2013) and Improving teacher quality: The U.S. teaching force in global context (Teachers College Press, 2009). Her research program has been funded by the National Science Foundation, Institutes of Education Sciences, and AERA Grant Program among others.
Jake Anders is an Associate Professor of Educational and Social Statistics in the Department of Learning and Leadership at UCL Institute of Education, and Director of CREATE (Conducting Research, Evaluations and Trials in Education) in UCL's Centre for Education Improvement Science. Jake's research interests focus on understanding the causes and consequences of educational inequality and the evaluation of policies and programmes aiming to reduce it. His research, which has been funded by the Economic and Social Research Council, the Nuffield Foundation, the Education Endowment [Page xv]Foundation, the Sutton Trust, and the UK Department for Education, among others, has been published in education, economics, sociology and psychology journals. Recent projects include experimental and quasi-experimental evaluations of school-based interventions, investigations of the importance of curriculum in explaining inequality in university access, and explorations of continuing inequalities into the labour market.
Courtney Bell is Principal Research Scientist in ETS's Global Assessment Center. She completed her doctorate at Michigan State University in Curriculum, Teaching, and Educational Policy after earning her B.A. in Chemistry at Dartmouth College. A former high school science teacher and teacher educator, Courtney's research looks across actors in the educational system to better understand the intersections of research, policy and practice. Her studies use mixed-methods to analyze the measurement of teaching and the validity of measures of teaching quality, especially observational measures. Current and recent studies investigate how administrators learn to use a high stakes observation protocol, how raters use subject specific and general protocols, how measures of teaching compare across countries, and the ways in which observation protocols capture high quality teaching for students with special needs. She has published in a variety of scholarly journals and also co-edited the 5th Edition of the AERA's Handbook of Research on Teaching.
Christopher C. Blakesley works as a Learning Engineer at the Eberly Center of Carnegie Mellon University. Dr. Blakesley consults with faculty and manages projects to improve learning opportunities with innovative technologies. Chris holds a Bachelor of Fine Arts degree in Media Arts from Brigham Young University, a Master of Science degree in Instructional Technology & the Learning Sciences from Utah State University, and a PhD in Curriculum & Instruction from the University of Wisconsin-Madison. His work focuses on designing curricula and evaluating learning programs to improve effectiveness and facilitate transformative experiences.
Mark Bray is a Distinguished Chair Professor in the Faculty of Education of East China Normal University (ECNU), Shanghai. Prior to joining ECNU in 2018, he worked for over three decades at the University of Hong Kong where he still holds the title of Emeritus Professor. His career commenced as a teacher in Kenya and Nigeria before moving to the Universities of Edinburgh, Papua New Guinea and London. Between 2006 and 2010, Professor Bray took leave from Hong Kong to work in Paris as Director of UNESCO's International Institute for Educational Planning (IIEP). Professor Bray is known for his pioneering studies of the so-called shadow education system of private supplementary tutoring in a wide range of countries.
Anna Burton is a doctoral student in Child, Youth and Family Studies at the University of Nebraska-Lincoln. Her research focuses on early childhood care and education quality, teacher training, and child development. She has consulted for the World Bank on studies of early childhood quality in preschools in several countries. Prior to beginning her doctoral studies, she was a lead teacher in a laboratory child development center at Baylor University. Ms Burton received the Rising Star award from the National Coalition of Campus Children Centers for her work. In the past, Ms Burton was the chair of the kindergarten readiness working group of Prosper Waco, a Collective Impact organization.[Page xvi]
Martin Carnoy is Vida Jacks Professor of Education and Economics at Stanford University. He is co-director of the Lemann Center for Brazilian Education at Stanford, a former president of the Comparative and International Education Society, a fellow of the National Academy of Education and of the International Academy of Education, and an associate of the Higher School of Economics’ Institute of Education in Moscow. He has written 40 books and more than 150 articles on the economic value of education, on the political economy of educational policy, on educational production, and on higher education. Much of his work is comparative and international and investigates the way educational systems are organized. Recent books include Cuba's Academic Advantage (2007), Vouchers and Public School Performance (2007), University Expansion in a Changing Global Economy (2014), and Transforming Comparative Education (2018).
Leland S. Cogan is a Senior Researcher in the Center for the Study of Curriculum Policy at Michigan State University. He earned undergraduate degrees in microbiology and psychology and a PhD in educational psychology from Michigan State University. He coordinated data collection and analyses for the Survey of Mathematics and Science Opportunities (SMSO), a multinational project that researched and developed the instruments used in the Third International Mathematics and Science Study (TIMSS). He has co-authored technical reports, articles, and books, including Characterizing Pedagogical Flow (1996), Facing the Consequences (1999), Why Schools Matter (2001), The Preparation Gap: Teacher Education for Middle School Mathematics in Six Countries (2007) and Schooling across the Globe: What We Have Learned from 60 Years of Mathematics and Science International Assessments (2019). His research interests include evaluation of mathematics and science curricula, mathematics and science classroom instruction, and the preparation of mathematics and science teachers.
Dawn Davis has a PhD in Child, Youth and Family Studies from the University of Nebraska where she is currently an Early Childhood Research Project Manager. Her research focuses on child development, measurement, program evaluation, and early interventions. Dr Davis has been part of the leadership teams on several large-scale intervention, evaluation, and longitudinal studies of low-income children and families in the US, including the Educare Learning Network and an IES-funded Reading for Understanding grant. In addition, she has consulted for the World Bank on studies of early childhood development and quality in preschools in several countries. Her responsibilities have included developing and facilitating trainings on measurement, adapting measurements to specific context, disseminating research findings, applying findings to policy, and working with programs to provide individualized professional development opportunities for staff to increase instructional practices and support program improvement.
Jessica L. Degol is an Assistant Professor of Human Development and Family Studies at Penn State, Altoona. Degol received her PhD in Applied Developmental Psychology at the University of Pittsburgh and completed a two-year postdoctoral fellowship at the University of Pittsburgh under the guidance of Dr Ming-Te Wang. Her research interests include understanding how the structure of child care environments impacts teacher stress and emotional well-being, as well as the development of preschool children's cognitive abilities and executive functioning skills. She also studies the sociocultural factors [Page xvii]that impact female decisions to pursue STEM (science, technology, engineering, and mathematics) majors and careers.
Siyuan Feng is a PhD candidate and in the Faculty of Education at the University of Hong Kong. He holds an MSc in education from the Moray House School of Education at the University of Edinburgh and has worked as a private admission consultant prior to his PhD study. His research interest is in private tutors’ perceptions of their occupational identities as well as relevant sociocultural factors shaping such identities. His other research interests and projects include policy research on Asia-Pacific countries’ regulations on private tutors’ occupational standards, competency modelling in outside-school education, and international student mobility and admissions.
Gerard Ferrer-Esteban is a Marie Skłodowska-Curie research fellow in the Department of Sociology at the Autonomous University of Barcelona, Catalonia, Spain. His educational background includes a PhD in Sociology and a BA in Pedagogy (UAB). He is involved in the ReformEd project of the Globalization, Education and Social Policies research center (GEPS), in which he complements the qualitative approach with quantitative analyses to disentangle the effects of school autonomy and performance-driven accountability policies on relevant outcomes, such as effectiveness, equity, and other non-cognitive outcomes. He has worked for seven years as a researcher on education policy and school effectiveness at the Agnelli Foundation, in Italy. He also worked as a full-time assistant professor in the Department of Social Pedagogy at UAB, and as a part-time instructor in the Department of Pedagogy at the University of Girona. His main fields of research are comparative education, education policy, and equity in school systems.
Gustavo E. Fischman is a Professor of Education Policy and Director of EdXchange, the knowledge mobilization initiative at Mary Lou Fulton Teachers College, Arizona State University. His scholarship has been distinguished and he has won several awards. He has been a visiting scholar in several universities in Europe and Latin America. He has authored over 100 articles, chapters, and books. He has been the lead editor of Education Policy Analysis Archives and is the editor of Education Review. Among his best-known works are Imagining Teachers: Rethinking Teacher Education and Gender Dumb Ideas Won't Create Smart Kids, co-authored with Eric M. Haas, and Made in Latin America: Open Access, Scholarly Journals, and Regional Innovations, co-edited with Juan P. Alperin.
Jiesi Guo is a Senior Research Fellow at the Institute for Positive Psychology and Education (IPPE) at the Australian Catholic University. His areas of interest include educational and developmental psychology, with a particular focus on how multiple systems on the cultural, social, motivational, and behavioural development of youth shape individual and gender difference in achievement choice. Jiesi completed his PhD and post-doctoral training at the Australian Catholic University and had multiple international research projects with fully funded scholarship to collaborate with prestigious researchers in the United States, Germany, and Finland. Jiesi has published in international leading journals in the fields of psychology and education.
Tanya Hathaway is a Lecturer in Coleg Llandrillo, in North Wales in the UK and an online instructor for Laureate Education B.V. She has a PhD in Higher Education from [Page xviii]Bangor University. She has extensive experience lecturing in the UK and Australia in teacher professional development and master's level education. She has pioneered master's level programmes in teaching and learning and educational research methods. She led the development of the Master's in Teaching and Learning at the University of Plymouth, UK, under the inspirational leadership of Professor Michael Totterdell. She has taught in a wide range of undergraduate and postgraduate courses in the areas of theories of learning, science research methods, values and beliefs in education, professional learning and development and early childhood education. She also has extensive experience of supervising postgraduate research studies. Her main research interests are in the field of personal epistemologies, teaching and learning in higher education and more recently in young children's play.
Cassandra Howard obtained a PhD from the Department of Educational Leadership and Policy Studies at Florida State University. She received her BA from the University of Mississippi and her MA in Latin American Studies from the University of Florida. She has taught in elementary, middle, and high schools and is interested in how to support and prepare teachers in ways that lead to enhanced, meaningful learning opportunities for all students. Her areas of research expertise include teacher professional learning, teacher leadership, teacher agency, and teacher policy and reform.
W. James Jacob is the Vice President of Innovation and International at the Collaborative Brain Trust (CBT). He has extensive administrative experience in higher education professional development and training programs, establishing international partnerships, and external research and program funding opportunities. His international networks span every major global region, where he has helped forge sustainable partnerships with universities, government agencies, nongovernmental organizations, and alumni groups. Dr Jacob holds master's degrees in Organizational Behavior (Marriott School of Management) and International Development (Kennedy Center for International Studies) from Brigham Young University and a PhD in Education from the University of California, Los Angeles.
Nina Jude is a Senior Researcher at the Leibniz Institute for Research and Information in Education (DIPF) in Frankfurt, Germany. Since 2001, Nina Jude has been involved in large-scale assessments, developing measures for cognitive and non-cognitive variables in national and international settings, as well as managing these projects. She has been the international project manager for questionnaire development in the Programmes for International Student Assessment (PISA) 2015 and 2018. For PISA 2021 she chairs the international questionnaire expert group. Her research focuses on the dimensionality of constructs in multi-level settings and the relevance of context factors for education. Recent publications include secondary analysis of large-scale context questionnaire data over time to track countries’ progress in different areas of educational quality.
Eckhard Klieme has trained as a mathematician and a psychologist. He is now Professor of Educational Research at Goethe University, Frankfurt am Main, Germany. He has been the Director of the Center for Research on Educational Quality and Evaluation at the German Institute for International Educational Research (DIPF) since 2001. His research interests focus on educational effectiveness and quality of teaching, classroom assessment, and international comparative educational research. Starting with [Page xix]TIMSS-Video 1995 (Trends in International Mathematics and Science Study) in Germany, Professor Klieme has led several video-based studies on teaching in mathematics, science, and language education. He has served as a consultant for national and international agencies and has been involved in international large-scale assessment programmes such as Programme for International Student Assessment (PISA), the OECD Teaching and Learning International Survey (TALIS), and currently the TALIS video study.
Susanne Kuger is Head of the Department of Social Monitoring and Methodology at the German Youth Institute in Munich (Germany). Her research interests and teaching topics are research on the effects of family, early childhood education and care, school and out-of-school environments on child and youth development, survey methodology, international comparisons in education, as well as refining modelling techniques for complex quantitative data analyses in education research.
Nancy Law is a Professor in the Division of Information and Technology in studies, Faculty of Education at the University of Hong Kong. She served as the Founding Director for the Centre for Information Technology in Education (CITE) for 15 years from 1998. She also led the Science of Learning Strategic Research Theme at the University of Hong Kong (2013–17). She is known globally as a learning scientist with a strong record and expertise in the integration of digital technology in learning and teaching to promote student-centred pedagogical innovations. Her research interests include international comparative studies of technology-enabled learning innovations, models of ICT integration in schools and change leadership, computer-supported collaborative learning, the use of expressive and exploratory computer-based learning environments, learning design and learning analytics. She received a Humanities and Social Sciences Prestigious Fellowship Scheme Award by the HKSAR Research Grants Council in 2014 in recognition of her outstanding research.
Wing On Lee is a Distinguished Professor at Zhengzhou University, China. He is concurrently serving as Director of the International and Comparative Education Research and the Central Plains Education Research Centre at the School of Education. In addition, he has been appointed Director of the Citizenship Education Research Centre as a National-based Centre established at Zhengzhou University. Professor Lee has over 20 years of senior management experience in higher education in different countries. He was previously Vice-President and Chair Professor of Comparative Education at the Open University of Hong Kong (2014–17) and Dean of Education Research at National Institute of Education, Singapore (2010–14). He has also previously served at Hong Kong Institute of Education as Vice President (Academic) and Deputy to President, Acting President and Chair Professor of Comparative Education, Founding Dean of the School of Foundations in Education, Head of two Departments and the Centre for Citizenship Education (2007–2010). In 2005, he was invited by the University of Sydney to be Professor and Director (International). Prior to his service in Australia, he had served at the University of Hong Kong as Associate Dean of Education and Founding Director of Comparative Education Research Centre. He has served on many strategic committees in his public services, such as Chair of Research Ethics Board on Population Health for the National Healthcare Group and Conference Ambassador for Singapore Tourism Board in Singapore, and [Page xx]Education Commission, Central Policy Unit, Curriculum Development Council and Quality Education Fund in Hong Kong. Currently, Professor Lee is appointed by the Hong Kong government to serve as Chair of the Award for Outstanding Practice in Moral Education (Primary Sector), Chair of the Steering Committee of PISA 2018, and member of Task Force on Curriculum Review.
Leming Liang is a PhD candidate completing his thesis entitled ‘A Multilevel and Multiscale Exploration of Teacher Learning in Technology-enhanced Pedagogical Innovations'. He is also the project manager for a learning design and analytics project named An Open Learning Design, Data Analytics and Visualization Framework for E-learning, which focuses on learning design, learning analytics, teacher inquiry, and developing a technology platform to support the synergistic process between the three aforementioned components in actual teaching practice. His research interests include technology-enabled pedagogical innovations, learning design, teacher learning, and technology use for teacher inquiry of student learning.
Guodong Liang is a Research Specialist at the Community Training and Assistance Center (CTAC) in Boston, Massachusetts, USA. He received his BA from the University of Science and Technology of China (USTC) and a PhD in Educational Policy Analysis from the University of Missouri. Dr Liang's research focuses on educational policy (e.g., performance-based compensation, teacher professional development, teacher evaluation, and principal leadership), especially from a comparative and international education perspective. His work has been published in such journals as Educational Policy, Journal of Educational Administration, Journal of Educational Research, Journal of School Leadership, and International Journal of Educational Research.
Donna J. Menke is an Assistant Professor at the University of Memphis College of Education in the Department of Leadership. She teaches courses in Higher Education and Student Affairs. Her research areas include the student athlete experience, college student academic advising, and career development. Articles from her research appear in the NACADA Journal, Journal for the Study of Sports and Athletes in Education, and the Journal of Loss and Trauma. She currently serves on the editorial review board of the First Year Experience, the Journal of College and Character, and recently completed service on the NACADA Journal editorial board. She maintains active memberships in NACADA and NASPA.
Monica Mincu is an Associate Professor in the Department of Philosophy and Educational Sciences at the University of Turin, Italy. She has published in high-profile journals, such as Comparative Education, Oxford Review of Education, and History of Education. She has engaged with education politics and governance from a social change and reform perspective and teacher education in Europe in various contexts. Her professional experience revolves around education politics and teacher education, through a comparative approach.
Laura M. O'Dwyer is a Professor in the Measurement, Evaluation, Statistics, and Assessment department. Her expertise is in the area of quantitative research methods and design, and advanced data analysis, and her research focuses primarily on examining the relationships between the organizational characteristics of schools and teachers and [Page xxi]student outcomes. She has contributed to numerous studies that examined ways for improving teacher quality and student outcomes, and her work has been funded by the NSF, the US Department of Education, and the Institute of Education Sciences. O'Dwyer has extensive experience in the design of large-scale observational and experimental research, and evaluation studies, and in the analysis of large-scale data sets such as PISA and TIMSS.
Catherine Paolucci is an Affiliate Research Scientist in the STEM Education Center at Worcester Polytechnic Institute. She began her career as a secondary mathematics teacher and earned an EdD in Mathematics Education from Teachers College, Columbia University (New York) in 2008. She has since served as a director of teacher education programs and professional development programs in the United States, Ireland, and South Africa. Her research and project work support program and policy development for mathematics teacher education, both in the US and abroad. Her current research in mathematics teacher education focuses on teachers’ development of mathematical knowledge and the impact of innovative field experiences in teacher education.
Jae Park reads at the Education University of Hong Kong. His research interests are in sociology and philosophy of education. He recently published in Comparative Education Review, Educational Philosophy and Theory, International Studies in Sociology of Education, Comparative Education, and Ethics & Behavior. He serves as the President of the Comparative Education Society of Hong Kong and as the Head of the International Education Research Group in the Centre for Lifelong Learning Research and Development of the Education University of Hong Kong. He is the Editor-in-Chief of the International Journal of Comparative Education and Development and Editorial Board member of the book series ‘Educational Leadership Theory’ for Springer.
Colin Power was Deputy Director-General of UNESCO from 1999 to 2000 and Assistant Director-General for Education from 1989 to 1998. As such, he was responsible for the overall policy and management of the education programmes of UNESCO, playing a central role in all of its major initiatives, such as International Literacy Year, Education for All and the International Commission on Education for the 21st Century, and in the UN's struggle to alleviate poverty, to defend human rights, to protect world heritage sites, and to promote education for sustainable development and a culture of peace and non-violence. Dr Power began his career teaching science and mathematics before taking up an academic post at the University of Queensland where he is an Adjunct Professor at the University and Alumnus of the Year 2002, and for ten years was Professor of Education at Flinders University of South Australia. He is author or co-author of 13 books and over 250 published works on education, learning and development. Currently he is Chair of the Commonwealth Consortium for Education and Director of the Eidos Institute (an international research network and think tank on social policy issues).
Anna-Katharina Praetorius is a Professor for Research on Learning, Instruction, and Didactics at the University of Zurich (Switzerland). She completed her doctorate at the University of Koblenz-Landau (Germany) after studying Educational Science, Psychology, and Elementary Educational Science at the University of Erlangen-Nuremberg (Germany). In her research, she focuses on issues around conceptualizing and measuring instructional [Page xxii]quality, both on a national and an international level. Additionally, she is doing research on teacher motivation and teachers’ judgment accuracy. She received several publication awards for her work.
Manfred Prenzel serves as Director of the Centre for Teacher Education at the University of Vienna since his retirement at the Technical University of Munich (in 2018), where he owned the Susanne Klatten Endowed Chair of Educational Research and also filled the position of the Founding Dean of the TUM School of Education. Before he had worked as Managing Director of the Leibniz Institute for Science and Mathematics Education (IPN) in Kiel. The main topics of his research relate to issues of learning and teaching in different domains (science, mathematics, medicine, economics). He was National Programme Manager in Germany for PISA 2003, 2006 and 2012. From 2005 until 2011 Manfred Prenzel was a Member of the European Science Foundation (ESF) Standing Committee Social Sciences, and from 2003 until 2009 a Member of the Senate and Joint Grants Committee of the German Research Foundation (DFG). Manfred Prenzel served also as Chair of the German Council of Science and Humanities (Wissenschaftsrat) from 2014 to 2017.
Abbie Raikes is an Associate Professor at the College of Public Health, University of Nebraska Medical Center. Dr Raikes’ recent work has focused on improving early childhood programs and policies in low- and middle-income countries. Her research background also includes a strong focus on young children's social/emotional development and leadership of the Measuring Early Learning and Quality Outcomes project. Previously, Abbie contributed to early childhood policy development in several countries as a program specialist for the United Nations Education, Science and Culture Organization (UNESCO) in Paris, where she also participated in UNESCO's process to develop indicators for the Sustainable Development Goals. Abbie was a senior program officer at the Bill & Melinda Gates Foundation, and has advised several organizations on early childhood development and education.
Wida Rogh is a Research Associate at the Department for Research on Learning, Instruction, and Didactics at the University of Zurich (Switzerland). She studied Educational Science, Art History and Psychology at the University of Münster (Germany) and completed her Master of Arts in Educational Science in 2014. Between 2013 and 2015 she worked as a Consultant at the Directorate for Education and Skills of the Organization for Economic Cooperation and Development (OECD). Since 2015 she has been working in various research projects on the creativity development and measurement in adolescence. Her current research focuses on the observational-based measurement of teaching and instruction.
David Rutkowski is an Associate Professor in Educational Policy and Educational Inquiry at Indiana University. Previously he was a Professor of Education at the Center for Educational Measurement (CEMO) at the University of Oslo, Norway, and a researcher for the International Association for the Evaluation of Educational Achievement (IEA) in Hamburg, Germany. He earned a PhD in educational policy with a research specialization in evaluation from the University of Illinois at Urbana-Champaign. His main areas of research are in the area of educational policy and educational measurement with [Page xxiii]specific emphasis on how large-scale assessments are used within policy debates. He has consulted for national and international organizations, including the US State Department, USAID, UNESCO, the IEA and the OECD, and has conducted evaluations in over 20 countries. He is the editor of the IEA policy brief series, serves on the IEA publication editorial committee (PEC) and is a board member of several academic journals. He teaches courses in evaluation, education policy, statistics and large-scale assessment.
Leslie Rutkowski is Associate Professor of Inquiry Methodology at Indiana University. She earned her PhD in Educational Psychology, specializing in Statistics and Measurement, from the University of Illinois at Urbana-Champaign. Leslie's research is in the area of international large-scale assessment. Her interests include latent variable modeling and examining methods for comparing heterogeneous populations in international surveys. In addition to a recently funded Norwegian Research Council FINNUT grant on developing international measurement methods, Leslie published the edited volume Handbook of International Large-Scale Assessment (Rutkowski, von Davier, and Rutkowski, 2014) with Chapman & Hall. She is also currently co-authoring a textbook on large-scale assessment under the Guilford stamp with David Rutkowski and Eugene Gonzalez. She teaches quantitative methods courses, including structural equation modeling/covariance structure analysis and related topics.
Pasi Sahlberg is a Professor of Education Policy and Research Director at the Gonski Institute for Education, University of New South Wales in Sydney, Australia. He has worked as schoolteacher, teacher-educator, researcher, and policy advisor in Finland and has analyzed education policies and advised education policy makers around the world. He has gained working knowledge in over 60 countries and he is a former senior education specialist at the World Bank in Washington, DC, lead education specialist at the European Training Foundation, director general of the Ministry of Education in Finland, and a visiting professor at Harvard University. He is recipient of the 2012 Education Award in Finland, the 2013 Grawemeyer Award in the US, the 2014 Robert Owen Award in Scotland, the 2016 Lego Prize in Denmark, and Rockefeller Foundation Bellagio Resident Fellowship in Italy in 2017. He has published widely in academic journals, professional magazines and public media about educational issues. Most recent books include Finnish Lessons: What Can the World Learn from Educational Change in Finland (2015), Hard Questions on Global Educational Change (2017), FinnishED Leadership: Four Big, Inexpensive Ideas to Transform Education (2018), and Let the Children Play: How more play will save our schools and help children thrive (with William Doyle, 2019). Pasi is a Finnish citizen, now living with his family in Sydney, Australia.
Christine Sälzer is Professor of education and Co-Director of the Professional School of Education at the University of Stuttgart, Germany. Prior to this position, Christine served as Germany's National Project Manager for PISA 2012, 2015 and 2018. In 2016, she graduated from Technical University of Munich with her habilitation on large-scale student assessments as an empirical point of reference for educational policy making. Christine's main research topics are large-scale student assessments, educational monitoring, student absenteeism and students with special educational needs. She focuses on teacher education in her university teaching and works on connecting educational research with educational practice.[Page xxiv]
William H. Schmidt is a University Distinguished Professor of statistics and education at Michigan State University. He serves as director of the Education Policy Center and holds faculty appointments in Statistics and Education. He served as National Research Coordinator and Executive Director of the US National Center which oversaw participation of the United States in the IEA-sponsored Third International Mathematics and Science Study (TIMSS). He has published in numerous journals including the Journal of the American Statistical Association, Journal of Educational Statistics, EEPA, Science, Educational Researcher and the Journal of Educational Measurement. He has co-authored 10 books, including Why Schools Matter (2001), Inequality for All (2012), and Schooling across the Globe: What We Have Learned from Sixty Years of Mathematics and Science International Assessments (2019). His current writing and research concerns issues of academic content in K-12 schooling, including the Common Core State Standards for Mathematics, assessment theory and the effects of curriculum on academic achievement. He is concerned with educational policy related to mathematics.
Michele Schweisfurth is Professor of Comparative and International Education and Director of the Robert Owen Centre for Educational Change at the University of Glasgow in Scotland, where she also leads a course on comparative and international education. Her research interests include ‘best practice’ pedagogies as travelling policies and ideas, global citizenship education, and the experiences of international students. She has published widely on comparative education as a field and methodology, and on the relationship between education and various forms of development in the Global South. She has lived, worked and researched in a wide range of countries in North America, Europe, Asia, and Africa. She is former Chair of the British Association for International and Comparative Education and former editor of the journal Comparative Education.
Iveta Silova is a Professor and Director of the Center for Advanced Studies in Global Education at Mary Lou Fulton Teachers College. Her research focuses on the study of globalization, post-socialist transformations, and knowledge production and transfer in education. More recently, she has been exploring the intersections of postsocialist, postcolonial, and decolonial perspectives in envisioning education beyond Western modernity. She is co-editor of ‘European Education: Issues and Studies’ and Associate Editor of ‘Education Policy Analysis Archives'.
Guillermo Solano-Flores is Professor of Education at Stanford University. He specializes in the intersection of educational assessment, language, culture, and cognition. His research on assessment development, translation, localization, and review integrates reasoning and methods from psychometrics, sociolinguistics, semiotics, and cognitive science. He is the author of the theory of test translation error – a theory of the inevitability of translation error and its impact on validity. Also, he has used generalizability theory – a psychometric theory of measurement error – to estimate the amount of measurement error due to language factors in testing. He has advised countries in Latin America, Asia, Europe, the Middle East, and Northern Africa on the development of national assessment systems and the translation and cultural adaptation of assessments. Current research projects examine formative assessment in linguistically-diverse science classrooms and the design of illustrations as visual accessibility resources in computer-administered tests for linguistically-diverse student populations.[Page xxv]
Amelia Marcetti Topper is an Assessment and Evaluation Specialist in the University of Rhode Island's Office of Student Learning, Outcomes Assessment and Accreditation, and an education consultant. She currently works with undergraduate, graduate, and general education program faculty to advance an effective and meaningful university-wide assessment process, and has also managed and collaborated on numerous quantitative, qualitative, and mixed-methods studies examining student access and success for the U.S. Department of Education and other governmental and non-governmental organizations. Her independent research draws on critical and human development frameworks to examine the conceptualization and measurement of student learning and outcomes at the institutional, national, and global levels. She completed her PhD in Education Policy and Evaluation at Arizona State University, and her dissertation received the American Educational Research Association's Division J (Postsecondary Education) 2016 Outstanding Dissertation of the Year award.
Lucia Tramonte is Professor of Sociology at the University of New Brunswick (UNB). Her research focuses on comparative education, equity, and equality in educational systems. She works on large-scale international assessments from two perspectives: she analyzes existing data to tease out inequalities and inequities associated with access and transition in education; and she designs contextual questionnaires, measures, and new tools. With Dr Jon Douglas Willms, she designed and developed the framework and questionnaires for the contextual assessment of 15 year olds, in and out of school, and the statistical analyses for the national and international reports of PISA for Development, an initiative for low- and middle-income countries aimed at tracking international educational targets in the post-2015 UN framework. As the Co-Director of the Canadian Research Institute for Social Policy (CRISP) at UNB, she led the analytical work on the Successful Transition Project for Human Resources and Skills Development Canada (HRSDC). Since 2004, she works internationally with large organizations like the OECD, AFD, and Unesco IIPE, national governments, and universities on questionnaire construction, secondary data analysis, measurement, and multilevel modelling of cross-sectional and longitudinal data.
David A. Turner is a Professor at the Institute for International and Comparative Education, Beijing Normal University. He gained his PhD from the University of London Institute of Education in 1981. He is a Fellow of the Academy of Social Sciences in the UK. His research interests include higher education policy, technical and vocational education, quality assurance, and leadership in international contexts. He has written dozens of scholarly articles and a number of books, including Theory and Practice of Education (2007). He has been a consultant to Ministries of Education in the Slovak Republic and Mexico. He has lived and worked in the UK, China, Japan and Mexico, as well as being an invited lecturer in conferences and institutions in many other countries. His book, Theory of Education (2005), was awarded the World Education Fellowship Book Award in 2007.
Fons J. R. van de Vijver is Professor Emeritus in cross-cultural psychology at Tilburg University, the Netherlands, and has an extraordinary chair at North-West University, South Africa, and the University of Queensland, Australia. He is senior researcher at the Higher School of Economics in Moscow, Russia. He has (co-)authored more than 550 [Page xxvi]publications, mainly in the domain of cross-cultural psychology. The main topics in his research involve bias and equivalence, psychological acculturation and multiculturalism, response styles, translations and adaptations. He is the former editor of the Journal of Cross-Cultural Psychology and serves on the board of various journals. He is a former president of Division 2 (Assessment and Evaluation) of the International Association of Applied Psychology, the European Association of Psychological Assessment, and Past-President of the International Association for Cross-Cultural Psychology. He has received various national and international prizes for his work. He was/is consultant to various large-scale international assessment projects.
Adriana Viteri is an economics consultant at the Inter-American Development Bank (IDB). Before joining the IDB, she worked as technical specialist at UNESCO at the Latin American Laboratory for Assessment of the Quality of Education (LLECE), where she took part in the organization of the fourth learning assessment study implemented across Latin American and Caribbean countries. From 2014 to mid-2018, she was responsible for the technical assistance to international teams for all the activities regarding the elaboration of several studies, such as comparative large-scale assessments and national reports. She has worked in both public and private sectors, as well as in civil society and international organizations, primarily performing work focused on quantitative topics.
Ming-Te Wang is a Professor of Psychology and Education and a Research Scientist in the Learning Research and Development Center at the University of Pittsburgh. He received a doctorate in Human Development and Psychology from Harvard University. His research program aims to inform practice and policy that improve human learning and development and address educational disparities across childhood and adolescence and in school and family contexts. His current research is centered around two primary domains: (a) creating supportive, responsive, and inclusive learning environments that can buffer students’ stress, foster engagement and resilience, and support positive development; and (b) elucidating the mechanisms and processes by which inequalities are propagated in learning environments, especially for students from disadvantaged backgrounds. His specific research interests include achievement motivation and engagement, racial/ethnic and gender disparities and biases, school/classroom climate and school discipline, and parental involvement in education and ethnic-racial socialization.
J. Douglas Willms is the President of the Learning Bar Inc. He is also the President of the International Academy of Education and a Member of the US National Academy of Education. From 2002 to 2017, he held the Tier 1 Canada Research Chair in Literacy and Human Development. Since receiving his PhD from Stanford in 1983, Dr Willms has published over 300 research articles and monographs pertaining to child development, children's health, youth literacy, the accountability of schooling systems, and the assessment of national reforms. He and his colleagues designed the Early Years Evaluation (EYE), an instrument for the assessment of children's early developmental skills, the OurSCHOOL evaluation system for the continuous monitoring of student outcomes, and Confident Learners, a whole-school literacy program. Dr Willms developed the assessment framework, Educational Prosperity, which several countries are using in their capacity-building efforts and for the development of educational policy.
Dominic Wyse is Professor of Early Childhood and Primary Education at University College London (UCL), Institute of Education (IOE), and Academic Head of the Department of Learning and Leadership. Dominic is a Fellow of the Academy of Social Sciences (FAcSS), incoming Vice-President, then President (2019–21) of the British Educational Research Association (BERA), and a fellow of the Royal Society for the encouragement of Arts, Manufactures and Commerce (RSA). The main focus of Dominic's research is curriculum and pedagogy. Key areas of work are the teaching of writing, reading and creativity (e.g. How Writing Works: From the Invention of the Alphabet to the Rise of Social Media. Cambridge University Press). Dominic has extensive experience of funded research projects which he has disseminated in numerous peer-reviewed research journal articles and books (e.g. his research paper: ‘Experimental trials and what works? in education: The case of grammar for writing’ (2017)). These books include major international research volumes for which he is the lead editor (e.g. The BERA/SAGE Handbook of Educational Research (2017) and The SAGE Handbook of Curriculum, Pedagogy and Assessment (2016)), and bestselling books for students, teachers and educators (e.g. Teaching English, Language and Literacy (4th edition, Routledge, 2018) and A Guide to Early Years and Primary Teaching (Sage, 2016). He has been an editor, and on the editorial board, of internationally recognised research journals. From 2012 to 2018 he was an editor of the Curriculum Journal, one of the journals of the British Educational Research Association (BERA).[Page xxvii][Page xxviii]
Rui Yang is a Professor in the Faculty of Education at The University of Hong Kong. With nearly three decades of academic career in China, Australia and Hong Kong, he has gained extensive experience and contributed to leadership, with an impressive track record on research at the interface of Chinese and Western traditions in education. He has established his reputation among scholars in English and Chinese languages in the fields of comparative and international education and Chinese higher education. Frequently called on to deploy his cross-cultural knowledge and expertise globally, his international reputation is evidenced by his extensive list of publications, research projects, invited keynote lectures in international and regional conferences, leadership in professional associations and membership in editorial boards of scholarly journals. His research bridges the theoretical thrust of comparative education and the applied nature of international education.
Pablo Zoido is education lead specialist at the Inter-American Development Bank (IDB), where he works to improve the education systems in Latin America and the Caribbean. Before joining the IDB, Pablo worked as an analyst at the Directorate for Education and Skills of the Organization for Economic Cooperation and Development (OECD), where he provided advice to governments and education stakeholders on how to use assessment and evaluation tools such as the Programme for International Student Assessment (PISA) to improve the quality, equity and efficiency of education systems. Pablo has published in academic journals in economics and education on issues ranging from democratic checks and balances, the informal economy or educational equality and opportunity to learn.
The field of Comparative Education research continues to produce a large volume of studies about all aspects of the practices of education in different cultures. One purpose of this new handbook is to critically examine the quality of claims about cross-national discoveries about learning and instruction. Another purpose is to provide guidance to anyone tempted to seek knowledge about educational practices in other nations. The rationale for this new handbook is based on the editors’ belief that comparative studies in education are frequently used to influence government and education policies at both national and international levels but that the interpretation of evidence by policy makers may not always be reliable, whereby authors and educational policy makers do not always share common understandings of the content of the studies that are the source of their discussion.
Other scholars have assembled syntheses of Comparative Education research and there are several excellent textbooks, encyclopedias and handbooks of international education that introduce the history and knowledge base of Comparative Education. Each of these outputs has a particular strength unique to the editors and writers, who, along with those who write for this handbook, provide evidence of great diversity in approach to the study of educational practices in different cultures.
The volume will critically assess the status of research methodology and knowledge of educational practices in the field of Comparative and International Education from multiple points of view. The authors of the chapters have been chosen for their expertise in methods of research and for their specific expert knowledge of disciplines such as economics, sociology, psychology, educational policy, philosophy, and political science. The authors have cast a critical lens on how curriculum, assessments and policies are organized today in Comparative Education research. They will discuss theoretical diversity within the discipline and examine integrity and intellectual coherence with a view of guiding future research. This assessment will provide practical guidance for students and experienced researchers for conducting future research about educational policies and practices across countries.
The editors of this handbook were stimulated to produce this volume by the expansive application of large-scale survey research methods to measuring student achievement in recent years. The study of educational practices around the world, once limited to observational descriptions like travel reports, has been injected with a large number of repeating ‘empirical’ surveys of students, teachers and parents. While these studies compose a small segment of the total number of publications about comparative education, they have received significant attention by policy makers in nearly all countries. By the turn of the 21st century, while survey methods had matured to reach high levels of quality in operationalization, the results from these studies were often at risk of oversimplified presentation and containing many hidden assumptions about student responses to self-reported surveys. This handbook was initiated by a belief by the three editors that a significant number of misunderstandings were to be found in public media and [Page xxx]among academic researchers about how to interpret studies about the nature and the practice of education in other nations. Handbooks such as these provide a source of direction for other researchers about what is important or what has been shown to be of less value. They contain definitions of complex areas of analysis and critical reviews of the current state of research that may not always be definitive but have authority in the excellence of the quality of thinking represented in their pages. This handbook was developed within this tradition of exploring the current state of knowledge in how we learn from one another in cross-national studies.
Who is the audience for a new handbook? We believe that novice researchers in every country have easy access to statistical and qualitative data about education in other countries because of the access to all forms of information allowed by modern technology. The number of such researchers is large and growing. From an organizational point of view, the field of Comparative Education is defined by the members of more than 70 professional associations and journals of Comparative Education around the world who publish in journals such as Compare, Comparative and International Education and Comparative Education, and by courses on International Comparative Studies taught in the world's leading universities. They, and their instructors, could benefit from listening to the wisdom of other researchers, as contained in this handbook. The editors do not claim that the authors of this book have reached final agreement on the complex issues of how educational knowledge is obtained; instead they represent a sample of the approaches to knowledge building.
Significant divisions in philosophies of knowledge are apparent among the authors of published articles about international comparative studies. The diversity of beliefs is especially large between those who conduct or use the results of large-scale survey research and those who examine the broad aspects of global economic, political, and educational systems through participatory observation or examination of historical records. Healthy debate of the tendency of government policy makers to rely on results of large-scale comparative research studies to drive educational policies has stimulated new research efforts to more clearly define the meaning of survey results. The history and consequences of different approaches to Comparative Education has been noted in several systematic reviews by leaders of the field of Comparative Education (Bray, 2010; Cowen, 2014). Researchers with a background in one scientific or historical domain may not be as likely to communicate frequently with researchers in other domains. Therefore, critical analyses of the intent and use of all types of comparative studies will be useful to sort out the possible implications for the best approaches to student learning. By including as many examples of these different approaches, and of critical analyses of each in this volume, we have attempted to address some of these gaps in communication and understanding.
The editors of this handbook worked together across great distances to organize and write the chapters of this handbook. Each editor brought unique professional experience and personal commitment to the project that required broad knowledge of educational content areas, familiarity with scholars in different geographic areas, and who held varying types of ideologies about methods of research found in the field of Comparative Education. These editors identified 55 authors for the 32 chapters who were at least partially representative of a broad spectrum of researchers in the field. Since the authors resided in 13 countries, writing and editing a handbook of such scope was possible because of the internet, which allowed connections between various electronic devices with ease. Communication between authors and editors about the content of the chapters occurred over long distances and over a dozen time zones.[Page xxxi][Page xxxii]
Our goal is to provide a bridge for those who industriously conduct all forms of research on student achievement, attitudes, and performances with those who outline the broader social, economic and political systems that also drive the development of educational theory and policy. The interaction of different approaches to the world's differences in preparing the new generation may inspire young researchers to create a new synthesis of educational knowledge.
Larry E. SuterReferences2010) Comparative education and international education in the history of Compare: Boundaries, overlaps and ambiguities. Compare: A Journal of Comparative and International Education, 40(6), 711–725.(2014) Comparative education: Stones, silences, and siren songs. Comparative Education, 50(1), 3–14, DOI: 10.1080/03050068.2013.871834 To link to this article: http://dx.doi.org/10.1080/03050068.2013.871834(
Methods and Practice in Comparative Education Research: Models, Ontologies and Epistemologies[Page xxxiii]Introduction
This handbook is addressed to educational researchers who are interested in the study of education theory and practice in countries around the world. This introductory chapter is an overview of the handbook chapters and a reflection on the status of research in comparative education in 2018. The editors have had a chance to review each chapter and to discuss the implications of their content for informing future research activities in comparative education. Here, the editors of the volume review the contributions of all authors and integrate their remarks with the other contributions to the field of comparative education. One premise of this review is that the different methods of research that are found in comparative education were established through lengthy development and are supported by networks of scholars who share common experiences and technical knowledge. Thereby, each of these approaches to understanding educational processes across cultures should be acknowledged as legitimate means of investigation and given an opportunity for explanation and debate. The hope of the handbook editors is that, by bringing different approaches to comparative education research together in one resource volume, a better understanding of the goals, methods, and values of all researchers may be gained.
The chapters give evidence that the field of comparative education research is a lively enterprise that is driven by a desire to improve educational practices and equity of access to educational opportunities in all corners of the world. Altogether, the chapters contain an overview of comparative educational research methods, ideologies, and policy uses. The subject matter includes [Page xxxiv]views on the status of the field itself; reviews and analyses of instruction and learning practices across different cultures; commentary on social justice issues such as gender and social-economic equity; and the discussion of the consequences of the dramatic growth in number of large-scale comparative surveys of education. Other topics include cross-national studies of pre-school practices, the content of primary and secondary curriculum, experiences in after-school learning, uses of technology, preparation for science and technology careers, the development of student attitudes, and the preparation of teachers.
The chapters were written by scholars who have extensive experience in conducting international comparative studies of education. Most of the authors are instructors in higher education institutions with specialties in the fields of educational psychology, social psychology, economics, statistics, teaching methods, curriculum, global education, and qualitative methods of research. Collectively, the authors have lived in, worked in and studied diverse cultures around the world. Therefore, comparative researchers are especially sensitive to the differences in living conditions across the globe and frequently discuss the responsibility of scholars to address with consistency the conditions of education in all systems. While many of the authors share common interests about how comparative studies can be useful for gaining knowledge about human behavior, they do not necessarily share common beliefs about how that knowledge can be gained or applied. Indeed, the recommendations for specific research designs are nearly always accompanied by cautions about the dilemmas and paradoxes found from conducting comparative research. Most researchers have a heightened recognition that over-simplification of country differences or inferences about causation could lead to errors in policy guidance.
The different chapters illustrate how comparative education researchers apply differing ontologies and paradigms for constructing their research agenda. We try here to identify the view of these authors and seek to find sources of conflicting conclusions. What can be learned from the accumulation of essays prepared by 57 diverse authors about what constitutes ‘good’ comparative education research (and what is good about it)? What have these chapters contributed to expanding the content and methods of ‘good research’ in comparative education? Have they, for example, defined standards of quality that can usefully guide future research endeavors in a field that includes diverse tribes and personal motivation for engaging in cross-national research (Becher & Trowler, 2001; Schweisfurth, 2014a)? The quantity of comparative education research has increased across all research paradigms. Has the increase in number of cross-national large-scale surveys provided useful frameworks for exchanging knowledge about learning, motivation, teaching practices across diverse cultures? Or has interjecting large amounts of survey data into the field simply created new diversions of arcane methodological topics that increase the complexity of understanding rather than settling issues?
The following discussion is organized by some of the most frequently addressed topics of debate and methods of research presented in this volume: the status of the field of comparative education; the variety of epistemologies found among the authors; the status of cross-national student assessment methodology; the use and abuse of qualitative and quantitative methods; the status of large-scale international research; what constitutes good research; and speculation about the types of studies and theory development that is likely to be conducted in the future.Status of the Field of Comparative Studies
The history, development, and theoretical assumptions of researchers in the field of comparative education were addressed in [Page xxxv]several chapters (Suter, Denman, Carnoy, Wang, Wang& Fischman, and Power). This collection of views is representative of the diverse approaches to research currently practiced because they reflect differences in ideology and basic beliefs in what should be accepted as knowledge of educational practice. Even though the number and scope of empirical studies of education across countries has increased greatly, there is little evidence contained in this volume that standardized statistical surveys have accomplished the hoped-for consensus among educators. More than one author finds, either from empirical examination of country educational practices or from ideological reasoning, that large-scale comparative studies of education tend to be homogeneous by design and therefore are likely to be blind to uniqueness in student learning and motivation in different countries. Some authors argue that qualitative and quantitative empirical observations should be interpreted to prove that countries and individuals do not easily conform to simple generalizations. Yet a significant proportion of comparative researchers continue to approach the study of comparative education using models of scientific discovery that seek to establish generalizations about human behavior. Some analyists in this volume warn researchers to pay attention to the differences in educational outcomes as much as to common features. Others in this volume focus their attention on enduring problems of inequality of opportunity for education between countries and within their borders and express concern that the research models of the field have not yet led to solutions to these social issues.
Some authors raised warnings that educational theory, methods, and policies may be too frequently driven by a few dominant models of explanation developed in ‘Western’ nations without acknowledging the cultural uniqueness of other, specifically ‘Eastern’ cultures. Others point out that the complexity and diversity of a global concept of learning and educational practices may require the development of completely new theories, frameworks, and practices that would maintain the uniqueness of individual cultures while acknowledging that macro changes in world access to information affects all countries.
The review of the scope of published research studies in comparative education over the past 25 years shows that the geographic breadth, theoretical scope, and application of research methods of international comparative studies have all expanded; that evidence gathered by researchers has been applied to support aspects of educational practice in some countries; and that scholarly debates about the meaning of and the value for cross-national analysis has contributed conceptual development and new measurement methods to the general field of educational research.
In a review of the history of research on comparative education, economics and psychometrics, Carnoy describes the changes in focus of educational policies that developed, along with the creation and publication of comparisons of student achievement. He provides a sweeping analysis of the theoretical and policy changes that occurred as the role of education in economic development became more apparent because of results of the empirical measurement of educational outcomes across all countries. He concludes that the empirical measurements from comparative surveys provide useful descriptive information without an accompanying causal understanding of what activities lead to higher student achievement in a given country. The existing studies are limited by their cross-sectional design and by measurement errors of the important educational practices that may be the cause of student performances. Thus, comparative studies as currently practiced do not provide a perfect source of scientific knowledge, but they do provide empirical evidence that enhance the development of new ideas and provide a check on beliefs that were formed without [Page xxxvi]evidence. Empirical evidence gathered by either qualitative or quantitative methods about topics such as the relationship between education and economic growth are most useful when they fail to support claims made by ideological beliefs. Developing a rationale for explaining negative evidence is a stronger guide to new knowledge than is seeking evidence to support an existing claim.Conflicting Theories, Frameworks, Ontologies, and Methodologies
The evolution of the field of comparative and international education continues to reflect ongoing challenges in bringing together the science of measurement (empirical studies) and discovery (qualitative studies). Feyerabend (1987) once argued that the distinguishing contexts of justification and discovery create a necessary blessing in dualism, as the former concerns a systematic approach to explain content and the reasons for accepting it, while the latter tells of a history of a particular piece of knowledge. While positivism may often be associated historically with Auguste Comte (1798–1857), the founding father of positivism and sociology, the nature of comparative education continues to maintain its interdisciplinary approaches, its historical roots in the social sciences, and its breadth and scope in modes of inquiry, suggesting that differences in space, time, reality, measurement, and change are all relative. While much of this handbook problematizes large-scale measurement, the qualitative aspects of this handbook tend to reflect how the field continues to mature and evolve. Park, Lee, and Yang share their current frustration that sound measurement is necessary for comparative education research to realize its importance in stages of educational development and advancement, and Denman reflects on the vacuous ‘shelf’ life of research, contending that there is still much opportunity in building greater robustness and veracity in comparative education research through increased collaboration, discussion, and depth. Turner, Hathaway, and Ferrer-Esteban, Suter, & Mincu actively engage in qualitative and quantitative studies in comparative education, which take ‘mixed methods’ design to the next level by addressing problems of experimental design, sampling, and data analysis by means of application.
Unabated dissatisfaction continues concerning the rate and speed of change for research in comparative studies in education. The field is no different, and most comparative scholars question the likelihood that the field will reach higher levels of understanding and awareness of its significance. Notwithstanding the need to explore how comparative education is taught as a field – with research centers promoting global, international, and multicultural education to help differentiate approaches – there is an increasing body of evidence suggesting that research elevates the potential within the field. Accordingly, the field is not necessarily to delimit, restrict, or control types of comparative education research, but to recognize that there may be limitations to existing methods and techniques in exploring education phenomena, different perspectives, and different discourses. This dissatisfaction is also a driver to excel and help the field to realize its potential. As such, the field will continue to evolve and mature.Measurement Epistemologies
About a fourth of the chapters in this volume contain discussions about qualities of different research methods. The significant attention to aspects of research methodology illustrates how much scholars in comparative education hold varying beliefs about the [Page xxxvii]fundamental basis of knowledge. Those who conduct research in and about cultures outside their own field of residence may be particularly drawn to holding a broad view of the nature of knowledge. Different epistemologies are clearly present in the entire field of comparative education, ranging from empirical positivism to ideological dogmatism. This epistemology of education research has been defined as ‘understanding about its core nature as a scientific endeavor’ (Shavelson & Towne, 2002: 15). A thorough review of methods of research about all forms of education found that even the basic understanding of the nature of knowledge has itself changed as knowledge of human behavior has accumulated and the depth of understanding of human nature has evolved (Shavelson & Towne, 2002).
Thus, the chapters in this volume reflect recommendations of each author about ways to improve methods of observation and reporting of educational practices in settings of widely diverse languages, historical backgrounds, geographic settings, and economic conditions. The methods of observation are an important aspect of comparative research that is continually evolving. One set of research methods attempts to design a common framework of learning and teaching that is applied rigorously in the study of diverse cultures, while other methods accept the diversity of learning and teaching as a given and seek methods to describe them within their own context.
The recent rapid growth of large-scale surveys, acknowledged and documented in these chapters, may have exacerbate differences of opinion among scholars about the source of knowledge. Certainly, the increased number of publications and sophistication of large-scale cross-national studies has increased the need for a larger number of researchers to appreciate the strengths and limitations of survey data collection and analysis. At least ten chapters in this volume (Ainley & Ainley, Praetorius, Rogh, Bell, & Klieme, Law & Liang, Rutkowski & Rutkowski, Sälzer & Prenzel, Cogan & Schmidt, Solano-Flores, van de Vijver, Jude, & Kuger, Wang, Degol, & Guo, and Willms & Tramonte) present a discussion of particular aspects of quantitative research methods in cross-national research. These authors reflect on the measurement of achievement, attitudes toward education, change over time, curriculum, language influence, and social status in large-scale survey research. Recommendations from these authors are based on extensive experience with conducting cross-national research. Their comments include cautions about drawing too many conclusions from cross-national studies as well as clarifications of techniques that improve quantitative analysis.The Application of Qualitative and Quantitative Research Methods
As in all social sciences, research methods may be divided into two research traditions: quantitative research that uses measurement and numbers and qualitative research that emphasizes meaning and words (Wikipedia, 2018; Wyse et al., 2017). A review of the content of chapters in this handbook shows that the comparative study of educational practices around the world cannot be undertaken without an understanding of the limitations and strengths of both qualitative and quantitative research methods. As Schweisfurth points out, making comparisons between countries’ type of educational practices has been underway for centuries (Schweisfurth, 2014b: 15) by means of travel reports. In the field of comparative education today, individual researchers differ in their degree of emphasis on words or numbers because researchers conduct and publish their studies using the traditions under which they were trained, such as the humanities (history and philosophy), the social sciences (psychology, anthropology, [Page xxxviii]sociology, economics, political science), or mathematics (statistics). Each of these disciplines represent a complex collection of belief systems, training periods, and research practices (Becher & Trowler, 2001; Schweisfurth, 2014a). No single set of established rules for research analysis has been consistently applied to evaluation of research in this field (Cowen, 2006).
Some policy makers in educational theory and research have argued that qualitative and quantitative research methods can co-exist in the same paradigm of scientific research in education (Feuer, Towne, & Shavelson, 2002). Yet, for practical as well as basic belief reasons, individual researchers are more likely to be more expert in one technique than the other. Professional life is limited by the time available to learn the rules of applying different models; the peer review process requires sophisticated knowledge and a specific vocabulary of a method; and the types of research question that interest an investigator may not invite the use of one method over the other.
Several chapters in this volume address specific issues in making quantitative measurements valid and reliable (Rutkowski & Rutkowski, van de Vijver, Jude, & Kuger, Willms & Tramonte, Ainley & Ainley, Praetorius, Rogh, Bell, & Klieme, and Power). The authors reliably define research procedures and techniques that quantify cognitive, non-cognitive, and student or teacher behaviors reliably across cultures as well as assess the successes of existing quantitative studies. Some of the authors are skeptical about the ability of student self-reported evidence of their characteristics to be consistent across different cultures (Rutkowski & Rutkowski). Others realize that errors of reporting occur naturally, and they seek to provide guidance for improving the reliability of the survey process in light of potential random errors (Willms & Tramonte and van de Vijver, Jude, & Kuger). The originating psychometricians and policy makers who initiated large-scale comparative research in the 1950s believed that quantitative measurement was preferred to the collection of stories and anecdotes about education. The objectivity of common survey instruments was believed to improve the accuracy of observation (Suter, this volume). However, some areas of educational outcomes, such as writing skills, were found to be too closely intertwined with cultural patterns to be analyzed across countries by a common evaluation framework – although certainly such efforts were made (Gorman, Purves, & Degenhart, 1989).
One chapter in this handbook addresses the meaning and use of qualitative methods directly. Schweisfurth writes that some researchers who apply qualitative methods believe that isolating specific variables, as is regularly done in quantitative research, loses the meaning of educational experiences. She explains that subjectivity is ‘paramount in understanding why students are successful'. Subjectivity would include how education is valued, the meaning of relationships between teachers and learners, and meanings shared between learners. She argues that comparative research must include both insider and outsider cultural perspectives to understand the ‘inner workings and influence of context'. She continues arguing that to understand why people are doing what they do, we need to go beyond what is readable, observable and countable to understand the wider and more subtle workings of context.’ Methods such as ethnography can help the researcher to understand these through the perspectives of the actors involved and to glean why differences between policy and practice and between different settings are evident. Qualitative research may be particularly valuable in comparative studies of education for showing how different contextual factors work together holistically (not simply exist or correlate, and not necessarily work as ‘causal factors'). Since educational practices may be influenced by several different levels of policy making (national, local, classroom, other students), it is necessary to examine the relationships [Page xxxix]between each of these variables to learn how individual policies are interpreted and acted upon. Finally, Schweisfurth points out that the methods of qualitative analysis of cross-national studies have been changing, and she describes how traditional approaches to analysis are being replaced by new revisions. Yet, she says, as others have also proclaimed, both quantitative and qualitative methods may be used together for the better understanding of educational policies.
The research studies reported in this handbook are rarely classifiable as either purely qualitative or quantitative. Although more chapters profess to discuss quantitative methods of data collection and interpretation of cross-national surveys, the content of the chapters often includes uses of qualitative methods, such as case studies, syntheses, or classroom observation, to generate hypotheses or explanations. The analysis of multiple research studies is a qualitative process which depends more upon the interpretation of words than on the discussion and analysis of numerical measurement. ‘Mixed methods’ has become a term that appears frequently in publications. A single research project is likely to apply methods that analyze words and methods to achieve the ability to count actions. Case studies and ethnographic analyses are sometimes summarized by counting frequencies of behaviors, for example. And vice versa, quantitative studies occasionally seek examples from case studies to illustrate the behaviors and overall systems that are captured by single statistical measurement.
As Darwin found in his exploration of finches in the Galapagos Islands, when one species of birds is confined to a single island, over time it is likely to multiply into many species rather than emerge as a single weeded-out species. Thus, it may be true also for research methods in comparative education. Over time, the number of combinations of different approaches to investigations increases as the types of discoveries and problems change.Assessing Student Achievement Across Countries
Large-scale internationally comparative surveys have been conducted at least once in over half of all countries in the world. These assessments of students in primary and secondary schools have been supported by a broad number of funding agencies from individual governments, international organizations (e.g. OECD, UNESCO, World Bank), and private foundations. Average individual country levels of student achievement are no longer confined to nations with large populations or advanced economies. The use of comparative assessments of student performance in countries of lower economic status has increased. Moreover, new comparative studies are being designed while previous studies are still in the process of analysis and interpretation (PISA 2021 and TIMSS, and the PISA new study for lower income countries, for example). Consequently, the body of knowledge about research practices for making comparisons is increasing as researchers learn from experience and develop improved technologies and conceptual frameworks for cross-national understanding. Whether knowledge about educational achievement or school management has increased because of the availability of extensive survey information about students is a question addressed by many of the chapters in this volume.
Researchers, educators, and psychometricians are concerned with establishing the validity and meaning of cognitive assessments designed in one culture and collected in another. The assessments are based on student reports of what they know about a field of study (such as mathematics, science, reading, or geography). Chapters in this volume address topics such as whether student assessments in mathematics, science, or reading are comparable from one country to another, and whether relationships between student school experiences and achievement in one country [Page xl](essentially a large, highly developed country) are of value for policy making in other, perhaps less developed, countries. Several chapters in this volume review the history and growth of large-scale international surveys of student achievement (Carnoy, Rutkowski & Rutkowski, van de Vijver, Jude, & Kuger, Fischman, Sahlberg, Silova, & Topper, Suter, and O'Dwyer & Paolucci). Each author reviews a particular aspect of the growth of these surveys with different analytical points of view, but all acknowledge that the methods of study have evolved over time in response to both policy needs and growth in research knowledge about the measurement and practices of student assessment.
Chapters by van de Vijver et al. and Rutkowski and Rutkowski address specific issues about using cross-national studies to reliably and validly measure student achievement. Van de Vijver et al. conclude that while valid measurement of achievement is theoretically possible, many details in the use of language must be considered when designing and conducting cross-national assessments. The authors conclude that the frameworks and systems of data collection have improved and progressed to higher levels of quality over the years of conducting international comparative assessments. However, the authors caution that differences occur in content coverage, student interpretation of items (because of dependence on self-response), differing perspectives of what education is, and that these perspectives change over time. Thus, making adjustments to measurement methods will be necessary on a continuous basis.
The Rutkowskis note that the use of the large-scale surveys PISA and TIMSS has expanded to as many as 90 countries at once, yet the test items have developed at an international rather than a local level. Thus, the assessment of student performance is less likely to be valid. They point out that these studies have progressed over the years and that future changes will be necessary as a larger number of countries are engaged. In a similar vein, O'Dwyer and Paolucci note that in studies of the relationship between teaching practices and student assessment the relationships found in large-scale surveys are weak and contradictory; probably, they say, because the assessments do not adequately reflect the unique educational experiences of students at the local level.
The discussion of the expansion of student assessment comparisons in Latin America by Viteri and Zoido illustrate the significant influence of global agencies in urging governments in all countries to apply methods of assessment and evaluation for the purposes of accountability. Their presentation is a useful illustration of how individual countries and regions may adopt portions of educational assessments from other countries but adapt them to the specific circumstances of the local cultures.Assessing Attitudes
While student achievement has been defined mainly by student performance on a test of skills in mathematics, science, reading, or technology, also of concern to educators are the ‘non-cognitive’ attributes or attitudes of students. As the authors of both chapters on attitudes in the volume state, comparative studies have shown a persistent across country relationship between a student's positive attitudes toward a subject and their test performance on that subject. Wang, Degol, and Guo explore the differences in both theory and evidence between Eastern and Western cultures in how student motivation is linked with achievement. They employ the ‘expectancy value’ theory as a means of untangling the relationships between attitudes and achievement. Expectancy beliefs are those that are related to an individual's expectations for success in the future and include a self-concept of ability. Task values, on the other hand, include attainment value, utility value, intrinsic value, and cost associated [Page xli]with a particular subject domain. They cite research studies that have tested the relationship of these motivational concepts to achievement in Eastern and Western cultures and find that the concepts apply equally, more or less, but that the task value measurements are especially important for predicting the achievement levels of females. They also cite evidence that expectations and task goals developed early in childhood are likely to predict the level of motivation later in life. However, in most cultures, interest in mathematics tends to decline between primary and secondary school, especially for boys.
Likewise, Ainley and Ainley discuss the development of motivational constructs that have been used in international comparative studies of reading, mathematics, and science. Their chapter pays particular attention to the achievement paradox first observed in TIMSS, in which the average level of achievement of a country is negatively related to the average level of interest in a subject; whereas, within countries, the relationship between interest and achievement is positive. Attempts to explain this paradox by examining biases in student reporting on the various attitude and achievement scales were unsuccessful. Ainley and Ainley then propose that cultural differences in the conception of attitudes explain the paradox. They write:
In sum, part of the explanation of the attitude–achievement paradox lies in the ways that macro-cultural values provide a context within which students develop their attitudes in relation to schooling domains. The examples described here locate these cultural context effects in the degree to which science and technology have been adopted and are embedded in the culture, more particularly, students’ access to science-relevant learning opportunities.
This observation of how the macro-economic and social conditions of a country affect how a student creates interests and goals is a theme that is repeated throughout this volume of international differences. Individuals react to the culture around them in ways that affect not only their achievement but also in the social-psychological processes that are a part of the determining features of their performance.
Some of the country-level characteristics that affect students’ career choices are explored in greater detail in the chapter by Suter and Smith. Their analysis of PISA 2015, for example, shows that the relationships between the country-level average of students’ interest in specific careers, such as scientist, engineer, or in the health fields, and the average achievement levels are positive for some occupations and negative for others. Their analysis of the interest of 15-year-olds in science and mathematics occupations or as natural scientists or software technicians, is higher in countries with high general achievement levels while students who are most likely to choose a career in engineering or technology professions are in lower achievement countries. Thus, the relationship of both interest levels and achievement levels of students to science career choices varies by specific occupation and the country's economic development level. These observations and the investigations of Ainley and Ainley suggest a rich area for future research on the aggregate conditions of countries that affect the performance and career choices of students.Equality of Opportunity and Social Justice
Equality of opportunity encompasses many types of social and economic circumstances. Not all of these have been addressed specifically in this handbook on international comparisons in education. But several chapters address the differences in student achievement and attitudes as they are related to their family social status and the economic conditions of the countries in which they reside. Some of the discussions are about individual-level status and others are discussed at the global level of economic differences between [Page xlii]countries and regions (e.g. Suter & Smith on careers, Willms & Tramonte on status measurement, Carnoy on economic development, and Lee on the balance of power).
The chapter by Willms and Tramonte is a lesson on how to use, apply, and interpret scales of social status in large-scale surveys. The scales are found in most large-scale surveys of student performance. The measure of social status includes the educational level of parents and their occupation. The income level of the student's family is also a potential measure of social status but is rarely collected in surveys of students in schools because students cannot be expected to report such sensitive information reliably. This chapter is useful to those who wish to examine survey data from the point of view of assessing differences in access to education by students at different levels of social status. The authors examine the characteristics of the scale itself, its construction, and how cut-off points might be created to measure sub-populations, such as those in poverty. Status levels have been used to set attainable goals for aggregates of students (those classrooms, schools, districts, or larger geographic units) but not individual students. The levels may assist in assessing the equity of provisions among advantaged and disadvantaged groups and the equality of outcomes. Status levels may also be useful for designing interventions aimed at reducing inequalities.
In his chapter on enduring issues, Wing On Lee evaluates the overall reduction in differences in educational opportunities. He particularly assesses successes of the efforts of the international organizations to set goals and achieve reductions in inequality. He finds that little has changed over the years because leaders have had a narrow vision of education, they have ignored childhood care (see also Raikes, Davis, & Burton, this volume), and they do not sufficiently emphasize the quality of learning. He finds that females have not achieved equality and says that investment in education worldwide is insufficient. Studies of the economic conditions of countries and the relationships between achievement scores and parental background conducted by PISA have shown that equity is higher in some countries than in others. Lee seems to believe that the evidence across countries illustrates that those countries which have high student achievement levels are most likely to grow economically and have fewer inequalities. His use of large-scale research to refine his understanding of the size of country-to-country differences and their relationship to underlying economic characteristics is worthy reading. Some of the same points are made in the chapter by Colin Power, who writes about how international organizations use ‘indicators’ to point out problems underlying a country's social issues. He writes that indicators only suggest where to look for areas of need but are not agents of the political will to make the necessary changes. Like Lee, Power uses available empirical data about education on a worldwide basis.Content of Education
Some specific educational activities of primary and secondary schools that are discussed in eight chapters. The chapters discuss preparation for early grades (pre-school), the issue of considering curriculum differences in analyses of achievement, the amount of after-school experiences, the use of technology, the types of instruction, differences in preparation for science occupations and careers, preparation of teachers, and preparation for careers in higher education. Only one of the chapters includes a statistical analysis of large-scale survey data while the others review case studies or international organizational policies toward educational practices and monitoring of worldwide changes. No author found evidence of improvements in the level of access to educational opportunities (although some research methods for [Page xliii]such discovery are discussed by Willms and Tramonte).
The analysis of cross-national differences in student interest in science careers finds paradoxes in relationships between interest and opportunity. For example, students in some countries (such as Peru) had a high level of interest in science occupations but were offered few opportunities for science occupations compared with those offered by other countries. On the other hand, highly advanced countries, such as France, have a lower proportion of students expecting to seek careers in science and technology.
Wyse and Anders examined whether and how international comparative research has paid attention to primary education curriculum as a matter of analysis. They point out that early discussions and analysis of curriculum have been qualitative and focused on the development of language itself through proper curriculum design. Later studies of curriculum design and presentation in schools used a quantitative design. The study, however, shows that the adoption of common concepts of primary curriculum that can be applied across countries has yet to be developed and used by educational researchers. In a similar vein, Cogan and Schmidt argue that many recent comparative studies fail to take into account the changes in opportunities to learn based on the content of the classroom curriculum. They had been engaged with measuring curriculum coverage of many countries in the TIMSS 1995 and in later PISA surveys. They believe that student achievement cannot be properly understood until the opportunities for students to learn specific content is accounted for in all large-scale studies of student performance and that measurement of the effects of student time devoted on a topic requires measurement of more than one point in time.
Two chapters include analyses of practices and change in how well young children are prepared for formal schooling around the world. One by Hathaway (chapter 24) is a case study analysis of pre-school practices and policy in British Columbia, Canada, and Singapore to learn about policies regarding transition into public education. The other by Raikes, Davis and Burton reviewed global policies of early childhood development, examining the policies and indicators conducted by UNESCO, the World Bank and the United States to improve early child development. Both chapters use cross-national experiences to argue for how to develop an adequate model for analyzing early childhood preparation for school and neither author believed that current statistical monitoring is sufficient to inform public policy about early child preparation for schooling.
Akiba, Howard, and Liang conducted a literature review of studies of teacher learning communities (such as ‘Lesson Study’ around the world). They reported that most published papers were published in a few English-speaking countries (although some were published in Chinese) and limited the analysis to a single country. They found an insufficient exchange of ideas across countries to develop the knowledge of learning communities into a valuable tool for education. They also examined a recent large-scale survey of teacher practices (TALIS by OECD) to describe country-to-country differences in teacher discussions about students. They reported that teachers in countries that reported concern about individual student performance were also more likely to exchange ideas with other teachers about how best to assess the students. Their statistical portraiture of country differences in the extent to which teachers have time to exchange teaching methods with each other is astounding and worthy of further study.
The use of computers and other forms of technology in classrooms has been a subject of comparative research since the birth of small-scale computing in the 1980s. Law and Liang (this volume) reviewed the results of 12 international comparative studies by the IEA and OECD on technology use. They find that early studies were concerned about the preparedness of students and schools to [Page xliv]use ICT. Later studies focused more on the pedagogical adoption patterns of schools and teachers and measured aspects of student knowledge of computer and information technology. They report that educational technologies have changed the priorities of student-centered learning to include a greater amount of collaboration and inquiry. They propose that future ICT studies examine the amount of progress in educational transformations of school systems.
Educational experiences outside of formal schooling occurs widely around the world in informal settings, such as museums and libraries, but also in organized study programs. Asian societies are especially likely to provide some form of tutoring to students who are preparing for exams. Feng and Bray (cp. 20) provide a thorough review of the types of outside-school-time (OST), sometimes called “shadow education”, that students in different countries have participated. They review the various ways that students may have experiences that affect their educational growth and they outline specific research areas that must be addressed in studying the significance of “shadow-education”.Conclusion: Influence of International
The authors of this volume are more likely to see the study of other cultures for the purposes of learning about ‘good educational practices’ to be a matter of contention than a guide for immediate policy implementation. Establishing causality with cross-national comparisons is difficult because of the wide range of conditions within countries that are not identified a priori and therefore not included in analyses. Much of comparative research leads to possible dead ends and mis-direction. It will be a challenge for future researchers to invent better methods of identifying the specific properties of countries that provide useful insight about practice.
The introduction of survey research into the province of cross-national comparison policy making has created a small industry of method development and policy development. The OECD and IEA promote more analysis. Many countries have participated, and the number of student assessments continues to grow. The impact of these studies on educational policy may be debated but their influence cannot be completely ignored. National leaders and education policy makers will need to become well informed about the strengths and limitations of large-scale quantitative studies (see Power, this volume).
Will the differences in interpretation of observations be solved by a more rigorous application of the scientific method? Testing hypotheses was not found to be a common activity by the authors of this volume. The acquisition of knowledge occurs through the exchange of ideas and argument more frequently than by the use of clearly aligned data.
What constitutes ‘good’ comparative education research? One of the leaders of the field of comparative education argues that the field is an academic exercise without the need to be relevant to the policy world. He wrote:
Comparative education as an academic field of study does not fix educational things when they are broken; it does not service the needs of Ministries of Education; it is not a branch of policy studies; it is not reducible to sociology, or to political science, or to history; it has not yet succumbed to the one true way of a specified methodology; nor has it accepted the seductive but corrosive position of claiming for itself disciplinary status in the terms defined so carefully by London philosophers of education. (Cowen, 2006: 592)
The use of any form of empirical approach to evaluate the effectiveness of social programs in education, health, or other systems has been found to be extremely difficult (Rossi, 1987). After years of evaluating United States social interventions during the 1970s and 1980s, Peter Rossi invented the ‘metallic laws of evaluation.’ He wrote in the Iron Law that the possibility that a social program [Page xlv]would have a positive effect is zero (Rossi, 1987). He later revised his Iron Laws of Evaluation to state that he believed that the evaluations of social programs were generally not believable (he wrote, ‘The findings of the majority of evaluations purporting to be impact assessments are not credible,’ Rossi, 2003: 5). While the wisdom of a researcher and evaluator developed four decades ago may appear dour today, Rossi's expressions of distress with the capability of empirical research to reach solutions should be taken seriously by those who are engaged in cross-national research. Educational research in general must contend with an extremely wide range of human issues and the conceptual frameworks research methods and techniques for studying student learning, teaching practices, or psychology, sociology, political science, economics, computer science, engineering, the physical sciences and the humanities such as history and philosophy. Each of these fields has its own set of standards and social networks that interact with the research in education. Moreover, the corpus of knowledge about education resides not only in university settings but also in government and non-governmental agencies that monitor, evaluate, and set goals for educational institutions. Therefore, establishing a single definition of what is ‘good’ in the field is nearly always a result of a compromise of goals and a recognition of the significant social networks that ultimately receive the results of the research, either passively or actively engaged in using the research results.
Recommendations for how to set standards for educational research can be found in many academic and policy-making resources. Publishing companies, such as Sage, have created ‘handbooks’ in nearly every field of science and social science to synthesize the field of research methods (such as The BERA/Sage Handbook on Educational Research Methods; Wyse et al., 2017). Shavelson and Towne, writing for a panel of the US National Academy of Sciences, which investigated the status of educational research to respond to attacks on it by politicians, claim that even the rigors of the scientific method are not agreed upon everywhere. The result: changes have occurred in understanding the elements of human nature and in how human knowledge grows and develops over time (Shavelson & Towne, 2002).
In the history of comparative educational research, several writers have recognized that examining other cultures may involve observational biases and that means must be established to improve the quality of these observations. In 1848, Julian de Paris argued for producing an ‘education science’ (Gautherin, 1993). In 1900, Sir Michael Sadler recognized observers of educational practices were not likely to all agree (Sadler, 1900/1964). In the 1960s, Noah and Eckstein again encouraged the field to adopt a ‘scientific method’ approach to the study of comparative education (Noah & Eckstein, 1969). At the beginning of the 21st century, the American Academy of Sciences sought to define the components of a ‘scientific discovery’ in education rather broadly (Shavelson & Towne, 2002) with the understanding that all rigorous research – quantitative and qualitative – embodies the same underlying logic of inference (King, Keohane, & Verba, 1994). This inferential reasoning is supported by clear statements about how the research conclusions are reached: What assumptions are made? How is evidence judged to be relevant? How are alternative explanations considered or discarded? How are the links between data and the conceptual or theoretical framework made?
A high-quality research study depends heavily on how well the research question itself is formulated. Well-respected research studies have formulated a research question that addresses a specific need for information. The questions must be clearly defined and articulated to ensure that they address all aspects of the issues under investigation. Its meaning, implications, and assumptions should be clear. The research problem should reduce broad questions into their constituent [Page xlvi]parts to make the problem more accessible to other research. The research question should reflect the content of relevant literature. All research questions should be capable of being found false and to be capable of being answered through practical data collection activities. The statement of purpose should not be biased toward the outcome of the research to avoid any tendency to predispose the outcome of the results.
Although none of the chapters in this handbook contains a recommendation for standards of ideal qualities in comparative educational research, the authors have provided several different models of good research design and execution in this field. For example, those supporting the use of empirical research methods of large-scale surveys, case studies, and rigorous observation of student activities in multiple cultures have adopted a model of discovery based on scientific methods. In scientific discovery, a claim is made about an educational phenomenon that is supported by previous research evidence or derived from theory and that claim is examined with new evidence such as from survey data, case study, or rigorous observation. On the other hand, some chapter authors argue that many assumptions made by educational researchers are inappropriate and should not be used to guide policy in local educational systems. Their beliefs are that standards adopted from what is known have been taken from dominant world cultures that do not accurately represent the meanings of educational interactions as understood by the local cultures (Lee, this volume).
Some proposed goals for international comparative educational research that may be taken from the chapters in this volume are these:
- First, contribute to the accumulation of knowledge of educational practices and methods of research in the field of comparative education;
- Second, provide guidance for educational policy and practice;
- Third, challenge, test, and replicate previous research studies;
- Fourth, foster the development of new theories and hypotheses about social, psychological, or economic conditions and educational achievement;
- Fifth, take into account ethical considerations about how they report on the behavior of others. Park suggests that, ‘our desired goal is not only external accountability but also the justification of moral coherence, integrity, and emotional satisfaction of the work performed.’ He, like others in the field, is especially sensitive to the ethics of explaining the behavior of others from the point of view of an outsider.
Following a period of public discussion of education following the release of international comparisons of student achievement in the 1990s, some officials of government believed that international comparisons were more authoritative than other forms of educational research. Thus, the US National Research Council (NRC) was asked to investigate whether the methodologies of large-scale surveys had become sufficiently sophisticated as to improve understanding of educational practices. In their conference discussion, the question was raised as to whether countries might ‘stand as existence proofs for the possibility of higher levels of achievement'? The reporters of the conference noted that countries may differ in so many ways that a simple interpretation of cause and effect would not be practical. Still, they believed that the studies could provide insights that would lead to developing hypotheses about more effective educational practices and they could be tested for feasibility (Porter & Gamoran, 2002: 5).The Future
The discussions found in this handbook provide some clues to predicting what types of changes in the field of comparative education may occur and how the field is likely to affect both policies and educational research [Page xlvii]topics in the future. The extensive and still expanding application of large-scale quantitative assessments of student achievement will continue and are likely to expand to new countries. The psychometricians working on these studies are likely to develop new techniques (since new surveys are already in the planning stages), and as long as the results of these studies are adopted by those in positions of power to enact policies based on them, they will cause debate.
So far, nearly all large-scale surveys are cross-sectional analyses of student performance measured at one point in time. While these studies permit the measurement of changes in overall achievement of a population over time, they do not allow describing the amount of change in student performances or attitudes within the same student over time. Some countries conduct their own individual longitudinal studies of student changes, but few of these allow comparisons of the nature of these changes across countries. Conducting longitudinal studies that follow up individual students over several years will be necessary to draw causal inferences. Longitudinal studies are very complex to administer, requiring methods to track individuals, and they are expensive. New methods of monitoring may be possible with the development of appropriate technological tools.
Future studies are likely to use new forms of observation, such as capturing classroom instruction through video methods (see Praetorius, Rogh, Bell, & Klieme, this volume). New technologies promise to provide cleaner information about observations of classroom practices and they will also bring new issues of interpretation.
The future of research in comparative education is likely to continue discussions of the consequences of gradual globalization as it impacts local institutions of education. The development of international organizations that monitor the condition of educational institutions in all parts of the world may lead toward better understanding of how economic growth and political stabilization is affected by the provision of education.
The future will likely continue to include strong differences in views about how to apply the observations from cross-national studies to educational policies. Diversity is more obvious than similarity. Researchers examining either quantitative or qualitative descriptions of education in different countries cannot avoid noting that the combination of many forms of organization and behavior exist.
Ideological and political issues will also continue to be discussed in the literature of comparative education. Some thinkers see individuals as integrated members of a large world system, while others see the world system as a problem that dominates little people. The problem of lagging economic development and how it has affected education will continue to be a dominant concern in the studies of comparative education researchers.
A continuing theme among nearly all the authors of this volume is that some aspects of educational and individual student characteristics are common across countries, but that the differences between countries in achievement and in cultural factors that lead to that achievement are dominating conditions that comparative researchers must find ways to incorporate into their analysis of global education.References2001) Academic Tribes and Territories. Buckingham, UK: The Society for Research into Higher Education and Open University Press. Downloaded from www.mheducation.co.uk/openup/chapters/0335206271.pdf (10/03/2018).& (2006) Acting comparatively upon the educational world: Puzzles and possibilities. Oxford Review of Education, 32(5), 561–573. ISSN 0305-4985 (print)/ISSN 1465-3915 (online)/06/050561–13. [Page xlviii](2002) Scientific culture and educational research. Educational Researcher, 318, 4–14. http://doi.org/10.3102/0013189X031008004., , & (1987) Farewell to Reason. London: Verso.(1993) Marc-Antoine Jullien de Paris. Perspectives: Revue trimestrielle d'éducation compare, 23(3–4), 783–798. Downloaded from http://www.nie.edu.sg/research/publication/cieclopedia-org/cieclopedia-org-a-to-z-listing/jullien-de-paris-marc-antoine (02/05/2017).(1989) The IEA Study of Written Composition I: The International Writing Tasks and Scoring Scales. College Composition and Communication, 40(2). DOI: 10.2307/358148., , & (1994) Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton, NJ: Princeton University Press., , & (1969) Towards a Science of Comparative Education (Part II, pp. 85–122). London: Macmillan.& (2002) Methodological Advances in Cross-National Surveys of Educational Achievement. Washington, DC: National Academy Press.& (Eds.) (1987) The Iron Law of Evaluation and other metallic rules. Research in Social Problems and Public Policy, 4, 3–20. Retrieved from www.gwern.net/docs/sociology/1987-rossi (03/10/2018).(2003) The ‘Iron Law of Evaluation’ reconsidered. Paper presented at the AAPAM (American Association for Public Administration and Management) Research Conference, Washington, DC. Retrieved from http://welfareacademy.org/rossi/Rossi_Remarks_Iron_Law_Reconsidered.pdf (10/05/2018).(1900/1964) How far can we learn anything of practical value from the study of foreign systems of education? Comparative Education Review, 73, 307–314.(2014a) Among the comparativists: Ethnographic perspectives. Comparative Education, 50(1), 102–111.(2014b) Data and dogma: Knowledge and education governance in the global age. Comparative Education, 50(2), 127–128. http://doi.org/10.1080/03050068.2014.892754.(2002) Scientific Research in Education. Washington, DC: National Academy of Sciences. http://doi.org/10.17226/10236.& (Wikipedia (2018) Quantitative and Qualitative Research Methods. https://en.wikipedia.org/wiki/Quantitative_research.2017) The BERA/SAGE Handbook of Educational Research. London: Sage., , , & (Eds.) (