Educational Assessment: Tests and Measurements in the Age of Accountability

My goal in writing this book is to provide students with an understandable and useful interpretation of the critical information related to educational measurement. There has never been a time when it has been more important for educators to have an understanding of educational assessment and measurement. Educational accountability has taken root in our nation, and our political leaders are all looking for “scientific” documentation of our successes. It is no longer prudent or even possible for educators to ignore this national Zeitgeist.

Our political leaders have decided that the high-stakes testing of public school children is the best way to provide this documentation. For that reason, educational testing now provides one unifying theme in the lives of all American educators. Today the testing industry in this country has literally become a billion-dollar business. No longer is it just the students who can “flunk” a test. In today's schools, educational measurements are all too often the driving force behind curriculum decisions, hiring practices, salary and benefit packages, and the politics of school boards. The annual release of average test scores to the news media has changed how schools operate and what goes on in the classrooms.

During the thirty years that I taught educational testing to graduate students, I was never completely comfortable with the textbooks that were available. It seemed that every year I adopted a different textbook, and every year I was never completely satisfied with my choice. It was my impression that the books were either too pedantic and designed to make students dislike this exciting field, or they were vacuous and devoid of the central core of knowledge required to develop the next generation of educational specialists and leaders.

[Page xxiv]This new book is the outcome of my quest for a readable and highly engaging textbook that does not compromise the core principles of measurement. In that regard, there are chapters covering traditional topics such as reliability, standard error of measurement, validity, classroom test construction, performance assessments, standardized achievement tests, item analysis, and the application of Item Characteristic Curves (ICC) for Differential Item Functioning (DIF).

Beyond these traditional concerns, this book is grounded in the real world of public schools and students. It is not designed for psychology or sociology courses but is targeted to meet the needs of educators and future educational leaders. This school-based focus was accomplished in several ways. As a case in point, a school focus pervades the numerous examples that describe the implications of measurement decisions on the lives of students and teachers. As another, more than 90% of the 860 references cited in this book are from either the educational literature or education-focused agencies.

Heretofore, textbooks in educational measurement have lost students in scientific discussions of the arcane and complex principles of psychometrics. Unlike what has gone before, this text provides an engaging, insightful, and highly readable introduction to the inner workings of educational measurement. Traditional topics are presented in approachable and understandable ways. This book employs an issue-oriented approach to the analysis and interpretation of complex measurement concerns, most of which are being debated in public forums today. As an example, this book examines issues such as the score gap, high-stakes tests and the dropout crisis, and the problem of grade retention versus social promotion. Intriguing real-life examples are introduced in each chapter, selected to demonstrate how the technical measurement principles actually affect those who are involved. The approach I used to write this textbook involved drawing from my public school experiences and years as a teacher-educator and then amalgamating my experiences into a narrative presentation designed to explain the science of educational measurement. This provides the text with a true school-based focus.

The grounding of this work in the real world of public education is evident from the book's emphasis on matters such as high-stakes testing. The narrative also presents the position that the justification for much that has occurred in the name of educational accountability was originally based on erroneous readings and interpretations of international academic achievement assessments. The point is also made that during the 1980s the concern for American education led inevitably to the Improving America's Schools Act (1994) and then to the No Child Left Behind Act (NCLB, 2002).

In this new book, the testing mandates of the NCLB Act are presented in their historical context. In this historical interpretation, they are seen as a [Page xxv]continuation of European social science traditions begun during the Victorian era. This approach to measurement-based accountability as prescribed by the NCLB mandates follows a psychometric model that was first proposed in 1923 by Lewis M. Terman. Today's high-stakes assessments are an extension of the American belief in an industrial model for management, which requires an “input-output” evaluation. This approach focuses on observable outputs and products and pays little attention to the processes or the initial condition of raw materials.

Another description presented in the text elaborates on the link between the No Child Left Behind Act and the so-called miracle of Texas. Additionally, the NCLB Act is discussed in the context of the goals of the original Elementary and Secondary Education Act (1965) and that legislation's mandate for a longitudinal educational evaluation, “The Nation's Report Card” (NAEP).

The book includes a lengthy discussion of ethical and unethical approaches used to improve scores on the mandated assessments. It also addresses the relationship between factors such as the child's personological characteristics, familial structures, teacher background, instructional approaches, and the leadership style of the principal and the outcome scores on mandated assessments.

This text also addresses the needs and problems incurred by students with significant learning problems who must take the mandated assessment tests. The difficulty of reconciling special education programs with the tenants of the No Child Left Behind law are explored in depth. The steps in the identification of significant learning problems and interventions that are called for as part of the special education entitlements are also described in some detail. Related issues such as the testing of English Language Learners and the NCLB mandate for the testing of four-year-old inner-city children are also included.

Administrative concerns linked to educational assessment are not ignored. I have included a chapter on school evaluation that includes value-added assessments and longitudinal data management. The book examines the traditional approaches used to integrate both formative and summative assessments into a “systematic educational program evaluation.” The chapter on evaluation also presents examples illustrating the design of an evaluation section for an application for external funding or subvention.

“Hot button” problems for school leaders such as the relationship between high-stakes testing and grade retention, the dropout problem, and even “academic red shirting” are part of this text. Also discussed are the concerns held by many high school administrators regarding “academic press” and the striving of students to be one of the “deserving ten” or even the one [Page xxvi]with the best GPA who is selected to give the valedictory speech. In this context, the text provides an examination of the grading policies associated with gifted students, AP classes, and even the report card options for children with disabilities.

Finally, the application of educational technology in testing and measurement is explored, and the future trends for the applications of educational technology are noted and discussed. A section is included describing the use of computer adaptive testing (CAT) and its potential impact on the achievement score gap with students who are part of ethnic minorities. This description includes a discussion of the problems associated with item development and test security. Other technological applications for measurement are also explored, including classroom clickers and real-time formative evaluations, on-line parent-teacher communications, and online grade books.

I have included a number of learning aids throughout this textbook. They have been incorporated in an effort to improve the quality and depth of learning that occurs for the reader.

• 1.
  Section descriptions running a page or two lead off each conceptual segment of the book. These section descriptors provide an overview and framework for the next few chapters.
• 2.
  A page or two at the start of every chapter provides a statement of “Issues and Themes.” This section provides the reader with insight into the perspective that I have taken in researching and writing the chapter.
• 3.
  The learning goals for each chapter following the Issues and Themes section are presented in a sequential list. Taken together these three elements provide the reader with a set of advanced organizers designed to provide him or her with an awareness of what is being learned and a structure for understanding how to interpret the new material.
• 4.
  Each chapter includes several sections labeled “Case in Point.” Each Case in Point provides a real-life example or application of the material presented in the text. This is included as a motivational component.
• 5.
  Another motivational device in every chapter is a cartoon addressing an aspect of the chapter's narrative. These cartoons were drawn by some of America's leading cartoon humorists, including those of the New Yorker.
[Page xxvii]
• 6.
  A total of 118 URLs are included in the chapters of the book. These Internet resources provide students with access to an expanded library of information associated with the material in the book.
• 7.
  Each chapter provides a list of discussion questions designed to initiate classroom discussions and motivate students to employ higher-order cognitive processes when considering the material herein.
• 8.
  A detailed glossary of terminology including over 600 technical terms and laws is also part of this book.
• 9.
  Over 860 references, including another 250 URLs, are included as support material for the book.

Educational tests and measurements constitute a topic that is now central to the preparation of educational leaders and specialists. It is also a germane topic for all classroom educators working in our public schools. For that reason I advocate using this book with classes that are composed of graduate and upper-division students in education and teacher preparation. Students who would gain the most from this book are first-year graduate students and sophisticated undergraduate students.

I also encourage you to review and consider the “Teaching on Point” components included on the Instructor's CD. They provide teaching and discussion ideas that are keyed to the real-world examples, known as “Case on Point” in each chapter (as mentioned above). That Instructor's CD also provides a multiple choice test (type-A format) for each chapter. I have also provided discussion questions for each chapter, which can spark classroom or seminar interactions and promote analytical and evaluative thinking.

Finally, I encourage you to contact me with your ideas and thoughts about this text. I am also open to interacting with students during one of your classes using Internet video technology or by conference call. E-mail me at order@sagepub.com to explore this further. (Include the word textbook as part of the subject line.)

Congratulations for electing to teach this course, and best wishes for a successful semester.

I accept the fact that very few students who enroll in collegiate and graduate courses in educational tests and measurements do so voluntarily Yet, the study of educational measurement can be an engaging and truly empowering experience for teachers and school leaders.

From my perspective, there are four things about any college class or graduate seminar that are central to its success. These include the students who take the course, the instructor who provides the class, the content of the discipline being studied, and the books and resources that are available to the students.

During the intersessions of my university I have heard colleagues comment on how much easier it is to be a professor when the students are away from campus. However, to be a professor is to profess to others. Students are the reason colleges exist, and students are the life force inspiring each of us who enter the lecture halls and classrooms. The only things a college instructor asks for are students who have an open mind to the discipline and who are truly willing to learn.

I took my first class in educational measurement as a graduate student in 1968. In that summer course, I was inspired by a good teacher. That experience eventually led me to earn my doctorate in educational psychology. Good teaching is always the key to success for every course.

Beyond the ability of an instructor to inspire and motivate students, the next most important component of a good learning experience is the topic itself. Educational testing and measurement is now the central focus of ongoing public policy debates and is also an ever-present concern for all public school educators. It is the “tests” that have become the bane of so many teachers and administrators. The continuing presence of mandated assessment programs in the schools is not debatable. There can be no argument that they will be with us for the foreseeable future. Having a solid foundation in the knowledge about educational measurement is therefore a matter of survival for those who earn their living in the front offices and classrooms of our schools. The topics that a course in educational tests and measurement presents are seen far beyond the lecture hall: These issues and concerns are seen throughout our popular culture, in our mass media, and even in the value of our real estate. Today it is axiomatic that no prudent educator can take the topic of educational tests and measurement lightly.

I have written this book with these concerns in mind. The text provides grounding in all of the aspects of measurement that a public school educator must have as background and goes on to provide insight into the inner workings of the state agencies involved in creating high-stakes educational [Page xxix]measurements. I have worked to write this book in a highly readable and engaging style. By reading and studying the material herein, you will gain the ability to maximize the test outcomes for your students and be better equipped to advocate for improved approaches to the measurement of learning.

A diligent and documented search has been made to assure that the copyright holders of all material included in this work have been contacted and have approved use here. Also, all cited reference materials have been checked for accuracy. The URLs listed in this book have been carefully examined for accuracy as well. If the reader finds that errors have been made in some of these efforts, please contact me at //widenerbob@hotmail.com.

Additional ancillary materials further support and enhance the learning goals of Educational Assessment: Tests and Measurements in the Age of Accountability. These ancillary materials include the following:

No project of this magnitude can be completed by one person working in isolation. This textbook would not have been possible without the encouragement, editorial assistance, intelligence and creativity, and freely given help of my wife and partner, Jeanne.

In addition I must provide a word of thanks to Ms. Molly Wolf, Education Collection Librarian at Widener University's Wolfgram Memorial Library; Gloria Floyd, my secretary; the librarians and assistants at the Bonita Springs, Florida, Library; and Drs. Richard and Ann St. John, my technology support team, critics, and friends.

Kudos are also due for the team at Sage Publications, including my editor, Dr. Diane McDaniel, whose belief in this project and support of my efforts has been invaluable. Also, I owe a special debt of gratitude to Ms. Ashley Plummer, the editorial assistant working with this manuscript. It was Ashley who coordinated this project, bringing together the many reviews and permissions that are part of this book. I also wish to express my gratitude to the many editorial reviewers who worked to make this the best possible textbook. This group of faculty reviewers includes the following:

• Morris I. Beers, State University of New York College at Brockport
• Tyrone Bynoe, University of the Cumberlands
• Ollie Daniels, Barry University
• Delisa K. Dismukes, Jacksonville State University
• Holmes Finch, Ball State University
• Sheryl R. Glausier, Southeastern Louisiana University
• Renée Í. Jefferson, The Citadel: The Military College of South Carolina
• Kae Keister, Wilmington College
• James Pelech, Benedictine University
• Kaye Pepper, University of Mississippi
• Thomas R. Scheira, Buffalo State College
• Thomas J. Sheeran, Niagara University
• John Shimkanin, California University of Pennsylvania
• Ellen Bennett Steiner, University of Denver
• John J. Venn, University of North Florida
• Colleen Willard-Holt, Penn State-Harrisburg