Scientifically Based Research

Scientifically based research is the systematic application of the scientific method resulting in reliable and valid information with the potential to improve educational practice. At the level of practice, scientifically based research is the clear development of best practice, as determined by rigorous and replicable empirical research. Scientifically based research became especially significant with the adoption of the No Child Left Behind (NCLB) legislation and represented one of the most significant aspects of the proposed NCLB reforms.

Although there are many formal definitions of scientifically based research (e.g., the American Educational Research Association, the Center for Comprehensive School Reform and Improvement), arguably the most influential definition appeared in 2001 when President Bush authorized the No Child Left Behind Act. Scientifically based research, according to the No Child Left Behind Act, “means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs.” Most formal definitions of scientifically based research, including that in the No Child Left Behind Act, include specific characteristics:

• Explores meaningful questions. Scientifically based research must seek answers to questions that are socially relevant to students and practitioners in today's schools and must be framed in such a way that allows the results to be generalized beyond the sample studied.
• Defines replicable procedures. Scientifically based research must include sufficient detail in its description of the dependent variables and steps for implementation so that other researchers can reproduce the research or build on the findings with further investigations.
• Uses valid and reliable measures and/or observational techniques. Scientifically based research must use sound and objective means of collecting the data, relying on established measures and observational techniques.
• Relies on appropriate design and analyses. Scientifically based research must thoughtfully match the research methodology to the question and apply appropriate data analysis strategies to draw valid conclusions.
• Allows independent review. Scientifically based research must allow for rigorous examination of the research methodology, tools, and conclusions, by qualified researchers not connected to, or invested in, the project's outcome.

Critics of scientifically based research have taken issue with both foundational and practical components. For instance, some have argued against the notion that one has the ability to know anything objectively. According to these critics, since creating definitions is in itself a subjective experience, there is no sensible way to objectively define the methodology or describe the research tools with the precision required for replication. More practical criticisms have been leveled at the value placed on different research methodologies. Since the Institute of Education Sciences values randomized trials (i.e., the random assignment of research subjects to treatment and control groups) as the “gold [Page 803]standard,” some researchers have felt that other methodologies (usually those that are more qualitative in design) are not seen as useful or valuable.

For example, there is little recognition by the Institute of Education Sciences that qualitative or single-subject design models contribute to the scientifically based research base of knowledge. Because scientifically based research is primarily concerned with establishing causal relationships (i.e., this intervention leads to this outcome), the focus must be on ensuring that the empirical investigation is of the highest quality, rather than narrowly focusing on the specific methodology. As Robert Horner and colleagues have clearly articulated, however, well-designed and -executed single-subject research can be appropriate and even desirable for establishing initial causal relationships when the affected population is small and strict controls and procedures are in place. As with any research, establishing a firm causal relationship requires that research be replicated over time, with a wide variety of subjects and in diverse settings. Recall in the medical field that the generally accepted “causal” relationship between smoking and cancer in humans was initially suggested by correlational investigations that demonstrated that smokers were much more likely to develop cancer of the lung, oral cavity, pharynx, larynx, and esophagus.

...