Replication

The Importance of Replication

[Page 1844]The life-course perspective refers to a multidisciplinary paradigm for the study of people’s lives in relation to structural, social, and cultural contexts. It can be challenging to make sense of the results of replications in longitudinal research studies because there are many factors that potentially lead to differing results. Failed attempts to replicate results can be due to differences in the population studied, including differences in birth cohort, age range, health, education, and culture of individuals in the sample. Additionally, differences in research methods, including the operationalization, measurement, and coding of variables of interest or to other aspects of study design, can affect the reproducibility of results. Finally, failed attempts to replicate may be due to the choice of analytic approach, including the selection of covariates and approach to handling rates of response, attrition, and mortality selection within and across studies. However, failure to replicate does not necessarily mean that a finding is false. Rather, the fundamental importance of replication practices is to allow the research community to accumulate a body of evidence on a given topic that can be used to inform, integrate, and advance life-span developmental theory.

As a field, life-span researchers must employ strategies for the evaluation of the reproducibility of results. As replications accumulate, it will become increasingly important to identify patterns in findings that emerge with variations in populations studied, research methods, and analytic approach. What kind of conclusions can be drawn if evidence is consistent and what can we learn from instances where results are inconsistent across replications?

Ways to Encourage Replication

One way to encourage replication is during the publication process. Articles under review may be judged in part by whether the results have been reconciled with already published research on the same topic. In irreconcilable cases, additional comparative estimation may be necessary to test the robustness of the main findings within the original data set or with a comparable secondary data set. In addition, novel research findings could be paired with additional replication steps within the article to test the sensitivity and robustness of results across demographic subgroups or multiple estimation techniques, perhaps involving one or more additional data set. For example, estimates derived from nationally representative data might be paired with an integrative statement regarding the generalizability of findings across key demographic subgroups within that data set. Similarly, effect sizes derived from alternative estimation strategies such as ordinary least squares regression along with sibling fixed effects or instrumental variables techniques might be presented along with an explanation of the assumptions and limitations associated with each method.

A second way to encourage replication is to promote open data and information sharing. To encourage responsible use of replication across studies of life-span development, transparency is imperative. When a researcher undertakes a research study, the entire process must be carried out in such a fashion that another researcher could easily replicate the findings or otherwise refute the underlying assumptions and/or conclusions. First and foremost, to facilitate transparency, the data need to be made available to other researchers. In many instances, the data themselves can be made publicly available along with strict requirements to ensure that disclosures are not made about study participants. Many of the funding agencies that support research on life-span development now encourage or require that data are made publicly available. For example, when substantial funding is requested from the National Institute of Health in the United States, applicants are required to include a plan for data sharing or to state why data sharing is not feasible. The National Institute of Health position allows the original investigator to benefit from first and continuing use but not from prolonged exclusive use of data and requires timely release immediately following acceptance for publication of the main findings from the final data set. Such policies recognize that large-scale investigations that produce rich and complex data sets are unlikely to be fully used by a single set of investigators. There is an emerging consensus that the open sharing and analysis of existing data sets is an essential strategy for increasing the probability of replication and advancing a cumulative science.

...