Health Intervention Research: Understanding Research Design and Methods


Souraya Sidani

  • Citations
  • Add to My List
  • Text Size

  • Chapters
  • Front Matter
  • Back Matter
  • Subject Index
  • Copyright

    About the Author

    Souraya Sidani, is Professor and Canada Research Chair at the School of Nursing, Ryerson University. Her areas of expertise are in quantitative research methods, intervention design and evaluation, treatment preferences, and measurement. Her research areas of interest focus on evaluating interventions and advanced practice roles, on examining patient preferences for treatments, and on refining research methods and measures for determining the clinical effectiveness of interventions.


    In an era characterized by an emphasis on evidence-informed practice, research plays a central role in helping us understand human conditions, behaviors, and emerging problems; and identifying interventions that effectively address the problems and promote well-being. Empirical evidence on the effects of treatments guides decision making regarding the selection, implementation, and evaluation of interventions to improve various domains of life at the local, national, and international levels. However, to be useful, research studies must be well planned and executed. Careful planning involves the selection of research designs and methods that 1) are appropriate to capture the problem under investigation, 2) are consistent with the study’s purpose and aims, 3) ensure validity of the inferences and minimize potential biases, and 4) are feasible within the context in which they are applied. Careful execution consists of developing a detailed study protocol, adhering to it when carrying out research activities (i.e., recruitment, data collection, implementation of the intervention, and data analysis), and closely monitoring the study in order to ensure quality of performance, and to identify and remedy any challenges or deviations.

    A range of research designs and methods is available to study the effects of interventions. Some have been commonly considered as the most useful in generating credible evidence, and others have been advanced as plausible alternatives in response to recent critique of commonly used designs and methods. The critique was prompted by the realization that most, if not all, designs and methods are based on assumptions and recommendations which have been taken for granted and not been systematically and critically evaluated. These are derived from logic that may no longer be tenable in light of accumulating experience and emerging empirical evidence. Specifically, the adoption of the experimental or randomized controlled trial as the ‘gold standard’ for determining the effects of interventions was based on theoretical reasons and intuitive attractiveness rather than a compelling evidence base of data. Empirical evidence derived from meta-analyses shows that results of randomized trials and well-designed non-randomized studies evaluating the same interventions are comparable in determining the success of the intervention. These findings raise questions about the necessity and utility of randomization in reducing selection bias and enhancing validity of causal inferences. Randomization increases the likelihood that study groups are similar at baseline, but it does not guarantee it. Further, it introduces biases related to who takes part in studies and the influence of their perception of the intervention on treatment adherence and outcomes. Practical, pragmatic trials and partially randomized clinical or preference trials have been proposed to enhance representativeness of the sample, account for participants’ treatment preferences, and reduce attrition. Similarly, evidence is emerging that questions the utility of other methods such as the use of placebo.

    This book represents a compendium of research designs and methods, encompassing commonly used ones and recent advances that can be used in the evaluation of interventions. The book content describes the theoretical, empirical, and practical knowledge required in choosing among designs and methods for intervention evaluation. Theoretical knowledge covers the logic underlying different designs and methods; it provides the rationale or the ‘why’ for methodological decisions. Empirical knowledge looks at the results of studies that investigate the effectiveness, utility, or efficiency of different methods; it informs the ‘what’, ‘when’, and ‘where’ of methodological decisions. Practical knowledge involves descriptions of the procedure for implementing different research methods; it points to the ‘how’ for carrying out selected methods. The aim is to inform researchers of the nature and effectiveness of various designs and methods. This information is essential to 1) make researchers aware of different designs and methods, each having its strengths and limitations at the theoretical and empirical levels, 2) assist researchers in making appropriate decisions related to the selection of most suitable methods that best fit the context of particular studies, 3) help researchers recognize that methodological decisions should be based on evidence rather than mere traditional recommendations which may not be well supported logically and empirically, and ultimately 4) move the research enterprise out of the ‘inertia’ of using commonly recommended designs and methods that produce empirical evidence of limited utility to decision making and policy development, and into the world of generating, testing, and using alternative relevant designs and methods.


    The author gratefully acknowledges the encouragement, support, and informal feedback from colleagues and students, and the instrumental support of staff at the Health Intervention Research Center (Sarah I. and Laura C.).

  • References

    Abel, U. and Koch, A. (1999) The role of randomization in clinical studies: Myths and beliefs. Journal of Clinical Epidemiology, 52: 487497.
    Ablon, J.S. and Marci, C. (2004) Psychotherapy process: The missing link: Comment on Westen, Novotny, and Thompson-Brenner (2004). Psychological Bulletin, 130: 664668.
    Adamson, S.J., Sellman, J.D. and Dore, G.M. (2005) Therapy preference and treatment outcome in clients with mild to moderate alcohol dependence. Drug and Alcohol Review, 24: 209216.
    Agodini, R. and Dynarski, M. (2004) Are experiments the only option? A look at dropout prevention programs. The Review of Economics and Statistics, 86: 180194.
    Ahern, K. and Le Brocque, R. (2005) Methodological issues in the affects of attrition: Simple solutions for social scientists. Field Methods, 17: 5369.
    Aiken, L.S., West, S.G., Schwaim, D.E., Carroll, J.L. and Hsiung, S. (1998) Comparison of a randomized and two quasi-experimental designs in a single outcome evaluation: Efficacy of a university-level remedial writing program. Evaluation Review, 22 (2): 207244.
    Ajzen, I. (1991) The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50 (2): 179211.
    Ames, S.C., Rock, E., Hurt, R.D., Patten, C.A., Craghan, J.T., Stoner, S.M., Decker, P.A., Offord, K.P. and Nelson, M. (2008) Development and feasibility of a parental support intervention for adolescent smokers. Substance Use and Misuse, 43: 497511.
    Amir, L.H., Lumley, J. and Garland, S.M. (2004) A failed RCT to determine if antibiotics prevent mastitis: Cracked nipples colonized with Staphylococcus aureus: A randomized treatment trial [ISRCTN65289389]. BMC Pregnancy and Childbirth, 4: 19.
    Anderson, T.R., Hogg, J.A. and Magoon, T.M. (1987) Length of time on a waiting list and attrition after intake. Journal of Counseling Psychology, 34: 9395.
    Armstrong, T.S., Cohen, M.Z., Erikson, L. and Cleeland, C. (2005) Content validation of self-report measurement instruments: An illustration from the development of a brain tumor module of the M.D. Anderson Symptom Inventory. Oncology Nursing Forum, 3: 669676.
    Aseltine, R.H., Carlson, K.J., Fowler, F.J. and Barry, M.J. (1995) Comparing prospective and retrospective measures of treatment outcomes. Medical Care, 33 (Suppl): AS–6776.
    Autret, A., Valade, D. and Debiais, S. (2012) Placebo and other psychological interactions in headache treatment. Journal of Headache and Pain, 13: 191198.
    Awad, M.A., Shapiro, S.H., Lund, J.P. and Feine, J.S. (2000) Determinants of patients’ treatment preferences in a clinical trial. Community Dental Oral Epidemiology, 28: 119125.
    Baekeland, F. and Lundwall, L. (1975) Dropping out of treatment: A critical review. Psychological Bulletin, 82: 738783.
    Baker, L., Lavender, T. and Tincello, D. (2005) Factors that influence women’s decisions about whether to participate in research: An exploratory study. Birth, 32: 6066.
    Baldwin, S.A., Wampold, B.E. and Imel, Z.E. (2007) Untangling the alliance-outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance. Journal of Consulting and Clinical Psychology, 75 (6): 842852.
    Balk, E.M., Bonis, P.A., Moskowitz, H., Schmid, C.H., Ioannidis, J.P., Wang, C. and Lau, J. (2002) Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA: The Journal of the American Medical Association, 287: 2973ȁ2982.
    Bamberger, M. and White, H. (2007) Using strong evaluation designs in developing countries: Experience and challenges. Journal of MultiDisciplinary Evaluation, 4 (8): 5873.
    Barkauskas, V.H., Lusk, S.L. and Eakin, B.L. (2005) Intervention research: Selecting control interventions for clinical outcome studies. Western Journal of Nursing Research, 27(3): 346363.
    Barrett, R. (2002) A nurses’ primer on recruiting participants for clinical trials. Oncology Nursing Forum, 29: 10911096.
    Basch, C.E. and Gold, R.S. (1986) The dubious effects of type V errors in hypothesis testing on health education practice and theory. Health Education Research, 1 (4): 299305.
    Baskin, T.W., Tierney, S.C., Minami, T. and Wampold, B.E. (2000) Establishing specificity in psychotherapy: A meta-analysis of structural equivalence of placebo controls. Journal of Consulting and Clinical Psychology, 71 (6): 973979.
    Becker, B.J. (1990) Coaching for the Scholastic Aptitude Test: Further synthesis and appraisal. Review of Educational Research, 60: 373417.
    Becker, H.S. (2008) Tricks of the Trade: How to Think about Your Research While You’re Doing it. Chicago: University of Chicago Press.
    Becker, H., Roberts, G. and Voelmeck, W. (2003) Explanations for improvement in both experimental and control groups. Western Journal of Nursing Research, 25: 746755.
    Bedi, N., Chilvers, C., Churchill, R., Dewey, M., Duggan, C., Fielding, K., Gretton, V., Miller, P., Harrison, G., Lee, A. and Williams, I. (2000) Assessing effectiveness of treatment of depression in primary care: Partially randomized preference trial. British Journal of Psychiatry, 177: 312318.
    Bellg, A.J., Borrelli, B., Resnick, B., Hecht, J., Sharp-Minicucci, D., Ory, M. et al. (2004) Enhancing treatment fidelity in health behaviour change studies: Best practices and recommendations from the NIH behaviour change consortium. Health Psychology, 23 (5): 443451.
    Benson, K. and Hartz, A.J. (2000) A comparison of observational studies and randomized, controlled trials. The New England Journal of Medicine, 342 (25): 18781886.
    Berwick, D.M. (2008) The science of improvement. The Journal of the American Medical Association,299 (10): 11821184.
    Bloom, H.S., Michalopoulos, C. and Hill, C.J. (2005) Using experiments to assess non-experimental comparison-group methods for measuring program effects. In H.S. Bloom (ed.), Learning More from Social Experiments. New York: Russell Sage Foundation, pp. 173235.
    Bluestein, D., Healy, A.C. and Rutledge, C.M. (2011) Acceptability of behavioral treatments for insomnia. Journal of the American Board of Family Medicine, 24: 272280.
    Bootzin, R.R. and Bailey, E.T. (2005) Understanding placebo, nocebo, and iatrogenic treatment effects. Journal of Clinical Psychology, 61 (7): 871880.
    Borglin, G. and Richards, D.A. (2010) Bias in experimental nursing research: Strategies to improve the quality and explanatory power of nursing science. International Journal of Nursing Studies, 47: 123128.
    Borkovec, T.D. (1993) Between-group therapy outcome research: Design and methodology. In L.S. Onken, J.D. Blaine and J.J. Oren (eds), Behavioral Treatments for Drug Abuse and Dependence. National Institute on Drug Abuse (NIDA) Research Monograph Series 137. Washington, DC: US Department of Health and Human Services, pp. 249289.
    Borradaile, K.E., Halpern, S.D., Wyatt, H.R., Klein, S., Hill, J.O., Bailer, B., Brill, C., Stein, R.I., Miller, B.V. and Foster, G.D. (2011) Relationship between treatment preference and weight loss in the context of a randomized controlled trial. Obesity, 20(6): 12181222.
    Borrelli, B., Sepinwall, D., Ernst, D., Bellg, A.J., Czajkowski, S., Breger, R. et al. (2005) A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behaviour research. Journal of Consulting and Clinical Psychology, 73 (5): 852860.
    Bottomley, A. (1997) To randomize or not to randomize: Methodological pitfalls of the RCT design in psychosocial intervention studies. European Journal of Cancer Care, 6: 222230.
    Boutron, I., Estellat, C. and Ravaud, P. (2005) A review of blinding in randomized controlled trials found results inconsistent and questionable. Journal of Clinical Epidemiology, 58: 12201226>.
    Bowers, T.G. and Clum, G.A. (1998) Relative contribution of specific and nonspecific treatment effects: Meta-analysis of placebo-controlled behavior therapy research. Psychological Bulletin, 103 (3): 313323.
    Bowling, A. (2005) Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health, 27 (3): 281291.
    Bowling, A. and Rowe, G. (2005) ‘You decide doctor’: What do patient preference arms in clinical trials really mean? Journal of Epidemiology and Community Health, 59: 914915.
    Boyd, N.F., Cousins, M. and Kriukov, V. (1992) A randomized controlled trial of dietary fat reduction: The retention of subjects and characteristics of drop outs. Journal of Clinical Epidemiology, 45 (1): 3138.
    Bradley, C. (1993) Designing medical and educational intervention studies. Diabetes Care, 16: 509518.
    Brandt, P.A., Davis Kirsch, S., Marcus Lewis, F. and Casey, S.M. (2004) Assessing the strength and integrity of an intervention. Oncology Nursing Forum, 31 (4): 833837.
    Breitenstein, S.M., Gross, D., Garvey, C.A., Hill, C., Fogg, L. and Resnick, B. (2010) Implementation fidelity in community-based interventions. Research in Nursing and Health, 33(2): 164173.
    Britton, A., McPherson, K., McKee, M., Sanderson, C., Black, N. and Bain, C. (1998) Choosing between randomised and non-randomised studies: A systematic review. Health Technology Assessment, 2 (13).
    Brown, S.J. (2002) Nursing intervention studies: A descriptive analysis of issues important to clinicians. Research in Nursing and Health, 25: 317327.
    Bruckenthal, P. and Broderick, J.E. (2007) Assessing treatment fidelity in pilot studies to assist in designing clinical trials: An illustration from a nurse practitioner community-based intervention for pain. Advances in Nursing Science, 30 (1): E72E84.
    Buckwalter, K.C., Grey, M., Bowers, B., McCarthy, A.M., Gross, D., Funk, M. and Beck, C. (2009) Intervention research in highly unstable environments. Research in Nursing and Health, 32: 110121.
    Buddelmeyer, H. and Skoufias, E. (2003) An Evaluation of the Performance of Regression Discontinuity Design on PROGRESA. Bonn: IZA.
    Burns, J.W. and Evon, D. (2007) Common and specific factors in cardiac rehabilitation: Independent and interactive effects of the working alliance and self-efficacy. Health Psychology, 26 (6): 684692.
    Butterfield, P.G., Yates, S.M., Rogers, B. and Healow, J.M. (2003) Overcoming subject recruitment challenges: Strategies for successful collaboration with novice research agencies. Applied Nursing Research, 16 (1): 4652.
    Califf, R.M., Zarin, D.A., Kramer, J.M., Sherman, R.E., Aberle, L.H. and Tasneem, A. (2012) Characteristics of clinical trials registered in, 2007–2010. Journal of the American Medical Association, 307 (17): 18381847.
    Campbell, J.P., Maxey, V. and Watson, W.A. (1995) Hawthorne effects: implications for prehospital research. Annals of Emergency Medicine, 26 (5): 590594.
    Campbell, M., Fitzpatrick, R., Haines, A., Kinmonth, A.L., Sandercock, P., Spiegelhalter, D. and Tyrer, P. (2000) Framework for design and evaluation of complex interventions to improved health. British Medical Journal, 321: 694696.
    Campbell, N.C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F. et al. (2007) Designing and evaluating complex interventions to improve health care. British Medical Journal, 334: 455459.
    Campbell, R., Peters, T., Grant, C., Quilty, B. and Dieppe, P. (2005) Adapting the randomized consent (Zelen) design for trials of behavioural interventions for chronic disease: Feasibility study. Journal of Health Sciences Research and Policy, 10 (4): 220225.
    Cappelleri, J.C., Darlington, R.B. and Trochim, W.M. (1994) Power analysis of cutoff-based randomized clinical trials. Evaluation Review, 18 (2): 141152.
    Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J. and Balain, S. (2007) A conceptual framework for implementation fidelity. Implementation Science, 2: 4048.
    Cartwright, N. (2007) Are RCT’s the gold standard? BioSocieties, 2: 1120.
    Cartwright, N. and Munro, E. (2010) The limitations of randomized controlled trials in predicting effectiveness. Journal of Evaluation in Clinical Practice, 16: 260266.
    Carver, R.P. (1974) Two dimensions of tests: Psychometric and edumetric. American Psychologist, 29: 512518.
    Chalmers, T.C., Celano, P., Sacks, H.S. and Smith, H. (1983) Bias in treatment assignment in controlled clinical trials. The New England Journal of Medicine, 309: 13581361.
    Charlson, M.E. and Horwitz, R.I. (1984) Applying results of randomised trials to clinical practice: Impact of losses before randomisation. British Medical Journal (Clinical research ed.), 289 (6454): 1281.
    Chatterji, M. (2007) Grades of evidence: Variability in quality of findings in effectiveness studies of complex field interventions. American Journal of Evaluation, 28 (3): 239255.
    Chung, L.K., Cimprich, B., Janz, N.K. and Mills-Wismeski, S.M. (2009) Breast cancer survivorship program: Testing for cross-cultural relevance. Cancer Nursing, 32 (3): 236245.
    Coday, M., Boutin-Foster, C., Goldman Sher, T., Tennant, J., Greaney, M.L., Saunders, S.D. and Somes, G.W. (2005) Strategies for retaining study participants in behavioural intervention trials: Retention experiences of the NIH behaviour change consortium. The Society of Behavioural Medicine, 29(Suppl): 5565.
    Cohen, J. (1988) Statistical Power Analysis for the Social Sciences (
    ed.). Hillsdale, NJ: Lawrence Erlbaum.
    Colditz, G.A., Miller, J.N. and Mosteller, F. (1988) The effect of study design on gain in evaluation of new treatments in medicine and surgery. Drug Information Journal, 22 (3): 343352.
    Concato, J. and Horwitz, R.I. (2004) Commentary: Beyond randomised versus observational studies. The Lancet, 363 (9422): 16601661.
    Concato, J., Shah, N. and Horwitz, R.I. (2000) Randomised, controlled trials, observational studies, and the hierarchy of research designs. New England Journal of Medicine, 342 (25): 18871892.
    Connolly, N.B., Schneider, D. and Hill, A.M. (2004) Improving enrollment in cancer clinical trials. Oncology Nursing Forum, 31 (3): 610614.
    Cook, T.D. (1993) A quasi-sampling theory of the generalization of causal relationships. New Directions for Program Evaluation, 57: 39–81.
    Cook, T.D. (2006) Describing what is special about the role of experiments in contemporary educational research: Putting the ‘gold standard’ rhetoric into perspective. Journal of Multi-Disciplinary Evaluation, 6: 17.
    Cook, T.D. (2007) Randomized experiments in education: Assessing the objections to doing them. Economics of Innovation and New Technology, 16 (5): 331355.
    Cook, T.D. and Campbell, D.T. (1979) Quasi-experimentation: Design and Analysis Issues for Field Settings. Boston, MA: Houghton Mifflin.
    Cook, T.D., Scriven, M., Coryn, C.L.S. and Evergreen, S.D.H. (2010) Contemporary thinking about causation in evaluation: A dialogue with Tom Cook and Michael Scriven. American Journal of Evaluation, 31 (1): 105117.
    Cook Gotay, C. (1991) Accrual to cancer clinical trials: Directions from the research literature. Social Science and Medicine, 33 (5): 569577.
    Cooley, M.E., Sarna, L., Brown, J.K., Williams, R.D., Chernecky, C., Padilla, G. and Danao, L.L. (2003) Challenges of recruitment and retention in multisite clinical research. Cancer Nursing, 26 (5): 376386.
    Cooper, L.A., Gonzales, J.J., Gallo, J.J., Rost, K.M., Meredith, L.S., Rubenstein, L.V. et al. (2003) The acceptability of treatment of depression among African-American, Hispanic and White primary care patients. Medical Care, 41(4): 479489.
    Copeland, A., Martin, P., Geiselman, P., Rash, C. and Kendzor, D. (2006) Predictors of pretreatment attrition from smoking cessation among pre- and postmenopausal, weight-concerned women. Eating Behaviours, 7: 243251.
    Coward, D.D. (2002) Partial randomization design in a support group intervention study. Western Journal of Nursing Research, 24: 406421.
    Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I. and Petticrew, M. (2008) Developing and evaluating complex interventions: The New Medical Research Council guidance. British Medical Journal, 337: a1655.
    Crits-Christoph, P. and Mintz, J. (1991) Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Counsulting and Clinical Psychology, 59: 2026.
    Crits-Christoph, P., Baranackie, K., Kurcias, J., Beck, A., Carroll, K., Perry, K. et al. (1991) Meta-analysis of therapist effects in psychotherapy outcome studies. Psychotherapy Research, 1 (2): 8191.
    Curtin, L., Brown, R. and Sales, S. (2000) Determinants of attrition from cessation treatment in smokers with a history of major depressive disorder. Psychology of Addictive Behaviours, 14 (2): 134142.
    D’Agostino, R.B. (2009) The delayed-start study design. New England Journal of Medicine, 361: 13041306.
    Damschroder, L.J., Aron, D.C., Keith, R.E., Kirsh, S.R., Alexander, J.A. and Lowery, J.C. (2009) Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4: 5064.
    Daunt, D.J. (2003) Ethnicity and recruitment rates in clinical research studies. Applied Nursing Research, 16 (3): 189195.
    Davidson, E.J. (2006) The RCTs only doctrine: Brakes on the acquisition of knowledge? Journal of MultiDisciplinary Evaluation, 6: iiv.
    Deacon, B.J. and Abramowitz, J.S. (2005) Patients’ perceptions of pharmacological and cognitive-behavioral treatments for anxiety disorders. Behavior Therapy, 36: 139145.
    De Maat, S., Dekker, J., Schoevers, R. and de Jonghe, F. (2007) The effectiveness of long term psychotherapy: Methodological research issues. Psychotherapy Research, 17 (1): 5965.
    Demissie, K., Mills, O.F. and Rhoads, G.G. (1998) Empirical comparison of the results of randomized controlled trials and case-control studies in evaluating the effectiveness of screening mammography. Journal of Clinical Epidemiology, 51 (2): 8191.
    DeVellis, R.F. (2003) Scale Development: Theory and Applications (
    ed.). Thousand Oaks, CA: Sage.
    Devine, D.A. and Ferrald, P.S. (1973) Outcome effects of receiving a preferred, randomly assigned, or nonpreferred therapy. Journal of Consulting and Clinical Psychology, 41: 104107.
    Deyo, R.A. and Centor, R.M. (1986) Assessing the responsiveness of functional scales to clinical change: An analogy to diagnostic test performance. Journal of Chronic Disease, 39 (11): 897906.
    DiMattio, M.J.K. (2001) Recruitment and retention of community-dwelling, aging women in nursing studies. Nursing Research, 50 (6): 369373.
    Dinger, U., Strack, M., Leichsenrig, F., Wilmers, F. and Schauenburg, H. (2008) Therapist effects on outcome and alliance in inpatient psychotherapy. Journal of Clinical Psychology, 64 (3): 344354.
    Donaldson, S.I. and Christie, C.A. (2004) Determining causality in program evaluation and applied research: Should experimental evidence be the gold standard? Journal of MultiDisciplinary Evaluation, 3: 6077.
    Donner, A. and Klar, N. (2000) Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold Publishers, pp. 8182.
    Donner, A. and Klar, N. (2004) Pitfalls of and controversies in cluster randomization trials. American Journal of Public Health, 94 (3): 416422.
    Donner, A., Taljaard, M. and Klar, N. (2007) The merits of breaking the matches: A cautionary tale. Statistics in Medicine, 26 (9): 20362051.
    Douglas, H.R., Normand, C.E., Higginson, I.J. and Goodwin, D.M. (2005) A new approach to eliciting patients’ preferences for palliative day care: The choice experiment method. Journal of Pain and Symptom Management, 29: 435445.
    Dumas, J.E., Lynch, A.M., Laughlin, J.E., Phillips Smith, E. and Prinz, R.J. (2001) Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the early alliance prevention trial. American Journal of Preventative Medicine, 20 (1S): 3847.
    Dumbrigue, H.B., Al-Bayat, M.I., Ng, C.C.H. and Wakefield, C. W. (2006) Assessment of bias in methodology for randomized controlled trials published on implant dentistry. Journal of Prosthodontics, 15 (4): 257263.
    Dumville, J.C., Torgerson, D.J. and Hewitt, C.E. (2006) Reporting attrition in randomized controlled trials. British Medical Journal, 332 (22): 969971.
    Durlack, J.A. and DuPre, E.P. (2008) Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41: 327350.
    Dwight-Johnson, M., Unutzer, J., Sherbourne, C., Tang, L. and Wells, K.B. (2001) Can quality improvement programs for depression in primary care address patient preferences for treatment? Medical Care, 39 (9): 934944.
    Earle, C.C., Tsai, J.S., Gelber, R.D., Weinstein, M.C., Neumann, P.J. and Weeks, J.C. (2001) Effectiveness of chemotherapy for advanced lung cancer in the elderly: Instrumental variable and property analysis. Journal of Clinical Oncology, 19 (4): 10641070.
    Eccles, M., Grimshaw, J., Walker, A., Johnston, M. and Pitts, N. (2005) Changing the behavior of healthcare professionals: The use of theory in promoting the uptake of research findings. Journal of Clinical Epidemiology, 58: 107112.
    Eckert, T.L. and Hintze, J.M. (2000) Behavioral conceptions and applications of acceptability: Issues related to service delivery and research methodology. School Psychology Quarterly, 15 (2): 123149.
    Edinger, J.D., Wohlgemuth, W.K., Radtke, R.A., Coffman, C.J. and Carney, C.E. (2007) Dose-response effects of cognitive-behavioral insomnia therapy: A randomized clinical trial. Sleep, 30 (2): 203212.
    Elander, G. and Hermérén, G. (1995) Placebo effect and randomized clinical trials. Theoretical Medicine, 16: 171182.
    Eldridge, S.M., Ashby, D., Feder, G.S., Rudnicka, A.R. and Ukoumunne, O.C. (2004) Lessons for cluster randomized trials in the twenty-first century: A systematic review of trials in primary care. Clinical Trials, 1: 8090.
    Eldridge, S., Spencer, A., Cryer, C., Parsons, S., Underwood, M. and Feder, G. (2005) Why modelling a complex intervention is an important precursor to trial design: Lessons from studying an intervention to reduce falls-related injuries in older people. Journal of Health Services Research and Policy, 10 (3): 133142.
    Elkin, I., Falconnier, L., Martinovich, Z. and Mahoney, C. (2006) Therapist effects in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychotherapy Research, 16 (2): 144160.
    Ellis, P.M. (2000) Attitudes towards and participation in randomized clinical trials in oncology: A review of the literature. Annals of Oncology, 11: 939945.
    Embi, P.J., Jain, A., Clark, J., Bizjack, S., Hornung, R. and Harris, M. (2005) Effect of a clinical trial alert system on physician participation in trial recruitment. JAMA Internal Medicine, 165: 22722277.
    Epstein, R.M. and Peters, E. (2009) Beyond information: Exploring patients’ preferences. Journal of the American Medical Association, 302 (2): 195197.
    Epstein, R.S. (2000) Responsiveness in quality-of-life assessment: Nomenclature, determinants, and clinical applications. Medical Care, 38 (9, Suppl II): 9194.
    Eremenco, S., Cella, D. and Arnold, B. (2005) A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Evaluation and the Health Professions, 28 (2): 212232.
    Ferriter, M. and Huband, N. (2005) Does the non-randomized controlled study have a place in the systematic review? A pilot study. Criminal Behaviour and Mental Health, 15 (2): 111120.
    Finnis, D.G., Kaptchuk, T.J., Miller, F. and Benedetti, F. (2010) Biological, clinical, and ethical advances of placebo effects. The Lancet, 375: 686695.
    Fischer, D., Stewart, A.L., Boch, D.A. et al. (1999) Capturing the patients’ view of change as a clinical outcome measure. Journal of the American Medical Association, 282: 11571162.
    Fitzpatrick, R., Ziebland, S., Jenkinson, C., Mowat, A. and Mowat, A. (1993) Transition questions to assess outcomes in rheumatoid arthritis. British Journal of Rheumatology, 32: 807811.
    Fixsen, D.L., Naoom, S.F., Blase, K.A., Friedman, R.M. and Wallace, F. (2005) Implementation Research: A Synthesis of the Literature. Tampa: University of South Florida.
    Fleiss, J.L. (1986) Analysis of data from multiclinic trials. Controlled Clinical Trials, 7 (4): 267275.
    Floyd, A.H.L. and Moyer, A. (2010) Effects of participant preferences in unblinded randomized controlled trials. Journal of Empirical Research on Human Research Ethics, 5: 8193.
    Forbes, A. (2009) Clinical intervention research in nursing. International Journal of Nursing Studies, 46: 557568.
    Forgatch, M.S., Patterson, G.R. and DeGarmo, D.S. (2005) Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behaviour Therapy, 36 (1): 313.
    Fraenkel, L., Rabidou, N., Wittink, D. and Fried, T. (2007) Improving informed decision-making for patients with knee pain. The Journal of Rheumatology, 34 (9): 18941898.
    Freedman, D.H. (2010) Lies, damned lies, and medical science. The Atlantic, November.
    Fuertes, J.N., Mislowack, A., Bennett, J., Paul, L., Gilbert, T.C., Fontan, G. and Boylan, L.S. (2007) The physician–patient working alliance. Patient Education and Counseling, 66: 2936.
    Ganguli, M., Lytle, M.E., Reynolds, M.D. and Dodge, H.H. (1998) Random versus volunteer selection for a community-based study. The Journals of Gerontology Series A: Biological Sciences and Medical Sciences, 53 (1): M39M46.
    Gearing, R.E., El-Bassel, N., Ghesquiere, A., Baldwin, S., Gillies, J. and Ngeow, E. (2011) Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clinical Psychology Review, 31: 7988.
    Gilbert, J.P., McPeek, B. and Mosteller, F. (1978) Statistics and ethics in surgery and anesthesia. Science, 78: 684689.
    Givens, J.L, Houston, T.K., Van Voorhees, B.W., Ford, D.E. and Cooper, L.A. (2007) Ethnicity and preferences for depression treatment. General Hospital Psychiatry, 29: 182191.
    Glasgow, R.E., Magid, D.J., Beck, A., Ritzwoller, D. and Estabrooks, P.A. (2005) Practical clinical trials for translating research to practice: Design and measurement recommendations. Medical Care, 43 (6): 551559.
    Glasziou, P., Chalmers, I., Rowlins, M. and McCulloch, P. (2007) When are randomized trials unnecessary? Picking signal from noise. British Medical Journal, 334: 349351.
    Glazerman, S., Levy, D.M. and Myers, D. (2003) Nonexperimental versus experimental estimates of earning impacts. The Annals of the American Academy, 589: 6393.
    Goldfried, M.R. and Eubanks-Carter, C. (2004) On the need for a new psychotherapy research paradigm: Comment on Westen, Novotny, and Thompson-Brenner (2004). Psychological Bulletin, 130 (4): 669673.
    Graham, J.W. and Donaldson, S.I. (1993) Evaluating interventions with differential attrition: The importance of nonresponse mechanisms and use of follow up data. Journal of Applied Psychology, 78 (1): 119128.
    Grapow, M.T.R., von Wattenwyl, R., Guller, U., Beyersdorf, F. and Zerkowski, H.R. (2006) Randomized controlled trials do not reflect reality: Real world analyses are critical for treatment guidelines. The Journal of Thoracic and Cardio Surgery, 132 (1): 57.
    Green, J. (2000) The role of theory in evidence-based health promotion practice. Health Education Research, 15: 125129.
    Grimshaw, J., Campbell, M., Eccles, M. and Steen, N. (2000) Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Family Practice, 17 (1): S11S16.
    Guentner, K., Hoffman, L.A., Happ, M.B., Kim, Y., DeVito Dabbs, A., Mendelson, A.B. and Chelluri, L. (2006) Preferences for mechanical ventilation among survivors of prolonged mechanical ventilation and tracheostomy. American Journal of Critical Care, 15: 6577.
    Gum, A.M., Arean, P.A., Hunkeler, E., Tang, L., Katon, W., Hitchcock, P., Steffens, D.C., Dickens, J. and Unützer, J. (2006) Depression treatment preferences in older primary care patients. The Gerontologist, 4: 1422.
    Guyatt, G.H., Haynes, B., Jaeschke, R., Cook, D., Greenhalgh, T., Meade, M, Green, L., Naylor, C.D., Wilson, M., McAlister, F. and Richardson, W.S. (2002) Introduction: The philosophy of evidence-based medicine. In G. Guyatt and D. Rennie (eds), Users’ Guides to the Medical Literature: A Manual for Evidence-based Clinical Practice. Chicago: AMA Press, pp. 312.
    Gwadry-Sridher, F., Guyatt, G.H., Arnold, M.O., Massel, D., Brown, J., Nadeau, L. and Lawrance, S. (2003) Instruments to measure acceptability of information and acquisition of knowledge in patients with heart failure. The European Journal of Heart Failure, 5: 783791.
    Haaga, D.A.F. (2004) A healthy dose of criticism for randomized trials: Comment on Western Novotny, and Thompson-Brenner. Psychological Bulletin, 130 (4): 674676.
    Haidich, A-B. and Ioannidis, J.P.A. (2001) Patterns of patient enrollment in randomized controlled trials. Journal of Clinical Epidemiology, 54: 877883.
    Hamunen, K. and Kalso, E. (2005) A systematic review of trial methodology, using the placebo groups of randomized controlled trials in pediatric postoperative pain. International Association for the Study of Pain, 116: 146158.
    Hardeman, W., Sutton, S., Griffin, S., Johnston, M., White, A., Wareham, N.J. and Kinmonth, A.L. (2005) A causal modelling approach to the development of theory-based behaviour change for programmes for trial evaluation. Health Education Research, 20 (6): 676687.
    Hark, T.T. and Stump, D.A. (2008) Statistical significance versus clinical significance. Seminars in Cardiothoracic and Vascular Anesthesia, 12 (1): 56.
    Harris, K.M., Florey, F., Tabor, J., Bearman, P.S., Jones, J. and Udry, J.R. (2003) The national longitudinal study of adolescent health: Research design. Carolina Population Center, University of North Carolina at Chapel Hill. Retrieved from: (accessed 3 September 2008).
    Harris, P.M. (1998) Attrition model. American Journal of Evaluation, 19 (3): 293305.
    Harris, R. and Dyson, E. (2001) Recruitment of frail older people to research: Lessons learnt through experience. Journal of Advanced Nursing, 36 (5): 643651.
    Hart, E. (2009) Treatment definition in complex rehabilitation interventions. Neuropsychological Rehabilitation, 19 (6): 824840.
    Hasson, H. (2010) Systematic evaluation of information fidelity of complex intervention in health and social care. Implementation Science, 5: 6775.
    Hawe, P., Schiell, A. and Riley, T. (2004) Complex interventions: How ‘out of control’ can a randomized controlled trial be? British Medical Journal, 328: 15611563.
    Hazlett-Stevens, H., Craske, M.G., Roy-Byrne, P.P., Sherbourne, C.D., Stein, M.B. and Bystritsky, A. (2002) Predictors of willingness to consider medication and psychosocial treatment for panic disorder in primary care patients. General Hospital Psychiatry, 24: 316321.
    Heaman, M. (2001) Conducting health research with vulnerable women: Issues and strategies. Canadian Journal of Nursing Research, 33 (3): 8186.
    Heinsman, D.T. and Shadish, W.R. (1996) Assignment methods in experimentation: When do nonrandomized experiments approximate answers from randomized experiments? Psychological Methods, 1 (2): 154169.
    Heit, M., Rosenquist, C., Culligan, P., Graham, C., Murphy, M. and Shott, S. (2003) Predicting treatment choice for patients with pelvic organ prolapse. The American College of Obstetrics and Gynecology, 101: 1279.
    Hertzog, M.A. (2008) Considerations in determining sample size for pilot studies. Research in Nursing and Health, 31: 180191.
    Hicks, C. (1998) The randomized controlled trial: A critique. Nurse Researcher, 6 (1): 1932.
    Hitchcock Noël, P., Marsh, G., Larme, A.C., Correa, A., Meyer, J. and Pugh, J.A. (1998) Patient choice in diabetes education curriculum. Diabetes Care, 21: 896901.
    Hofmann, S.G., Barlow, D.H., Papp, L.A., Detweiler, M.F., Ray, S.E., Shear, M.K. et al. (1998) Pretreatment attrition in a comparative treatment outcome study on panic disorder. American Journal of Psychiatry, 155: 4347.
    Hollis, S. and Campbell, F. (1999) What is meant by intention to treat analysis? Survey of published randomised controlled trials. British Medical Journal, 319 (7211): 670674.
    Holtz, A. (2007) Comparative effectiveness of health interventions: Strategies to change policy and practice. Report from ECRI Institute’s 15th Annual Conference, pp. 18.
    Homer, C.S. (2002) Using the Zelen design in randomized controlled trials: Debates and controversies. Journal of Advanced Nursing, 38 (2): 200207.
    Horn, S.D., DeJong, G., Ryser, D.K., Veazie, P.J. and Teraoka, J. (2005) Another look at observational studies in rehabilitation research: Going beyond the holy grail of the randomized controlled trial. Archives of Physical Medicine and Rehabilitation, 86 (2): S8S15.
    Horwitz, R.I. (1987) Complexity and contradiction in clinical trial research. The American Journal of Medicine, 82: 498510.
    Howard, L. and Thornicroft, G. (2006) Patient preference and randomized controlled trials in mental health research. British Journal of Psychiatry, 188: 303304.
    Huibers, M.J.H., Bleijenberg, G., Beurskens, A.J.H.M., Kant, I., Knottnerus, J.A., Van der Windt, S.A.W.M. et al. (2004) An alternative trial design to overcome validity and recruitment problems in primary care research. Family Practice, 21 (2): 213218.
    Huppert, J.D., Bufka, L.F., Barlow, D.H., German, J.M., Shear, M.K. and Woods, S.W. (2001) Therapists, therapist variables, and cognitive-behavioral therapy outcome in a multicenter trial for panic disorder. Journal of Consulting and Clinical Psychology, 69: 747755.
    Hussey, M.A. and Hughes, J.P. (2007) Design and analysis of stepped wedge cluster randomized trials. Contemporary Clinical Trials, 28: 182191.
    Hyde, P. (2004) Fool’s gold: Examining the use of gold standards in the production of research evidence. British Journal of Occupational Therapy, 67 (2): 8994.
    Ibrahim, S. and Sidani, S. (2013) Strategies to recruit minority persons: A systematic review. Journal of Immigrant and Minority Health, 5 (11): 17841790.
    Ilnyckyj, A., Shanahan, F., Anton, P.A., Cheang, M. and Bernstein, C.N. (1997) Quantification of the placebo response in ulcerative colitis. Gastroenterology, 112: 18541858.
    Imbens, G.W. and Lemieux, T. (2008) Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142 (2): 615635.
    Institute of Medicine (2001) Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press.
    Ioannidis, J.P.A. (2005) Why most published research findings are false. PLoS Medicine, 2 (8): 696701.
    Ioannidis, J.P.A., Haidich, A.B., Pappa, M., Pantazis, N., Kokori, S.I., Tektonidou, M.G. et al. (2001) Comparison of evidence of treatment effects in randomized and nonrandomized studies. Journal of the American Medical Association, 286 (7): 821830.
    Jacobs, G.D., Pace-Schott, E.F., Stickgold, R. and Otto, M.W. (2004) Cognitive behavior therapy and pharmacotherapy for insomnia: A randomized controlled trial and direct comparison. Archives of Internal Medicine, 164: 18881986.
    Janevic, M.R., Janz, M.K., Dodge, J.A., Lin, X., Pan, W., Sinco, B.R. and Clark, N.M. (2003) The role of choice in health education intervention trials: A review and case study. Social Science and Medicine, 56: 15811594.
    Jenkins, V. and Fallowfield, L. (2000) Reasons for accepting or declining to participate in randomized clinical trials for cancer therapy. British Journal of Cancer, 82 (11): 17831788.
    Jepsen, P., Johnsen, S.P., Gillman, M.W. and Sorensen, H.T. (2004) Interpretation of observational studies. Heart, 90: 956960.
    Jo, B. (2002) Statistical power in randomized intervention studies with noncompliance. Psychological Methods, 7 (2): 178193.
    Jo, B., Muthen, B.O., Asparouhov, T., Ialongo, N.S. and Hendricks-Brown, C. (2008) Cluster randomized trials with treatment noncompliance. Psychological Methods, 13 (1): 118.
    Jobe, J.B. (2003) Cognitive psychology and self-reports: Models and methods. Quality of Life Research, 12: 219227.
    Jolly, K., Lip, G.Y., Taylor, R.S., Mant, J.W.,Lane, D.A., Lee, K.W. and the BRUM Steering Committee (2005) Recruitment of ethnic minority patients to a cardiac rehabilitation trial: The Birmingham Rehabilitation Uptake Maximisation (BRUM) study. BMC Medical Research Methodology, 5 :18.
    Joyce, A.S., Ogradniczuk, J.S., Piper, W.E. and McCallum, M. (2003) The alliance as mediator of expectancy effects in short-term individual therapy. Journal of Clinical and Consulting Psychology, 71 (4): 672679.
    Jüni, P., Witschi, A., Bloch, R. and Egger, M. (1999) The hazards of scoring the quality of clinical trials for meta-analysis. Journal of the American Medical Association, 282: 10541060.
    Kang, J., Robertson, R.J., Hagberg, J.M., Kelley, D.E., Goss, F.L., Dasilva, S.G., Suminski, R.R. and Utter, A.C. (1996) Effect of exercise intensity on glucose and insulin metabolism in obese individuals and obese NIDDM patients. Diabetes Care, 19 (4): 341349.
    Kaptchuk, T.J. (2001) The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf? Journal of Clinical Epidemiology, 54: 541549.
    Kaptchuk, T.J., Friedlander, E., Kelley, J.M., Sanchez, M.N., Kokkotou, E., Singer, J. P. et al. (2010). Placebos without deception: A randomized controlled trial in irritable bowel syndrome. Public Library of Science One, 5 (12): 17.
    Karanicolas, P.J., Montori, V.M., Devereaux, P.J., Schünemann, H. and Guyatt, D.H. (2009) A new ‘mechanistic-practical’ framework for designing and interpreting randomized trials. Journal of Clinical Epidemiology, 62: 479484.
    Kaufman, J.S., Kaufman, S. and Poole, C. (2003) Causal inference from randomized trials in social epidemiology. Social Science and Medicine, 57: 23972409.
    Kaul, S. and Diamond, G.A. (2010) Trial and error: How to avoid commonly encountered limitations of published clinical trials. Journal of the American College of Cardiology, 55: 415427.
    Kazdin, A.E. (1980) Acceptability of alternative treatments for deviant child behavior. Journal of Applied Behavior Analysis, 13: 259273.
    Kazdin, A.E. (2003) Research Design in Psychology. Boston: Allyn and Bacon.
    Keith, R.E., Hopp, F.P., Subramanian, U., Witala, W. and Lowery, J.C. (2010) Fidelity of implementation: Development and testing of a measure. Implementation Science, 5: 99109.
    Keller, C.S., Gonzales, A. and Fleuriet, K. J. (2005) Retention of minority participants in clinical research studies. Western Journal of Nursing Research, 27 (3): 292306.
    Keller, C., Fleury, J., Sidani, S. and Ainsworth, B. (2009) Fidelity to theory in physical activity intervention research. Western Journal of Nursing Research, 31 (3): 289311.
    Kemmer, L. and Júlia Paes da Silva, M. (2007) How can one choose what is unknown? A study on the image of nurses by high school students. Acta Paulista de Enfermagem, 20 (2): 125130.
    Kerlikowske, K. (2012) Screening mammography in women less than age 50 years. Current Opinion in Obstetrics and Gynecology, 24 (1): 3843.
    Kernan, W.N., Viscoli, C.M., Makuch, R.W., Brass, L.M. and Horwitz, R.I. (1999) Stratified randomization for clinical trials. Journal of Clinical Epidemiology, 52 (1): 1926.
    Kiesler, D.J. and Auerbach, S.M. (2006) Optimal matches of patient preferences for information, decision-making and interpersonal behavior: Evidence, models, and interventions. Patient Education and Counseling, 61: 319341.
    Kim, D-M., Wampold, B.E. and Bolt, D.M. (2006) Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Program data. Psychotherapy Research, 16 (2): 161172.
    King, M., Nazareth, I., Lampe, F., Bower, P., Chandler, M., Morou, M., Sibbald, B. and Lai, R. (2005) Impact of participant and physician intervention preferences on randomized trials: A systematic review. Journal of the American Medical Association, 293: 10891099.
    Kjaergaard, J., Clemmensen, I.H. and Storm, H.H. (2001) Validity and completeness of registration of surgically treated malignant gynaecological diseases in the Danish National Hospital Registry. Journal of Epidemiology and Biostatistics, 6 (5): 387392.
    Klaber-Moffett, J.K., Torgerson, D., Bell-Syer, S., Jackson, D., Llewlyn-Phillips, H., Farrin, A. and Barber, J. (1999) Randomised controlled trial of exercise for low back pain: Clinical outcomes, costs, and preferences. British Medical Journal, 319 (7205): 279283.
    Kleinman, P.H., Woody, G.E., Todd, T.C., Millman, R.B., Kang, S-Y., Kemp, J. and Lipton, D.S. (1990) Crack and cocaine abusers in outpatient psychotherapy. In L.S. Onken and J.D. Blaine (eds), Psychotherapy and Counseling in the Treatment of Drug Abuse. National Institute on Drug Abuse (NIDA) Research Monograph Series 104. Washington, DC: US Department of Health and Human Services, pp. 2435.
    Koedoot, C.G., de Haan, R.J., Stiggelbout, A.M., Stalmeier, P.F.M., de Graeff, A., Bakker, P.J.M. et al. (2003) Palliative chemotherapy or best supportive care? A prospective study explaining patients’ treatment preference and choice. British Journal of Cancer, 89: 22192226.
    Kok, G., Schaalma, H., Ruiter, R.A.C., Van Empelen, P. and Brug, J. (2004) Intervention mapping: A protocol for applying health psychology theory to prevention programmes. Journal of Health Psychology, 9 (1): 8598.
    Kovach, C.R. (2009) Some thoughts on the hazards of sloppy science when designing and testing multicomponent interventions, including the Kitchen Skin phenomenon. Research in Nursing and Health, 32: 13.
    Kownacki, R.J. (1997) A meta-analysis of the alcoholism treatment outcome literature: Selection with randomized and quasi-experiments (Doctoral dissertation, University of Memphis).
    Kraemer, H.C. and Fendt, K.H. (1990) Random assignment in clinical trials: Issues in planning (infant health and development program). Journal of Clinical Epidemiology, 43 (11): 11571167.
    Kraemer, M.S., Martin, R.M., Sterne, J.A.C., Shapiro, S., Dahhou, M. and Platt, R.W. (2009) The double jeopardy of clustered measurement and cluster randomization. British Medical Journal, 339: 503505.
    Kunz, R. and Oxman, A.D. (1998) The unpredictability paradox: Review of empirical comparisons of randomised and non-randomised clinical trials. British Medical Journal, 317: 11851190.
    Kwan, B.M., Dimidjian, S. and Rizvi, S.L. (2010) Treatment preferences, engagement, and clinical improvement in pharmacotherapy versus psychotherapy for depression. Behavior Research and Therapy, 48: 799804.
    LaLonde, R. (1986) Evaluating the econometric evaluations of training programs. American Economic Review, 76: 604620.
    Lambert, N., Rowe, G., Bowling, A., Ebrahim, S., Laurence, M. et al. (2004) Reasons underpinning patients’ preferences for various angina treatments. Health Expectations, 6: 246256.
    Lancaster, G.A., Dodd, S. and Williamson, P.R. (2004) Design and analysis of pilot studies: Recommendations for good practice. Journal of Evaluation in Clinical Practice, 10 (2): 307312.
    Larzelere, R.E., Kuhn, B.R. and Johnson, B. (2004) The intervention selection bias: An under recognized confound in intervention research. Psychological Bulletin, 130 (2): 289303.
    Lawlor, D.A., Smith, G.D., Bruckdorfer, K.R., Kundu, D. and Ebrahim, S. (2004) Those confounded vitamins: What can we learn from the differences between observational versus randomized trial evidence? The Lancet, 363: 19241927.
    Leach, M.J. (2003) Barriers to conducting randomized controlled trials: Lessons learnt from the Horsechestnut and Venous Leg Ulcer Trial (HAVLUT). Contemporary Nurse, 15: 3747.
    Lee, E., McNally, D.L. and Zuckerman, I.H. (2004) Evaluation of a physician-focused educational intervention on Medicaid children with asthma. The Annals of Pharmacotherapy, 38 (6): 961966.
    Leeman, R., Quiles, Z., Molinelli, L., Terwal, D., Nordstrom, B., Garvey, A. and Kinnunen, T. (2006) Attrition in a multi-component smoking cessation study for females. Tobacco Induced Disease, 3 (2): 5971.
    Lehman, A.F. and Steinwachs, D.M. (2003) Evidence-based psychosocial treatment practices in schizophrenia: Lessons from the Patient Outcomes Research Team (PORT) Project. Journal of the American Academy of Psychoanalysis and Dynamic Psychiatry, 31 (1): 141154.
    Leigh, J.P., Ward, M.M. and Fries, J.F. (1993) Reducing attrition bias with an instrumental variable in a regression model: Results from a panel of rheumatoid arthritis patients. Statistics in Medicine, 12 (11): 10051018.
    Leon, A.C., Davis, L.L. and Kraemer, H. C. (2011) The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research, 45 (5): 626629.
    Leventhal, H. and Friedman, M.A. (2004) Does establishing fidelity of treatment help in understanding treatment efficacy? Comment on Bellg et al. (2004). Health Psychology, 23 (5): 452456.
    Lewis, B.A., Napolitano, M.A., Whiteley, J.A. and Marcus, B.H. (2006) The effect of preferences for print versus telephone interventions on compliance and attrition in a randomized controlled physical activity trial. Psychology of Sport and Exercise, 7: 453462.
    Leykin, Y., DeRubeis, R.J., Gallop, R., Amsterdam, J.D., Shelton, R.C. and Hollon, S.D. (2007) The relation of patients’ treatment preferences to outcome in a randomized clinical trial. Behavior Therapy, 38: 209217.
    Liang, M.H. (2000) Longitudinal construct validity: Establishment of clinical meaning in patient evaluative instruments. Medical Care, 38 (9, Suppl II): 8490.
    Linde, K., Scholz, M., Ramirez, G., Clausius, N., Melchart, D. and Jonas, W.B. (1999) Impact of study quality on outcome in placebo-controlled trials of homeopathy. Journal of Clinical Epidemiology, 52 (7): 631636.
    Lindsay, B. (2004) Randomized controlled trials of socially complex nursing interventions: Creating bias and unreliability? Journal of Advanced Nursing, 45 (1): 8494.
    Lindsay Davis, L., Broome, M.E. and Cox, R.P. (2002) Maximizing retention in community-based clinical trials. Journal of Nursing Scholarship, 34: 4753.
    Lippke, S. and Ziegelman, J.P. (2008) Theory-based health behavior change: Developing, testing, and applying theories for evidence-based interventions. Applied Psychology: An International Review, 57 (4): 698716.
    Lipsey, M.W. (1990) Design Sensitivity: Statistical Power for Experimental Research. Thousand Oaks, CA: Sage.
    Lipsey, M.W. and Wilson, D.B. (1993) The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48: 11811209.
    Little, R.J.A. and Rubin, D.B. (2002) Statistical Analysis with Missing Data (
    ed.). Hoboken, NJ: John Wiley and Sons.
    Lowenstein, P.R. and Castro, M.G. (2009) Uncertainty in the translation of preclinical experiments to clinical trials: Why do most Phase III clinical trials fail? Current Gene Therapy, 9 (5): 368374.
    Luborsky, L., McLellan, A.T., Diguer, L., Woody, G. and Selignman, D.A. (1997) The psychotherapist matters: Comparison of outcomes across twenty two therapists and seven patient samples. Clinical Psychology: Science and Practice, 4: 5365.
    Lugtenberg, M., Burgers, J.S. and Westert, G.P. (2009) Effects of evidence-based clinical practice guidelines on quality of care: A systematic review. Quality and Safety in Health Care, 18: 385392.
    Lutz, W., Leon, S.C., Martinovich, Z., Lyons, J.S. and Stiles, W.B. (2007) Therapist effects in outpatient psychotherapy: A three-level growth curve approach. Journal of Counseling Psychology, 54 (1): 3239.
    Lyons, K.S., Carter, J.H., Carter, E.H., Rush, K.N., Stewart, B.J. and Archbold, P.G. (2004) Locating and retaining research participants for follow-up studies. Research in Nursing and Health, 27: 6368.
    Macias, C., Barreira, P., Hargreaves, W., Bickman, L., Fisher, W. and Aronson, E. (2005) Impact of referral source and study applicants’ preference for randomly assigned service on research enrollment, service engagement, and evaluative outcomes. American Journal of Psychiatry, 162: 781787.
    MacKinnon, D.P. and Fairchild, A.J. (2009) Current directions in mediation analysis. Current Directions in Psychological Science, 18 (1): 1620.
    MacLehose, R.R., Reeves, B.C., Harvey, I.M., Sheldon, T.A., Russell, I.T. and Black, A.M.S. (2000) A systematic review of comparisons of effect sizes derived from randomized and non-randomised studies. Health Technology Assessment, 4 (34): 1154.
    Maclure, M. (2009) Explaining pragmatic trials to pragmatic policymakers. Journal of Clinical Epidemiology, 62: 476478.
    Man-Son-Hing, M., Hart, R.G., Berquist, R., O’Connor, A.M. and Laupacis, A. (2001) Differences in treatment preferences between persons who enroll and do not enroll in clinical trials. Annals of the Royal College of Physicians and Surgeons, 34 (5): 292296.
    Marcellus, L. (2004) Are we missing anything? Pursuing research on attrition. Canadian Journal of Nursing Research, 36 (3): 8298.
    Margo, C.E. (1999) The placebo effect. Survey of Ophthalmology, 44 (1): 3144.
    Martin, K.A., Bowen, D.J., Dunbar-Jacob, J. and Perri, M.G. (2000) Who will adhere? Key issues in the study and prediction of adherence in randomized controlled trials. Controlled Clinical Trials, 21: 195S199S.
    McCarney, R., Warner, J., Iliffe, S., Van Haselen, R., Griffin, M. and Fisher, P. (2007) The Hawthorne Effect: A randomized controlled trial. BMC Medical Research Methodology, 7: 30.
    McLean, S. (1997) No consent means not treating the patient with respect. British Medical Journal, 314:1076.
    McClelland, G.H. (1997) Optimal design in psychological research. Psychological Methods, 2 (1): 319.
    McDonald, A.M., Knight, R.M., Campbell, M.K., Entwhistle, V.A., Grant, A.M., Cook, J.A. et al. (2006) What influences recruitment to randomized controlled trials? A review of trials funded by two UK funding agencies. Trials, 7: 9.
    McEwen, A. and West, R. (2009) Do implementation issues influence the effectiveness of medications? The case of nicotine replacement therapy and bupropion in UK Stop Smoking Services. BMC Public Health, 9: 2836.
    McKay, J.R., Alterman, A.I., McLellan, T., Boardman, C.R., Mulvaney, F.D. and O’Brien, C.P. (1998) Journal of Consulting and Clinical Psychology, 66 (4): 697701.
    McKee, M., Britton, A., Black, N., McPherson, K., Sanderson, C. and Bain, C. (1999) Interpreting the evidence: Choosing between randomised and non-randomised studies. British Medical Journal, 19: 312315.
    McKnight, P.E., McKnight, K.M., Sidani, S. and Figueredo, A.J. (2007) Missing Data: A Gentle Introduction. New York: Guilford Publications.
    McRae, A.D., Weijer, C., Binik, A, Grimshaw, J.M., Baruch, R., Brehaut, J.C., Donner, A., Eccles, M.P., Saginur, R., White, A. and Taljaard, M. (2011a) When is informed consent required in cluster randomized trials in health research? Trials, 12: 202.
    McRae, A.D., Weijer, C., Binik, A., White, A., Grimshaw, J.M., Baruch, R., Brehaut, J.C., Donner, A., Eccles, M., Saginur, R., Zwarenstein, M. and Taljaard, M. (2011b) Who is the research subject in cluster randomized trials in health research? Trials, 12: 183.
    Melde, C., Esbensen, F.A. and Tusinski, K. (2006) Addressing program fidelity using onsite observations and program provider descriptions of program delivery. Evaluation Review, 30 (6): 714740.
    Messer, S.B. and Wampold, B.E. (2002) Let’s face facts: Common factors are more potent than specific therapy ingredients. Clinical Psychology and Scientific Practice, 9: 2125.
    Messick, S. (1995) Validity of psychological assessment. American Psychologist, 50 (9): 741749.
    Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI) (2012) Methodological standards and patient-centeredness in comparative effectiveness research. The PCORI perspective. Journal of the American Medical Association, 307 (15): 16361640.
    Miao, L.L. (1977) Gastric freezing: An example of the evaluation of medical therapy by randomized clinical trials. In J.P. Bunker, B.A. Barnes and F. Mosteller (eds), Costs, Risks, and Benefits of Surgery. New York: Oxford University Press, pp. 198211.
    Mihalic, S. (2002) The importance of implementation fidelity. 442 UCB. Retrieved from:
    Millat, B., Borie, F. and Fingerhut, A. (2005) Patients’ preference and randomization: New paradigm of evidence-based clinical research. World Journal of Surgery, 29: 596600.
    Miller, J.N., Colditz, G.A. and Mosteller, F. (1989) How study design affects outcomes in comparisons of therapy. II: Surgical. Statistics in Medicine, 8: 455466.
    Mills, N., Donovan, J.L., Wade, J., Hamdy, F.C., Neal, D.E. and Lane, J.A. (2011) Exploring treatment preferences facilitated recruitment to randomized controlled trials. Journal of Clinical Epidemiology, 64: 11271136.
    Miranda, J. (2004) An exploration of participants’ treatment preferences in a partial RCT. Canadian Journal of Nursing Research, 36 (3): 100114.
    Moffett, J.A., Jackson, D.A., Richmond, S., Hahn, S., Coulton, S., Farrin, A., Mance, A. and Torgerson, D.J. (2005) Randomised trial of a brief physiotherapy intervention compared with usual physiotherapy for neck pain patients: Outcomes and patients’ preferences. British Medical Journal, 330: 7580.
    Montoro-Rodriguez, J. and Smith, G.C. (2010) When cost meets efficiency: rethinking ways to sample a rare population (pp. 136146). In D.L. Streiner and S. Sidani (eds), When Research goes off the Rails. New York: Guilford Press.
    Moser, D.K., Dracup, K. and Doering, L.V. (2000) Factors differentiating dropouts from completers in a longitudinal, multicenter clinical trial. Nursing Research, 49 (2): 109116.
    Mowbray, C.T., Holter, M.C., Teague, G.B. and Bybee, D. (2003) Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24 (3): 315340.
    Murray, D.M., Varnell, S.P. and Blitstein, J.L. (2004) Design and analysis of group randomized trials: A review of recent methodological developments. American Journal of Public Health, 94 (3): 423432.
    Najavits, L.M. and Weiss, R.D. (1994) Variations in therapist effectiveness in the treatment of patients with substance use disorders: An empirical review. Addiction, 89: 679688.
    Nallamothu, B.K., Hayward, R.A. and Bates, E.R. (2008) Beyond the randomized clinical trial: The role of effectiveness studies in evaluating cardiovascular therapies. Circulation, 118: 12941303.
    Nápoles-Springer, A.M., Santayo-Olsson, J., O’Brien, H. and Stewart, A.L. (2006) Using cognitive interviews to develop surveys in diverse populations. Medical Care, 44: S21S30.
    National Institute of Neurological Disorders and Stroke [NINDS] (1995) Tissue plasminogen activator for acute ischemic stroke. The New England Journal of Medicine, 333: 15811587.
    National Institutes of Health, National Cancer Institute (2005) Theory at a Glance: A Guide for Health Promotion Practice (
    ed.). Retrieved from:
    Noar, S.M. and Mehrotra, P. (2011) Toward a new methodological paradigm for testing theories of health behavior and health behavior change. Patient Education and Counseling, 82 (3): 468474.
    Nock, M.K. (2007) Conceptual and design essentials for evaluating mechanisms of change: Alcoholism. Clinical and Experimental Research, 31 (S3): 4S12S.
    Norman, G.R. and Streiner, D.L. (2008) Biostatistics: The Bare Essentials (
    ed.). Shelton, CT: PMPH US.
    Norris, S.L. and Atkins, D. (2005) Challenges in using randomized studies in systematic reviews of treatment interventions. Annals of Internal Medicine, 142 (12): 11121119.
    North West Uro-Oncology Group (2002) A preliminary report on patient-preference study to compare treatment options in early prostate cancer. British Journal of Urology International, 90: 253256.
    Nunnally, J.C. and Bernstein, I.H. (1994) Psychometric Theory (
    ed.). New York: McGraw-Hill.
    Nystuen, P. and Hagen, K.B. (2004) Telephone reminders are effective in recruiting nonresponding patients to randomized controlled trials. Journal of Clinical Epidemiology, 57: 773776.
    O’Reilly, P. Martin, L. and Collins, G. (1999) Few patients with prostate cancer are willing to be randomized to treatment. British Medical Journal, 318 (197): 1556.
    Oakley, A., Strange, V., Torogan, T., Wiggins, M., Roberts, I. and Stephenson, J. (2003) Using random allocation to evaluate social interventions: The recent U.K. examples. The Annals of American Academy of Political and Social Science, 589: 170189.
    Okiishi, J., Lambert, M.J., Nielson, S.L. and Ogles, B.M. (2003) Waiting for supershrink: An empirical analysis of therapist effects. Clinical Psychology and Psychotherapy, 10: 361373.
    Ong, J.C., Kuo, T.F. and Manber, R. (2008) Who is at risk for dropout from group cognitive-behavior therapy for insomnia? Journal of Psychosomatic Research, 64: 419425.
    Ong-Dean, C., Hofsletter, C.H. and Strick, B.R. (2011) Challenges and dilemmas in implementing random assignment in educational research. American Journal of Evaluation, 32 (1): 2949.
    Oremus, M., Cosby, J.L. and Wolfson, C. (2005) A hybrid qualitative method for pre-testing questionnaires: The example of a questionnaire to caregivers of Alzheimer’s disease patients. Research in Nursing and Health, 28: 419430.
    Osoba, D., Hsu, M.A., Copley-Merriman, C., Coombs, J., Johnson, F.R., Hauber, B., Manjunath, R. and Pyles, A. (2006) Stated preferences of patients with cancer for health-related quality-of-life (HRQOL) domains during treatment. Quality of Life Research, 15: 273283.
    Ottenbacher, K. (1992) Impact of random assignment on study outcome: An empirical examination. Controlled Clinical Trials, 13: 5061.
    Oxman, A.D., Lombard, C., Treweek, S., Gagnier, J.J., Maclure, M. and Zwarenstein, M. (2009) Why we will remain pragmatists: Four problems with the impractical mechanistic framework and a better solution. Journal of Clinical Epidemiology, 62: 485488.
    Oxman, T.E., Schulberg, H.C., Greenberg, R.L., Dietrich, A.J., Williams, J.W., Nutting, P.A. and Bruce, M.L. (2006) A fidelity measure for integrated management of depression in primary care. Medical Care, 44 (11): 10301037.
    Padhye, N.S., Cron, S.G., Gusick, G.M., Hamlin, S.K. and Hanneman, S.K. (2009) Randomization for clinical research: An easy-to-use spreadsheet method. Research in Nursing and Health, 32: 561566.
    Painter, J.E., Borba, C.P.C., Hynes, M., Mays, D. and Glanz, K. (2008) The use of theory in health behavior research from 2000–2005: A systematic review. Annals of Behavioral Medicine, 35: 358362.
    Pallesen, S., Mitsem, M., Kvale, G., Johnsen, B.H. and Molde, H. (2005) Outcome of psychological treatments of pathological gambling: A review and analysis. Addiction, 100 (10): 14121422.
    Pawson, R. and Tilley, N. (1997) Realistic Evaluation. London: Sage.
    Peabody, J.W., Luck, J., Jain, S., Bertenthal, D. and Glassman, P. (2004) Assessing the accuracy of administrative data in health information systems. Medical Care, 42 (11): 10661072.
    Pearson, M.L., Wu, S., Schaefer, J., Bonomi, A.E., Shortell, S.M., Mendel, P.J. et al. (2005) Assessing the implementation of the chronic care model in quality improvement collaboratives. Health Services Research, 40 (4): 978996.
    Peduzzi, P., Henderson, W., Hartigan, P. and Lavori, P. (2002) Analysis of randomized controlled trials. Epidemiologic Reviews, 24 (1): 2638.
    Pincus, T. (2002) Limitations of randomized clinical trials in chronic diseases: Explanations and recommendations. Advances, 18 (2): 1421.
    Podsakoff, P.M., MacKenzie, S.B., Lu, J-Y. and Podsakoff, N.P. (2003) Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88 (5): 879903.
    Polit, D.F., Beck, C.T. and Owen, S.V. (2007) Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing and Health, 30 (4): 459467.
    Porter, R., Frampton, C., Joyce, P.R. and Mulder, R.T. (2003) Randomized controlled trials in psychiatry. Part 1: Methodology and critical evaluation. Australian and New Zealand Journal of Psychiatry, 37: 257264.
    Preference Collaborative Review Group (2009) Patients’ preferences within randomized trials: Systematic review and patient level meta-analysis. British Medical Journal Online. Retrieved 3 February 2009.
    Proctor, R.W. and Capaldi, E.J. (2001) Empirical evaluation and justification of methodologies in psychological science. Psychological Bulletin, 127 (6): 759772.
    Project MATCH Research Group (1998) Therapist effects in three treatments for alcohol problems. Psychotherapy Research, 8: 455474.
    Pruitt, R.H. and Privette, A.B. (2001) Planning strategies for the avoidance of pitfalls in intervention research. Journal of Advanced Nursing, 35 (4): 514520.
    Quitkin, F.M. (1999) Placebos, drug effects, and study design: A clinician’s guide. American Journal of Psychiatry, 156 (6): 829836.
    Raudenbush, S.W. and Bryk, A.S. (2002) Hierarchical Linear Models: Applications and Data Analysis Methods (
    ed.). Thousand Oaks, CA: Sage.
    Raue, P.J., Schulberg, H.C., Heo, M., Klimstra, S. and Bruce, M.L. (2009) Patients’ depression treatment preferences and initiation, adherence, and outcome: A randomized primary care study. Psychiatric Services, 60: 337343.
    Relton, C., Torgerson, D., O’Cathain, A. and Nicholl, J. (2010) Rethinking pragmatic randomized controlled trials: Introducing the ‘cohort multiple randomized controlled trial’ design. British Medical Journal, 340: 963967.
    Resio, M.A., Baltch, A. and Smith, R.P. (2004) Mass mailing and telephone contact were effective in recruiting veterans into an antibiotic treatment randomized clinical trial. Journal of Clinical Epidemiology, 57: 10631070.
    Ribisl, K.M., Walton, M.A., Mowbray, C.T., Luke, D.A., Davidson, W.S. and Bootsmiller, B.J. (1996) Minimizing participant attrition in panel studies through the use of effective retention and tracking strategies: Review and recommendations. Evaluation and Program Planning, 19(1): 125.
    Robey, R.R. (2004) A five phase model for clinical-outcome research. Journal of Communication Disorder, 37: 401411.
    Robinson, E.J., Kerr, C.E.P., Stevens, A.J., Lilford, R.J., Brauholtz, D.A., Edwards, S.J., Beck, S.R. and Rowley, M.G. (2005) Lay public’s understanding of equipoise and randomisation in randomised controlled trials. Health Technology Assessment, 9 (8).
    Rohrer, J.E. (2000) Rigor at the expense of relevance equals rigidity: Where to next in the study of medical care utilization? Health Services Research, 34 (6): 13071314.
    Rosenberger, W.F. (1999) Randomized play-the-winner clinical trials: Review and recommendations. Controlled Clinical Trials, 20: 328342.
    Ross, S., Grant, A., Counsell, C., Gillespie, W., Russell, I. and Prescott, R. (1999) Barriers to participation in randomized controlled trials: A systematic review. Journal of Clinical Epidemiology, 52 (12): 11431156.
    Rossi, P.H., Freeman, H.E. and Lipsey, M.W. (2004) Evaluation: A Systematic Approach (
    ed.). Thousand Oaks, CA: Sage.
    Rothwell, P.M. (2005) External validity of randomized controlled trials: ‘To whom do the results of this study apply?’ The Lancet, 365: 8293.
    Rücker, G. (1989) A two-stage trial design for testing treatment, self selection and treatment preference effects. Statistics in Medicine, 8(4): 477485.
    Russell, K. and Walsh, D. (2009) Can the use of behavioural intervention studies support change in professional practice behaviours? A literature review. Evidence Based Midwifery, 7 (2): 5459.
    Sackett, D.L. (2007) Commentary: Measuring the success of blinding in RCTs: Don’t, must, can’t or needn’t? International Journal of Epidemiology, 36: 664665.
    Sacks, H., Chalmers, T.C. and Smith, H. (1982) Randomized versus historical controls for clinical trials. The American Journal of Medicine, 72: 233240.
    Sacks, H.S., Chalmers, T.C. and Smith, H. (1983) Sensitivity and specificity of clinical trials: Randomized v. historical controls. Archives of Internal Medicine, 143: 753755.
    Salyer, J., Geddes, N., Smith, C.S. and Mark, B.A. (1998) Commitment and communication: Keys to minimizing attrition in multisite longitudinal organizational studies. Nursing Research, 47 (2): 123125.
    Saunders, R.P., Evans, M.H. and Joshi, P. (2005) Developing a process-evaluation plan for assessing health promotion program implementation: A how to guide. Health Promotion Practice, 6 (2): 134147.
    Savard, J., Simard, S., Hervouet, S., Ivers, H., Lacombe, L. and Fradet, Y. (2005) Insomnia in men treated with radical prostatectomy for prostate cancer. Psycho-Oncology, 14: 147156.
    Say, R.E. and Thomson, R. (2003) The importance of patient preferences in treatment decisions: Challenges for doctors. British Medical Journal, 327: 542545.
    Schafer, J.L. and Kang, J. (2008) Average causal effects from nonrandomized studies: A practical guide and simulation example. Psychological Methods, 13 (4): 279313.
    Schellings, R., Kessels, A.G., ter Riet, B., Knottnerus, J.A. and Stumans, F. (2006) Randomized consent designs in randomized controlled trials: systematic literature search, Contemporary Clinical Trials, 27 (4): 320332.
    Schmidt, F. (2010) Detecting and connecting the lies that data tell. Perspectives on Psychological Science, 5 (3): 233242.
    Schochet, P.Z. (2008) Statistical power for random assignment evaluations of education programs. Journal of Educational and Behavioral Statistics, 33(1); 6287.
    Schulz, K.F. and Grimes, D.A. (2002a) Generation of allocation sequences in randomised trials: Chance, not choice. The Lancet, 359: 515519.
    Schulz, K.F. and Grimes, D.A. (2002b) Allocation concealment in randomised trials: Defending against deciphering. The Lancet, 359: 614618.
    Schulz, K.F., Chalmers, I., Hayes, R.J. and Altman, D.G. (1995) Empirical evidence of bias: Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Journal of the American Medical Association, 273(5): 408412.
    Schwartz, C.E. and Fox, B.H. (1995) Who says yes? Identifying selection biases in a psychosocial intervention study of multiple sclerosis. Social Science and Medicine, 40 (3): 359370.
    Schwartz, C.E., Chesney, M.A., Irvine, J. and Keefe, F.J. (1997) The control group dilemma in clinical research: Applications for psychosocial and behavioural medicine trials. Psychosomatic Medicine, 59: 362371.
    Schwartz, C.E., Merriman, M.P., Reed, G.W. and Hammes, B.J. (2004) Measuring patient treatment preferences in end-of-life care research: Applications for advance care planning interventions and response shift research. Journal of Palliative Medicine, 7: 233245.
    Sellors, J., Cosby, R., Trim, K., Kaczorowski, J., Howard, M., Hardcastle, L. et al. for the Seniors Medication Assessment Research Trial (SMART) Group (2002) Recruiting family physicians and patients for a clinical trial: Lessons learned. Family Practice, 19 (1): 99104.
    Sepucha, K. and Ozanne, E.M. (2010) How to define and measure concordance between patients’ preferences and medical treatments: A systematic review of approaches and recommendations for standardization. Patient Education and Counseling, 78: 1223.
    Shadish, W.R. (2010) Campbell and Rubin: A primer and comparison of their approaches to causal inference in field settings. Psychological Methods, 15 (1): 317.
    Shadish, W.R. and Ragsdale, K. (1996) Random versus nonrandom assignment in controlled experiments: Do you get the same answer? Journal of Consulting and Clinical Psychology, 64 (6): 12901305.
    Shadish, W.R., Hu, H., Glaser, R.R., Kownacki, R. and Wong, S. (1998) A method for exploring the effects of attrition in randomized experiments with dichotomous outcomes. Psychological Methods, 3 (1): 322.
    Shadish, W.R., Matt, G.E., Navarro, A.M. and Phillips, G. (2000) The effects of psychological therapies under clinically representative conditions: A meta-analysis. Psychological Bulletin, 126 (4): 512529.
    Shadish, W.R., Cook, T.D. and Campbell, D.T. (2002) Experimental and Quasi-experimental Design for Generalized Causal Inference. Boston: Houghton-Mifflin.
    Shadish, W.R., Galino, R., Wong, V.C., Steiner, P.M. and Cook, T.D. (2011) A randomized experiment comparing random and cutoff-based assignment. Psychological Methods, 16: 179191.
    Shapiro, D.A. and Shapiro, D. (1982) Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychological Bulletin, 92: 581604.
    Shapiro, S.L., Figueredo, A.J., Caspi, O., Schwartz, G.E., Bootzin, R.R., Lopez, A.M. and Lake, D. (2002) Going quasi: The premature disclosure effect in a randomized clinical trial. Journal of Behavioural Medicine, 25 (6): 605621.
    Shetty, N., Friedman, J.F., Kieburtz, K., Marshall, F.J., Oakes, D. and the Parkinson Study Group. (1999) The placebo response in Parkinson’s disease. Clinical Neuropharmacology, 22 (4): 207212.
    Shumaker, S.A., Dugan, E. and Bowen, D.J. (2000) Enhancing adherence in randomized controlled clinical trials. Controlled Clinical Trials, 21: S226S232.
    Sidani, S. (2006) Random assignment: A systematic review. In R.R. Bootzin and P.E. McKnight (eds), Strengthening Research Methodology: Psychological Measurement and Evaluation. Washington, DC: American Psychological Association Books, pp. 125141.
    Sidani, S. and Braden, C.J. (1998) Evaluating Nursing Interventions: A Theory-driven Approach. Thousand Oaks, CA: Sage.
    Sidani, S. and Braden, C.J. (2011) Design, Evaluation, and Translation of Nursing Interventions. Ames, IA: Wiley-Blackwell.
    Sidani, S., Epstein, D.R., Bootzin, R.R., Moritz, P. and Sechrest, L. (2007) Alternative Methods for Clinical Research (NP 05075). Final report submitted to the National Institute of Nursing Research.
    Sidani, S., Epstein, D.R. and Miranda, J. (2006) Eliciting patient treatment preferences: a strategy to integrate evidence-based and patient-centred care, Worldviews on Evidence-Based Nursing, 3(3): 116123.
    Sidani, S., Miranda, J., Epstein, D. and Fox, M. (2009a) Influence of treatment preferences on validity: A review. Canadian Journal of Nursing Research, 41 (4): 5267.
    Sidani, S., Epstein, D.R., Bootzin, R.R., Moritz, P. and Miranda, J. (2009b) Assessment of preferences for treatment: Validation of a measure. Research in Nursing and Health, 32: 419431.
    Sidani, S., Guruge, S., Miranda, J., Ford-Gilboe, M. and Varcoe, C. (2010) Cultural adaptation and translation of measures: An integrated method. Research in Nursing and Health, 33: 133143.
    Silverman, E. and skinner, J. (2004) Medicare upcoding and hospital ownership, Journal of Health Economics, 23: 369389.
    Simmons, J.P., Nelson, L.D. and Somonsohn, U. (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22 (11): 13591366..
    Sinnot, P.L., Joyce, V.R. and Barnett, P. (2007) Preference Measurement in Economic Analysis. Guidebook. Menlo Park, CA: VA Palo Alto, Health Economic Resource Center.
    Slater, M.D. (2006) Specification and misspecification of theoretical foundations and logic models for health communication campaigns. Health Communication, 20 (2): 149157.
    Smith, M.L., Glass, G.V. and Miller, T.I. (1980) The Benefits of Psychotherapy. Baltimore, MD: Johns Hopkins University Press.
    Sørensen, H.T., Lash, T.I. and Rothman, K.J. (2006) Beyond randomized controlled trials: A critical comparison of trials with non-randomized studies. Hepatology, 44: 10751082.
    Sox, H.C., Helfand, M., Grimshaw, J., Dickerson, K., The PLoS Medicine, Torvey, D. et al. (2010) Comparative effectiveness research: Challenges for medical journals. Trials, 7 (4): e1000269.
    Spencer, S.M. and Patrick, J.H. (2010) Revisiting traditional survey methodology to recruit and surevy lesbian, gay and bisexual older adults (pp.211218). In D.L. Streiner and S. Sidani (eds), When Research goes off the Rails. New York: Guilford Press.
    Spillane, V., Byrne, M.C., Byrne, M., Leathem, C.S., O’Malley, M. and Cupples, M.E. (2007) Monitoring treatment fidelity in a randomized controlled trial of a complex intervention. Journal of Advanced Nursing, 60 (3): 343352.
    Spiro, S.G., Gower, N.H., Evans, M.T., Facchini, F.M. and Rudd, R.M., on behalf of the Big Lung Trial Steering Committee (2005) Recruitment of patients with lung cancer into a randomised clinical trial: Experience at two centres. Thorax, 55: 463465.
    Staines, G.L., Cleland, C.M. and Blankertz, L. (2006) Counselor confounds in evaluations of vocational rehabilitation methods in substance dependency treatment. Evaluation Review, 30 (2): 139170.
    Stalmeier, P.F.M., Van Tol-Geerdink, J.J., Van Lin, E.N.J.T., Schimmel, E., Huizenga, H., Van Daal, W.A.J. et al. (2007) Doctors’ and patients’ preferences for participation and treatment in curative prostate cancer radiotherapy. Journal of Clinical Oncology, 25 (21): 30963100.
    Stanford Encyclopedia of Philosophy (2008) Counterfactual theories of causation.
    Stein, K.F., Sargeant, J.T. and Rafaels, N. (2007) Establishing fidelity of the independent variable in nursing clinical trials. Nursing Research, 56 (1): 5462.
    Steinke, E.E. (2004) Research ethics, informed consent, and participant recruitment. Clinical Nurse Specialist, 18 (2): 8897.
    Stevens, T. and Ahmedzai, S.H. (2004) Why do breast cancer patients decline entry into randomized trials and how do they feel about their decision later: A prospective, longitudinal, in-depth interview study. Patient Education and Counseling, 52: 341348.
    Stewart-Williams, S. and Podd, J. (2004) The placebo effect: Dissolving the expectancy versus conditioning debate. Psychological Bulletin, 130 (2): 324340.
    Stirman, S.W., DeRubeis, R.J., Brody, P.E. and Crits-Christoph, P. (2003) Are samples in randomized clinical trials of psychotherapy representative of community outpatients? A new methodology and initial findings. Journal of Consulting and Clinical Psychology, 71 (6): 963972.
    Stirman, S.W., DeRubeis, R.J., Crits-Christoph, P. and Rothman, A. (2005) Can the randomized controlled trial literature generalize to nonrandomized patients? Journal of Consulting and Clinical Psychology, 73 (1): 127135.
    Stone, G.W. and Pocock. S.J. (2010) Randomized trials, statistics, and clinical inference. Journal of the American College of Cardiology, 55 (5): 428431.
    Streiner, D.L. (2002) The 2 ‘Es’ of research: Efficacy and effectiveness trials. Canadian Journal of Psychiatry, 47 (6): 552556.
    Streiner, D.L. and Norman, G.R. (2008) Health Measurement Scales: A Practical Guide to Their Development and Use (
    ed.). Oxford: Oxford University Press.
    Streiner, D.L. and Sidani, S. (eds) (2010) When Research Goes Off the Rails: Why it Happens and What You Can Do About it. New York: The Guilford Press.
    Swift, J.K., Callahan, J.L. and Vollmer, B.M. (2011) Preferences. Journal of Clinical Psychology: In Session, 67 (2): 155165.
    Tacher, J.A., Morey, E. and Craighead, W.E. (2005) Using patient characteristics and attitudinal data to identify depression treatment preference groups: A latent-class model. Depression and Anxiety, 21: 4754.
    Tarlov, A.R., Ware, J.E., Greenfield, S., Nelson, E.C., Perrin, E. and Zubkoff, M. (1989) The medical outcomes study: An application of methods for monitoring the results of medical care. Journal of the American Medical Association, 262 (7): 925930.
    Tarrier, N., Liversidge, T. and Gregg, L. (2006) The acceptability and preference for the psychological treatment of PTSD. Behaviour Research and Therapy, 44: 16431656.
    TenHave, T.R., Coyne, J., Salzer, M. and Katz, I. (2003) Research to improve the simple randomized clinical trial. General Hospital Psychiatry, 25: 115123.
    Thomas, E., Croft, P.R., Paterson, S.M., Dziedzic, K. and Hay, E.M. (2004) What influences participants’ treatment preference and can it influence outcome? Results from a primary care-based randomized trial for shoulder pain. British Journal of General Practice, 54: 9396.
    Thomas Becker, P. (2008) Publishing pilot intervention studies. Research in Nursing and Health, 31(1): 13.
    Thoolen, B., de Ridder, D., Bensing, J., Gorter, K. and Rutten, G. (2007) Who participates in diabetes self-management interventions? Issues of recruitment and retainment. The Diabetes Educator, 33 (3): 465474.
    Thorpe, K.E., Zwarenstein, M., Oxman, A.D., Treweek, S., Furberg, C.D., Altman, D.G., Tunis, S., Bergel, E., Harvey, I., Magid, D.J. and Chalkidou, K. (2009) A pragmatic – explanatory continuum indicator summary (PRECIS): A tool to help trial designers. Journal of Clinical Epidemiology, 62: 464475.
    Tilley, N. (2000) Realistic evaluation: An overview. Paper presented at the Founding Conference of the Danish Evaluation Society.
    Torgerson, C.J. and Torgerson, D.J. (2004) The need for pragmatic experimentation in educational research. Economics of Innovation and New Technology, 16 (5): 323330.
    Torgerson, D.J., Klaber-Moffatt, J. and Russell, I.T. (1996) Patient preference in randomized trials: Threat or opportunity? Journal of Health Science Research and Policy, 1 (4): 194197.
    Towne, L. and Hilton, M. (2004) Implementing randomized field trials in education: Report of a workshop. Committee on Research in Education, National Research Council. Washington, DC. National Academy Press.
    Travaodo, L., Grassi, L., Gil, F., Ventura, C., Martins, C. and the Southern European Psycho-Oncology Study (SEPOS) Group (2005) Physician–patient communication among southern European cancer physicians: The influence of psychosocial orientation and burnout. Psycho-Oncology, 14: 661670.
    Tunis, S.R., Stryer, D.B. and Clancy, C.M. (2003) Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. Journal of the American Medical Association, 290 (12): 16241632.
    US Government Accountability Office (2009) Program evaluation: A variety of rigorous methods can help identify effective interventions. Retrieved from:
    Valentine, J.C. and McHugh, C.M. (2007) The effects of attrition on baseline comparability in randomized experiments in education: A meta analysis. Psychological Methods, 12 (3): 268282.
    Vallance, J.K., Courneya, K.S., Taylor, L.M., Plotnikoff, R.C. and Mackey, J.R. (2008) Development and evaluation of a theory-based physical activity guidebook for breast cancer survivors. Health Education and Behavior, 35 (2): 174189.
    Vandelanotte, C., Bourdeaudhuij, I. and Brug, J. (2004) Acceptability and feasibility of an interactive computer-tailored fat intake intervention in Belgium. Health Promotion International, 19 (4): 463470.
    Van der Berg, M., Timmermans, D.R.M., Knol, D.L., Van Eijk, J.M., de Smit, D., Van Vugt,J.M.G. and Van der Wal, G. (2008) Understanding pregnant women’s decision making concerning prenatal screening. Health Psychology, 27 (4): 430437.
    Van der Wejden, T., Légaré, F., Boivin, A., Burgers, J.S., Van Veenendaal, H., Stiggelbout, A.M., Faber, M. and Elwyn, G. (2010) How to integrate individual patient values and preferences in clinical practice guidelines? A research protocol. Implementation Science, 5: 1018.
    Van Die, D.M., Bone, K.M., Burger, H.G. and Teede, H.J. (2009) Are we drawing the right conclusions from randomized placebo-controlled trials? A post-hoc analysis of data from a randomized controlled trial. BioMed Central Medical Research Methodology, 9: 4147.
    Veitch, C., Hollins, J., Worley, P. and Mitchell, G. (2001) General practice research: Problems and solutions in participant recruitment retention. Australian Family Physician, 30 (4): 399406.
    Victora, C.G., Habicht, J.-P. and Bryce, J. (2004) Evidence based public health: Moving beyond randomized trials. American Journal of Public Health, 94 (3): 400406.
    Vist, G.E., Hagen, K.B., Devereaux, P.J., Bryant, D., Kristofferson, D.T. and Oxman, A.D. (2005) Systematic review to determine whether participation in a trial influences outcome. British Medical Journal, 330: 11751181.
    Vuorma, S., Teperi, J., Hurskainen, R., Aalto, A., Rissanen, P. and Kujansuu, E. (2003) Correlates of women’s preferences for treatment of heavy menstrual bleeding. Patient Education Counseling, 49: 125132.
    Waller, G. (2009) Evidence-based treatment and therapist drift, Behavioural Research and Therapy, 47: 119127.
    Waltz, J., Addis, M.E., Koerner, K. and Jacobson, N.S. (1993) Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61 (4): 620630.
    Wampold, B.E. and Brown, G.S. (2005) Estimating variability in outcomes attributable to therapists: A naturalistic study of outcomes in managed care. Journal of Consulting and Clinical Psychology, 73 (5): 914923.
    Watson, B., Procter, S. and Cochrane, W. (2004) Using randomized controlled trials (RCTS) to test service interventions: Issues of standardisation, selection and generalisability. Nurse Researcher, 11 (3): 2842.
    Weijer, C., Grimshaw, J.M., Taljaard, M., Binik, A., Boruch, R., Brehaut, J.C., Donner, A., Eccles, M.P., Gallo, A., McRae, A.D., Saginur, R. and Zwarenstein, M. (2011) Ethical issues posed by cluster randomized trials in health research. Trials, 12: 100.
    Weinberger, M., Oddone, E.Z., Henderson, W.G., Smith, D.M., Huey, J., Giobbie-Hurder, A. and Feussner, J.R. (2001) Multisite randomized controlled trials in health services research: Scientific challenges and operational issues. Medical Care, 39 (6): 627634.
    Weinberger, M., Oddone, E.Z., Henderson, W.G., Smith, D.M., Huey, J., Giobbie-Hurder, A. and Feussner, J.R. (2002a) Multisite randomized controlled trials in health services research: Scientific challenges and operational issues. Medical Care, 39: 627634.
    Weinberger, M., Murray, M.D., Marrero, D.G., Brewer, N., Lykens, M., Harris, L.E. et al. (2002b) Issues in conducting randomized controlled trials of health services research interventions in nonacademic practice settings: The case of retail pharmacies. Health Services Research, 37(4): 10671077.
    Welleck, S. and Blettner, M. (2012) On the proper use of the crossover design in clinical trials: part 18 of a series on evaluation of scientific publications. Deutsches Arzteblatt International, 109(15): 276281.
    Wertz, R.T. (1995) Intention to treat: Once randomized, always analyzed. Clinical Aphasiology, 23: 5764.
    West, S.G. and Thoemmes, F. (2010) Campbell’s and Rubin’s perspectives on causal inference. Psychological Methods, 15 (1): 1837.
    Westen, D., Novotny, C.M. and Thompson-Brenner, H. (2004) The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials. Psychological Bulletin, 130 (4): 631663.
    Westerberg, V.S., Miller, W.R. and Tonigan, J.S. (2000) Comparison of outcomes for clients in randomized versus open trials of treatment for alcohol use disorders. Journal of Studies on Alcohol, 6(5): 720732.
    Wierzbicki, M. and Pekarik, G. (1993) A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice, 24 (2): 190195.
    Wilder, E.T. and Hollister, R. (2002) How Close is Close Enough? Testing Non-experimental Estimates of Impact against Experimental Estimates of Impact with Education Test Scores as Outcomes. Madison, WI: Institute for Research on Poverty.
    Williams, A.C., Nicholas, M.K., Richardson, P.H., Pither, C.E. and Fernandes, J. (1999) Generalizing from a controlled trial: The effects of patient preference versus randomization on the outcome of impatient versus outpatient chronic pain management. Pain, 83: 5765.
    Wilson, D.B. and Lipsey, M.W. (2001) The role of method in treatment effectiveness research: Evidence from meta-analysis. Psychological Methods, 6 (4): 413429.
    Wiltsey-Stirman, S., DeRubeis, R.J., Crits-Christoph, P. and Brody, P.E. (2003) Are samples in randomized controlled trials of psychotherapy representative of community outpatients? A new methodology and initial findings. Journal of Consulting and Clinical Psychology, 71 (6): 963972.
    Wingerson, D., Sullivan, M., Dager, S., Flick, S., Dunner, D. and Roy-Byrne, P. (1993) Personality traits and early discontinuation from clinical trials in anxious patients. Journal of Clinical Psychopharmacology, 13 (3): 194197.
    Wood, L., Egger, M., Gluud, L.L., Schulz, K.F., Jüni, P., Altman, D.G., Gluud, C., Martin, R.M., Wood, A.J. and Sterne, J.A. (2008) Empirical evidence of bias in treatment effect sizes in controlled trials with different interventions and outcomes: Meta-epidemiological study. British Medical Journal, 336: 601.
    Woods, M., Harris, K., Mayo, M., Catley, D., Scheibmeir, M. and Ahluwalia, J. (2002) Participation of African Americans in a smoking cessation trial: A quantitative and qualitative study. Journal of the National Medical Association, 94 (7): 609618.
    Worrall, J. (2002) What evidence in evidence-based medicine? Philosophy of Science, 69 (S3): S316S330.
    Yong, E. (2012) Bad copy. Nature, 485: 1298300.
    Young, B., Beidel, D., Turner, S., Ammerman, R., McGraw, K. and Coaston, S. (2006) Pretreatment attrition and childhood social phobia. Anxiety Disorders, 20: 11331147.
    Zatzick, D.F., Kang, S.-M., Hinton, L. et al. (2001) Posttraumatic concerns: A patient-centered approach to outcome assessment after traumatic physical injury. Medical Care, 39 (4): 327339.
    Zelen, M. (1979) A new design for randomized clinical trials. The New England Journal of Medicine, 300: 12421245.
    Zoeller, L.A., Feeny, N.C., Cochran, B. and Pruitt, L. (2003) Treatment choice for PTSD. Behaviour Research and Therapy, 41: 879886.
    Zwarenstein, M. and Treweek, S. (2009) What kind of randomized trials do we need? Canadian Medical Association Journal, 180 (10): 9981000.

    • Loading...
Back to Top

Copy and paste the following HTML into your website