Pilot Study

Importance

All researchers seek to obtain reliable and valid data to answer their research questions or hypotheses. However, although researchers generally try to be quite meticulous, measurement error can still easily occur as a result of problems with questionnaire design, improperly trained interviewers, and so on. Therefore, pilot studies should be a normal component of good research design. Such studies can save researchers both time and money because logistical problems and other design deficiencies can be identified prior to the real study, and corrections and adjustments can be made before the main study is executed.

In some cases, more than one pilot study may be necessary. For example, the first pilot study may be an expert review or focus group to help determine and/or refine the types of questions that should be included in a particular questionnaire. The research instrument can then be prepared and a second pilot study conducted to evaluate other issues, such as clarity of instructions, clarity of questions, and so on. Alternatively, if the initial pilot study had started with an evaluation of the questionnaire, this may have revealed such an abundance of problems that it may be necessary to conduct a second pilot to ensure that all or most of the identified problems are properly addressed to minimize or avoid measurement error.

Pilot studies have been in use since the 1940s. They are suitable for both quantitative and qualitative studies and are helpful for determining suitability of instruments, data collection procedures, and sample population, to name a few. Additionally, pilot studies also serve other useful purposes. For example, pilot tests may help to convince funders that the particular study is worthy of funding. Or, in the case of a trial run, the pilot may reveal that the proposed relationship between variables may not exist, thus signaling to the researcher that the main study is no longer warranted. Pilot studies may even signal problems with local politics.

Procedure

The sample population for a pilot study should closely mirror the intended targeted population. Thus, if the targeted population is university graduates, then the pilot participants should also be [Page 1033]university graduates. Generally, a convenience sample is used with about 50 to 100 participants—some studies may use even fewer participants. Compared to a pretest, a trial run or feasibility study is much more comprehensive because it is, in effect, a miniature version of the real research study. This means that it is executed using the same administration procedures that would be used to carry out the real study. Thus, if the research study intended to use interviews as the primary data collection method, the trial run should not use only experienced interviewers for the pilot because these interviewers would already be trained to deal with problematic situations.

...