Visual Data Displays

Background

Criticism from both inside and outside sources, as well as the proliferation of qualitative methods in educational research, has led to considerable controversy about standards for the design and conduct of qualitative research. Discussions regarding these standards have failed to address one very important dilemma—questions concerning the credibility and status of qualitative inquiry as related to the privatization of this type of analysis. In short, the basic premise involves how researchers account for and disclose their approach to all aspects of the research process. Central to this premise are the core elements of classical science—replicability and refutability.

Three observations form the basis for this call for public disclosure of decisions made during the conduct of research. First, what exactly does it mean when a researcher writes that “themes emerged”? Readers are expected to take the word of the researcher that he or she did a credible job in data analysis—that the themes that were developed and reported actually have some congruence (verisimilitude) with the reality of the phenomenon studied. Typically, little if any evidence is provided for readers to assess the development of the findings that are offered.

Second, although triangulation, member checks, and other qualitative techniques are mentioned frequently in design or methods sections of research articles, rarely is there evidence of exactly how these were achieved. They are invoked as if magical incantations, and readers must simply believe and trust the researcher—a leap of faith that is sometimes hard to accomplish. Many qualitative researchers will note that they used triangulation to increase the reliability of their research findings. Unfortunately, usually little if any evidence of this is found in the data analysis presented. Results of member checks are rarely reported, and where multiple sources of data are used, the data that are presented do not always adequately represent all of the data sources.

Third, rarely are readers privy to the protocols that are used to collect data. For example, many authors do not provide readers with the interview questions or any hint of those questions. Typically, the analysis that is presented actually leads readers to wonder what the data collection protocols included.

These three observations have led to the conclusion that, in all the discussions of validity in qualitative research, there is one major element missing—the public disclosure of processes. One way in which to achieve this public disclosure is through the creation and inclusion of visual data displays either in the body of the research report or in an appendix. Good qualitative research shows the hand and opens the mind of the investigator to his or her readers. The [Page 931]efforts to do this might not always be successful, but there should be clear paths indicating the attempt has been made.

...