Error Rates

Signal Detection Theory

One particularly powerful data-analytic approach employing error rates is Signal Detection Theory (SDT). SDT is applied in situations where the task involves judging whether a signal exists (e.g., “Was a word presented previously, or is it a new word?”). Using error rates in a series of trials in a task, SDT mathematically derives characteristics of participants’ response patterns such as sensitivity (the perceived distinction between a signal and noise) and judgment criterion (the tendency to respond in one way rather than the other).

Typically, SDT is based on the following assumptions. First, in each trial, either a signal exists or it does not (e.g., a given word was presented previously or not). Even when there is no signal (i.e., the correct response would be negative), the perceived intensity of the stimuli varies randomly (caused by factors originating from the task or from the perceiver), which is called “noise.” Noise follows a normal distribution with a mean of zero. Noise always accompanies signal, and because noise is added to signal, the distribution of perceived intensity of signal has the same (normal) shape. Each perceiver is assumed to have an internal set criterion (called threshold) used to make decisions in the task. If the perceived intensity (e.g., subjective familiarity) of the stimulus is stronger than the threshold, the perceiver will decide that there is a signal (respond affirmatively—e.g., indicate that the word was presented previously); otherwise, the perceiver will respond negatively. When the response is not consistent with the objective properties of the stimulus (e.g., a negative response to a word that was presented previously or an affirmative response to a word that was not), it is an error.

...