Decision Boundary

A decision boundary is a partition in n-dimensional space that divides the space into two or more response [Page 236]regions. A decision boundary can take any functional form, but it is often useful to derive the optimal decision boundary that maximizes long-run accuracy.

The use of decision boundaries is widespread and forms the basis of a branch of statistics known as discriminant analysis. Usually, discriminant analysis assumes a linear decision bound and has been applied in many settings. For example, the clinical psychiatrist might be interested in identifying the set of factors that best predicts whether an individual is likely to evidence some clinical disorder. To achieve this goal, the researcher identifies a set of predictor variables taken at Time 1 (e.g., symptoms, neuropsychological test scores, etc.) and then constructs a linear function of these predictors that best separates depressed from nondepressed or schizophrenic from nonschizophrenic patients diagnosed at Time 2. The resulting decision bound then can be applied to symptom and neuropsychological test data collected on new patients to determine whether they are at risk for that clinical disorder later in life. Similar applications can be found in machine learning (e.g., automated speech recognition) and several other domains.

To make this definition more rigorous, suppose we have two categories of clinical disorders, such as depressed and nondepressed individuals with predictor variables in n-dimensional space. Denote the two multivariate probability density functions fD(x) and fND(x) and the two diagnoses RD and RND. To maximize accuracy, it is optimal to use the following decision rule:

Notice that the optimal decision bound is the set of points that satisfies

It is common to assume that fD(x) and fND(x) are multivariate normal. Suppose that μD and μND denote the depressed and nondepressed means, respectively, and that ΣD and ΣND denote the multivariate normal covariance matrices. In addition, suppose that ΣD = ΣND =Σ. Under the latter condition, the optimal decision bound is linear.

Expanding Equation 1 yields

Taking the natural log of both sides of Equation 2 yields

which is linear in x.

As a concrete example, suppose that the objects are two-dimensional with μD = [100 200]′, μND = [200 100], ΣD = ΣND = Σ 50I (where I is the identify matrix). Applying Equation 3 yields