Discriminant Analysis

The need for classification arises in most scientific pursuits. Typically, there is interest in classifying an entity, say, an individual or object, on the basis of some characteristics (feature variables) measured on the entity. This classification is usually undertaken in the context where there is a finite number, say, g, of predefined distinct populations, categories, classes, or groups, and the entity to be classified is assumed to belong to one (and only one) of these g possible groups. In order to assist with the construction of a classification rule or classifier for this purpose, there are usually available so-called training data from each group; that is, these training data comprise the features measured on some entities that are classified with respect to the g underlying groups. In statistical terminology, this classification process is referred to as discriminant analysis, whereas in pattern recognition and machine learning, it is referred to as supervised learning or learning with a teacher.

We let G1,…, Gg denote the g possible groups, and we suppose that a (feature) vector x containing p variables can be measured on the entity. The group membership of the entity is denoted by the categorical variable z, where z = i implies that the entity belongs to Gi (i = 1,…, g). The problem is to estimate or predict z solely on the basis of x and the associated training data. An example in which an outright assignment is required concerns the rejection or acceptance of loan applicants by a financial institution. For this decision problem, there are two groups: G1 refers to applicants who will service their loans satisfactorily, and G2 refers to those who will not. The feature vector x for an applicant contains information such as age, income, and marital status. A rule based on x for allocating an applicant to either G1 or G2 (that is, either accepting or rejecting the loan application) can be formed from an analysis of the feature vectors of past applicants from each of the two groups. In some applications, no assignment of the entity to one of the possible groups is intended. Rather, the problem is to draw inferences about the relationship between z and the feature variables in x. An experiment might be designed with the specific aim to provide insight into the predictive structure of the feature variables. For example, a political scientist may wish to determine the socioeconomic factors that have the most influence on the voting patterns of a population of voters.