Category Learning, Computational Perspectives

A Balancing Act between Flexibility and Bias

Any learning system faces a trade-off that is known in statistics as the bias-variance dilemma. This trade-off involves finding the right balance of inductive bias and flexibility when learning the category function from a limited set of examples (as people do). Inductive bias guides a model's interpretation of data. To make an analogy, people have an inductive bias to view events co-occurring in time (e.g., smoke and fire) as causally related. A strong bias constrains the form of the category function that a model considers. For example, prototype models are strongly biased to only learn linear mappings (i.e., functions) from stimuli to categories, because prototype models represent categories by a single [Page 139]average (i.e., abstraction) of category members. For example, a prototype model would represent the category of birds as a single point (i.e., the prototype) that is the average of the features (e.g., size, color) of all birds (e.g., eagles, robins, penguins, sparrows). In practice, prototype models are best for learning categories that have a common family resemblance structure. For example, for the category birds, most birds have characteristics in common—they tend to be small, have wings, can fly, and so on. However, other items violate this structure (e.g., penguins, bats). Thus, the prototype model will have trouble with these items as they go against its bias of categories consisting of one single clump of items. Other models, such as exemplar models, are weakly biased. Rather than averaging items together in memory, the exemplar model stores each item separately, which allows it to learn any possible function. For example, an exemplar model would represent the category of birds as the collection of all birds (i.e., one point for each category member). Exemplar models can learn any category function, whether it be linear or nonlinear. However, even the exemplar model has biases because it will learn some functions more rapidly (i.e., require fewer training examples) than others.

...