Neural Network Models

A neural network, also referred to as an artificial neural network, consists of a network of units or artificial neurons. The network is based on an analogy to the human brain, where information is processed by networks of interconnected neurons. There are two main characteristics of an artificial neural network that make it analogous to a biological network of neurons. Each biological neuron receives information from various neighboring neurons through inhibitory and excitatory presynaptic connections. The neuron's activation will depend on the combined input from neighboring neurons.

This combined input is processed by the neuron and transmitted to its neighboring neurons. Likewise, each neuron in a neural network model acts like an information processing unit that receives information from neighboring neurons and transmits information to neighboring neurons. The excitatory and inhibitory connections to a specific neuron are modeled by connection weights that may be positive or negative. Additionally, the output from each unit, also known as its activation, is set to a specific range, typically between 0 and 1. This is done by applying a “squashing function” to the summed input from all neighboring units.

Research in the field of neural network models reached a peak in the mid-1990s. Neural network models are used for a wide variety of tasks, comprising pattern recognition or classification and function approximation. They are considered one [Page 806]among several machine learning techniques used for classification and regression. A neural network models the human brain. Analogous to the human brain, it acquires domain-specific knowledge through repeated exposure (i.e., learning) and uses that knowledge for classification or prediction when presented with new inputs within that domain. Therefore, it learns to generalize previously learned information to new situations. For example, a neural network might learn to classify a predetermined set of happy and sad melodies into its respective emotion categories based on the acoustic features within these melodies. Then, it can use that learned knowledge to classify a new set of melodies into happy and sad categories.

The architecture of a neural network typically consists of an input layer, one or more hidden layers, and an output layer. Each layer consists of a predetermined number of units. Neural networks may be supervised or unsupervised. In a classification network that is supervised, the output categories are already determined. So, the network is trained on input sets with known outputs such that the error in prediction is considerably reduced. This error is calculated by computing the difference between the network's predicted outputs for a given set of inputs, and the desired outputs for that input set. A training algorithm is applied to update the network's connection weights based on the error. Once the error is sufficiently minimized, the network is then tested on a new set of inputs to see how well its prediction matches the desired outputs. Here, the network is supervised because the categories are already specified. In an unsupervised classification network, the network learns to cluster melodies into different categories that are not labeled using specific emotions.