Artificial Neural Networks

Research on artificial neural network modeling started in the early 1940s when the first scientific paper by Warren McCulloch and Walter Pitts was published. The motivation came from the fields of artificial intelligence and neuroscience when initial investigators attempted to model the workings of neurons in the human brain. One of the hypothesized reasons for the brain's superiority compared with common computers lies in the fact that neurons function in parallel. There are approximately 1012 neurons in the human brain, all interconnected and receiving input from many other neurons, as well as stimulating many others in a conglomeration of complex interconnections. Thus, neural networks are able to perform highly complex computing tasks in an efficient and powerful manner. In addition, they are able to integrate newly acquired data, or experiences, into existing ones, thus allowing for efficient learning and inference. Figure 1 illustrates a basic representation of the neuron and how it gets activated to fire (stimulate) other connected neurons.

As shown in Figure 1, the neuron collects and processes input from structures referred to as dendrites. It then sends out electrical activity through a long strand called an axon. This axon splits into multiple branches, and at the end of a branch, a synapse converts the electrical activity from the axon and sends stimuli to the neighboring neuron. This activity is either excitatory or inhibitory. Prior information affects signal transfer functions and influences how neurons respond to any future stimuli; synaptic processing mimics learning in this sense. Information transmission and processing across multiple neurons influence the development of artificial neural network models.

The simplest representation of a single-layer artificial neural network is shown in Figure 2. Similar to neuronal processing, information is passed between nodes (neurons) interconnected by links (synapses) with modifiable weights. In the case of a single-layer neural network, input into the node is often represented as a vector of features X = (x1, x2, …, xn). (A single-layer network is also referred to as a two-layer network corresponding to the number of layers of input and output units. Often, it is referred to as a single-layer because there is only one layer of modifiable weights.) Each of these feature values, xi, is multiplied by a corresponding weight, wi. Thus, the effective input at the output unit would be the sum of all the products Σwixi. Adding a constant bias term (x0 = 1) with a corresponding weight w0 produces the formula for a single-unit perceptron.

This could then be represented as

A clinical scenario where an artificial neural network would be useful would be in predicting mortality after a procedure (e.g., angioplasty) in patients with chronic renal failure. The input might comprise several clinical features, such as age, gender, hypertension, diabetes, heart failure, and coronary artery involvement. The output would be mortality after 6 months, which is binary in this example although not necessarily so for artificial neural networks. The artificial neural network is useful for achieving increased accuracy of prediction when the features might have nonlinear interactions.

The input into the node is then processed to generate an optimal output. This is determined by a function y = g(f(x,w)). Typically, f(x, w) is linear as in Equation 1. The function g, on the other [Page 30]hand, is commonly referred to as the activation function. It is chosen from a selection of functions, including the

...