Intelligence artificial

Learning and Training In the last chapter, we presented an overview of different neural network models. In this chapter, we continue the broad discussion of neural networks with two important topics: Learning and Training. Here are key questions that we would like to answer: • • • • How do neural networks learn? What does it mean for a network to learn ? What differences are there betweensupervised and unsupervised learning ? What training regimens are in common use for neural networks?

Objective of Learning There are many varieties of neural networks. In the final analysis, On network modeling, all neural networks do one or more of the following : • Pattern classification • Pattern completion • Optimization • Data clustering • Approximation • Function evaluation A neural network, inany of the previous tasks, maps a set of inputs to a set of outputs. This nonlinear mapping can be thought of as a multidimensional mapping surface. The objective of learning is to mold the mapping surface according to a desired response, either with or without an explicit training process. Learning and Training A network can learn when training is used, or the network can learn also in theabsence of training. The difference between supervised and unsupervised training is that, in the former case, external prototypes are used as target outputs for specific inputs, and the network is given a learning algorithm to follow and calculate new connection weights that bring the output closer to the target output. Unsupervised learning is the sort of learning that takes place without a teacher.For example, when you are finding your way out of a labyrinth, no teacher is present. You learn from the responses or events that develop as you try to feel your way through the maze. For neural networks, in the unsupervised case, a learning algorithm may be given but target outputs are not given. In such a case, data input to the network gets clustered together; similar input stimuli cause similarresponses. When a neural network model is developed and an appropriate learning algorithm is proposed, it would be based on the theory supporting the model. Since the dynamics of the operation of the neural network is under study, the learning equations are initially formulated in terms of differential equations. After solving the differential equations, and using any initial conditions that areavailable, the algorithm could be simplified to consist of an algebraic equation for the changes in the weights. These simple forms of learning equations are available for your neural networks. At this point of our discussion you need to know what learning algorithms are available, and what they look like. We will now discuss two main rules for learning—Hebbian learning, used with unsupervisedlearning and the delta rule, used with supervised learning. Adaptations of

these by simple modifications to suit a particular context generate many other learning rules in use today. Following the discussion of these two rules, we present variations for each of the two classes of learning: supervised learning and unsupervised learning. Hebb’s Rule Learning algorithms are usually referred to aslearning rules. The foremost such rule is due to Donald Hebb. Hebb’s rule is a statement about how the firing of one neuron, which has a role in the determination of the activation of another neuron, affects the first neuron’s influence on the activation of the second neuron, especially if it is done in a repetitive manner. As a learning rule, Hebb’s observation translates into a formula for thedifference in a connection weight between two neurons from one iteration to the next, as a constant [mu] times the product of activations of the two neurons. How a connection weight is to be modified is what the learning rule suggests. In the case of Hebb’s rule, it is adding the quantity [mu]aiaj, where ai is the activation of the ith neuron, and aj is the activation of the jth neuron to the...