RBF Networks

One kind of field transformation, which is very useful and may be quite common in the brain, is similar to a radial basis function (RBF) neural network. The input field is a computational map, which encodes significant stimulus values by the location of peak activity within the field (similar to the direction fields already discussed). The transformation has two stages. The first stage is a convolution between the input field and a local field (such as a Gaussian); this ``coarse codes'' the stimulus as a pattern of activity. (We do not require the local field to be strictly radial, although it commonly is.) This stage is implemented by a layer of neurons with identical receptive field profiles given by the local field.

The second stage is a linear transformation of the coarse-coded field, which yields the output field; it is also implemented by a single layer of neurons. Thus the transformation is given by $L(\rho \otimes \phi)$), where $\phi$ is the input, $\rho$ is the local field, and L is the linear transformation.

Notice that this transformation is linear in its input field (which does not imply, however, that it is a linear function of the stimulus values). Since, if there are several significant stimuli, the input field will be a superposition of fields if the individual stimuli, the output will likewise be a superposition of the corresponding individual outputs. Thus this transformation supports a limited kind of parallel computation in superposition. This is especially useful when the output, like the input, is a computational map.

It has been shown (Lowe 1991, Moody & Darken 1989, Wettschereck & Dietterich 1992) that simple networks of this form are universal in an important sense, and can adapt through a simple learning algorithm. For example, as we saw for direction fields, and input vector ${\bf r}$ can be coded by a vector field ${\bf D}$ to yield a scalar field ${\bf r}\cdot{\bf D}$, which is linearly transformed $L({\bf r}\cdot{\bf D})$. Learning proceeds by slow adaptation of the encoding vector field ${\bf D}$ and by fast adaptation of the kernel field L.