next up previous
Next: Learning Up: Field Dynamics Previous: Linear Dynamics

Nonlinear Dynamics

 

In the nonlinear case, the variation in the state field is a nonlinear function F of the state and the input field : [ .(t) = F[ (t), (t) ] . ]

  Many computational processes, especially optimization processes, can be described by gradient descent; this is most commonly seen in low-dimensional vector spaces, but applies as well to field computation, as will now be explained. Often the suitability of a field for some purpose can be measured by a scalar function (for reasons that will become apparent, we will take lower numbers to represent greater suitability). For example, might represent an interpretation of sensory data and might represent the internal incoherence of that interpretation (so that the lowest gives the most coherent ). More relevantly, might represent a motor plan of some kind, and the difficulty, in some sense, of that plan. Then minimizing gives an optimal plan. By analogy with physical processes, is called a potential function.

One way to find a state that minimizes U is by a gradient-descent process, that is, a process that causes to follow the gradient of the potential. The gradient is defined: [ (U)_x = U_x ] (where, for notational convenience, we treat the field as a high-dimensional vector). The gradient is a field (over the same domain as ) giving the ``direction'' of change that most rapidly increases U, that is, the relative changes to areas of that will most rapidly increase U. Conversely, the negative gradient gives the direction of change that most rapidly decreases U. (This is because is linear and so .)

In a gradient-descent process the change of state is proportional to the negative gradient of the state's potential: [ . = -r U() . ] (The constant r determines the rate at which the process takes place.) The resulting ``velocity'' field is called a potential flow.

It is easy to show that a gradient-descent process cannot increase the potential, and indeed it must decrease it unless it is at a (possibly local) minimum (or other saddle point). In this way gradient-descent can be used for optimization (although, in general, we cannot guarantee that a global minimum will be found).

A common, special case occurs when the potential is a quadratic function: [ U() = Q + + , ] where by we mean the quadratic form: [ Q = ___x Q_xy _y y x . ] The coupling field Q, which is of higher type than (i.e., Q is a field over ), is required to be symmetric ( ). In this case the gradient has a very simple (first degree) form: [ U () = 2 Q + , ] where, as usual, is the integral operator . In many cases and gradient descent is a linear process: [ . = -r Q . ]

Notice that represents the coupling between regions x and y of the state field and therefore how the potential varies with coherence between activity in these parts of the field. If then the potential will be lower to the extent and covary (are positive at the same time or negative at the same time) since then ; if , the potential will be lower to the extent they contravary. Thus gives the change to that maximally decreases the potential according to the covariances and contravariances requested by Q.


next up previous
Next: Learning Up: Field Dynamics Previous: Linear Dynamics

Bruce MacLennan
Wed Oct 2 16:55:07 EDT 1996