In the nonlinear case, the variation in the state field
is a nonlinear function F of the state and the input
field
:
[ .(t) = F[ (t), (t) ] . ]
Many computational processes, especially optimization
processes, can be described by gradient descent; this
is most commonly seen in low-dimensional vector spaces, but
applies as well to field computation, as will now be
explained.
Often the suitability of a field
for some purpose can
be measured by a scalar function
(for reasons that
will become apparent, we will take lower numbers to
represent greater suitability).
For example,
might represent an interpretation of
sensory data and
might represent the internal
incoherence of that interpretation (so that the lowest
gives the most coherent
).
More relevantly,
might represent a motor plan of some
kind, and
the difficulty, in some sense, of that
plan. Then minimizing
gives an optimal plan.
By analogy with physical processes,
is called a
potential function.
One way to find a state
that minimizes U is by a
gradient-descent process, that is, a process that
causes
to follow the gradient
of the
potential.
The gradient is defined:
[
(U)_x
= U_x
]
(where, for notational convenience, we treat the field
as a high-dimensional vector).
The gradient
is a field (over the same
domain as
) giving the ``direction'' of change that
most rapidly increases U, that is, the relative changes to
areas of
that will most rapidly increase U.
Conversely, the negative gradient
gives the
direction of change that most rapidly decreases U.
(This is because
is linear and so
.)
In a gradient-descent process the change of state is
proportional to the negative gradient of the state's
potential:
[ . = -r U() . ]
(The constant r determines the rate at which the process
takes place.)
The resulting ``velocity'' field
is called a
potential flow.
It is easy to show that a gradient-descent process cannot increase the potential, and indeed it must decrease it unless it is at a (possibly local) minimum (or other saddle point). In this way gradient-descent can be used for optimization (although, in general, we cannot guarantee that a global minimum will be found).
A common, special case occurs when the potential is a
quadratic function:
[
U() =
Q + + ,
]
where by
we mean the quadratic form:
[
Q
= ___x Q_xy _y y x .
]
The coupling field Q, which is of higher type than
(i.e., Q is a field over
),
is required to be symmetric
(
).
In this case the gradient has a very simple (first degree)
form:
[
U () = 2 Q + ,
]
where, as usual,
is the integral operator
.
In many cases
and gradient descent is a linear
process:
[ . = -r Q . ]
Notice that
represents the coupling between regions
x and y of the state field and therefore how the
potential varies with coherence between activity in these
parts of the field.
If
then the potential will be lower to the
extent
and
covary (are positive at the
same time or negative at the same time) since then
;
if
, the potential will be lower to the extent
they contravary.
Thus
gives the change to
that maximally
decreases the potential according to the covariances and
contravariances requested by Q.