Most of the linear operators of interest to neuroscience can
be computed efficiently by neural networks.
This is because such operators have an orthonormal set
of eigenfunctions 
with
associated eigenvalues 
.
Therefore the operator can be written as a summation:
[
L=
_k _k (_k ) _k ,
]
a procedure we call factoring a linear
operator through a discrete space.
This is an infinite sum, but there are only a finite number
of eigenvalues greater than any fixed bound, so that the
operator can be approximated by finite sums.
The computation 
is accomplished in two steps.
In the first, inner products are formed between the input
field and each of the eigenfunctions 
yielding a
finite-dimensional vector 
, given by
.
Each of these inner products could, in principal, be computed
by a single neuron.
This step effectively represents the input in a
finite-dimensional vector space, that is, in a space with no
significant topology (i.e., the axes are independent, none
are nearer to each other than to the others).
In the second step, the computed coefficients are used to
amplitude-modulate the generation of fixed fields
(specifically, the eigenfunctions), which are superposed to
yield the output field:
.
This computation, likewise, can be computed by a single layer
of neurons.
Even if the eigenfunctions of the operator are not known, in
practical cases the operator can still be factored through a
discrete space, since it can be computed via a
finite-dimensional representation in terms of any orthonormal basis
for the input space.
First compute the coefficients by inner products with the
basis functions,
(accomplished by neurons with receptive fields 
).
A finite-dimensional matrix product, 
is
computed by a single-layer neural network with fixed
interconnection weights:
[ M_jk = _j L_k . ]
Again, topological relations between the vector and matrix
elements are not significant, so there are few constraints on
their neural arrangement.
The output is a superposition of basis functions weighted by
the computed 
,
(accomplished by neurons with output weight patterns
).
Computing the linear operator by means of the low-dimensional
space spanned by the basis functions avoids the biologically
unrealistic dense (all-to-all) connections implicit in the
direct computation of the operator:
.
(The preceding results are easily extended to the case where
the input and output spaces have different basis fields.)