Abstract.

For a continuous unit endowed with a neuron-like nonlinearity (monotonically increasing with threshold and saturation), the transformation of the information entropy in the input-output transmission is derived. A principle of maximization of the output entropy is introduced, that we associate to an optimal representation of information. With the neural nonlinearity, Gaussian input signals lead nearly to the absolute maximum of the output entropy, suggesting an optimal adaptation of this nonlinearity to the physical environment. For arbitrary signals, adaptive maximization of the output entropy leads to a synaptic plasticity algorithm of Hebbian character.