Networks of Analog, Discrete, Noisy Neurons

Analog neurons, spike rates, two-state neural models

The two-state representation of neural output, which enjoys such a wide popularity among modelers of neural networks, is often considered oversimplified both by biologists and device designers. Biologists prefer to describe relevant neural activity by firing rates. These are continuous variables describing mean spike activity of neurons, rather than the discrete variables which describe the presence or the absence of an individual spike. Device designers prefer sometimes to think in terms of operational amplifiers, currents, capacitances, resistors, and continuous time equations. It turns out that in a wide range of parameters the performance of a network as an ANN is largely independent of the representation[1]. As we shall see in the following chapters, e.g., Chapter 5, the discrete representation provides a more transparent framework for structured manipulations of attractors.

Eventually the gap between the different descriptions, closes, because in our formulation of the output mechanism, Section 1.4.4, a significant event is said to occur when a time average over spike activity is found to be high, which is nothing but a measure of mean firing rates. On the other hand, in the analog description, in terms of electronic components, which is deterministic in structure, one eventually generates spikes, stochastically, at a mean instantaneous rate proportional to the continuous variable at hand, e.g., the potential excess over the threshold.