Excitation-Inhibition Balanced Neural Networks for Fast Signal Detection

Excitation-inhibition (E-I) balanced neural networks are a classic model for modeling neural activities and functions in the cortex. The present study investigates the potential application of E-I balanced neural networks for fast signal detection in brain-inspired computation. We first theoretically analyze the response property of an E-I balanced network, and find that the asynchronous firing state of the network generates an optimal noise structure enabling the network to track input changes rapidly. We then extend the homogeneous connectivity of an E-I balanced neural network to include local neuronal connections, so that the network can still achieve fast response and meanwhile maintain spatial information in the face of spatially heterogeneous signal. Finally, we carry out simulations to demonstrate that our model works well.

1. Introduction

To survive in natural environments, animals have developed, through millions of years evolution, the ability to process sensory inputs rapidly. For instance, studies have shown that human subjects can perform complex visual analyses within 150 ms (Thorpe et al., 1996), and the response latency of neurons in the visual cortex of monkeys is as short as tens of milliseconds (Raiguel et al., 1999; Sugase et al., 1999).

Meanwhile, many artificial engineering systems have high demands for real-time processing of rapidly varying signals. This is exemplified by the recently developed Spike Camera (Dong et al., 2017), which has a sampling rate of up to 40, 000 frames per second (fps), far surpassing conventional cameras' 60 fps. This allows it to capture high-speed objects and their textual details, which can be used on real-time motion detection, tracking, and recognition if we have the appropriate algorithms and computing platforms. However, the processing speed of traditional algorithms often cannot meet such demands.

The balance of excitation and inhibition is a general property of neural systems. The excitation-inhibition (E-I) balanced neural network was first proposed to explain the irregular firing of cortical neurons widely observed in the cortex (Softky and Koch, 1993; Shadlen and Newsome, 1994), and was later confirmed by a large amount of experimental data (Haider et al., 2006; Okun and Lampl, 2008; Dorrn et al., 2010; Graupner and Reyes, 2013). Theoretical studies have found that the asynchronous irregular firing state spontaneously emerges in a network of excitatory and inhibitory neurons with random connections satisfying some very loose balancing conditions (van Vreeswijk and Sompolinsky, 1996; van Vreeswijk and Sompolinsky, 1998; Renart et al., 2010). The effects of this chaotic state on optimal coding (Denève and Machens, 2016), working memory (Lim and Goldman, 2014), and neuronal tuning (Hansel and van Vreeswijk, 2012), as well as its coexistence with attractor dynamics (Litwin-Kumar and Doiron, 2012) have been widely studied.

In the present study, we focus on the fast tracking ability of E-I balanced networks, where the population firing rate of the network is proportional to the input amplitude and tracks input changes rapidly (van Vreeswijk and Sompolinsky, 1996; Renart et al., 2010), and investigate how E-I balanced neural networks can be used for fast signal detection in brain-inspired computation. Neuromorphic computing, which mimics the structures and computational principles of the neural system, is receiving increasing attention in artificial intelligence (AI), as it has the potential to overcome the von Neumann bottleneck in modern computers that limits their processing speed (Indiveri and Liu, 2015). The fast response property of the E-I balanced network makes it a naturally compatible candidate to be implemented in neuromorphic systems to achieve rapid information processing.

In the following sections, we show that the asynchronous firing state of the network generates an optimal noise structure which enables the network to track input changes rapidly. We then extend the homogeneous connectivity of the classical E-I balanced neural network to include local neuronal connections, so that the network can achieve fast response and meanwhile maintain the spatial information when presented with spatially heterogeneous signals. Finally, we carry out simulations to demonstrate the performance of our model.

2. Fast Response of a Homogeneous E-I Balanced Network

To illustrate the mechanism of the fast response property, we first investigate a homogeneously connected E-I balanced network.

2.1. Intuition on the Mechanism of Fast Response

The fast response property of an E-I balanced network is at the population level. To understand this, let us consider a non-leaky linear integrate-and-fire neuron, whose dynamics is given by

$\begin{array}{ll}\tau \frac{\text{d}v}{\text{d}t}=I,& (1)\end{array}$

where τ is the integration time constant of the neuron, v the membrane potential, and I the input current. When v reaches the threshold θ, the neuron generates an action potential, and v is reset to the reset potential v₀. Thus, for a constant input I₀, the time it takes for a neuron to generate a spike starting from v₀ is

$\begin{array}{l}T=\tau \frac{\theta -v_0}{I_0}.\end{array}$

It can be seen that the response time of a single neuron is limited by τ (Figure 1A).

FIGURE 1

Figure 1. An illustration of the mechanism of fast response for a neural population. (A) The integration and firing process of a neuron receiving a noiseless input. The integration time is constrained by the membrane time constant. (B) A distribution of membrane potentials across a neural population enables it to respond to input changes rapidly. Red dots represent neurons whose potentials are close to the firing threshold, which are the first ones to respond to input changes.

However, when a neural population receives a signal, if the noise in the system keeps membrane potentials of different neurons at different levels, there will always be a few neurons whose potentials are near the threshold that can quickly respond to input changes. In such a case, the network as a whole can respond to input changes very fast, whose reaction time is only restricted by insurmountable factors such as axonal conduction delays, rather than the membrane time constant τ of individual neurons (Figure 1B). The key of this mechanism is to prevent synchronous firing of neurons and maintain a stable distribution of membrane potentials in the neural population, and asynchronous firing happens to be one of the hallmarks of an E-I balanced network (Renart et al., 2010), which we shall discuss in more detail below.

2.2. The Balancing Condition

We first present the conditions for maintaining an E-I balanced neural network and the stationary population firing rates under those conditions in the large N limit, where N is the number of neurons (van Vreeswijk and Sompolinsky, 1998; Rosenbaum et al., 2017). Consider a network of size N, with N_E = q_EN being excitatory and N_I = q_IN inhibitory, where q_E + q_I = 1. The input current received by neuron i in population a (a = E being excitatory and a = I being inhibitory) can be written as

$\begin{array}{ll}{I}_i^a(t)=F_i^a(t)+R_i^a(t),a=E,I,& (2)\end{array}$

where $F_i^a$ is the feedforward (i.e., external) input, and $R_i^a$ is the recurrent input from other neurons in the network with the form

$\begin{array}{ll}{R}_i^a(t)={\displaystyle \sum _{b=E,I}}{\displaystyle \sum _j}{J}_{ij}^{ab}{\displaystyle \sum _k}\frac{1}{{\tau }_{b,s}}{e}^{-(t-t_{j,k})/{\tau }_{b,s}},a=E,I,& (3)\end{array}$

where j indexes presynaptic neurons, τ_b,s is the synaptic time constant of the presynaptic population b, and t_j,k is the spike time of the k'th spike of neuron j.

Since in both the cortex and industrial applications, the number of neurons in a network is large, we may examine the balanced network in the N → ∞ limit. Expressing the relevant quantities in orders of N can help elucidate the mechanism. Neurons in the network are connected randomly, with the connection probability determined solely by the neuron types. The probability that neuron j in population b connects to neuron i in population a is p_ab for all i, j. Note that here p_ab is constant, and does not tend to 0 as N → ∞. This regime is usually referred to as dense connectivity, in contrast to sparse connectivity where the number of presynaptic neurons for each postsynaptic neuron is kept constant as N → ∞ (van Vreeswijk and Sompolinsky, 1996; Brunel, 2000). If a connection exists, its strength is set to be $J_{ij}^{ab}=j_{ab}/\sqrt{N}$; otherwise $J_{ij}^{ab}=0$. Here $j_{ab}~{O}(1)$. (${O}$ denotes scaling with respect to N → ∞ throughout this paper.) This scaling is a hallmark of balanced networks. Note that in some earlier works, especially those that employ a sparse connectivity regime (van Vreeswijk and Sompolinsky, 1996), this scaling is often written as $J~{O}(\sqrt{K_{ab}})$, where K_ab is the average number of presynaptic inputs from population b for a neuron in population a. Here, since we have K_ab = p_abN and $p_{ab}~{O}(1)$, these two scalings are essentially the same.

Using the mean-field approximation, the time- and population-averaged input current received by a neuron in population a can be written as,

$\begin{array}{ll}\overline{I_a}=\overline{F_a}+\overline{R_a}=\sqrt{N}(f_a{\mu }_0+w_{aE}{r}_E+w_{aI}{r}_I),a=E,I,& (4)\end{array}$

where r_b is the mean firing rate of population b, b = E, I, and $w_{ab}=p_{ab}{j}_{ab}{q}_b~{O}(1)$. Here, we have written $\overline{F_a}$ as $\overline{F_a}=\sqrt{N}{f}_a{\mu }_0$, where $f_a,{\mu }_0~{O}(1)$, because if we notice that long-distance projections are mainly excitatory, and assume that the feedforward inputs originated from another neural population of size ${O}(N)$ and that the feedforward synaptic strength is also of order ${O}(1/\sqrt{N})$, then $\overline{F_a}~{O}(\sqrt{N})$ is a natural consequence. This is exactly the case in the Spike Camera data scenario that we shall examine later in section 3.

Therefore, to keep I (and thus r) bounded when N → ∞, we must have

$\begin{array}{l}{w}_{aE}{r}_E+w_{aI}{r}_I+f_a{\mu }_0~{O}(\frac{1}{\sqrt{N}}),a=E,I.\end{array}$

Letting N → ∞, we get approximate firing rates in the large N limit

$\begin{array}{ll}\begin{array}{c}{\displaystyle \underset{N\to \infty }{lim}}{r}_E=\frac{f_E{w}_{II}-f_I{w}_{EI}}{w_{EI}{w}_{IE}-w_{EE}{w}_{II}}{\mu }_0,\\ {\displaystyle \underset{N\to \infty }{lim}}{r}_I=\frac{f_I{w}_{EE}-f_E{w}_{IE}}{w_{EI}{w}_{IE}-w_{EE}{w}_{II}}{\mu }_0.\end{array}& (5)\end{array}$

To keep the above limits positive and yield a stable solution, it is necessary and sufficient to let (van Vreeswijk and Sompolinsky, 1998)

$\begin{array}{l}\frac{f_E}{f_I}>\frac{w_{EI}}{w_{II}}>\frac{w_{EE}}{w_{IE}}.\end{array}$

This is the condition for the balanced firing state.

It is worth noting that whatever the neuronal transfer function is, the population firing rate in the large N limit is always linearly proportional to μ₀. That is, Equation (5) always holds. This is a direct result of Equation (4), where the total input current is the linear sum of the three ${O}(\sqrt{N})$ order terms. The balanced firing state is a stable solution dynamically formed by the network (van Vreeswijk and Sompolinsky, 1998; Renart et al., 2010), and therefore requires no fine tuning of parameters such as j_ab, which is different from some other models that also try to recreate the asynchronous irregular firing state (e.g., Brunel, 2000).

It should be pointed out that Equation (5) only gives the ${O}(1)$ order term of r_a. To satisfy the specific transfer function of neurons while maintaining the balance of the ${O}(1)$ order term, the firing rates are adjusted by an ${O}(1/\sqrt{N})$ term, which results in a ${O}(1)$ order correction to I (Equation 4). We will come back to this in the specific case presented in the next section.

2.3. The Mechanism of Fast Response

As previously mentioned, the asynchronous firing of neurons is the key for fast response of the network. When the balancing conditions presented in the previous section are met, the network can achieve asynchronous irregular firing (Renart et al., 2010). We next use a network of non-leaky linear integrate and fire neurons to study the mechanism of fast response in more detail. Notably, this simple neuron model has already been implemented in a neuromorphic system (Fusi and Mattia, 1999). While not biologically realistic, this model captures the key characteristics of integrate-and-fire neurons crucial for neuromorphic computing.

The neuronal dynamics is given by Equation (1). For simplicity, let v₀ = 0. It can be easily seen that the transfer function of this neuron is threshold-linear, i.e.,

$\begin{array}{ll}r=\{\begin{array}{cc}\frac{\overline{I}}{\theta \tau },& \overline{I}⩾0,\\ 0,& \overline{I}<0.\end{array}& (6)\end{array}$

Substituting this into Equation (4) yields the population firing rates of excitatory and inhibitory neurons

$\begin{array}{ll}\begin{array}{c}{r}_E=\frac{(f_E{w}_{II}-f_I{w}_{EI})-\frac{1}{\sqrt{N}}{f}_E\theta {\tau }_I}{(w_{EI}{w}_{IE}-w_{EE}{w}_{II})+\frac{1}{\sqrt{N}}\theta (w_{EE}{\tau }_I+w_{II}{\tau }_E)-\frac{1}{N}{\theta }^2{\tau }_I{\tau }_E}{\mu }_0,\\ r_I=\frac{(f_I{w}_{EE}-f_E{w}_{IE})-\frac{1}{\sqrt{N}}{f}_I\theta {\tau }_E}{(w_{EI}{w}_{IE}-w_{EE}{w}_{II})+\frac{1}{\sqrt{N}}\theta (w_{EE}{\tau }_I+w_{II}{\tau }_E)-\frac{1}{N}{\theta }^2{\tau }_I{\tau }_E}{\mu }_0.\end{array}& (7)\end{array}$

Comparing the above result with Equation (5), we can see that they are indeed ${O}(1/\sqrt{N})$ order corrections to the N → ∞ limit, as stated at the end of the last section. Note that the firing rates still linearly encode the external input, which is a result of the threshold-linear transfer function. We also check that even when the external input is small or the number of neurons is not large, the linear encoding property still holds, which expands the dynamic range of the network. However, for other non-linear neuron models, this linear encoding property may not hold.

Equation (7) is derived from the mean-field approximation, that is, it is the result of averaging over time and neurons when the system reaches a stable state. To study how the instantaneous firing rate of the population changes with time when external input changes, we need more detailed analysis. We shall use the Fokker-Planck equation (Risken, 1996) to study the membrane potential distribution p_a(v, t) (Brunel and Hakim, 1999; Fusi and Mattia, 1999; Brunel, 2000; Huang et al., 2011).

First, we examine the input received by a single neuron as described in Equation (2). We consider an external input signal with additive white Gaussian noise

$\begin{array}{ll}{F}_i^a(t)=\sqrt{N}{f}_a{\mu }_F(t)+{\sigma }_{aF}(t){\xi }_i^{aF}(t),a=E,I,i=1,\cdots ,N_a,& (8)\end{array}$

where ${\xi }_i^{aF}$ is a Gaussian white noise of magnitude 1 that is independent across neurons. Note that the signal mean is of order ${O}(\sqrt{N})$, while the variance is of order ${O}(1)$. This is because if we continue to use the settings considered before, and view the feedforward input as coming from Poisson spike trains generated by ${O}(N)$ neurons firing at rates of order ${O}(1)$, and transmitted through synapses with the strength of order ${O}(1/\sqrt{N})$, then the resulting input's variance is the sum of ${O}(N)$ number of terms with the same order as the square of synaptic strengths (${O}(1/N)$), and is therefore of order ${O}(1)$. This characteristic is also present in the later analysis of recurrent inputs.

Next, we examine the recurrent inputs. When the network enters the balanced state, since the neurons fire asynchronously (Renart et al., 2010), and the effect of each spike is small, we could use Gaussian white noise to approximate the variations of recurrent inputs, and rewrite the second term in Equation (2) as (Brunel, 2000)

$\begin{array}{ll}{R}_i^a(t)=\sqrt{N}{\mu }_{aR}(t)+{\sigma }_{aR}(t){\xi }_i^{aR}(t),a=E,I,& (9)\end{array}$

where

$\begin{array}{ll}{\mu }_{aR}=w_{aE}{r}_E+w_{aI}{r}_I,{\sigma }_{aR}^2=j_{aE}{w}_{aE}{r}_E+j_{aI}{w}_{aI}{r}_I,& (10)\end{array}$

and ${\xi }_i^{aR}$ is Gaussian noise of magnitude 1. The terms ${\xi }_i^{aR}$ and ${\xi }_i^{aF}$ are independent due to the asynchronous firing state, and can therefore be merged into one noise source. Thus, we transform Equation (2) into

$\begin{array}{ll}{I}_i^a(t)={\mu }_a(t)+{\sigma }_a(t){\xi }_i^a(t),a=E,I,& (11)\end{array}$

where

$\begin{array}{l}{\mu }_a=\sqrt{N}(w_{aE}{r}_E+w_{aI}{r}_I+f_a{\mu }_F)\\ {\sigma }_a^2=j_{aE}{w}_{aE}{r}_E+j_{aI}{w}_{aI}{r}_I+{\sigma }_{aF}^2,& (12)\end{array}$

and ${\xi }_i^a$ is Gaussian white noise of magnitude 1. Also note that the mean of the signal is consistent with Equation (4), and the mean and variance are both of order ${O}(1)$.

Since the balanced state implies asynchronous firing (Renart et al., 2010), the noise ${\xi }_i^a$ of different neurons can be seen as independent. Then, the excitatory (inhibitory) population can be viewed as i.i.d. samples of the same random process. The membrane potential distribution of population a, p_a(v, t), can thus be derived from Equation (11). We obtain the Fokker-Planck equation (Brunel, 2000; Huang et al., 2011)

$\begin{array}{ll}{\tau }_a\frac{\partial p_a(v,t)}{\partial t}=-{\mu }_a\frac{\partial p_a(v,t)}{\partial v}+\frac{{\sigma }_a^2}{2{\tau }_a}\frac{{\partial }^2{p}_a(v,t)}{\partial v^2},a=E,I.& (13)\end{array}$

A few boundary conditions can be naturally imposed (Brunel and Hakim, 1999; Brunel, 2000):

$\begin{array}{ll}{p}_a(v,t)=0,\forall v⩾\theta .& (14)\end{array}$

$\begin{array}{ll}{p}_a(0^{-},t)=p_a(0^{+},t),& (15)\end{array}$

$\begin{array}{ll}\frac{\partial p_a(0^{+},t)}{\partial v}-\frac{\partial p_a(0^{-},t)}{\partial v}=\frac{\partial p_a(\theta ,t)}{\partial v}.& (16)\end{array}$

$\begin{array}{ll}{\displaystyle {\int }_{-\infty }^{\theta }}{p}_a(v,t)\text{d}v=1.& (17)\end{array}$

In Equation (13), letting ∂p_a/∂t = 0, and using the above boundary conditions, we get the stationary solution

$\begin{array}{ll}{p}_{a0}(v)=\{\begin{array}{cc}\frac{1}{\theta }\left[1-exp\left(-2{\tau }_a{\beta }_a\right)\right]exp\left(\frac{2{\tau }_{a}v}{{\beta }_a}\right),& \hfill v<0\\ \frac{1}{\theta }\left[1-exp\left(\frac{-2{\tau }_a(\theta -v)}{{\beta }_a}\right)\right],& \hfill 0⩽v⩽\theta \\ 0,& \hfill v>\theta \end{array}& (18)\end{array}$

where ${\beta }_a:\; ={\sigma }_a^2/{\mu }_a$ is the variance-to-mean ratio (VMR). This result is confirmed by simulations (Figure 2A).

FIGURE 2

Figure 2. Simulation results of an uncoupled neural population. (A) The membrane potential distribution of a neural population receiving independent white noise-corrupted signals with a constant VMR of 1. The red curve is the theoretical prediction given by Equation (18), and the blue histogram is the actual simulation result. (B) The tracking performance of a neural population depends on the input noise structure. The blue curve is the theoretical prediction of steady-state firing rate given by Equation (19). The red curve is the network performance when the VMR is constant (β = 1), which tracks the input change almost instantaneously. The green curve is the network performance when the noise variance, rather than the VMR, is constant (σ ≡ 1), where a significant delay is present. Other parameters are: N = 2, 500, τ = 1, θ = 1, and μ changing from 1 to 5 at time t = 5.

The population firing rate, i.e., the flux at θ, is

$\begin{array}{ll}{r_a=-\frac{{\sigma }_a^2}{2{\tau }_a^2}\frac{\partial p_{a0}(v)}{\partial v}|}_{\theta }=\frac{{\mu }_a}{\theta {\tau }_a}.& (19)\end{array}$

which is consistent with Equation (6).

It can be seen from Equation (18) that the membrane potential distribution is determined by the VMR β_a. The ideal noise structure is thus obtained when VMR stays constant (Huang et al., 2011), because it ensures that when the external input μ_F changes, the system remains in a stationary state where Equation (19), and thus Equation (7), always holds. In this way, the population rate can track input changes instantaneously and linearly encode μ_F at all times. Figure 2B illustrates how the response time of the population rate is determined by input noise structure.

From Equations (12) and (19), we know that when the network is at the stationary state,

$\begin{array}{l}{\beta }_a=\frac{{\sigma }_a^2}{{\mu }_a}=\frac{j_{aE}{w}_{aE}{r}_E+j_{aI}{w}_{aI}{r}_I+{\sigma }_{aF}^2}{\theta {\tau }_a{r}_a},a=E,I.\end{array}$

From Equation (7), we know r_E, r_I ∝ μ_F. For the ${\sigma }_{aF}^2$ term, if we continue to assume that the external input comes from the Poisson spike trains of another population of neurons, and the changes in μ_F are due to the firing rate of that population, then we have ${\sigma }_{aF}^2\propto {\mu }_F$. Thus, when μ_aF changes, β_a remains constant. This is the ideal noise structure, and the population rate of the network can track the external input instantaneously. In reality, the ideal noise structure can only be approximately satisfied, but the tracking speed of the network is still reasonably fast, as confirmed by Figure 3.

FIGURE 3

Figure 3. Simulation results of a homogeneous E-I balanced network tracking a time-varying input. The network receives a sinusoidal input centered at μ_F = 0.1 with an amplitude of 0.05. ${\sigma }_{aF}^2/{\mu }_F=0.1$ remains constant. The blue curve is the theoretic prediction given by Equation (7). The red curve is the instantaneous average firing rate of excitatory neurons. The parameters are: $N=1\times 10^4,q_I=0.2,p_{ab}=0.25,\theta =15,$${\tau }_E=15,{\tau }_I=10,{\tau }_{E,s}=6,{\tau }_{I,s}=5,f_E=3,$$f_I=2,j_{EE}=0.25,j_{EI}=-1,j_{IE}=0.4,j_{II}=-1$.

It should be pointed out that the neuron model we used in this section does not have a lower bound to its membrane potential. In real applications, a reflecting barrier can be imposed at the reset potential v₀ (Fusi and Mattia, 1999). We verify that this does not affect our main results. The neuron model used in the following sections has a reflecting barrier.

3. Processing Spatially Heterogeneous Input With Local Connectivity

In the above, we have studied an E-I balanced neural network with homogeneous connectivity, which is able to track input changes rapidly. However, when the external input is spatially heterogeneous, that is, when different neurons receive inputs of different magnitudes, this homogeneous connectivity generates statistically equivalent recurrent inputs for each neuron that cannot balance the external inputs. The same $R_a^i$'s cannot balance different $F_a^i$'s, causing neurons to receive inputs of order ${O}(\sqrt{N})$ and fire pathologically. In addition, the random long-range connections between neurons spread out local activities to the entire network, which blurs the spatial location of inputs. In applications, however, we often need to know not only when the signal occurs but also where it occurs. To solve this problem, we need to introduce local connectivity in the network. Previous studies have shown that if appropriate local connectivity is included, the network can maintain the balanced firing state as well as retain the spatial information of the input (Rosenbaum and Doiron, 2014; Rosenbaum et al., 2017), which enables the network to achieve both fast tracking and spatial location encoding. Below, we briefly introduce the balancing conditions and the response property of an E-I balanced neural network with local connectivity.

Here, each neuron is assigned a location (x, y) on the plane, and local connectivity is achieved by a connection probability that decays with the spatial distance between pairs of neurons instead of being homogeneous as in the previous sections, so that neurons closer to each other have higher probabilities to connect with each other. Specifically, the probability of a connection between neurons i and j follows

$\begin{array}{ll}\mathbb{P}(j\text{ connects to }i)\propto G_b(d_{ij}),& (20)\end{array}$

where G_b is a 2-dimensional Gaussian shaped function whose spatial spread is determined by the presynaptic population b, and d_ij is the distance between the neurons.

Similar to Equation (4), we again utilize the mean-field approximation. Only this time, we do not average over the entire population, but rather approximate the neural activity of population a near location x with the neural field

$\begin{array}{l}\overline{I_a}(\text{x})=\overline{F_a}(\text{x})+\overline{R_a}(\text{x})=\sqrt{N}\left[f_a(\text{x})+w_{aE}*r_E(\text{x})-w_{aI}*r_I(\text{x})\right],\\ a=E,I,& (21)\end{array}$

where the feedforward input ${\overline{F}}_a(\text{x})=\sqrt{N}{f}_a(\text{x})$, w_ab(x) = q_bj_abp_abG_b(x) is the mean connectivity a neuron in population a receives from neurons in population b at location x, and r_a(x) is the firing rate. The symbol * denotes the spatial convolution against x.

Similar to section 2.2, we have

$\begin{array}{ll}{w}_{aE}*r_E(\text{x})-w_{aI}*r_I(\text{x})+f_a(\text{x})~{O}(1/\sqrt{N}),a=E,I.& (22)\end{array}$

Let N → ∞ and perform 2-dimensional Fourier transform against x, and we get

$\begin{array}{l}{\tilde{w}}_{aE}{\tilde{r}}_E-{\tilde{w}}_{aI}{\tilde{r}}_I+{\tilde{f}}_a=0,a=E,I,\end{array}$

where the symbol ˜ denotes the spatial Fourier transform. This gives

$\begin{array}{ll}{\tilde{r}}_E=\frac{{\tilde{f}}_E{\tilde{w}}_{II}-{\tilde{f}}_I{\tilde{w}}_{EI}}{{\tilde{w}}_{EI}{\tilde{w}}_{IE}-{\tilde{w}}_{EE}{\tilde{w}}_{II}},{\tilde{r}}_I=\frac{{\tilde{f}}_E{\tilde{w}}_{IE}-{\tilde{f}}_I{\tilde{w}}_{EE}}{{\tilde{w}}_{EI}{\tilde{w}}_{IE}-{\tilde{w}}_{EE}{\tilde{w}}_{II}}.& (23)\end{array}$

To ensure that the above Fourier transform exists, it is necessary that ${\tilde{r}}_a$ tends to 0 as the frequency tends to infinity. This requires that the external input f be “wider” than recurrent input w. This can be understood intuitively from Equation (22), where we see convolution makes w_ab*r_b(x) wider than w_ab(x), so for the terms to balance each other, f has to be “wider” than w. Also, to get a positive stable solution, the following condition has to be met:

$\begin{array}{ll}\frac{{\overline{f}}_E}{{\overline{f}}_I}>\frac{{\overline{w}}_{EI}}{{\overline{w}}_{II}}>\frac{{\overline{w}}_{EE}}{{\overline{w}}_{IE}},& (24)\end{array}$

where the bar represents spatial average. Also, to make the solution stable, w_aE has to be “wider” than w_aI. For a more detailed account of these conditions, see Rosenbaum and Doiron, 2014; Pyle and Rosenbaum, 2017.

Rosenbaum et al. (2017) proved the asynchronous firing state of the network with local connections under the above conditions. Thus, with the premise of asynchronous firing satisfied, our results regarding the optimum noise structure in section 2.3 still holds. Let the total input variance of the neuron in population a at location x be ${\sigma }_a^2(\text{x})$, and the VMR be β_a(x), and we have

$\begin{array}{l}{\sigma }_a^2(\text{x})=j_{aE}{w}_{aE}*r_E(\text{x})+j_{aI}{w}_{aI}*r_I(\text{x}).\end{array}$

The threshold-linear transfer function gives us $\overline{I_a}(\text{x})=\theta {\tau }_a{r}_a(\text{x})$, so we have

$\begin{array}{l}{\beta }_a(\text{x})=\frac{j_{aE}{w}_{aE}*r_E(\text{x})+j_{aI}{w}_{aI}*r_I(\text{x})}{\theta {\tau }_a{r}_a(\text{x})},\end{array}$

Here the division is point-wise at each x. If β_a(x) is constant at each x for arbitrary external input f_a(x), it must be spatially invariant, that is, β_a(x) ≡ β_a. We can thus move the denominator on the r.h.s. to the left, and perform Fourier transform to get

$\begin{array}{l}{\beta }_a\theta {\tau }_a{\tilde{r}}_a=j_{aE}{\tilde{w}}_{aE}*{\tilde{r}}_E(\text{x})+j_{aI}{\tilde{w}}_{aI}*{\tilde{w}}_I(\text{x}),a=E,I.\end{array}$

Substituting it in Equation (23), we get

$\begin{array}{l}\begin{array}{c}-\frac{j_{EE}{\tilde{w}}_{EE}}{j_{EI}{\tilde{w}}_{EI}}=\frac{{\tilde{f}}_E{\tilde{w}}_{IE}-{\tilde{f}}_I{\tilde{w}}_{EE}}{{\tilde{f}}_E{\tilde{w}}_{II}-{\tilde{f}}_I{\tilde{w}}_{EI}},\\ -\frac{j_{II}{\tilde{w}}_{II}}{j_{IE}{\tilde{w}}_{IE}}=\frac{{\tilde{f}}_E{\tilde{w}}_{II}-{\tilde{f}}_I{\tilde{w}}_{EI}}{{\tilde{f}}_E{\tilde{w}}_{IE}-{\tilde{f}}_I{\tilde{w}}_{EE}}.\end{array}\end{array}$

The above equations cannot be satisfied for all f_a, so this network structure cannot maintain an optimum noise structure and track any input instantly. However, for input changes that only concerns magnitude and not the spatial shape, β_a(x) can remain constant and allow instant tracking. For other kinds of input changes, although instantaneous tracking is not possible, the response speed of the network is still significantly smaller than what the neuronal time constant allows, as we shall explore in the next section.

4. Simulation Results

One of the potential applications of the balanced network's fast response property is to process Spike Camera data in real time. Spike Camera is a newly developed neuromorphic hardware that encodes visual signals with spikes (Dong et al., 2017). It consists of artificial ganglion cells, each corresponding to a pixel, that linearly integrate the luminance intensity and fire a spike upon reaching the threshold, converting continuous visual information to discrete spikes. This event-based data transmission method significantly reduces the data volume and allows for a sampling rate of as high as 40,000 fps. Compared to another extremely high-speed camera, the Dynamic Vision Sensor (DVS) (Serrano-Gotarredona and Linares-Barranco, 2013), which only transmits changes in light intensity, Spike Camera can directly encode the absolute value of the luminance signal with its spiking rate while having an even higher sampling rate. In this section, we explore the tracking performance of our network under the setting of processing Spike Camera-like data.

4.1. Network Structure

We use a feedforward layer consisting of 50 × 50 non-leaky linear integrate-and-fire neurons to mimic the Spike Camera. Each neuron in this layer receives visual signal from its corresponding pixel location, and connects to the balanced network layer through feedforward connections $J_{ij}^{aF},a=E,I$. The balanced network layer consists of 80 × 80 excitatory neurons and 40 × 40 inhibitory neurons. The neurons of each population is placed uniformly on a square area with a side length of 1. The neurons in the feedforward layer obeys Equation (1), and have a neuronal time constant of τ_F. To reflect the high sampling rate of Spike Camera, τ_F is set to be very small. The connection probability of the network obeys

$\begin{array}{l}\mathbb{P}(J_{ij}^{ab}=j_{ab}/\sqrt{N})=p_{ab}{G}_b(d_{ij}^{ab}),b=F,E,I,a=E,I,\end{array}$

where F stands for the feedforward layer, G_b is a 2-dimensional Gaussian distribution centered at 0 with scale parameter A_b. To satisfy the balancing conditions, we let A_F > A_E ⩾ A_I and make sure that Equation (24) holds. Since spatial location is discretized in the network, to keep the total connection probability from population b to population a at p_ab, we normalize G_b by letting ${\sum }_i{G}_b(d_{ij}^{ab})=1,\forall j$. Figure 4 demonstrates this structure.

FIGURE 4

Figure 4. Schematic of the network structure in the context of processing Spike Camera data.

4.2. Tracking Time-Varying Stimuli

We test the tracking performance of our network with four example input stimuli. The first stimulus is the sudden appearance of an object, modeled as an abrupt change in input magnitude at the object's location. Figure 5A shows the network's response to this change summarized by the population rate of the excitatory neurons corresponding to the location of interest. We see that in this case, the network's activity tracks the stimulus change very quickly.

FIGURE 5

Figure 5. Performance of the network with local connections in response to time-varying stimuli. (A) Network response to the sudden appearance of an object. The Spike Camera layer receives a disc-shaped visual input centered at (0.25, 0.5) with a radius of 0.05, whose magnitude changes abruptly from 1.5 to 15 at t = 75. A background noise is added. The blue curve is the firing rate of the area corresponding to the visual input in the Spike Camera layer. The red curve is the rate of the excitatory neurons at the same area in the balanced network layer, which is normalized for better comparison with the blue curve. (B) Same as panel (A), except that the input amplitude follows the sinusoidal function μ(t) = A(sin(B*2πt/T)) + C, A = 30, B = 3/2, C = 30. (C,D) The stimulus is an object moving across the visual field in constant velocity. The object has the same shape as panels (A,B), with a magnitude of 10. Panels (C,D) show the tracking of the x and y coordinates, respectively. The blue curve is the object location decoded from the activity of the Spike Camera layer, and the red curve is that of the balanced network layer. (E,F) Same as panels (C,D), except that the stimulus moves counterclockwise on a circle in constant speed. The network parameters are θ = 15, τ_F = 1, τ_E = 15, τ_I = 10, τ_F,s = τ_E,s = 5, τ_I,s = 2.5, p_EF = 0.05, p_IF = 0.025, p_EE = 0.02, p_EI = 0.08, p_IE = 0.06, p_II = 0.08, A_F = 0.05, A_E = 0.02, A_I = 0.02, j_EF = 140, j_IF = 93.3, j_EE = 80, j_EI = −320, j_IE = 40, j_II = −320.

The second stimulus is similar to the previous one, except that the input magnitude continuously changes in a sinusoidal manner. Figure 5B shows the tracking performance of the network. It can be seen that the network can track the stimulus almost instantaneously, which is expected since β_E is constant here.

The third stimulus is an object moving quickly from left to right in the field of vision, which can be seen as a model of a typical motion tracking task. We use the coordinates of the center of the circular object to represent the location of the stimulus. The coordinates calculated from the Spike Camera data and the balanced network activity are then compared in Figures 5C,D. The network activity closely tracks the input, and the spatial information is preserved.

The last stimulus is similar to the previous one, except that the motion is circular instead of linear, which implies a constantly changing velocity. The same method is used to locate the stimulus, and the results are shown in Figures 5E,F. The performance is again very good.

4.3. Trackable Speeds

To explore the extent of the network's tracking ability, we next evaluate the temporal and spatial lags of the response. We first change the frequency of the sinusoidal signal in the second task in the previous section (Figure 5B) and calculate the phase lag of the balanced network's response. As can be seen in Figure 6A, while the phase lag |ϕ| increases when the signal frequency 1/T is higher, the delay is still very small overall.

FIGURE 6

Figure 6. Quantifying the network's performance with temporal and spatial phase lag. To examine the effect of the synaptic time constant on tracking performance, we define τ_b,s = kτ_{b, s0}, b = F, E, I, where τ_{b, s0} is the set of parameters used in Figure 5. (A) Temporal phase lag in the second task (Figure 5B) with different signal periods. (B) Spatial phase lag in the fourth task (Figures 5E,F) with different circular motion periods. Ten trials for each data point. Error bars show standard deviations.

Next, we vary the speed of the object's circular motion in the fourth task in the previous section (Figures 5E,F) and evaluate the spatial phase lag of the object location decoded from the balanced network activity compared to that of the Spike Camera layer. As shown in Figure 6B, the tracking error is small even when the object is moving very quickly.

Since the encoding happens at the population level, input changes have to be propagated through the population to be successfully tracked, and this process is mediated by synaptic interactions. This lead us to suspect that the synaptic time constants τ_b,s could be a limiting factor for tracking performance. To study this, we varied τ_b,s in both the temporal and spatial tracking tasks. Indeed, as can be seen in Figure 6, a shorter synaptic time constant leads to better performance. In practice, the shape of the synaptic current can be designed to have a τ_b,s as small as possible. The real constricting factor is the synaptic transmission delay, which corresponds to the communication speed of the hardware, but this is expected to be insignificant given the highly compact nature of neuromorphic chips.

5. Discussion and Conclusion

This paper proposed an algorithm for fast response in neuromorphic systems based on E-I balanced networks, systematically analyzed its fast response mechanism, and introduced local connections to maintain balance and retain spatial information in the face of spatially heterogeneous inputs. Simulations verified that the network indeed performs well with rapidly changing input stimuli.

There are still some questions left to explore. For instance, we have mentioned that the network cannot keep an optimal noise structure at all times, and thus the membrane potential distribution will change with the input. A study of the transient dynamics during such changes could help us further improve the network performance. As another example, notice that most of the theoretical analyses in the paper were conducted in the limit of N → ∞. In real-world applications, we often have to track small objects, during which the number of neurons encoding it usually does not exceed a few hundred. Studying the finite-size effect could help us better understand the network dynamics.

Although we mainly discussed the case where the input comes from Spike Camera, the network structure we proposed is not limited to processing visual signal. The “location” of neurons can also correspond to tuning to different variables or representation of abstract features. To achieve real-time processing of high-frequency data, the fast response property is required for each computational process. There has been a lot of research discussing how to implement various computations on top of a balanced network (Barrett, 2012; Hansel and van Vreeswijk, 2012; Litwin-Kumar and Doiron, 2012; Lim and Goldman, 2014; Denève and Machens, 2016; Pyle and Rosenbaum, 2017). The asynchronous irregular state can be taken as a model of the spontaneous state in the cortex. With the spontaneous state as a global attractor, and the specific computations and memories as input-sensitive local attractors (Amit and Brunel, 1997; Litwin-Kumar and Doiron, 2012), the chaos in the network's balanced firing state can allow it to respond to specific inputs very rapidly and initiate the required computation. Besides the fast response property, the balanced state also has other computational advantages such as stochastic resonance (Barrett, 2012).

Neuromorphic computing systems colocalize computation and memory by mimicking neural structures like neurons and synapses. This allows it to circumvent the von Neumann bottleneck, granting it enormous potentials in processing speed (Indiveri and Liu, 2015). There has been a lot of work investigating possible mechanisms for fast neural response (e.g., Bharioke and Chklovskii, 2015; Yu et al., 2015) which could potentially complement the processing speed of neuromorphic systems, and the balance of excitation and inhibition we explored here is one of them. The model we proposed here, with its simple neuron model and connectivity structure, can be readily implemented in hardware and serve as a fast-responding module integrated in a general neuromorphic system for rapid information processing. This paper thus lays the groundwork for realizing various kinds of fast computation using balanced networks, especially in neuromorphic systems.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

SW designed the project. GT, SL, and SW wrote the paper. GT did the theoretical analyses. GT, SL, and SW carried out simulations and data analysis. TH contributed important ideas.

Funding

This work was supported by the National Natural Science Foundation of China (No: 61425025, TH) and Guangdong Province with Grant (No. 2018B030338001, SW). This work also received support from Huawei Technology Co., Ltd (YBN2019105137). The authors declare that this study received funding from Huawei Technology Co., Ltd. The funder was not involved in the study design, collection, analysis, interpretation of data, the writing of this article or the decision to submit it for publication.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Amit, D. J., and Brunel, N. (1997). Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cereb. Cortex 7, 237–252. doi: 10.1093/cercor/7.3.237

PubMed Abstract | CrossRef Full Text | Google Scholar

Barrett, D. G. T. (2012). Computation in balanced networks (Ph.D. thesis). University College London, London, United Kingdom.

Google Scholar

Bharioke, A., and Chklovskii, D. B. (2015). Automatic adaptation to fast input changes in a time-invariant neural circuit. PLoS Comput. Biol. 11:e1004315. doi: 10.1371/journal.pcbi.1004315

PubMed Abstract | CrossRef Full Text | Google Scholar

Brunel, N. (2000). Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comput. Neurosci. 8, 183–208. doi: 10.1023/A:1008925309027

PubMed Abstract | CrossRef Full Text | Google Scholar

Brunel, N., and Hakim, V. (1999). Fast global oscillations in networks of integrate-and-fire neurons with low firing rates. Neural Comput. 11, 1621–1671. doi: 10.1162/089976699300016179

PubMed Abstract | CrossRef Full Text | Google Scholar

Denéve, S., and Machens, C. K. (2016). Efficient codes and balanced networks. Nat. Neurosci. 19, 375–382. doi: 10.1038/nn.4243

PubMed Abstract | CrossRef Full Text | Google Scholar

Dong, S., Huang, T., and Tian, Y. (2017). “Spike camera and its coding methods,” in 2017 Data Compression Conference (DCC) (Snowbird, UT), 437. doi: 10.1109/DCC.2017.69

CrossRef Full Text | Google Scholar

Dorrn, A. L., Yuan, K., Barker, A. J., Schreiner, C. E., and Froemke, R. C. (2010). Developmental sensory experience balances cortical excitation and inhibition. Nature 465, 932–936. doi: 10.1038/nature09119

PubMed Abstract | CrossRef Full Text | Google Scholar

Fusi, S., and Mattia, M. (1999). Collective behavior of networks with linear (VLSI) integrate-and-fire neurons. Neural Comput. 11, 633–652. doi: 10.1162/089976699300016601

PubMed Abstract | CrossRef Full Text | Google Scholar

Graupner, M., and Reyes, A. D. (2013). Synaptic input correlations leading to membrane potential decorrelation of spontaneous activity in cortex. J. Neurosci. 33, 15075–15085. doi: 10.1523/JNEUROSCI.0347-13.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Haider, B., Duque, A., Hasenstaub, A. R., and McCormick, D. A. (2006). Neocortical network activity in vivo is generated through a dynamic balance of excitation and inhibition. J. Neurosci. 26, 4535–4545. doi: 10.1523/JNEUROSCI.5297-05.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

Hansel, D., and van Vreeswijk, C. (2012). The mechanism of orientation selectivity in primary visual cortex without a functional map. J. Neurosci. 32, 4049–4064. doi: 10.1523/JNEUROSCI.6284-11.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, L., Cui, Y., Zhang, D., and Wu, S. (2011). Impact of noise structure and network topology on tracking speed of neural networks. Neural Netw. 24, 1110–1119. doi: 10.1016/j.neunet.2011.05.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Indiveri, G., and Liu, S. (2015). Memory and information processing in neuromorphic systems. Proc. IEEE 103, 1379–1397. doi: 10.1109/JPROC.2015.2444094

CrossRef Full Text | Google Scholar

Lim, S., and Goldman, M. S. (2014). Balanced cortical microcircuitry for spatial working memory based on corrective feedback control. J. Neurosci. 34, 6790–6806. doi: 10.1523/JNEUROSCI.4602-13.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

Litwin-Kumar, A., and Doiron, B. (2012). Slow dynamics and high variability in balanced cortical networks with clustered connections. Nat. Neurosci. 15, 1498–1505. doi: 10.1038/nn.3220

PubMed Abstract | CrossRef Full Text | Google Scholar

Okun, M., and Lampl, I. (2008). Instantaneous correlation of excitation and inhibition during ongoing and sensory-evoked activities. Nat. Neurosci. 11, 535–537. doi: 10.1038/nn.2105

PubMed Abstract | CrossRef Full Text | Google Scholar

Pyle, R., and Rosenbaum, R. (2017). Spatiotemporal dynamics and reliable computations in recurrent spiking neural networks. Phys. Rev. Lett. 118:018103. doi: 10.1103/PhysRevLett.118.018103

PubMed Abstract | CrossRef Full Text | Google Scholar

Raiguel, S. E., Xiao, D.-K., Marcar, V. L., and Orban, G. A. (1999). Response latency of macaque area MT/V5 neurons and its relationship to stimulus parameters. J. Neurophysiol. 82, 1944–1956. doi: 10.1152/jn.1999.82.4.1944

PubMed Abstract | CrossRef Full Text | Google Scholar

Renart, A., de la Rocha, J., Bartho, P., Hollender, L., Parga, N., Reyes, A., et al. (2010). The asynchronous state in cortical circuits. Science 327, 587–590. doi: 10.1126/science.1179850

PubMed Abstract | CrossRef Full Text | Google Scholar

Risken, H. (1996). Fokker-Planck Equation. Berlin; Heidelberg: Springer. doi: 10.1007/978-3-642-61544-3_4

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosenbaum, R., and Doiron, B. (2014). Balanced networks of spiking neurons with spatially dependent recurrent connections. Phys. Rev. X 4:021039. doi: 10.1103/PhysRevX.4.021039

CrossRef Full Text | Google Scholar

Rosenbaum, R., Smith, M. A., Kohn, A., Rubin, J. E., and Doiron, B. (2017). The spatial structure of correlated neuronal variability. Nat. Neurosci. 20, 107–114. doi: 10.1038/nn.4433

PubMed Abstract | CrossRef Full Text | Google Scholar

Serrano-Gotarredona, T., and Linares-Barranco, B. (2013). A 128 × 128 1.5 mw asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers. IEEE J. Solid State Circ. 48, 827–838. doi: 10.1109/JSSC.2012.2230553

CrossRef Full Text | Google Scholar

Shadlen, M. N., and Newsome, W. T. (1994). Noise, neural codes and cortical organization. Curr. Opin. Neurobiol. 4, 569–579. doi: 10.1016/0959-4388(94)90059-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Softky, W. R., and Koch, C. (1993). The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci. 13, 334–350. doi: 10.1523/JNEUROSCI.13-01-00334.1993

PubMed Abstract | CrossRef Full Text | Google Scholar

Sugase, Y., Yamane, S., Ueno, S., and Kawano, K. (1999). Global and fine information coded by single neurons in the temporal visual cortex. Nature 400, 869–873. doi: 10.1038/23703

PubMed Abstract | CrossRef Full Text | Google Scholar

Thorpe, S., Fize, D., and Marlot, C. (1996). Speed of processing in the human visual system. Nature 381, 520–522. doi: 10.1038/381520a0

PubMed Abstract | CrossRef Full Text | Google Scholar

van Vreeswijk, C., and Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726. doi: 10.1126/science.274.5293.1724

PubMed Abstract | CrossRef Full Text | Google Scholar

van Vreeswijk, C., and Sompolinsky, H. (1998). Chaotic balanced state in a model of cortical circuits. Neural Comput. 10, 1321–1371. doi: 10.1162/089976698300017214

PubMed Abstract | CrossRef Full Text | Google Scholar

Yu, L., Wang, L., Jia, F., and Jia, D. (2015). Stimulus-dependent frequency modulation of information transmission in neural systems. arXiv [preprint]. arXiv:1507.08269.

Google Scholar