The effect of synaptic plasticity on orientation selectivity in a balanced model of primary visual cortex

Orientation selectivity is ubiquitous in the primary visual cortex (V1) of mammals. In cats and monkeys, V1 displays spatially ordered maps of orientation preference. Instead, in mice, squirrels, and rats, orientation selective neurons in V1 are not spatially organized, giving rise to a seemingly random pattern usually referred to as a salt-and-pepper layout. The fact that such different organizations can sharpen orientation tuning leads to question the structural role of the intracortical connections; specifically the influence of plasticity and the generation of functional connectivity. In this work, we analyze the effect of plasticity processes on orientation selectivity for both scenarios. We study a computational model of layer 2/3 and a reduced one-dimensional model of orientation selective neurons, both in the balanced state. We analyze two plasticity mechanisms. The first one involves spike-timing dependent plasticity (STDP), while the second one considers the reconnection of the interactions according to the preferred orientations of the neurons. We find that under certain conditions STDP can indeed improve selectivity but it works in a somehow unexpected way, that is, effectively decreasing the modulated part of the intracortical connectivity as compared to the non-modulated part of it. For the reconnection mechanism we find that increasing functional connectivity leads, in fact, to a decrease in orientation selectivity if the network is in a stable balanced state. Both counterintuitive results are a consequence of the dynamics of the balanced state. We also find that selectivity can increase due to a reconnection process if the resulting connections give rise to an unstable balanced state. We compare these findings with recent experimental results.

Introduction

Neurons in primary visual cortex are characterized by being selective to several stimulus features, such as orientation, ocular dominance or retinotopy. One of the most interesting aspects of the primary visual cortex is that these cortical features can be spatially organized, i.e., nearby neurons tend to have similar optimal stimuli. One important receptive field property, such as orientation preference, can be mapped rather smoothly across the cortical surface. This was found in cats in Hubel and Wiesel (1961, 1962), Bonhoeffer and Grinvald (1991) and also confirmed in primates (Wiesel and Hubel, 1974). For these systems we say that an orientation map is present.

More recently it was found that in rodents the visual cortex behaves in a very different way. Cells in the primary visual cortex are selective to orientation (Dräger, 1975; Scholl et al., 2013) but there are no orientation maps (Ohki et al., 2005; van Hooser et al., 2005). In fact, neighboring cells display completely different preferred orientations, giving rise to the salt-and-pepper organization of orientation selectivity (Ohki et al., 2007). This structure can appear even for highly visual animals (van Hooser et al., 2005).

Having high degree of orientation selectivity for such different structures leads to question the structural organization of the intracortical connections. It is known that neurons tend to be connected with other neurons in their neighborhood (Holmgren et al., 2003; Stepanyants et al., 2008); for systems with columnar organization this means neurons with similar selectivity properties. However, for the salt-and-pepper structure the situation is not so clear. Neurons with similar response properties could be more strongly connected independently of the physical distance (Ko et al., 2011; Cossell et al., 2015). Moreover, this functional connectivity could be the result of plasticity processes. For instance, in Ko et al. (2013) it was found that functional microcircuits are generated during development, but their presence leads only to a small increment of selectivity.

From a theoretical point of view, it was recently found that even if the connectivity patterns are totally random, selectivity can be maintained from layer 4 to layer 2/3 (Hansel and van Vreeswijk, 2012). This happens because as the network is populated by both excitatory and inhibitory neurons operating in a balanced activity regime, untuned excitatory and inhibitory inputs roughly cancel each other giving rise to a sizable net tuned input. The theory of balanced state has been proposed to explain the temporal variability of the response in systems that are highly connected (Softky and Koch, 1993). This theory explains the variability without doing fine tuning of parameters (van Vreeswijk and Sompolinsky, 1998) and was later supported by experimental works (Destexhe et al., 2003; Shu et al., 2003; Haider et al., 2006).

Even if balanced networks with totally random connections can display a substantial degree of selectivity, it has been suggested (Corey and Scholl, 2012) that increasing the connection probability of nearby neurons with similar orientation preference might further enhance their selectivity. This increment could be generated by plasticity processes at the synaptic level.

Here we intend to clarify the problem of the effect of plasticity on orientation selectivity. The questions we address are: under which conditions can plasticity improve selectivity? Is the effect of plasticity different for the salt-and-pepper organization and for systems with orientation maps? If plasticity does improve selectivity, what is the main mechanism responsible for that change?

We study these questions by analyzing a computational model of layer 2/3 and a reduced one-dimensional model of orientation selective neurons. Since we are modeling cortical activity (characterized by both highly irregular response and large connectivity), our computational and reduced models are in the balanced state. We analyze two plasticity mechanisms, one of them involves spike-timing dependent plasticity (STDP) and the other considers the reconnection of neuronal interactions according to the preferred orientations of the pre- and post-synaptic neurons.

We find that spike-timing dependent plasticity improves selectivity but it works in a somehow unexpected way, that is effectively decreasing the modulated part of the intracortical connectivity as compared to its non-modulated part. We find this conclusion to be valid both for systems with salt-and-pepper organization and with orientation maps. The effect of reconnection of interactions is also non-intuitive: As predicted by the one dimensional model, if the network dynamics is in a stable balanced state, functional connectivity leads to a decrease in selectivity. However, if the balanced state is unstable, functional connectivity can indeed improve selectivity.

Materials and Methods

The Model Network

The model is composed of one layer representing a square patch of layer 2/3 of size M × M, where we assume M to be approximately 1 mm. It has N_E excitatory and N_I inhibitory neurons (see Figure 1). We denote neuron i = 1, …, N_A of population A = E, I, with neuron (i, A). The neurons are arranged on a square grid and the position (x_iA, y_iA) of neuron (i, A) in its layer is given by $x_{iA}=i_{x}M∕\sqrt{N_A},y_{iA}=i_{y}M∕\sqrt{N_A}$, where $i_x=(i-1)mod\sqrt{N_A}$ and $i_y=\lfloor (i-1)∕\sqrt{N_A}\rfloor$. Here, ⌊x⌋ is the largest integer equal or smaller than x. We choose N_A in such a way that $\sqrt{N_A}$ is an integer number. Thus, indices i_x and i_y go from 0 to $\sqrt{N_A}-1$. Unless otherwise specified in the simulations presented here, we take N_E = 8100 and N_I = 2025.

FIGURE 1

Figure 1. The network model. Each one of the cells of layer 2/3 (lower panels) receives a feed-forward input whose preferred orientation is shown in the corresponding position of the upper panel. The circle in layer 2/3 represents the width of the recurrent connectivity matrices. Left: orientation map; Right: salt-and-pepper layout.

Single Neuron Dynamics of Layer 2/3 Cells

Neurons in layer 2/3 are described by a leaky integrate-and-fire model. The membrane potential V_iA of neuron (i, A), A = E, I, evolves in time according to:

$\begin{array}{ll}\tau \frac{d{V}_{iA}}{dt}=-V_{iA}+R_m(I_{L4,iA}+I_{rec,iA}+I_{back,iA}),& \left(1\right)\end{array}$

where τ is the membrane time constant, R_m the membrane resistance (see Section Default Parameters), I_{L4, iA} is the input current from layer 4, I_{rec, iA} is the recurrent input from all the other neurons in layer 2/3 and I_{back, iA} represents a background input from other cortical regions. When the membrane potential reached the threshold value V_T = 30 mV, it was immediately reset to V_rest = 0 mV.

The Model of the Feed-forward Input

The input is considered to have a firing rate that depends on the visual stimulus, which is assumed to be a sinusoidal grating with a fixed wavelength. For a grating with orientation θ, the input to neuron (i, A) is given by:

$\begin{array}{cc}{I}_{L4,iA}=g_{AL}{K}_{AL}f{r}_L(1+2{\rho }_{iA}\cos (2(\theta -{\theta }_{iA})),& \left(2\right)\end{array}$

where g_AL is the individual efficacy of the feed-forward synapse, K_AL is the number of feed-forward synapses, fr_L is the average firing rate of cells in layer 4, ρ_iA is the effective modulation of the feed-forward input with respect to the orientation of the stimulus and θ_iA is the stimulus orientation for which the feed-forward input to neuron (i, A) takes its maximal value. In the following we assume that all the neurons in each population receive inputs with the same modulation, i.e., ρ_iA = ρ_A.

We analyze two different possibilities for the orientations θ_iA:

1. Random orientations. The values of θ_iA are independent random variables with uniform distribution between 0 and π. This is known as the salt-and-pepper distribution.

2. Continuous orientation map. The values of the optimal feed-forward inputs are given by:

$\begin{array}{rcl}{\theta }_{iA}& =& arctan(sin(2\pi y_{iA}∕M)∕sin(2\pi x_{iA}∕M))∕2+\\ & & \pi ∕2+\pi (1+sign(x_{iA}∕M-0.5))∕4.& \left(3\right)\end{array}$

This condition guarantees a continuous distribution of the optimal feed-forward inputs (except at the pinwheels) while respecting periodic boundary conditions see Figure 1.

Recurrent Interactions

We assume that the recurrent interactions in layer 2/3 are random. The probability of connection between neuron (j, B) and (i, A) (A, B = E, I) is given by:

$\begin{array}{ll}{p}_{iA,jB}=Z_{AB}G(x_{iA}-x_{jB},{\sigma }_{AB})G(y_{iA}-y_{jB},{\sigma }_{AB})& \left(4\right)\end{array}$

where G is the periodic Gaussian with period M, $G(x,\sigma )=\sum _{k=-\infty }^{\infty }exp(-{(x-Mk)}^2∕(2{\sigma }^2))$. Z_AB insures that the average number of connections from population B to population A is given by K_AB. In the following we take K_AB = K.

In principle, the probability of having a connection depends on the distance between the pre- and-postsynaptic neurons (Holmgren et al., 2003; Stepanyants et al., 2008). For the system with an orientation map, however, nearby neurons tend to have similar preferred inputs. This implies that we necessarily have functional connectivity, i.e., neurons with similar feed-forward inputs are more likely to be connected. On the other hand for the salt-and-pepper structure there is no correlation between the position and the preferred stimulus. Functional connectivity can be introduced by reconnecting neurons with similar preferred orientations (see Section Reconnection Probability).

Synaptic Currents

After the connection probabilities are evaluated the connectivity matrices C^AB (A, B = E, I) can be determined. Their elements, $C_{ij}^{AB}$, are 0 or 1 according to the probabilities of Equation (4).

The recurrent currents are given by:

$\begin{array}{ll}{I}_{rec,iA}={\displaystyle \sum _B}{g}_{AB}{\displaystyle \sum _{j=1}^{N_B}}{w}_{iA,jB}{C}_{ij}^{AB}{\displaystyle \sum _k}\frac{exp(-(t-t_{k,j})∕{\tau }_{syn,B})}{{\tau }_{syn,B}},& \left(5\right)\end{array}$

where t_{k, j} is the time of the k^th spike of presynaptic neuron j. The time scale of the synaptic interactions is controlled by τ_{syn, A} and their strength is determined by the coupling parameters g_AB and by the normalized synaptic efficacies w_{iA, jB}. These parameters are set to 1 at the beginning of the simulation and can be modified in the presence of synaptic plasticity (see Section Spike-timing Dependent Plasticity).

Let us note that, in order to have a well defined scaling in the limit of large connectivity, while keeping a highly variable temporal activity, the couplings have to be scaled with the inverse of the square root of the connectivity (van Vreeswijk and Sompolinsky, 1996, 1998). For that reason we define $g_{AB}=\frac{G_{AB}}{\sqrt{K_{AB}}}$ (A, B = E, I).

For the background current we use the expression:

$\begin{array}{ll}{I}_{back,iA}=\sqrt{K}{I}_{back,A}.& \left(6\right)\end{array}$

This represents the effect of K excitatory only inputs, each one with synaptic strength that scales as $1∕\sqrt{K}$.

Numerical Procedures and Analysis of the Results

The numerical simulations were performed using an Euler scheme to integrate the neuronal dynamics, the neuronal interactions and the synaptic plasticity (Press et al., 1992). The time step is δt = 0.05 ms. The typical simulation run was 20 s long and the transient period was 0.1 s. The transient was chosen in order to let the firing rate stabilize after the beginning of the simulation.

For stimulus angle θ the firing rate of the neuron was estimated by averaging its spike response over the whole run (after the transient). We took 9 stimulus angles: θ = 0, 20, 40, 60, 80, 100, 120, 140, 160°. The selectivity was quantified using the Orientation Selectivity Index (OSI) of the activity:

$\begin{array}{cc}OS{I}_{iA}=\frac{\sqrt{{({\displaystyle {\sum }_{\theta }{f}_{iA}}(\theta )\cos (2\theta ))}^2+{({\displaystyle {\sum }_{\theta }{f}_{iA}}(\theta )\sin (2\theta ))}^2}}{{\displaystyle {\sum }_{\theta }{f}_{iA}}(\theta )},& \left(7\right)\end{array}$

where f_iA(θ) is the firing rate of neuron (i, A) when the stimulus angle is θ. Notice that this definition of the orientation selectivity index corresponds to the ratio between the first and the zeroth Fourier components of the firing rate. This number goes between 0 and 1. It is 0 for flat tuning curves and becomes 1 when the tuning curve is a delta function.

The preferred orientation of neurons in layer 2/3 was estimated using the population vector. The angle of the population vector PO_iA of neuron (i, A) indicates its preferred orientation and is evaluated according to:

$\begin{array}{cc}\tan (2P{O}_{iA})=\frac{{\displaystyle {\sum }_{\theta }{f}_{iA}}(\theta )\sin (2\theta )}{{\displaystyle {\sum }_{\theta }{f}_{iA}}(\theta )\cos (2\theta )}.& \left(8\right)\end{array}$

To check that the time step in the numerical simulations was sufficiently small, we performed several simulations with a smaller time step, δt = 0.025 ms, and verified that the results were not affected. We also performed some simulations on networks with approximately twice as many neurons (N_E = 16129, N_I = 4096), keeping the average number of connections into excitatory and inhibitory cells the same, to verify that our results were not an artifact of a small network size.

Spike-timing Dependent Plasticity

The excitatory synapses undergo a process of plasticity according to the following rule (Bi and Poo, 1998; Abbott and Nelson, 2000). When neuron j in population E generates a spike at time t_pre and neuron i in population A (A = E, I) (which is connected post-synaptically to (j, E)) generates a spike at time t_post, the normalized synaptic efficacy between the two neurons is modified according to:

$\begin{array}{ll}{w}_{iA,jE}=w_{iA,jE}+\Delta w_{iA,jE},& \left(9\right)\end{array}$

where

$\begin{array}{ll}\Delta w_{iA,jE}=a_{+}exp(-(t_{post}-t_{pre})∕{\tau }_{+})(2-w_{iA,jE})& \left(10\right)\end{array}$

if t_post > t_pre or

$\begin{array}{ll}\Delta w_{iA,jE}=a_{-}exp(-(t_{pre}-t_{post})∕{\tau }_{-})w_{iA,jE}& \left(11\right)\end{array}$

if t_post < t_pre. We take a₊ > 0, a₋ < 0 (see Figure 2). This implies that if the postsynaptic spike comes after the presynaptic spike the connection becomes stronger and for the reverse order it becomes weaker. The multiplicative dependencies (Rubin et al., 2001) of the synaptic modification on the synaptic strengths (2 − w_{iA, jE} in Equation 10 and w_{iA, jE} in Equation 11) insure that the modifications become very weak as w_ij approach the bounds 0 or 2. Let us note that inhibitory synapses are not plastic, i.e., w_{iA, jI} is always equal to 1. The plasticity mechanism is applied during the first 2/3 of the simulation (13.33 s). During the last third of the simulation (6.66 s) the plasticity process is stopped and the activity is recorded in order to evaluate the selectivity properties and other parameters such as the values of the synaptic efficacies. During the whole simulation, only one stimulus orientation is applied.

FIGURE 2

Figure 2. STDP window: the relative changes of synaptic efficacy depend on the time difference between the post-synaptic (t_post) and the pre-synaptic (t_pre) spikes. Green indicates facilitation and red depression.

Reconnection Probability

In the way it is described above, spike-timing dependent plasticity can give rise to the strengthening or weakening of the synaptic connections but not to a change in the connection probability (i.e., it does not create nor destroy connections). However, this change is observed in rodent visual cortex in Ko et al. (2013). Thus, in order to implement it, we first measure the preferred orientation of each neuron according to the population vector (see Equation 8). Then the probability of connection between neuron (j, B) and neuron (i, A) is given by:

$\begin{array}{cc}{\widehat{p}}_{iA,jB}=p_{iA,jB}(1+{ϵ}_{cB}\cos (2(P{O}_{jB}-P{O}_{iA})),& \left(12\right)\end{array}$

where p_{iA, jB} is given by Equation (4). If parameter ϵ_cB > 0, neurons with similar preferred orientations are more likely to be connected than neurons with orthogonal preferred orientations. We assume that the reconnection parameter depends only on the presynaptic population. After the reconnection process is applied, the simulation continues for 6.66 s to measure the selectivity properties and the preferred orientations of the neurons. As before, only one stimulus angle is presented during each simulation.

Default Parameters

The membrane time constant is τ = 20 ms (Somers et al., 1995) and the membrane resistance is R_m = 38.3 MΩ (Koch, 1999). The parameters of the synapses are G_EE = 32, G_EI = −96, G_IE = 96, G_II = −128, g_EL = 1.65, g_IL = 1.65 nA ms, τ_{syn, E} = 25 ms, τ_{syn, I} = 4 ms (Hansel and van Vreeswijk, 2012). We choose a longer time constant for the excitation to represent a mixture of AMPA and NMDA synapses. Unless otherwise specified, the average recurrent connectivity is K = 500 and the feed-forward connectivity is K_AL = 250. The standard deviations of the recurrent connections are σ_AB = σ_AL = 0.2M (A = E, I). The firing rate of cells in layer 4 is fr_L = 15 Hz and the default values of the selectivity of the feed-forward input are ρ_E = ρ_I = 0.06. The values of the background currents are I_{back, E} = 0.12 nA, I_{back, I} = 0.12 nA. The default parameters of the spike-timing dependent plasticity process are given by: a₊ = 0.0128, a₋ = −0.0045, τ₊ = 30 ms, τ₋ = 40 ms (Bi and Poo, 1998).

The Reduced Model

The reduced model is composed by two populations of spiking neurons evenly spaced in the state space (0, 1]. The variable θ in this space denotes the preferred orientation of the external input divided by π. There are N neurons, half of them are excitatory and half of them inhibitory (Rosenbaum and Doiron, 2014). The location of the kth neuron is θ = k∕N. The input current to the kth excitatory (A = E) or inhibitory (A = I) neuron is given by:

$\begin{array}{ll}{I}_{Ak}(t)={\displaystyle \sum _{j=1}^N}{g}_{Ak,Ej}{s}_{E,j}(t)-g_{Ak,Ij}{s}_{I,j}(t)+i_{ext,Ak}(\theta ),& \left(13\right)\end{array}$

where g_{Ak, Bj} is the connection strength between neuron j in population B and neuron k in population A (A, B = E, I). $s_{A,j}(t)=\sum _l\delta (t-t_{A,j}^l)$ is the spike train of neuron j in population A. The l^th spike of neuron (j, A) occurs at time $t_{A,j}^l$. External input is provided by i_{ext, Ak}(θ). The synaptic weights g_{Ak, Bj} are equal to the constant g_AB with probability KC_AB(θ − ψ) or 0 else, where θ and ψ are the locations of neurons k and j, respectively. We take $C_{AB}(\theta -\psi )=\sum _{n=-\infty }^{\infty }{c}_{AB}(\theta -\psi -n)$ to insure periodic boundary conditions and we fix it in such a way that the average number of connections is always K, with 1 ≪ K ≪ N. Let us remark that this is the probability of having a connection in the state space, not in the physical space as in the previous model. A system with salt-and-pepper organization is represented by functions C_AB(θ − ψ) that are totally flat. In contrast, a system with an orientation map has to be modeled with a modulated probability C_AB(θ − ψ).

The synaptic weights are scaled according to:

$\begin{array}{ll}{g}_{AB}=\frac{G_{AB}}{\sqrt{K}},& \left(14\right)\end{array}$

and the external inputs according to:

$\begin{array}{ll}{i}_{ext,Ak}(\theta )=I_{ext,Ak}(\theta )\sqrt{K}.& \left(15\right)\end{array}$

The mean firing rate of neuron k in the network is denoted by ν_A(x) = 〈s_{A, k}(t)〉, where <. > denotes average over time. Under these conditions, neuron k in population A will receive a total input given by:

$\begin{array}{ll}{\mu }_{Ak}=\sqrt{K}\left[{\displaystyle \sum _j}(J_{Ak,Ej}{\nu }_{Ej}-J_{Ak,Ij}{\nu }_{Ij})+I_{ext,Ak}(\theta )\right]& \left(16\right)\end{array}$

where J_{Ak, Bj} = G_ABN_{Ak, Bj} and N_{Ak, Bj} = 0, 1 is the number of connections between neuron j in population B and neuron k in population A. In the continuum limit, i.e., in the limit when the number of neurons N becomes very large, the interaction term can be written as J_AB(θ, ψ) = G_AB(C_AB(θ − ψ)+ ξ_AB(θ, ψ)), where ξ_AB(θ, ψ) are Gaussian random variables with 0 mean and correlation $<{\xi }_{AB}(\theta ,\psi ){\xi }_{A^{\prime }{B}^{\prime }}({\theta }^{\prime },{\psi }^{\prime })>={\delta }_{A{A}^{\prime }}{\delta }_{B{B}^{\prime }}\delta (\theta -{\theta }^{\prime })\delta (\psi -{\psi }^{\prime })C_{AB}(\theta -\psi )∕K$. In the limit of large K, the interaction term becomes translational invariant (J_AB(θ, ψ) = J_AB(θ − ψ)) and the input can be written as:

$\begin{array}{ll}{\mu }_A(\theta )=\sqrt{K}\left[J_{AE}*{\nu }_E(\theta )-J_{AI}*{\nu }_I(\theta )+I_{ext,A}(\theta )\right],& \left(17\right)\end{array}$

where * stands for the circular convolution.

In order to have a finite result in the limit of large K the following equation has to be satisfied:

$\begin{array}{ll}{J}_{AE}*{\nu }_E(\theta )-J_{AI}*{\nu }_I(\theta )+I_{ext,A}(\theta )=O(1∕\sqrt{K})& \left(18\right)\end{array}$

for A = E, I.

Taking the Fourier transform these integral equations become:

$\begin{array}{ll}{\tilde{J}}_{AE}(n){\tilde{\nu }}_E(n)-{\tilde{J}}_{AI}(n){\tilde{\nu }}_I(n)+{Ĩ}_{ext,A}(n)=O(1∕\sqrt{K}),& \left(19\right)\end{array}$

where $\tilde{f}(n)={\displaystyle {\int }_0^{1}f(\theta )\exp (-2i\pi n\theta )d\theta }$. The solution of this linear system at the leading order is given by:

$\begin{array}{rcll}{\tilde{\nu }}_E(n)& =& \frac{{\tilde{J}}_{II}(n){Ĩ}_{ext,E}(n)-{\tilde{J}}_{EI}(n){Ĩ}_{ext,I}(n)}{{\tilde{J}}_{IE}(n){\tilde{J}}_{EI}(n)-{\tilde{J}}_{EE}(n){\tilde{J}}_{II}(n)}& \left(20\right)\end{array}$

$\begin{array}{rcll}{\tilde{\nu }}_I(n)& =& \frac{{\tilde{J}}_{IE}(n){Ĩ}_{ext,E}(n)-{\tilde{J}}_{EE}(n){Ĩ}_{ext,I}(n)}{{\tilde{J}}_{IE}(n){\tilde{J}}_{EI}(n)-{\tilde{J}}_{EE}(n){\tilde{J}}_{II}(n)}.& \left(21\right)\end{array}$

Stability of the Balanced State

The balanced state solution from Equations (20) and (21) is stable only if the eigenvalues of the matrix:

$\begin{array}{cc}A(n)=\left|\begin{array}{cc}{\tilde{J}}_{EE}(n)& -{\tilde{J}}_{EI}(n)\\ {\tilde{J}}_{IE}(n)& -{\tilde{J}}_{II}(n)\end{array}\right|& \left(22\right)\end{array}$

have negative real part (Rosenbaum and Doiron, 2014). This condition, that is valid in the mean-field approximation to firing rate dynamics, implies that the following equations must be satisfied for each Fourier mode n:

$\begin{array}{rcll}{\tilde{J}}_{EE}(n)-{\tilde{J}}_{II}(n)& <& 0& \left(23\right)\end{array}$

$\begin{array}{rcll}{\tilde{J}}_{IE}(n){\tilde{J}}_{EI}(n)-{\tilde{J}}_{EE}(n){\tilde{J}}_{II}(n)& >& 0.& \left(24\right)\end{array}$

Results

Orientation Selectivity in the Reduced Model

We first analyze the selectivity properties of a one-dimensional network of spiking neurons in the balanced state. We assume that we are in the condition for which there is a stable solution with non-zero average firing rate. From Equations (20) and (21), this requires that (Rosenbaum and Doiron, 2014):

$\begin{array}{ll}\frac{{Ĩ}_{ext,E}(0)}{{Ĩ}_{ext,I}(0)}>\frac{{\tilde{J}}_{EI}(0)}{{\tilde{J}}_{II}(0)}>\frac{{\tilde{J}}_{EE}(0)}{{\tilde{J}}_{IE}(0)}.& \left(25\right)\end{array}$

Moreover, we will take the simple case where the spatial structure of the connectivity probability depends only on the presynaptic population, (${\tilde{J}}_{AB}(n)=G_{AB}{\tilde{J}}_B(n)$) and the external currents have the same spatial profile in both populations (Ĩ_{ext, A}(n) = Ĩ_ext(n)I_{ext, A}). In that situation, Equations (20) and (21) become:

$\begin{array}{rcll}{\tilde{\nu }}_E(n)& =& \frac{(G_{II}{I}_{ext,E}-G_{EI}{I}_{ext,I}){Ĩ}_{ext}(n)}{(G_{IE}{G}_{EI}-G_{EE}{G}_{II}){\tilde{J}}_E(n)}& \left(26\right)\end{array}$

$\begin{array}{rcll}{\tilde{\nu }}_I(n)& =& \frac{(G_{IE}{I}_{ext,E}-G_{EE}{I}_{ext,I}){Ĩ}_{ext}(n)}{(G_{IE}{G}_{EI}-G_{EE}{G}_{II}){\tilde{J}}_I(n)}.& \left(27\right)\end{array}$

The feed-forward input current on the neuron located at coordinate θ when the stimulus orientation is θ₀ will be taken as:

$\begin{array}{ll}{I}_{ext,A}(\theta ,{\theta }_0)=(1+2\rho cos(2\pi (\theta -{\theta }_0)))I_{ext,A}.& \left(28\right)\end{array}$

Let us remark that the state space does not necessarily correspond with the physical space. For a salt-and-pepper organization they are, in fact, independent: The physical location of a neuron is unrelated to its preferred orientation. This means that the connection probabilities must have a flat spatial structure.

According to Equation (28) and using the definition $\tilde{f}(n)={\displaystyle {\int }_0^{1}f(\theta )\exp (-2i\pi n\theta )d\theta }$, the following result can be obtained: Ĩ_ext(0) = I_{ext, A}, Ĩ_ext(1) = ρI_{ext, A} and Ĩ_ext(n) = 0 for all n > 1. Therefore, ρ is simply the orientation selectivity index of the external current (see Equation 7). According to Equations (26) and (27), ${\tilde{\nu }}_E(n)={\tilde{\nu }}_I(n)=0$ for all n > 1 while:

$\begin{array}{rcll}\frac{{\tilde{\nu }}_E(1)}{{\tilde{\nu }}_E(0)}& =& \rho \frac{{\tilde{J}}_E(0)}{{\tilde{J}}_E(1)}& \left(29\right)\end{array}$

$\begin{array}{rcll}\frac{{\tilde{\nu }}_I(1)}{{\tilde{\nu }}_I(0)}& =& \rho \frac{{\tilde{J}}_I(0)}{{\tilde{J}}_I(1)}.& \left(30\right)\end{array}$

This result implies that the selectivity of the cortical activity ($\frac{{\tilde{\nu }}_A(1)}{{\tilde{\nu }}_A(0)}$) is directly proportional to the selectivity of the feed-forward input (ρ) but inversely proportional to the spatial modulation of the intracortical connections (${\tilde{J}}_E(1)$). This somehow surprising behavior was already discussed in van Vreeswijk and Sompolinsky (2005). The cortical network receives an excitatory input of order $\sqrt{K}$ that has to be canceled by the intracortical activity giving rise to a net input of order 1. The intracortical input to a given neuron is a convolution of the cortical activity and the connectivity matrices. Then, if the connectivity matrices have a wide spatial profile, the cortical activity must be narrower to keep a fixed spatial profile of the intracortical input. This leads to strong cortical selectivity since it contributes to the term $\tilde{\nu }(1)$ of Equations (29) and (30). This result is in contrast with the one found in Ben-Yishai et al. (1995), where the modulation of the excitatory connections leads to increasing selectivity. The difference is due to the fact that in this last case the network is not in a balanced state. Thus, the recurrent connections do not lead to the cancelation of the leading term of the input but, instead, to its amplification.

Comparison of Orientation Selectivity in Salt-and-pepper and Orientation Map

For a given feed-forward input, the selectivity of the cortical activity is determined by the ratio $\frac{{\tilde{J}}_A(0)}{{\tilde{J}}_A(1)}$. For both structures (salt-and-pepper and orientation map) the numerator is equal to the average number of intracortical connections K. In contrast, the behavior of the denominator is completely different.

For the salt-and-pepper structure the connection probability is, on average, independent of the distance in the state space. According to this, selectivity should be infinite in the limit of large K (in that limit ${\tilde{J}}_A(1)=0$), but this is impossible because selectivity cannot be larger than 1. However, as fluctuations of the connectivity patterns generate contributions to the selectivity that are proportional to $1∕\sqrt{K}$ (see Supplementary Material), and these contributions become dominant for very weakly modulated interactions, the selectivity remains bounded between 0 and 1. In practice, numerical simulations must be performed in order to test whether a reasonable value of connectivity is compatible with the balanced state. In this scenario, the balanced state can be preserved in the limit of large K if the modulation of the input, ρ, also scales as $1∕\sqrt{K}$. This is studied in Hansel and van Vreeswijk (2012).

In contrast, in systems with an orientation map the numerator and the denominator are both proportional to K. As a consequence, for large connectivity, and keeping the rest of the parameters the same, one should expect salt-and-pepper structures to be more selective than systems with an orientation map, as the denominator of orientation map (~K) is larger than the denominator of salt-and-pepper (($~1∕\sqrt{K}$)). Moreover, for systems with an orientation map, selectivity could increase by taking a broader connection probability profile.

Effect of Plasticity on Orientation Selectivity in the Reduced Model

The plasticity rules of Equations (10) and (11) depend on the precise timing of the pre- and post-synaptic spikes. However, in the balanced state, significant cross-correlations between different spike trains are not expected (van Vreeswijk and Sompolinsky, 1998; Renart et al., 2010). In this situation, the net change in the synaptic strength will be controlled by the total number of spikes, by the integral of the STDP window and by the synaptic strength itself (through the terms w_{iA, jE} and 2 − w_{iA, jE} in Equations 10, 11). As the normalized value of the synapses is initially w_{iA, jE} = 1, the integral is equal to α = a₊ τ₊ + a₋ τ₋. In order to estimate the effect of plasticity on selectivity we neglect the effect of the non-linear terms in the plasticity rule. Thus, the change in the synaptic strength per unit time between neurons (j, E) and (k, A), A = E, I, is given by:

$\begin{array}{ll}\Delta w_{Ak,Ej}=\alpha {\nu }_{Ak}{\nu }_{Ej}.& \left(31\right)\end{array}$

If the feed-forward input is described by Equation (28) and we neglect the changes of the firing rates generated by the changes in the synaptic weights, the firing rates can be approximated as a function of the stimulus orientation θ:

$\begin{array}{cc}{\nu }_{Ak}(\theta )={\nu }_{Ak}^{°}(1+2{\xi }_{Ak}\cos (2\pi (\theta -{\theta }_{Ak}))& \left(32\right)\end{array}$

$\begin{array}{cc}{\nu }_{Ej}(\theta )={\nu }_{Ej}^{°}(1+2{\xi }_{Ej}\cos (2\pi (\theta -{\theta }_{Ej}))& \left(33\right)\end{array}$

where θ_Ak, θ_Ej are the preferred orientations, ${\nu }_{Ak}^{°}$, ${\nu }_{Ej}^{°}$ are the firing rates averaged over the stimulus orientations and ξ_Ak, ξ_Ej are the selectivities of neurons (k, A) and (j, E), respectively. Notice that the approximation will be valid only when the changes of the synaptic weights are small. Replacing Equations (32), (33) in Equation (31) and averaging over the stimulus orientation we obtain:

$\begin{array}{cc}\overline{\Delta w_{Ak,Ej}}=\alpha {\nu }_{Ak}^{°}{\nu }_{Ej}^{°}(1+2{\xi }_{Ak}{\xi }_{Ej}\cos (2\pi ({\theta }_{Ak}-{\theta }_{Ej})).& \left(34\right)\end{array}$

A similar result is obtained for a fixed stimulus orientation after averaging over θ_Ak and θ_EJ, keeping θ_Ak − θ_EJ constant, and assuming that the mean rates and the selectivities of the neurons are independent from the position on the network. This means that the specific protocol for the stimulus presentation does not affect the synaptic weights.

For the salt-and-pepper architecture this result implies the generation of functional connectivity, i.e., a connectivity pattern that is linked to the relative preferred orientation of the neurons. For positive values of α, neurons with similar preferred orientations will have stronger synapses than neurons with orthogonal preferred orientations. In terms of the Fourier components of the connectivity matrices, the first Fourier component will be increased by a term that is proportional to the square of the selectivity of the cortical activity. But, at the same time, the zeroth Fourier component will be reinforced by a term that is order 1 in the selectivity. Thus, for initial small values of cortical selectivity, plasticity will improve it. It is interesting to note that this improvement does not emerge because of the generation of functional connectivity, but in spite of it.

For systems with cortical maps the situation is similar: the modulated component of the connectivity matrices will be reinforced, but typically less than the non-modulated part. This mechanism leads to an increment of the selectivity.

Let us remark that these conclusions are valid only at the initial stages of learning, where the non-linear terms can be neglected. The long term behavior must be analyzed on the network model with numerical simulations.

Effect of Reconnection Probability on Orientation Selectivity in the Reduced Model

For the salt-and-pepper structure with a connection probability that depends only on the physical distance between the neurons, the connection probability in the space of preferred orientations is totally flat, i.e.,

$\begin{array}{ll}{p}_{iA,jB}=\frac{K}{N∕2},& \left(35\right)\end{array}$

so that we always have in average K connections per neuron for each one of the N∕2 neurons in population A incoming from population B (A, B = E, I). The reconnection probability rule of Equation (12) gives rise to modulation of the probability in this state space. If the preferred orientations of the neurons are perfectly correlated with the orientations that maximize the feed-forward input the probability of connection between neuron (i, A) and (j, B) will be:

$\begin{array}{cc}{\tilde{p}}_{iA,jB}=\frac{K}{N/2}(1+{ϵ}_{cB}\cos (2\pi ({\theta }_{jB}-{\theta }_{iA})).& \left(36\right)\end{array}$

This implies that C_AB(θ − ψ) = 1 + ϵ_cBcos(2π(θ − ψ)) and (in the limit of large K) that J_AB(θ − ψ) = G_AB(1 + ϵ_cBcos(2π(θ − ψ))). The first Fourier components are:

$\begin{array}{rcll}{\tilde{J}}_{AE}(1)& =& G_{AE}{ϵ}_{cE}∕2& \left(37\right)\end{array}$

$\begin{array}{rcll}{\tilde{J}}_{AI}(1)& =& G_{AI}{ϵ}_{cI}∕2,& \left(38\right)\end{array}$

while the other Fourier components are not affected by the reconnection. By replacing Equations (37) and (38) into Equations (29) and (30) we can see that more functional modulation implies a reduction of the orientation selectivity:

$\begin{array}{rcll}\frac{{\tilde{\nu }}_E(1)}{{\tilde{\nu }}_E(0)}& \propto & \frac{1}{{ϵ}_{cE}}& \left(39\right)\end{array}$

$\begin{array}{rcll}\frac{{\tilde{\nu }}_I(1)}{{\tilde{\nu }}_I(0)}& \propto & \frac{1}{{ϵ}_{cI}}.& \left(40\right)\end{array}$

We can also observe by replacing Equations (37) and (38) into Equation (23) that changes in functional connectivity may lead to the loss of stability of the balanced state. This will happen when:

$\begin{array}{ll}\frac{{ϵ}_{cE}}{{ϵ}_{cI}}>\frac{G_{II}}{G_{EE}},& \left(41\right)\end{array}$

since Equation (23) will not be satisfied for n = 1. This means that the excitatory modulated recurrent input is too strong to be compensated by the inhibitory–inhibitory interactions. Under these conditions, the predictions of Equations (29) and (30) are not valid anymore and numerical simulations are required to assess selectivity.

Orientation Selectivity in the Network Model

We now analyze the behavior of the network model in order to check some of the predictions done by the reduced model. First, we studied the dynamical state from the point of view of the total input on an individual neuron. We performed simulations with connectivity K = 250 and 1000. For each value of K, we chose a neuron and depicted its excitatory and inhibitory input (see Figure 3). On one hand, as the theory of balanced state predicts, as connectivity grows both contributions to the total input increase proportionally to $\sqrt{K}$. On the other hand, the net result is almost the same, showing the cancelation between the two components. The average firing rate of the excitatory population has changed from f_E = 3.02 Hz for K = 250 to f_E = 2.61 Hz for K = 1000. For the inhibitory population, the firing rate has gone from f_I = 7.81 Hz to f_I = 5.71 Hz.

FIGURE 3

Figure 3. The excitatory and inhibitory inputs to one neuron tend to cancel even as they grow stronger with larger connectivity. (A) K = 250, (B) K = 1000. N_E = 16129, N_I = 4096. The rest of the parameters are set to default.

We also studied the selectivity properties of the two cortical architectures mentioned above: orientation map and salt-and-pepper. For the system with an orientation map, we found that the mean orientation selectivity index of the excitatory population is < OSI_E > = 0.27 (see Figure 4A). If exactly the same network is considered, but now it has a salt-and -pepper structure instead of an orientation map, < OSI_E > = 0.57 (see Figure 4B). Note that the change in selectivity appears without a strong change in the average firing rate, that has gone from f_E = 2.51 Hz (orientation map) to f_E = 2.62 (salt-and-pepper). This is a somehow counterintuitive result, but it agrees with the predictions of the reduced model. In systems with an orientation map, the correlation between the position of the neurons in the network and their preferred orientations gives rise to functional connectivity. Therefore, there will be a significant modulation of the connectivity matrix in the functional space, that will reduce selectivity according to Equations (29) or (30). This does not necessarily imply that systems with salt-and-pepper organization should always have more selectivity than systems with orientation map. This could only be the case if all the other parameters are kept constant. For instance, the modulation of the external input ρ. A given value of orientation selectivity in layer 2/3 will require a more modulated input in a system with an orientation map than in a network with a salt-and-pepper structure.

FIGURE 4

Figure 4. Distribution of orientation selectivity index for the neurons of the excitatory population. Systems with orientation maps (A) are less selective than networks with a salt-and-pepper organization (B). Parameters set to default values.

Effect of STDP on Orientation Selectivity in the Network Model

Here, we analyze the influence of STDP on orientation selectivity for both architectures (salt-and-pepper and orientation map). In Figure 5 we show the distribution of OSI of the neurons in the excitatory population. In both cases, the average selectivity increases once plasticity is applied, although in a much smaller degree in salt-and-pepper (< OSI > from 0.57 to 0.63) than in orientation map (< OSI > from 0.27 to 0.36). We checked that these values are stable by comparing simulations of 20 s and 40 s long.

FIGURE 5

Figure 5. Spike-timing dependent plasticity increases selectivity. (A) Salt-and-pepper before synaptic modifications < OSI > = 0.57, (B) salt-and-pepper after synaptic modifications < OSI > = 0.63, (C) orientation map before synaptic modifications < OSI > = 0.27, (D) orientation map after synaptic modifications < OSI > = 0.36. In all the cases we show the distribution of orientation selectivity index for the neurons of the excitatory population. Parameters as in the previous figure.

What is the reason for the different behaviors? In the balanced state, the change in selectivity is controlled by the ratio between the mean value (${\tilde{J}}_A(0)$) and the modulation $({\tilde{J}}_A(1))$ of the connectivity matrices in the orientation space (see Equations 29 and 30). These two quantities can be estimated from the simulations by evaluating the change in the synaptic strength as a function of the difference of the preferred orientations between the pre- and post-synaptic neurons. Those changes are shown in Figure 6 for salt-and-pepper (Figures 6A,B) and orientation map (Figures 6C,D).

FIGURE 6

Figure 6. Only for the system with salt-and-pepper organization STDP gives rise to a significant degree of functional connectivity. (A) Final value of the normalized synaptic efficacy w_{i, j} (see Equations 10 and 11) between neurons (i, E) and (j, E) as a function of the difference in the preferred orientations of the same neurons. (B) Average of the previous points in intervals of 10°. Green line: fit with the function $F(x)=a_0+2a_{1}cos(\pi x∕90^{°})$. The values of the fitted parameters are a₀ = 0.14, a₁ = 0.016. (C,D) As in (A,B) but for the orientation map. The values of the fitted parameters are now a₀ = 0.15, a₁ = −0.00084. All the rest of parameters as in the previous figure.

Figures 6A,C show how the strength of the connections changes as a function of the difference of the preferred orientations. As those changes display a great amount of variability, in Figures 6B,D we show the average for all the pairs of neurons whose differences in preferred orientations fall within an interval of 10 degrees (i.e., [0°, 10°), [10°, 20°), …). The resulting averages were fitted with the function $a_0+2a_{1}cos(\pi x∕90^{°})$. The value of a₀ gives an estimation of the change in the mean synaptic strength $\Delta {\tilde{J}}_A(0)$, while a₁ approximates the change in the modulation $\Delta {\tilde{J}}_A(1)$. In both cases a₀ is positive (see Figure 6), this means the connections are strengthened in presence of STDP. For the orientation map the change in the modulation is very small, what leads to a significant improvement of selectivity as the numerator of Equation (29) grows much more as compared with the denominator. For the salt-and-pepper structure both the mean value and the modulation of the synaptic strength grow significantly. This modification gives rise to opposite contributions to the selectivity: While the increase of the mean value tends to improve selectivity, the strong modulation reduces it. For this particular case, the net effect is small but positive, although we cannot rule out a situation where the net result is negative.

Effect of Reconnection Probability on Orientation Selectivity in the Salt-and-pepper Structure

As presented here, the spike-timing dependent plasticity rule can either strengthen or weaken existing connections but cannot create or destroy existing ones (although destruction could be implemented by including an absorbing barrier for a zero value of the synapse). Experimental results (Ko et al., 2013) indicate that a substantial reconnection process takes place in the initial stage of development in rodents. After eye opening, local connectivity reorganizes extensively in such a way that new connections between neurons with similar visual responses arise. However, connectivity rate does not significantly change during this process and only a modest increment in orientation selectivity takes place, taking average OSI from 0.62 to 0.68 (Ko et al., 2013). Another motivation for introducing a reconnection mechanism as implemented in Equation (12), is that the average synaptic efficacy is automatically kept constant. In the balanced state the mean firing rates are determined by the mean connectivity (see Equations 26, 27 for n = 0). As the reconnection process does not affect the total number of connections, there should be no change in the average firing rates. Thus, any change in the selectivity properties will be given by a modification of the modulated part of the tuning curves.

We analyze the effect of this reconnection process in our computational model. We first calculate the preferred orientation of each neuron using the population vector (Equation 8) with a simulation of 20 s. Then, according to that preferred orientation, the connection probability is recalculated (Equation 12) and a new connectivity matrix is generated. Note that even for the salt-and-pepper organization this process will generate a new network with a significant degree of functional connectivity. This procedure will work only if there is a significant degree of correlation between the preferred orientations both before and after the reconnection. This is verified for both populations in Figure 7, where we show that the preferred orientations are strongly correlated with the preferred orientation of the feed-forward input (see Figures 7A,B), and that the are mostly preserved by the reconnection process (see Figures 7C,D).

FIGURE 7

Figure 7. Preferred orientations of the neurons (PO_iA) are strongly correlated to preferred orientations of the feed-forward inputs (θ_iA) and they are conserved after reconnection. All the graphs correspond to the salt-and-pepper organization with default parameters. (A,C) Excitatory neurons. (B,D) Inhibitory neurons. In all the panels we show 2025 neurons. Parameters set as default.

The reconnection parameters (ϵ_cE, ϵ_cI) determine how much modulated the corresponding connectivity matrix is. It is important to remark that if the degree of modulation in the excitatory interactions is too large as compared to the modulation in the inhibitory interactions, the balanced state might become unstable (see Equation 41). The results of Ko et al. (2013) indicate a degree of functional connectivity between excitatory neurons compatible with a value of ϵ_cE ≈ 0.44 (see Figure 2i in Ko et al., 2013). According to the coupling values we are using, and asking for the network to remain in the stable balanced state, ϵ_cI is required to be at least equal to 0.11 (because |G_EE ∕G_II| = 1∕4, see Equation 41). In Figure 8 we show the mean values of selectivity of both populations for combinations of ϵ_cE, ϵ_cI that preserve the stability of the balanced state. We observe that increasing the functional connectivity in the excitatory population leads to a decrease in the excitatory selectivity (Figure 8A), while increasing the functional connectivity in the inhibitory population conveys a reduction in the inhibitory selectivity (Figure 8B); Both effects were predicted by the reduced model. Note, however, that in both cases there is a change of selectivity for the population whose interactions have not been affected by the reconnection process (population I in Figure 8A and population E in Figure 8B). This behavior is not predicted by the reduced model and deserves further investigation to fully understand it. It is probably due to the different topology between the two models. Let us note that in all the cases, the reconnection rule generates a change in the mean firing rates always smaller than 5%.

FIGURE 8

Figure 8. When the balanced state is stable increasing functional connectivity in the excitatory interactions leads to a loss of selectivity in the excitatory population and an increase of selectivity in the inhibitory population. In contrast, increasing functional connectivity in the inhibitory interactions always leads to reduction of selectivity. (A) Average orientation selectivity index for the excitatory population (blue) and for the inhibitory population (red) as a function of the reconnection parameter ϵ_cE keeping ϵ_cI = 0.22. (B) The same averages as a function of ϵ_cI with ϵ_cE = 0.44.

It is also possible to choose a combination of the reconnection parameters so that the balanced state becomes unstable, for instance ϵ_cE = 0.44 and ϵ_cI = 0.03. Using those parameters, the mean value of orientation selectivity is < OSI_E > = 0.67 (see Figure 9). This result represents an increment of selectivity from the control situation, at which < OSI_E > = 0.57. This increase cannot be predicted by the theory of the reduced model in the balanced state, since the network is not balanced anymore. Note that the loss of stability for the first Fourier mode does not lead to a significant change of the mean firing rates, that are given by f_E = 2.38 Hz and f_I = 6.19 Hz for ϵ_cE = 0.44 and ϵ_cI = 0.03 (compared to f_E = 2.61 Hz and f_I = 6.05 Hz in the control case). As the balanced state is not longer stable, the results we show here may depend on the details of the network dynamics since the non-linearities of the neuron transfer curve are not washed out by the balanced equations anymore.

FIGURE 9

Figure 9. When the balanced state is unstable for the first Fourier mode, increasing functional connectivity in the inhibitory interactions leads to reduction of selectivity. Average orientation selectivity index for the excitatory population (blue) and for the inhibitory population (red) as a function of ϵ_cI with ϵ_cE = 0.44.

Discussion

We studied the selectivity properties of systems with orientation map or salt-and-pepper structure. We found that, keeping all the rest of the parameters the same, systems with an orientation map were less selective for orientation than systems with a salt-and-pepper organization. If we define functional connectivity as the correlation between the connectivity structure and the responses of the neurons, the result we found is due to the fact that the more functional connectivity the system has, the more modulated its connectivity profile is. Therefore, according to Equations (29) and (30), the system becomes less selective to orientation. This does not mean that systems with salt-and-pepper architecture should always be more selective than systems with an orientation map. For instance, high selectivity in a system with an orientation map can be achieved by increasing the selectivity of the feed-forward input.

We worked in the framework of balanced networks. Several experimental works support the idea that cortical systems operate in a balanced regime, i.e., excitation is compensated by inhibition (Destexhe et al., 2003; Shu et al., 2003; Haider et al., 2006). This idea was proposed to explain why neurons in vivo fire so irregularly (Holt et al., 1996) in spontaneous as well as in sensory-evoked activity (van Vreeswijk and Sompolinsky, 1996, 1998). It also intends to explain the highly irregular neuronal firing pattern in persistent activity by introducing synaptic non-linearities (Hansel and Mato, 2013).

For a network to operate in the balanced regime, the recurrent and the feed-forward inputs have to be large as compared to the threshold of the neurons (van Vreeswijk and Sompolinsky, 1996, 1998; Hansel and van Vreeswijk, 2012). Under very general conditions, the balance emerges from the recurrent dynamics of the network so that the net input is much smaller than its excitatory and inhibitory components. Thus, as the mean value of the net input is below the threshold neurons fire because of the fluctuations of the input. Therefore, the firing properties are essentially determined by the statistical properties of the input and the intrinsic dynamic of the neuron is not relevant (Hansel and van Vreeswijk, 2002). Moreover, in Rauch et al. (2003) it was found that the firing properties of cortical neurons in vivo can be well approximated by an integrate-and-fire model. These results imply that the findings we here present do not depend on the integrate-and-fire model, but they apply to other neuron models too. However, at high firing rates this conclusion does not longer hold because of the refractory period of the neurons, but these effects are irrelevant for systems that are as far from the saturation of the firing rate as our results are.

In the balanced state the main role of the recurrent connections is to cancel the leading order of the feed-forward input. One of the consequences of this behavior is that increasing all the coupling constants by a given factor leads to a decrease in the firing rates by the same factor if the external inputs are kept constant (van Vreeswijk and Sompolinsky, 1998). As at the population level the system is linear, the same behavior is found for each one of the Fourier components of the activity (see Equations 26 and 27). In other words, there is an inverse relation between the modulation of the activity profile and the modulation of the connectivity profile. This fact was already observed in van Vreeswijk and Sompolinsky (2005), where it was analyzed in terms of the cancelation of the leading terms of the external input. In the limit of large size the recurrent input is equal to the convolution between the network activity and the connectivity matrices. Therefore, in order to keep the recurrent input constant, wider connectivity profiles and narrower activities must be balanced out. This counterintuitive behavior is not always observed nor well understood. For instance, in Corey and Scholl (2012), it was stated that increasing the connection probability of nearby neurons with similar orientation preference might further enhance their selectivity. However, here we show that this is not necessarily true. In fact, for networks in the balanced state the opposite result is the one that should be expected.

In a recent work (Ko et al., 2013) it was shown that after eye opening, local connectivity reorganized extensively without changing the overall connectivity. This means that connections between neurons with similar visual response were created and strengthened, but that the mean value of connections remained the same. This reconnection process must coexist with plasticity mechanisms that affect the synaptic strength without changing the number of synapses. Such a plasticity mechanism could be STDP that, in fact, has also been found in cortical systems (Markram et al., 1997; Feldman, 2000).

We investigated both mechanisms separately: The STDP process and the reconnection rule. Regarding STDP, this rule leads to a significant increment in the orientation selectivity for systems with orientation map (~50%), while it results in a very small effect for salt-and-pepper (~10%). We also found that in the first case the plasticity rule did not generate significant functional connectivity, while in the second one, it did. The observed change in selectivity is consistent with the one found with the theoretical analysis of the reduced model. If there is a positive effect on the selectivity it is because the mean value of the connections increases more than its modulation. For instance, as for the salt-and-pepper structure STDP generates a strong modulation of the connectivity profile, the generation of functional connectivity results in a negative effect on selectivity. On the other hand, for systems with an orientation map the most significant effect of STDP is the increment of the average connectivity without giving rise to functional connectivity. This combination strongly increases selectivity. Note that despite the fact that STDP can generate functional connectivity, this feature does not contribute to an increment of selectivity. The rise in selectivity is dominated by the change of the mean value of the connectivity profile as compared to the modulation that it acquires after plasticity is applied. This non-trivial result is a consequence of the dynamics of the balanced regime.

It is important to remark that the difference in the growth of selectivity for both architectures that we found, is probably due to the small degree of selectivity that systems with an orientation map have in the first place in our simulations. For the STDP rule, the amount of functional connectivity generated is proportional to the square of the selectivity before learning (see Equation 34). As the initial value of selectivity in salt-and-pepper is more than twice as large as the original value of selectivity in orientation map, much more functional connectivity is generated in the first case. However, note that some kind of self-regulatory mechanism arises: Systems with strong selectivity develop more functional connectivity as STDP is applied, but the more functional connectivity is generated, the less the selectivity increases. In other words, for systems that are highly selective to orientation, STDP does not produce such an increment of selectivity as it does for systems that are initially less selective.

Though in some systems inhibitory connections exhibit STDP (Haas et al., 2006) too, for simplification we here assume that only excitatory synapses are plastic. However, the effect of plasticity on inhibitory connections can be easily analyzed with the same tools presented here. For instance, an increase of the modulated part of the inhibitory interactions should lead to some degree of reduction of the orientation selectivity in the inhibitory population if the non-modulated part is kept constant (see Equation 30). An increase of the non-modulated part, in contrast, should lead to more selectivity. These effects are the same as the ones found for the excitatory interactions. The difference between both of them is that an increment of the modulation of the inhibitory interactions cannot lead to the loss of the stability of the balanced state. If Equation (23) is initially satisfied, then an increase in the modulation would only make the second term of the left side more negative.

Similarly, the effect of homeostatic processes (Turrigiano and Nelson, 2000, 2004) that tend to keep constant the mean firing rate could be analyzed too. These mechanisms might affect selectivity even if the total amount of modulation of the tuning curves is kept constant. For instance, an additive homeostatic process would modify the zeroth Fourier component of the connectivity matrices but not the other ones. If this mechanism acts at the same time as STDP on the excitatory interactions, it will cancel the change of the mean value of the connections while keeping constant the rise of the modulated part. This will lead to the loss of orientation selectivity. In contrast, a global synaptic rescaling that equalizes the postsynaptic firing rates (Turrigiano, 1999; van Rossum et al., 2000) will affect by the same factor all the synapses independently of the position of the neurons in the state space. Therefore, the ratio between the zeroth Fourier component and the first one will not be affected and the change of selectivity will be the same as without homeostasis.

Finally, we want to remark that in order to preserve the stability of the balanced state, some constraints between the modulation of the excitatory interactions and the modulation of the inhibitory interactions must be satisfied (see Equation 56). If the modulation of the excitatory interactions is large enough, then the balanced state becomes unstable for the first Fourier mode. Remarkably, the loss of stability does not give rise to a significant change of the average firing rate, implying that the instability does not affect the zeroth Fourier mode. This comment is also applicable to the generation of functional connectivity using STDP only on the excitatory interactions: Increasing the modulation of the excitatory population gives rise to an increase in selectivity, but even a small increment of the modulation of the inhibitory interactions might cancel the effect. This could explain the results of Ko et al. (2013), where it was found that a significant increase in functional connectivity of the excitatory interactions generates a very small change of orientation selectivity. In any case, it is clear that more information on the functional connectivity of the inhibitory interactions is needed to understand completely the behavior of the system.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Inés Samengo for a careful reading of the manuscript and David Hansel for useful discussions.

Supplementary Material

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fncir.2015.00042

Sample software codes implemented in C are available at: http://fisica.cab.cnea.gov.ar/estadistica/matog/v1-stdp/software.html

References

Abbott, L. F., and Nelson, S. B. (2000). Synaptic plasticity: taming the beast. Nat. Neurosci. 3, 1178–1183. doi: 10.1038/81453

PubMed Abstract | CrossRef Full Text | Google Scholar

Ben-Yishai, R., Bar-Or, R. L., and Sompolinsky, H. (1995). Theory of orientation tuning in visual cortex. Proc. Natl. Acad. Sci. U.S.A. 92, 3844–3848. doi: 10.1073/pnas.92.9.3844

PubMed Abstract | CrossRef Full Text | Google Scholar

Bi, G.-Q., and Poo, M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464–10472.

PubMed Abstract | Google Scholar

Bonhoeffer, T., and Grinvald, A. (1991). Iso-orientation domains in cat visual cortex are arranged in pinwheel-like patterns. Nature 353, 429–431. doi: 10.1038/353429a0

PubMed Abstract | CrossRef Full Text | Google Scholar

Cossell, L., Iacaruso, M. F., Muir, D. R., Houlton, R., Sader, E. N., Ko, H., et al. (2015). Functional organization of excitatory synaptic strength in primary visual cortex. Nature 518, 399–403. doi: 10.1038/nature14182

PubMed Abstract | CrossRef Full Text | Google Scholar

Corey, J., and Scholl, B. (2012). Cortical selectivity through random connectivity. J. Neurosci. 32, 10103–10104. doi: 10.1523/JNEUROSCI.2463-12.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Destexhe, A., Rudolph, M., and Paré, D. (2003). The high-conductance state of neocortical neurons in vivo. Nat. Rev. Neurosci. 4, 739–751. doi: 10.1038/nrn1198

PubMed Abstract | CrossRef Full Text | Google Scholar

Dräger, U. C. (1975). Receptive fields of single cells and topography in mouse visual cortex. J. Comp. Neurol. 160, 269–290. doi: 10.1002/cne.901600302

PubMed Abstract | CrossRef Full Text | Google Scholar

Feldman, D. E. (2000). Timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. Neuron 27, 45–56. doi: 10.1016/S0896-6273(00)00008-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Haas, J. S., Nowotny, T., and Abarbanel, H. D. I. (1996). Spike-timing-dependent plasticity of inhibitory synapses in the entorhinal cortex. J. Neurophysiol. 96, 3305–3313. doi: 10.1152/jn.00551.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

Haider, B., Duque, A., Hasenstaub, A. R., and McCormick, D. A. (2006). Neocortical network activity in-vivo is generated through a dynamic balance of excitation and inhibition. J. Neurosci. 26, 4535–4545. doi: 10.1523/JNEUROSCI.5297-05.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

Hansel, D., and Mato, G. (2013). Short-term plasticity explains irregular persistent activity in working memory tasks. J. Neurosci. 33, 133–149. doi: 10.1523/JNEUROSCI.3455-12.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Hansel, D., and van Vreeswijk, C. (2002). How noise contributes to contrast invariance of orientation tuning in cat visual cortex. J. Neurosci. 22, 5118–5128. Available online at: http://www.jneurosci.org/content/22/12/5118.full

Hansel, D., and van Vreeswijk, C. (2012). The mechanism of orientation selectivity in primary visual cortex without a functional map. J. Neurosci. 32, 4049–4064. doi: 10.1523/JNEUROSCI.6284-11.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Holmgren, C., Harkany, T., Svennenfors, B., and Zilberter, Y. (2003). Pyramidal cell communication within local networks in layer 2/3 of rat neocortex. J. Physiol. 551, 139–153. doi: 10.1113/jphysiol.2003.044784

PubMed Abstract | CrossRef Full Text | Google Scholar

Holt, G. R., Softky, W. R., Koch, C., and Douglas, R. J. (1996). Comparison of discharge variability in vitro and in vivo in cat visual cortex neurons. J. Neurophysiol. 75, 1806–1814.

PubMed Abstract | Google Scholar

Hubel, D. H., and Wiesel, T. N. (1961). Integrative action in the cat's lateral geniculate body. J. Physiol. 155, 385–398. doi: 10.1113/jphysiol.1961.sp006635

PubMed Abstract | CrossRef Full Text | Google Scholar

Hubel, D. H., and Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 160, 106–154. doi: 10.1113/jphysiol.1962.sp006837

PubMed Abstract | CrossRef Full Text | Google Scholar

Ko, H., Hofer, S. B., Pichler, B., Buchanan, K. A., Sjöström, P. J., Mrsic-Flogel, T. D. et al. (2011). Functional specificity of local synaptic connections in necortical networks. Nature 473, 87–91. doi: 10.1038/nature09880

PubMed Abstract | CrossRef Full Text | Google Scholar

Ko, H., Cossell, L., Baragli, C., Antolik, J., Clopath, C., and Hofer, S. (2013) The emergence of functional microcircuits in visual cortex. Nature 496, 96–100. doi: 10.1038/nature12015

PubMed Abstract | CrossRef Full Text | Google Scholar

Koch, C. (1999). Biophysics of Computation. Oxford: Oxford University Press.

Google Scholar

Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213–215. doi: 10.1126/science.275.5297.213

PubMed Abstract | CrossRef Full Text | Google Scholar

Ohki, K., Chung, S., Ch'ng, Y. H., Kara, P., and Reid, R. C. (2005). Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex. Nature 433, 597–603. doi: 10.1038/nature03274

PubMed Abstract | CrossRef Full Text | Google Scholar

Ohki, K., and Reid, R. C. (2007). Specificity and randomness in the visual cortex. Curr. Opin. Neurobiol. 17, 401–407. doi: 10.1016/j.conb.2007.07.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Press, W., Vetterling, W., Teukolsky, S., and Flannery, B. (1992). Numerical ecipes in FORTRAN 77: The Art of Scientific Computing. Cambridge, UK: Cambridge University Press.

Google Scholar

Rauch, A., La Camera, G., Luscher, H. R., Senn, W., and Fusi, S. (2003). Neocortical pyramidal cells respond as integrate-and-fire neurons to in-vivo like input currents. J. Neurophysiol. 90, 1598–1612. doi: 10.1152/jn.00293.2003

PubMed Abstract | CrossRef Full Text | Google Scholar

Renart, A., de la Rocha, J., Bartho, P., Hollender, L., Parga, N., Reyes, A., et al. (2010). The asynchronous state in cortical circuits. Science 327, 587–590. doi: 10.1126/science.1179850

PubMed Abstract | CrossRef Full Text | Google Scholar

Rubin, J., Lee, D. D., and Sompolinsky, H. (2001). Equilibrium properties of temporally asymmetric hebbian plasticity. Phys. Rev. Lett. 86, 364–367. doi: 10.1103/PhysRevLett.86.364

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosenbaum, R., and Doiron, B. (2014). Balanced networks of spiking neurons with spatially dependent recurrent connections. Phys. Rev. X 4:021039. doi: 10.1103/PhysRevX.4.021039

CrossRef Full Text | Google Scholar

Scholl, B., Tan, A. Y. Y., Corey, J., and Priebe, N. J. (2013). Emergence of orientation selectivity in the mammalian visual pathway. J. Neurosci. 33, 10616–10634. doi: 10.1523/JNEUROSCI.0404-13.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Shu, Y., Hasenstaub, A., and McCormick, D. A. (2003). Turning on and off recurrent balanced cortical activity. Nature 423, 288–293. doi: 10.1038/nature01616

PubMed Abstract | CrossRef Full Text | Google Scholar

Softky, W. R., and Koch, C. (1993). The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci. 13, 334–350.

PubMed Abstract | Google Scholar

Somers, D. C., Nelson, S., and Sur, M. (1995). An emergent model of orientation selectivity in cat visual cortical simple cells. J. Neurosci. 15, 5448–5465.

PubMed Abstract | Google Scholar

Stepanyants, A., Hirsch, J. A., Martinez, L. M., Kisvárday, Z. F., Ferecskó, A. S., and Chklovskii, D. B. (2008). Local potential connectivity in cat primary visual cortex. Cereb. Cortex 18. 13–28. doi: 10.1093/cercor/bhm027

PubMed Abstract | CrossRef Full Text | Google Scholar

Turrigiano, G. G. (1999). Homeostatic plasticity in neuronal networks: the more things change, the more they stay the same. Trends Neurosci. 22, 221–227. doi: 10.1016/S0166-2236(98)01341-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Turrigiano, G. G., and Nelson, S. B. (2000). Hebb and homeostasis in neuronal plasticity. Curr. Opin. Neurobiol. 10, 358–364. doi: 10.1016/S0959-4388(00)00091-X

PubMed Abstract | CrossRef Full Text | Google Scholar

Turrigiano, G. G., and Nelson, S. B. (2004). Homeostatic plasticity in the developing nervous system. Nat. Rev. Neurosci. 5, 97–107. doi: 10.1038/nrn1327

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Hooser, S. D., Heimel, J. A. F., Chung, S., Nelson, S. B., and Toth, L. J. (2005). Orientation selectivity without orientation maps in visual cortex of a highly visual mammal. J. Neurosci. 25, 19–28. doi: 10.1523/JNEUROSCI.4042-04.2005

PubMed Abstract | CrossRef Full Text | Google Scholar

van Rossum, M. C., Bi, G. Q., and Turrigiano, G. G. (2000). Stable Hebbian learning from spike timing-dependent plasticity. J. Neurosci. 20, 8812–8821. Available online at: http://www.jneurosci.org/content/20/23/8812.full

van Vreeswijk, C., and Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726. doi: 10.1126/science.274.5293.1724

PubMed Abstract | CrossRef Full Text | Google Scholar

van Vreeswijk, C., and Sompolinsky, H. (1998). Chaotic balanced state in a model of cortical circuits. Neural Comput. 10, 1321–1372. doi: 10.1162/089976698300017214

PubMed Abstract | CrossRef Full Text | Google Scholar

van Vreeswijk, C., and Sompolinsky, H. (2005). “Irregular activity in large networks of neurons,” in Les Houches Lectures LXXX on Methods and Models in Neurophysics, eds C. Chow, B. Gutkin, D. Hansel, C. Meunier, and J. Dalibard (London: Elsevier), 341–402.

Wiesel, T. N., and Hubel, D. H. (1974). Ordered arrangement of orientation columns in monkeys lacking visual experience. J. Comp. Neurol. 158, 307–318. doi: 10.1002/cne.901580306

PubMed Abstract | CrossRef Full Text | Google Scholar