Information-geometric measures estimate neural interactions during oscillatory brain states

The characterization of functional network structures among multiple neurons is essential to understanding neural information processing. Information geometry (IG), a theory developed for investigating a space of probability distributions has recently been applied to spike-train analysis and has provided robust estimations of neural interactions. Although neural firing in the equilibrium state is often assumed in these studies, in reality, neural activity is non-stationary. The brain exhibits various oscillations depending on cognitive demands or when an animal is asleep. Therefore, the investigation of the IG measures during oscillatory network states is important for testing how the IG method can be applied to real neural data. Using model networks of binary neurons or more realistic spiking neurons, we studied how the single- and pairwise-IG measures were influenced by oscillatory neural activity. Two general oscillatory mechanisms, externally driven oscillations and internally induced oscillations, were considered. In both mechanisms, we found that the single-IG measure was linearly related to the magnitude of the external input, and that the pairwise-IG measure was linearly related to the sum of connection strengths between two neurons. We also observed that the pairwise-IG measure was not dependent on the oscillation frequency. These results are consistent with the previous findings that were obtained under the equilibrium conditions. Therefore, we demonstrate that the IG method provides useful insights into neural interactions under the oscillatory condition that can often be observed in the real brain.

Introduction

The dynamics of neural interactions have been conjectured to play an important role in neural information processing. One way to investigate the neural interactions is to record multi-neuronal firing activity from a freely behaving animal, and analyze the correlations between individual units. In past decades, electrophysiological studies have significantly been advanced by the use of multi-electrode recording techniques (Wilson and McNaughton, 1993; Chapin et al., 1999; Kudrimoti et al., 1999; Laubach et al., 2000; Hoffman and McNaughton, 2002; Buzsaki, 2004; Tatsuno et al., 2006; Euston et al., 2007; Davidson et al., 2009; Peyrache et al., 2009b; Dragoi and Tonegawa, 2011, 2013). In order to analyze such high-dimensional multi-neuronal datasets, a number of statistical methods have also been developed (Gerstein and Perkel, 1969; Abeles and Gerstein, 1988; Aertsen et al., 1989; Zhang et al., 1998; Panzeri and Schultz, 2001; Grun et al., 2002a,b; Brown et al., 2004; Fellous et al., 2004; Czanner et al., 2005; Shimazaki and Shinomoto, 2007; Amari, 2009; Gilestro et al., 2009; Peyrache et al., 2009a; Shimokawa and Shinomoto, 2009; Lopes-Dos-Santos et al., 2011). Recently, a method based on information geometry (IG) has been applied to the analysis of neural data (Amari and Nagaoka, 2000; Amari, 2001; Nakahara and Amari, 2002; Amari et al., 2003; Tatsuno and Okada, 2004; Eleuteri et al., 2005; Ikeda, 2005; Miura et al., 2006; Nakahara et al., 2006; Gilestro et al., 2009; Tatsuno et al., 2009; Ince et al., 2010; Lovette et al., 2011; Nie and Tatsuno, 2012). It has been demonstrated that IG provides a powerful statistical tool for analyzing spiking data. Some of the advantages of IG approach include the orthogonal decomposition of neural interactions (Amari, 2001; Nakahara and Amari, 2002), and its direct relationship to underlying connections (Tatsuno and Okada, 2004; Tatsuno et al., 2009; Nie and Tatsuno, 2012); the single-IG measure is related to the amount of external inputs and the pairwise-IG measure is related to the amount of direct neural interactions between two neurons.

These IG properties were often investigated under the assumption that the network is in an equilibrium state. However, in the brain, the equilibrium assumption does not hold true. Instead, the brain undergoes a variety of non-equilibrium states such as oscillations. For example, the slow-wave oscillation (~1 Hz) was discovered during non-REM sleep (Steriade et al., 1993; Crunelli and Hughes, 2010), and evidence suggests that it plays an important role in memory consolidation (Huber et al., 2004; Stickgold, 2005; Diekelmann and Born, 2010). The theta (6–10 Hz) rhythm is a prominent coherent oscillation observed in the hippocampus, and its surrounding area during rat spatial navigation (Vanderwolf, 1969; Bland, 1986; Buzsaki, 2002). The theta rhythm has also been observed in various human neocortical areas during the delay period of working memory tasks (Raghavachari et al., 2001; Meltzer et al., 2008). The beta (15–30 Hz) oscillation is conjectured to play a key role in action preparation and inhibitory control in the motor system (Baker et al., 1997). The gamma (30–80 Hz) oscillation has been shown to play a role in the integration of sensory information (Gray et al., 1989; Singer and Gray, 1995). The fast hippocampal sharp wave ripples (100–200 Hz) were also reported during an animal's awake immobility and slow-wave sleep (Buzsaki et al., 1992). Therefore, it is important to investigate if the IG measures can be applied to neural data under oscillatory conditions.

In this study, we investigated how the single- and pairwise-IG measures are influenced by a network oscillation. Under an equilibrium assumption, previous studies have shown that the single- and pairwise-IG measures provide a robust estimation of the magnitude of external input and direct neural interactions (Tatsuno et al., 2009; Nie and Tatsuno, 2012). We also focused on these IG measures in this study because the external inputs and intrinsic neural interactions are the two main factors for characterizing network dynamics. For the oscillation mechanisms, we have considered two representative cases; one is an external driven oscillation where a network is influenced by external oscillatory inputs. The other is an internally induced oscillation where interactions between excitatory and inhibitory neuron populations produce an oscillation. By computer simulations using simple binary model neurons or more biologically plausible spiking neurons, we investigated whether the properties of the IG measures that were established with the equilibrium condition still hold true under oscillatory network states.

In section Methods, we briefly introduce an information-geometric analysis of neural spikes. In section Results, we describe the model and network structure used in the numerical simulation. In section Discussion, the simulation results for both externally driven and internally induced oscillations are described in detail. In section Acknowledgments, we summarize our findings and discuss future directions of research on this topic.

Methods

Information-Geometric Method

We briefly introduce the information-geometric method for spiking data analysis (for details see Amari and Nagaoka, 2000). Generally, in an N-neuron system, the state of i-th(i = 1, …, N) neuron is represented by a binary random variable x_i, where x_i = 1 or 0 representing neuronal firing or silence, respectively. The joint probability distribution of the N-neuron system can be described by a fully expanded N-th order log-linear model (LLM)

$\begin{array}{cc}\begin{array}{l}\log p_{x_1{x}_2\cdots x_N}={\displaystyle \sum _i{\theta }_i^{(N,N)}}{x}_i+{\displaystyle \sum _{i<j}{\theta }_{ij}^{(N,N)}}{x}_i{x}_j+\cdots \\ +{\theta }_{12,\cdots N}^{(N,N)}{x}_1{x}_2\cdots x_N-\psi {(\theta )}^{(N,N)},\end{array}& (1)\end{array}$

where θ^{(N, N)}_{ij,… m} (1 ≤ m ≤ N) represents the m-neuron interaction and ψ(θ)^{(N, N)} with θ = {θ^{(N, N)}_i, θ^{(N, N)}_ij, …, θ^{(N, N)}_12,…N } is a normalization constant such that ∑ p_x1x2 … x_N = 1. The first and the second superscripts in θ^{(N, N)}_{ij, … m} represent the order of the LLM and the number of neurons in the system. We use θ^{(N, N)}_i, θ^{(N, N)}_ij, and θ^{(N, N)}_{ij, … m} to describe the single-IG measure, the pairwise-IG measure and the m-neuron IG measure with the N-th order LLM for a N-neuron system, respectively, (Nie and Tatsuno, 2012). The joint probability of N neurons is calculated by

$\begin{array}{cc}{p}_{x_1{x}_2,\dots ,x_N}=\frac{c_{x_1{x}_2,\dots ,x_{\text{N}}}}{{\displaystyle {\sum }_{x_1{x}_2,\dots ,x_{\text{N}}}{c}_{x_1{x}_2,\dots ,x_{\text{N}}}}},& (2)\end{array}$

where c_x1x₂,…,x_N is the count of events (X₁ = x₁, X₂ = x₂, …,X_N = x_N) that occur.

However, in reality, it is difficult to calculate the statistical information from all neurons in a large network. Therefore, the partially expanded LLM is often used for the estimation of neuronal interactions. The partially expanded k-th order LLM in an N-neuron network is given by

$\begin{array}{cc}\begin{array}{l}\log p_{x_1{x}_2\cdots x_{k,\ast \cdots \ast }}={\displaystyle \sum _i{\theta }_i^{(k,N)}}{x}_i+{\displaystyle \sum _{i<j}{\theta }_{ij}^{(k,N)}}{x}_i{x}_j+\cdots \\ +{\theta }_{12,\cdots k}^{(k,N)}{x}_1{x}_2\cdots x_k-\psi {(\theta )}^{(k,N)}\end{array}& (3)\end{array}$

where θ = {θ^{(k, N)}_i, θ^{(k, N)}_ij, …, θ^{(k, N)}_{12,… k}}. The first few terms of θ and normalization factor are given as follows:

$\begin{array}{cc}\begin{array}{l}{\theta }_i^{(k,N)}=\log \frac{p_{x_1=0,\cdots ,x_i=1,\cdots ,x_k=0,\ast \cdots \ast }}{p_{x_1=0,\cdots ,x_k=0,\ast \cdots \ast }},\\ {\theta }_{ij}^{(k,N)}=\log \frac{\begin{array}{c}{p}_{x_1=0,\dots ,x_i=1,\dots ,x_j=1,\cdots ,x_k=0,\ast \cdots \ast }\\ p_{x_1=0,\cdots ,x_k=0,\ast \cdots \ast }\end{array}}{\begin{array}{c}{p}_{x_1=0,\dots ,x_i=1,\dots ,x_j=0,\dots ,x_k=0,\ast \cdots \ast }\\ p_{x_1=0,\dots ,x_i=0,\dots ,x_j=1,\dots ,x_k=0,\ast \cdots \ast \cdots \cdots \cdots }\end{array}}\\ {\psi }^{(k,N)}=-\log p_{x_1=0,\dots ,x_k=0,\ast \cdots \ast },\end{array}& (4)\end{array}$

where ‘* … *′, represents the marginalization over the (N − k) neurons.

The single-IG measure θ^{(k, N)}_i and the pairwise-IG measure θ^{(k, N)}_ij are the two main focuses in this study because θ^{(k, N)}_i is related to the amount of external inputs and θ^{(k, N)}_ij is related to the amount of direct neural interactions between two neurons (Tatsuno and Okada, 2004; Tatsuno et al., 2009; Nie and Tatsuno, 2012). Using a network of simple binary neurons, and the assumption of an equilibrium state, the previous study has shown that the single-IG measure θ^{(2, N)}_i and the pairwise-IG measure θ^{(2, N)}_ij with the 2nd-order LLM are related to the network parameters as

$\begin{array}{cc}{\theta }_i^{(2,N)}\propto 2h_i+O\left(\frac{1}{N}\right),{\theta }_{ij}^{(2,N)}\propto (J_{ij}+J_{ji})+O\left(\frac{1}{N}\right),& (5)\end{array}$

where h_i represents the magnitude of constant external input to a neuron i, and J_ij (J_ji) is the connection weight from a neuron j to i (from a neuron i to j), respectively, (Tatsuno et al., 2009). If a network receives correlated inputs, the relationship for θ^{(2, N)}_ij in Equation 5 does not hold true anymore. However, we have also shown that θ^{(k, N)}_ij with the higher k-th order LLM provides a better estimation of neural interactions (Nie and Tatsuno, 2012). For example, θ^{(4, N)}_ij was shown to have the relationship

$\begin{array}{cc}{\theta }_{ij}^{(4,N)}\propto \left(J_{ij}+J_{ji}\right),& (6)\end{array}$

within approximately a 10% error if the number of neurons N is ~10³ or larger; a typical size of network in a cortical column (Urban et al., 2001). We have also confirmed that the relationship θ^{(4, N)}_i α2h_i holds true within approximately a 10% error (unpublished data).

These properties could be useful for the field of neuroscience because the IG measures can estimate the changes of underlying network parameters (h_i and J_ij) separately, while other correlation measures have not yet been shown to have such a property (Amari, 2009). However, these results were derived under the equilibrium limit, and little is known if the similar relationship holds under the oscillatory condition.

Neuron Model and Network Structure

Neuron model

We investigated the influence of oscillations using a network of simple binary neurons with stochastic dynamics (Ginzburg and Sompolinsky, 1994) and biologically plausible spiking neurons (Izhikevich, 2003). Using simple binary model neurons, we first investigated whether the property of the IG measures that were shown under the equilibrium condition also held true for the oscillatory condition. We then extended our investigation to more realistic spiking neurons.

For a binary model neuron, the transition between the binary states is given by the transition rate w as

$\begin{array}{cc}\begin{array}{c}w(x_i=0\to x_i=1)=\frac{g(u_i)}{{\tau }_0},\\ w(x_i=1\to x_i=0)=\frac{1-g(u_i)}{{\tau }_0},\\ w(x_i=0\to x_i=0)=1-w(0\to 1),\\ w(x_i=1\to x_i=1)=1-w(1\to 0),\end{array}& (7)\end{array}$

where τ₀ is a microscopic characteristic time and u_i represents the total input to the i-th neuron.

$\begin{array}{cc}g(u_i)=\frac{1+\tanh (u_i-m)}{2}& (8)\end{array}$

is the sigmoidal function in the bounded interval [0, 1] where m is a parameter controlling the firing probability of a model neuron.

For a biologically more plausible neuron model, we adopted the Izhikevich model because it is known to be computationally efficient and biologically plausible (Izhikevich, 2003). The Izhikevich model reduced the complex dynamics of the Hodgkin–Huxley (HH) neuronal models to two coupled differential equations as

$\begin{array}{cc}\text{}\text{}\text{}\frac{d{V}_i}{dt}=0.04V_i^2+5V_i+140-U_i+I_i,\frac{d{U}_i}{dt}=a_i(b_i{V}_i-U_i).& (9)\end{array}$

Here the variable V_i represents the membrane potential of neuron i, and U_i represents a membrane recovery variable which correlates with the activation of K⁺ ionic currents and inactivation of Na⁺ [for detail see (Izhikevich, 2003)]. U_i and V_i are reset after a spike: if V_i ≥ 30 mV, then V_i ← c_i, U_i ← U_i + d_i. I_i represents a total input to neuron i; a_i, b_i, c_i, d_i are dimensionless adjustable parameters which are usually taken as (a_i, b_i) = (0.02, 0.2) and (c_i, d_i) = (−65, 8) + (15, −6)r²_i for excitatory neurons, (a_i, b_i) = (0.02, 0.25) + (0.08, −0.05r_i) and (c_i, d_i) = (−65, 2) for inhibitory neurons. r_i is a uniformly distributed random variable on the interval [0, 1] (Izhikevich, 2003).

Network structure

We considered two mechanisms for generating oscillatory network states; one is the oscillation driven by external inputs (Figure 1A), and the other is the oscillation induced by the intrinsic interaction between excitatory and inhibitory neuron populations (Figure 1B). The former mechanism can be a model for hippocampal theta oscillation in which the projections from the medial septum to the hippocampus play a central role (Dragoi et al., 1999). The latter structure where excitatory and inhibitory neuron pools interact is widely observed in cortical areas (Buzsaki and Wang, 2012). It can be a model for cortical oscillations (such as in a gamma-range) that rely on the interplay between excitatory and inhibitory neuron pools.

FIGURE 1

Figure 1. A schematic of two mechanisms for generating network oscillations. (A) Oscillation is generated by an external oscillatory input (externally driven oscillation). A neuron i in the network of N neurons with recurrent connections J_ij receives a sinusoidal external input h_i = h₀ sin(ωt + φ_i) where h₀, ω, and φ_i represent the amplitude, angular speed, and phase of the sinusoidal input, respectively. (B) Oscillation is generated by the interaction between excitatory and inhibitory neuron pools (internally induced oscillation). Excitatory neurons are connected by positive connections J_EE, inhibitory neurons are connected by negative connections J_II, inhibitory neurons receive positive connections J_IE from excitatory neurons, and excitatory neurons receive negative connections J_EI from inhibitory neurons. In addition, excitatory and inhibitory neurons receive external constant input H^E_i and H^I_k, respectively.

In the first scenario (externally driven oscillation, Figure 1A), a sinusoidal external input h_i(t) = h₀ sin(ωt + φ_i) for the i-th neuron was used to generate oscillatory states in a network, where h₀, ω, and upvarphi_i represent the amplitude, angular speed, and phase of sinusoidal signals, respectively. Note that h₀ and ω are common to all neurons, but φ_i can be different for individual neurons. The explicit expression of an input signal allows one to produce different network oscillations systematically. For the binary neuron model, the total input to the i-th neuron is written as,

$\begin{array}{cc}{u}_i(t)={\displaystyle \sum _j{J}_{ij}}{x}_j(t)+h_i(t).& (10)\end{array}$

where J_ij represents a connection weight from the j-th neuron to the i-th neuron. The neuronal state x_i(t) was then updated following the transition rate w in Equation 7. Note that model neurons are identical, whether they are excitatory or inhibitory.

For the Izhikevich model in the first scenario, we considered a population of excitatory neurons. Although it has been demonstrated that a network of excitatory neurons can synchronize, a network of Izhikevich neurons that were connected in this particular way cannot produce an intrinsic oscillation (Mirollo and Strogatz, 1990; Hansel et al., 1995). This allows us to investigate the relationship between the IG measures and an externally driven oscillation in a more realistic setting. The total input to an Izhikevich neuron i is given by,

$\begin{array}{cc}{I}_i^E(t)={\displaystyle \sum _j{J}_{ij}^{EE}}{s}_j^E(t)+h_i(t),& (11)\end{array}$

where J^EE represent positive weights between excitatory neurons and s^E_j = δ(t − t^f_j) is the delta function representing the existence of a spike emitted from an excitatory neuron j at time t^f_j. The neuronal state was then updated by Equation 9 and the associated reset dynamics. In the numerical simulation, we used a small bin width of 1 ms so that it would contain no more than one spike.

Figures 2A–C show example spike trains and multi-unit activity of binary model neurons driven by external sinusoidal inputs of 1, 6, and 100 Hz oscillations, respectively. Izhikevich neurons also exhibited very similar activity (data not shown). It can be clearly seen that neural activity is entrained to external input. Using these two models, we investigated how the IG measures are affected by externally driven oscillations.

FIGURE 2

Figure 2. Average firing probability and raster plot of representative oscillatory activity. (A) Average firing probability of 1000 binary neurons (top panel) and a raster plot of 100 randomly sampled neurons (bottom panel) over 1000 ms under the influence of an external sinusoidal input of 1 Hz (slow oscillation) are shown. (B) Average firing probability of 1000 binary neurons (top panel) and a raster plot of 100 randomly sampled neurons (bottom panel) for an external sinusoidal input of 6 Hz (theta oscillation) are shown. (C) Average firing probability of 1000 binary neurons (top panel) and a raster plot of 100 randomly sampled neurons (bottom panel) for an external sinusoidal input of 100 Hz (ripple oscillation) are shown. (D) 1000 ms of average firing probability of 1250 Izhikevich neurons (top panel) and a raster plot (bottom panel) with approximately a 6-Hz oscillation are shown. In the top panel, spikes from an excitatory neuron and an inhibitory neuron are represented by a black dot and a gray dot, respectively. (E) 1000 ms of average firing probability of 1250 Izhikevich neurons (top panel) and a raster plot (bottom panel) with approximately a 40-Hz oscillation are shown.

In the second scenario (internally induced oscillation, Figure 1B), interaction between excitatory and inhibitory neuron pools generates an oscillation. For the binary neuron model, the total input to the i-th excitatory neuron and the k-th inhibitory neuron are written as,

$\begin{array}{cc}\begin{array}{l}{u}_i^E(t)={\displaystyle \sum _j{J}_{ij}^{EE}{x}_j^E(t)}+{\displaystyle \sum _j{J}_{ij}^{EI}}{x}_j^E(t)+H_i^E,\\ u_k^I(t)={\displaystyle \sum _j{J}_{kj}^{II}{x}_j^I(t)}+{\displaystyle \sum _j{J}_{kj}^{IE}{x}_j^I(t)}+H_k^I,\end{array}& (12)\end{array}$

where J^II, J^IE, and J^EI represent negative weights between inhibitory neurons, positive weights from excitatory neurons to inhibitory neurons, and negative weights from inhibitory neurons to excitatory neurons, respectively. The excitatory and inhibitory neurons receive constant external inputs H^E_i and H^I_i, and maintain sustained oscillatory activity. The neuronal state was updated following the transition rate w in Equation 7.

For the Izhikevich model in the second scenario, a similar relationship exists for the total inputs for the i-th excitatory neuron and the k-th inhibitory neuron,

$\begin{array}{cc}\begin{array}{l}{I}_i^E(t)={\displaystyle \sum _j{J}_{ij}^{EE}{s}_j^E(t)}+{\displaystyle \sum _j{J}_{ij}^{EI}}{s}_j^E(t)+H_i^E,\\ I_k^I(t)={\displaystyle \sum _j{J}_{kj}^{II}}{s}_j^I(t)+{\displaystyle \sum _j{J}_{kj}^{IE}}{s}_j^I(t)+H_k^I.\end{array}& (13)\end{array}$

The neuronal state was then updated following Equation 9 and the associated reset dynamics. Figures 2D,E provide examples of spike trains and multi-unit activity of Izhikevich neurons that exhibited ~6 and 40 Hz oscillations, respectively. Binary neurons also exhibited a very similar activity (data not shown). Neural activity was synchronized, but the degree of entrainment was weaker than the externally driven mechanisms. Using these two models, we investigated how the IG measures were influenced by the internally induced oscillation.

Results

Externally Driven Oscillation

We investigated the relationship between the IG measures, θ^{(4, N)}_i and θ^{(4, N)}_ij, and the connection weights, (J_ij + J_ji), using a network of 1000 binary neurons and 1000 Izhikevich neurons. We focused on the IG measures with 4th-order LLM because they have been shown to estimate connection weights (Nie and Tatsuno, 2012) and external inputs (unpublished data) within a 10% error under an equilibrium assumption. In the simulation, we kept the amplitude of external input at a value such that the overall network firing probability is relatively low (p_xi ~ 0.1). Connection weights were set to the order of 1/N to prevent saturation of neuronal activity. For a binary neuron model, we used J_ij = 1/N + ε_ij where ε_ij is a random variable from a normal distribution N(0, 1/N) with a mean of 0 and the standard deviation of 1/$\sqrt{N}$. For the Izhikevich model, we restricted the simulations to a pool of only excitatory neurons to ensure that no internally induced oscillation occurred. The connection weights were assigned as J^EE_Ij = 1/N + ε′_ij where ε′_ij is a random variable following uniform distribution U(0, 1/N) within the interval of [0, 1/N]. θ ^{(4, N)}_i and θ^{(4, N)}_ij were calculated by 10⁶ updates of the network. With the time resolution of 1 ms, the simulation corresponds to ~15 min of recordings. To obtain the mean and variances of the IG measures, we performed 100 independent simulations. Error bars in the figure represent the standard error of mean (SEM).

We investigated the oscillation frequencies that have often been observed in the brain; slow oscillation (~1 Hz), theta oscillation (6–10 Hz), and ripple oscillation (100–200 Hz). The left column of (Figures 3A,D,G) shows the results for the slow oscillation. The multi-unit activity of the binary neurons exhibits a slow oscillation of the frequency of external input (Figure 3A) and the neurons were entrained to this frequency (Figure 2A). The spiking activity of Izhikevich neurons also showed almost identical activity (data not shown). To investigate how θ^{(4, N)}_i and θ^{(4, N)}_ij are related to the change of connection weights, we systematically modified the sum of connection weights between two neurons (1 and 2) from −9/N to 9/N. Due to the randomness of the connectivity, focusing the neurons (1 and 2) did not affect the generality. We found that θ ^{(4, N)}₁₂ was linearly related to the sum of the connection weights, and that the values of θ^{(4, N)}₁₂ for both the binary and Izhikevich models were very close (Figure 3D, black line for a binary model and gray line for the Izhikevich model). On the other hand, θ^{(4, N)}₁ and θ^{(4, N)}₂ were independent from the change of synaptic weights (Figure 3G). These results are consistent with the previous findings under the equilibrium assumption; showing that IG measures can also provide useful insights in conditions where the network oscillates. The middle and right columns of Figure 3 show the results for theta oscillations (Figures 3B,E,H) and ripple oscillations (Figures 3C,F,I), respectively. We found that the relationship between the IG measures and connection weights was robust against a different frequency of external inputs. This confirmed that the IG measures can also provide useful information for externally driven theta and ripple oscillations.

FIGURE 3

Figure 3. Relationship between the IG measures and the sum of connection weights for an externally driven oscillation. (A) Average firing probability of 1000 binary neurons with a 1-Hz oscillatory modulation is shown. (B) Average firing probability of 1000 binary neurons with a 6-Hz oscillatory modulation is shown. (C) Average firing probability of 1000 binary neurons with a 100-Hz oscillatory modulation is shown. (D) Relationship between the pairwise IG measure (θ^{(4, 1000)}₁₂) and the sum of connection weights (J₁₂ + J₂₁) under a 1-Hz oscillation. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. (E) Relationship between the pairwise IG measure (θ ^{(4, 1000)}₁₂) and the sum of connection weights (J₁₂ + J₂₁) under a 6-Hz oscillation. (F) Relationship between the pairwise IG measure (θ^{(4, 1000)}₁₂) and the sum of connection weights (J₁₂ + J₂₁) under a 100-Hz oscillation. (G) Relationship between the single IG measure (θ^{(4, 1000)}₁) and the sum of connection weights (J₁₂ + J₂₁) under a 1-Hz oscillation. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. (H) Relationship between the single IG measure (θ^{(4, 1000)}₁) and the sum of connection weights (J₁₂ + J₂₁) under a 6-Hz oscillation. (I) Relationship between the single IG measure (θ^{(4, 1000)}₁) and the sum of connection weights (J₁₂ + J₂₁) under a 100-Hz oscillation.

To further investigate if the robust property of the IG measures for slow, theta, and ripple oscillations holds true for other frequencies, we varied the frequency over 1–200 Hz, the range that can be typically observed in the brain. We set (J₁₂ + J₂₁) = 2J. Figure 4 shows that θ^{(4, N)}₁₂ and θ^{(4, N)}₁ did not depend on oscillation frequencies (Figures 4A,C for a binary model, and Figures 4B,D for Izhikevich model). The results confirmed that the IG measures would be useful for neural data analysis when the brain exhibits a variety of oscillations depending on cognitive demands and the sleep stages.

FIGURE 4

Figure 4. Dependency of the IG measures for an oscillation frequency for an external drive oscillation. The values of the single-IG measure θ^{(4, 1000)}₁ and the pairwise-IG measure θ^{(4, 1000)}₁₂ were calculated for different oscillatory frequencies (1, 5, 6, 10, 20, 50, 100, and 200 Hz). Parameters were set as J₁₂ + J₂₁ = 2J, h₀ = 0.05, and φ_i = 0. (A) The relationship between θ^{(4, 1000)}₁₂ and the oscillation frequency by binary model neurons. (B) The relationship between θ^{(4, 1000)}₁₂ and the oscillation frequency by Izhikevich neurons. (C) The relationship between θ^{(4, 1000)}₁ and the oscillation frequency by binary model neurons. (D) The relationship between θ^{(4, 1000)}₁ and the oscillation frequency by Izhikevich neurons.

The previous analyses (Figures 3, 4) were performed under the zero relative phase difference between two neurons i and j, namely δφ_ij = |φ_i − φ_j| = 0. This corresponds to the synchronous neural firings that were depicted in Figures 2A–C. Neurons can, however, exhibit phase differences. For instance, sequential neural activity was observed in the natural and anesthetized brain states (Lee and Wilson, 2002; Euston et al., 2007; Luczak et al., 2007; Bermudez Contreras et al., 2013). Therefore, we calculated IG measures with phase differences. Figure 5 shows the results of the 6-Hz simulations in which the phase difference between sinusoidal inputs to the neurons 1 and 2 was set to π/6 (Figures 5A,C) and π/2 (Figures 5B,D). The rest of the neuron pairs have random phases in the range of [0, 2π]. Figures 5A,B show that θ^{(4, 1000)}₁₂ is linearly related to the sum of synaptic weights, suggesting that the relationship observed in zero phase difference condition also holds for the non-zero phase difference condition. Similarly, Figures 5C,D show that θ^{(4, 1000)}₁ does not depend on the connection weights, even when neurons fire with phase differences. By comparing these results with Figures 3E,H where there was no phase difference, we also found that phase difference produced the shift of the actual values of IG measures. This suggests that if the phase relationship drastically changes between the two recording epochs, the values of the IG measures cannot be directly comparable. However, if their difference is not large or if phase difference can be estimated beforehand, we could use the information for adjusting the IG values. We also confirmed that these relationships held true for slow (1 Hz) and ripple (100 Hz) frequencies (data not shown).

FIGURE 5

Figure 5. Relationship between the IG measures and the sum of connection weights for non-zero phase differences. An external oscillation mechanism was used, and the oscillation frequency was set to 6 Hz. (A) Relationship between the pairwise IG measure (θ^{(4, 1000)}₁₂) and the sum of connection weights (J₁₂ + J₂₁) for the phase difference of δφ₁₂ = π/6. Black and gray curves represent the simulation by binary neurons and Izhikevich neurons, respectively. (B) Relationship between the pairwise IG measure θ^{(4, 1000)}₁₂) and the sum of connection weights, (J₁₂ + J₂₁) for the phase difference of δφ₁₂ = π /2. (C) Relationship between the single-IG measure (θ^{(4, 1000)}₁) and the sum of connection weights (J₁₂ + J₂₁) for the phase difference of δφ₁₂ = π/6. (D) Relationship between the single-IG measure (θ^{(4, 1000)}₁) and the sum of connection weights (J₁₂ + J₂₁) for the phase difference of δ φ₁₂ = π /2.

So far we have focused on the relationship between the IG measures and the connection weights. Another important parameter is the magnitude of sinusoidal input h₀. Therefore, we have analyzed how θ^{(4, 1000)}₁ and θ^{(4, 1000)}₁₂ are related to h₀. Figure 6 shows the result when the external sinusoidal input has a frequency of 6 Hz (theta oscillation). We found that θ^{(4, 1000)}₁₂ was nearly independent from the change of h₀ (Figure 6A), but θ^{(4, 1000)}₁ was almost linearly related to it (Figure 6B). We also confirmed that almost identical relationship holds true for other frequencies such as slow oscillation and ripple oscillation if there is no phase difference. For non-zero phases between neurons, we observed that the IG values were shifted, like the case for the IG values and connection weights, but that the same linear and independent relationship in Figure 6 was sustained. The results under the oscillatory condition are consistent with the previous findings under the equilibrium condition; θ^{(4, N)}_i was linearly related to the magnitude of the constant input and that θ^{(4, N)}_ij was almost independent from it (Nie and Tatsuno, 2012). The investigation here provides further evidence that θ^{(4, N)}_i is useful for the estimation of the magnitude of external input.

FIGURE 6

Figure 6. Relationship between the IG measures and the amplitude of an external sinusoidal input for an externally driven oscillation. Oscillation frequency was set to 6 Hz and phase difference was δφ₁₂ = 0. (A) Relationship between the pairwise-IG measure, θ ^{(4, 1000)}₁₂, and the amplitude of a sinusoidal input, h₀. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. (B) Relationship between the single-IG measure, θ ^{(4, 1000)}₁, and the amplitude of a sinusoidal input, h₀.

In summary, we investigated how the IG measures were influenced by an externally driven oscillation. Using a simple binary neuron model, and a more realistic Izhikevich model, we found that θ ^{(4, N)}_ij had a linear relationship with the sum of the connection weights, and that it was almost independent from the magnitude of a sinusoidal input. In contrast, θ^{(4, N)}_i was almost independent from the connection weights, but was linearly related to the magnitude of the sinusoidal input. These properties were not affected by the frequency of the oscillations or the relative phase differences between neurons.

Internally Induced Oscillation

As another mechanism for generating an oscillatory network behavior, we also investigated the interactions between excitatory and inhibitory neuron pools. We analyzed the network structure in Figure 1B by simple binary model neurons and Izhikevich neurons. Unlike the first oscillation mechanism, where an oscillation frequency and phase differences could be explicitly controlled, it was not easy to generate an oscillation with desired parameters. However, we were able to generate two examples that were often observed in the brain. Figures 7A,B show multi-unit activity corresponding to theta frequency (~8 Hz) and gamma frequency (~40 Hz), respectively. The same examples with a raster plot were also depicted in Figures 2D,E. To avoid saturation in neural activity, we have set the connection weights to the order of 1/N. For a theta oscillation, we set the connection as J^EE_ij = J₁ · ϵ′_ij, J^IE_ij = 5J₁ · ϵ′_ij, J^II_ij = −J₂ · ϵ′_ij and J^EI_ij = −J₂ · ϵ′_ij where J₁ = 1/N_e, J₂ = 1/N_i, and ϵ′_ij is a random variable following a uniform distribution U(0, 1) within the interval [0,1]. For a gamma oscillation, we used J^EE_ij = J₁ · ϵ′_ij, J^IE_ij = 6J₁ · ϵ′_ij, J^II_ij = −J₂ · ϵ′_ij and J^EI_ij = −2J₂ · ϵ′_ij. The stronger J^IE was necessary to induce oscillation (Adini et al., 1997). The external constant inputs to excitatory and inhibitory neurons were set as H^E_i = 0.05 and H^I_i = 0.02, respectively. The simulation of 10⁶ update was performed with N_e = 1000 excitatory neurons and N_i = 250 inhibitory neurons. The mean and variance was estimated using 100 independent simulations.

FIGURE 7

Figure 7. Relationship between the IG measures and the sum of connection weights for an internally induced oscillation. Both connections, J₁₂ and J₂₁, were positive for the range of (J₁₂ + J₂₁) ≥ 0 and they were negative for (J₁₂ + J₂₁) < 0. (A) Average firing probability of 1250 Izhikevich neurons with approximately a 6-Hz oscillatory oscillation is shown. (B) Average firing probability of 1250 Izhikevich neurons with approximately a 40-Hz oscillatory oscillation is shown. (C) Relationship between the pairwise-IG measure (θ ^{(4, 1000)}₁₂) and the sum of connection weights (J₁₂ + J₂₁) under a 6-Hz oscillation. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. (D) Relationship between the pairwise-IG measure (θ^{(4, 1000)}₁₂) and the sum of connection weights (J₁₂ + J₂₁) under a 6-Hz oscillation. (E) Relationship between the single-IG measure (θ^{(4, 1000)}₁) and the sum of connection weights (J₁₂ + J₂₁) under a 6-Hz oscillation. (F) Relationship between the single-IG measure (θ ^{(4, 1000)}₁) and the sum of connection weights (J₁₂ + J₂₁) under a 40-Hz oscillation.

To investigate the relationship between the IG measures and the sum of connection weights, without losing the generality, we modified (J₁₂ + J₂₁) between the neurons (1 and 2) in the range of [−9J, 9J]. Firstly, we focused on the connections within the excitatory neuron population and within the inhibitory neuron population. In other words, both connections, J₁₂ and J₂₁, were positive for the range of (J₁₂ + J₂₁) ≥ 0 and both were negative for (J₁₂ + J₂₁) < 0. Figures 7C,E show the relationship between θ^{(4, 1250)}₁₂ and θ^{(4, 1250)}₁, and the sum of connection strengths (J₁₂ + J₂₁) under the theta oscillation. The results clearly show that θ^{(4, 1250)}₁₂ is linearly related to the sum of the connection weights and that θ ^{(4, 1250)}₁ was independent from the modulation of the connection weights. Furthermore, the dependency of the IG measures on the connection weights was continuous in both positive and negative ranges. This suggests that IG measures can be applicable to both positive and negative connections. Figures 7D,F show results for gamma oscillation. Similar results were obtained for both θ^{(4, 1250)}₁₂ and θ^{(4, 1250)}₁.

Secondly, we investigated the interaction between excitatory and inhibitory neurons. Namely, we selected the neuron 1 from the excitatory neuron pool and the neuron 2 from the inhibitory neuron pool. The sum of connection weights was modified from −9J to 9J. Figures 8A,B are the same with Figures 7A,B, showing the multi-unit activity for theta and gamma oscillation, respectively. Figures 8C,E show the relationship between the IG measures and the sum of the connection weights under the theta oscillation. Similarly, Figures 8D,F are for gamma oscillation. The results show that the linear dependency of θ^{(4, 1250)}₁₂ on the sum of the connection weights holds true for an excitatory and inhibitory neuron pair. We also found that θ^{(4, 1250)}₁ had almost no relationship with the sum of connection weight. For the relationship between the IG measures and the magnitude of constant input H^E_i and H^I_i, we confirmed that θ ^{(4, 1250)}₁ was linearly related to their magnitude, but θ^{(4, 1250)}₁₂ was independent from them (data not shown).

FIGURE 8

Figure 8. Relationship between the IG measures and the sum of connection weights for an internally induced oscillation. Neuron 1 was selected from the excitatory neuron pool and Neuron 2 was selected from the inhibitory neuron pool. In other words, one of the connections in J₁₂ + J₂₁ was positive and the other was negative. (A) Average firing probability of 1250 Izhikevich neurons with approximately a 6-Hz oscillatory oscillation is shown. (B) Average firing probability of 1250 Izhikevich neurons with approximately a 40-Hz oscillatory oscillation is shown. (C) Relationship between the pairwise-IG measure (θ^{(4, 1000)}₁₂) and the sum of connection weights (J₁₂ + J₂₁) under a 6-Hz oscillation. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. (D) Relationship between the pairwise-IG measure (θ ^{(4, 1000)}₁₂) and the sum of connection weights (J₁₂ + J₂₁) under a 6-Hz oscillation. (E) Relationship between the single-IG measure (θ^{(4, 1000)}₁) and the sum of connection weights (J₁₂ + J₂₁) under a 6-Hz oscillation. (F) Relationship between the single-IG measure (θ^{(4, 1000)}₁) and the sum of connection weights (J₁₂ + J₂₁) under a 40-Hz oscillation.

In summary, for internally generated oscillations, we demonstrated that the relationship between the IG measures and the connection weights that were found under equilibrium assumption also held true.

Discussion

Previous studies have shown that the IG measures provided useful information about network structures (Tatsuno and Okada, 2004; Tatsuno et al., 2009; Nie and Tatsuno, 2012). Specifically, the single-IG measure θ^{(4, N)}_i was related to the magnitude of external constant input, and the pairwise-IG measure θ^{(4, N)}_ij was related to the sum of the connection strengths. Although these studies were conducted under the equilibrium assumption, the real neural signals exhibit various oscillations depending on cognitive demand of the task or the state of the brain. Therefore, we studied the relationship between the IG measures and the neural network parameters under oscillatory network states.

We have considered two general oscillation mechanisms; one was the oscillation driven by external input, and the other was the oscillation induced internally due to interactions between excitatory and inhibitory neuron pools. Numerical simulation was performed by the network of a simple binary neuron model and the Izhikevich neuron model. The former model was used so as to compare the results with that of previous studies, and the latter was used to investigate the relationship with more realistic model neurons.

For the external oscillation, our investigation showed that θ^{(4, N)}_ij was linearly related to the sum of the connection strengths, and that θ^{(4, N)}_i was independent from it over a wide range of frequency from 1 to 200 Hz. We also showed that the relationship holds true when there are phase differences between neurons. In addition, we demonstrated that θ^{(4, N)}_i was almost linearly related to the magnitude of sinusoidal input, but that θ^{(4, N)}_ij was almost independent from it. For the internally induced oscillation, we have also confirmed that θ^{(4, N)}_ij was linearly related to the sum of the connection strengths, and that θ^{(4, N)}_i was independent from it. We have also shown that the same relationship holds true for any neuron pairs (within excitatory population, within inhibitory population, and across excitatory and inhibitory populations).

In summary, this study and previous studies have demonstrated that the IG measure provides useful information for analyzing neural circuits; not only for the equilibrium condition, but also for the oscillatory condition. The single-IG measure is useful for estimating the relative strength of external inputs. In addition, the single-IG measure is better than using the change in firing rate because the firing rate can be modulated both by the change in synaptic coupling strength and the magnitude of external inputs. Studies show that the appropriately selected single-IG measure is capable of estimating the external inputs with relatively small influence from synaptic interactions. Similarly, the pairwise-IG measure can provide more direct information about the synaptic interactions between neurons than other correlation measures (Amari, 2009). It has been also shown that the pairwise-IG measure is statistically independent from the change in firing rate and that it provides pure neural interactions (Amari, 2001; Nakahara and Amari, 2002). Together with the findings in this study, the pairwise-IG measure is a very useful measure to study direct neural interactions between neurons.

This study suggests that the actual values of the IG measures depend on the mechanisms of oscillation. For an externally driven oscillation, θ^{(4, N)}₁₂ ~ 0.2 was obtained for (J₁₂ + J₂₁) ~ 1/N. For an internally induced oscillation, the same connection strength produced θ^{(4, N)}₁₂ ~ 0.002. Within the same oscillation mechanism, the selection of model neurons (binary model or Izhikevich model), or a small difference in network parameters such as phase differences also produced a difference in the actual value of the IG measures. Nonetheless, as long as the network is in one of the oscillation mechanisms, and the phase difference is kept the same, the IG measures can provide useful insights into network structures regardless of the oscillation frequencies.

In the study of memory consolidation, one of the key questions is to understand how the changes in synaptic connections are related to learning and memory formation. Evidence suggests that neural activity during slow-wave sleep plays an important role in learning (Diekelmann and Born, 2010). Specifically, there is increasing evidence supporting the hypothesis that replay of neural activity during subsequent sleep is positively correlated with memory formation (Pavlides and Winson, 1989; Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Lee and Wilson, 2002; Euston et al., 2007; Girardeau et al., 2009; Peyrache et al., 2009b; Ego-Stengel and Wilson, 2010). However, the direct information about synaptic change is not available from multi-unit recordings of a freely behaving animal because spikes and local field potentials are the two main observables. In this study, we showed that θ^{(4, N)}_ij was linearly related to the sum of the connection weights, and that θ ^{(4, N)}_i was linearly related to the magnitude of external inputs, even under the oscillatory conditions. We have also verified these relationships not only with a simple binary model neuron, but also with a more realistic spiking model neuron. This finding would allow us to analyze neural activity during slow-wave sleep before and after the task; θ^{(4, N)}_ij would be a good measure for the change of connection weight, and θ^{(4, N)}_i for the magnitude of background input that would be influenced by local field potentials. By comparing the relative change of θ^{(4, N)}_ij between slow-wave sleeps before and after the task, and the strength of memory replay/improvement of behavior performance, the IG measure may provide a way to estimate the relationship between the synaptic modification and memory formation without having direct access to information of synaptic change.

As a related approach to the IG method, the maximum entropy (MaxEnt) has attracted much attention recently (Schneidman et al., 2006; Tang et al., 2008; Tyler et al., 2012). The philosophy of the MaxEnt approach is not to assume anything other than what we know from the data. For example, if firing rate and pairwise correlation are the only information we have, the distribution with maximum entropy is given as the Boltzmann distribution,

$\begin{array}{cc}{P}^{(2)}\left\{x\right\}=\frac{1}{Z}\exp \left({\displaystyle \sum _i{h}_i^{\prime }{x}_i}+{\displaystyle \sum _{i<j}{J}_{ij}^{\prime }}{x}_i{x}_j\right),& (14)\end{array}$

where h′_i is a bias term for the neuron i, J′_ij is the symmetric coupling strength between neurons i and j, and the partition function Z is given by,

$\begin{array}{cc}Z={\displaystyle \sum _{\left\{x\right\}}\exp }\left({\displaystyle \sum _i{h}_i^{\prime }}{x}_i+{\displaystyle \sum _{i<j}{J}_{ij}^{\prime }}{x}_i{x}_j\right).& (15)\end{array}$

We see that the MaxEnt is equivalent to the IG with the 2nd-order LLM,

$\begin{array}{cc}\log p_{x_1{x}_2\ast \cdots \ast }={\displaystyle \sum _i{\theta }_i^{(2,N)}}{x}_i+{\displaystyle \sum _{i<j}{\theta }_{ij}^{(2,N)}}{x}_i{x}_j-\psi {(\theta )}^{(2,N)},& (16)\end{array}$

where the relationship between the parameters are given as,

$\begin{array}{cc}{\theta }_i^{(2,N)}=h_i^{\prime },{\theta }_{ij}^{(2,N)}=J^{\prime }{}_{ij},\psi {(\theta )}^{(\text{2},\text{N})}=\log Z.& (17)\end{array}$

As was discussed in Tatsuno and Okada (2004) and in Tatsuno et al. (2009), it is possible to relate these IG measures, θ^{(2, N)}_i and θ^{(2, N)}_ij, to the network structure even for a network with asymmetric connections (Equation 5). However, under the influence of correlated inputs, we have also shown that the relationship in Equation 5 broke down, and that it was necessary to use the IG measures with the higher-order LLM such as the 4th-order (Equation 6) (Nie and Tatsuno, 2012). In other words, it was necessary to take into account neural activity of two additional neurons to estimate the direct neural interaction between neuron i and j. In summary, we see that the MaxEnt approach and the IG method are closely related. In addition, we also see that the MaxEnt can be considered a part of the IG method that provides a more general analysis framework for the space of the probability distributions.

In this study, we used the synchronous neural activity for estimating the direct neural interaction as the form of (J_ij + J_ji). However, in the real learning processes such as sequential learning, it is possible that synaptic modification occurs differently for each direction; e.g., J_ij increases, while J_ji decreases. The proposed method is not able to estimate the directed synaptic change. As one possible remedy for this difficulty, calculation of the pairwise-IG measure using the time-lagged spiking activity between neurons was suggested (Tatsuno and Okada, 2004; Nie and Tatsuno, 2012). Another limitation of the present study is not including the effect of delay; e.g., axonal conduction delay or synaptic transmission delay. It is possible that these delays dramatically change the firing patterns as well as increase a variety of coexisting patterns (Izhikevich, 2006). Little is known about the relationship between the IG measures and direct neural interactions with conduction delay. In addition, it has not been clear how IG measures with more neuronal interactions such as triplewise-IG measures θ^{(k, N)}_ijk or quadruple-IG measures θ^{(k, N)}_ijkl behave under oscillatory conditions. It would be interesting to extend the current study to include more neuronal interactions.

Despite these limitations, the IG method is one of the most promising statistical tools for spike train analysis (Amari, 2001; Nakahara and Amari, 2002). Its direct relationship with the network parameters would provide useful information for the estimation of structural changes (Tatsuno and Okada, 2004; Tatsuno et al., 2009; Nie and Tatsuno, 2012). We hope that an advancement of novel analysis methods including IG will lead to a break-through finding in neuroscience.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We are grateful to Karim Ali for his support with the numerical simulations and to Amanda Mauthe-Kaddoura for proofreading the manuscript. This work was supported by Alberta Innovates Technology Futures (SCH001), the National Science Foundation (CRCNS-1010172), and Alberta Innovates Health Solutions (Polaris Award).

References

Abeles, M., and Gerstein, G. L. (1988). Detecting spatiotemporal firing patterns among simultaneously recorded single neurons. J. Neurophysiol. 60, 909–924.

Pubmed Abstract | Pubmed Full Text

Adini, Y., Sagi, D., and Tsodyks, M. (1997). Excitatory-inhibitory network in the visual cortex: psychophysical evidence. Proc. Natl. Acad. Sci. U.S.A. 94, 10426–10431. doi: 10.1073/pnas.94.19.10426

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Aertsen, A. M., Gerstein, G. L., Habib, M. K., and Palm, G. (1989). Dynamics of neuronal firing correlation: modulation of “effective connectivity.” J. Neurophysiol. 61, 900–917.

Pubmed Abstract | Pubmed Full Text

Amari, S. (2001). Information geometry on hierarchy of probability distributions. IEEE Trans. Inf. Theory 47, 1701–1711. doi: 10.1109/18.930911

CrossRef Full Text

Amari, S. (2009). Measure of correlation orthogonal to change in firing rate. Neural Comput. 21, 960–972. doi: 10.1162/neco.2008.03-08-729

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Amari, S., and Nagaoka, H. (2000). Methods of Information Geometry. New York, NY: Oxford University Press.

Amari, S., Nakahara, H., Wu, S., and Sakai, Y. (2003). Synchronous firing and higher-order interactions in neuron pool. Neural Comput. 15, 127–142. doi: 10.1162/089976603321043720

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Baker, S. N., Olivier, E., and Lemon, R. N. (1997). Coherent oscillations in monkey motor cortex and hand muscle EMG show task-dependent modulation. J. Physiol. 501(pt 1), 225–241. doi: 10.1111/j.1469-7793.1997.225bo.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bermudez Contreras, E. J., Schjetnan, A. G., Muhammad, A., Bartho, P., McNaughton, B. L., Kolb, B., et al. (2013). Formation and reverberation of sequential neural activity patterns evoked by sensory stimulation are enhanced during cortical desynchronization. Neuron 79, 555–566. doi: 10.1016/j.neuron.2013.06.013

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bland, B. H. (1986). The physiology and pharmacology of hippocampal formation theta rhythms. Prog. Neurobiol. 26, 1–54. doi: 10.1016/0301-0082(86)90019-5

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Brown, E. N., Kass, R. E., and Mitra, P. P. (2004). Multiple neural spike train data analysis: state-of-the-art and future challenges. Nat. Neurosci. 7, 456–461. doi: 10.1038/nn1228

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Buzsaki, G. (2002). Theta oscillations in the hippocampus. Neuron 33, 325–340. doi: 10.1016/S0896-6273(02)00586-X

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Buzsaki, G. (2004). Large-scale recording of neuronal ensembles. Nat. Neurosci. 7, 446–451. doi: 10.1038/nn1233

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Buzsaki, G., Horvath, Z., Urioste, R., Hetke, J., and Wise, K. (1992). High-frequency network oscillation in the hippocampus. Science 256, 1025–1027. doi: 10.1126/science.1589772

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Buzsaki, G., and Wang, X. J. (2012). Mechanisms of gamma oscillations. Annu. Rev. Neurosci. 35, 203–225. doi: 10.1146/annurev-neuro-062111-150444

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Chapin, J. K., Moxon, K. A., Markowitz, R. S., and Nicolelis, M. A. (1999). Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat. Neurosci. 2, 664–670. doi: 10.1038/10223

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Crunelli, V., and Hughes, S. W. (2010). The slow (<1 Hz) rhythm of non-REM sleep: a dialogue between three cardinal oscillators. Nat. Neurosci. 13, 9–17. doi: 10.1038/nn.2445

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Czanner, G., Grun, S., and Iyengar, S. (2005). Theory of the snowflake plot and its relations to higher-order analysis methods. Neural Comput. 17, 1456–1479. doi: 10.1162/0899766053723041

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Davidson, T. J., Kloosterman, F., and Wilson, M. A. (2009). Hippocampal replay of extended experience. Neuron 63, 497–507. doi: 10.1016/j.neuron.2009.07.027

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Diekelmann, S., and Born, J. (2010). The memory function of sleep. Nat. Rev. Neurosci. 11, 114–126. doi: 10.1038/nrn2762

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Dragoi, G., Carpi, D., Recce, M., Csicsvari, J., and Buzsaki, G. (1999). Interactions between hippocampus and medial septum during sharp waves and theta oscillation in the behaving rat. J. Neurosci. 19, 6191–6199.

Pubmed Abstract | Pubmed Full Text

Dragoi, G., and Tonegawa, S. (2011). Preplay of future place cell sequences by hippocampal cellular assemblies. Nature 469, 397–401. doi: 10.1038/nature09633

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Dragoi, G., and Tonegawa, S. (2013). Distinct preplay of multiple novel spatial experiences in the rat. Proc. Natl. Acad. Sci. U.S.A. 110, 9100–9105. doi: 10.1073/pnas.1306031110

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ego-Stengel, V., and Wilson, M. A. (2010). Disruption of ripple-associated hippocampal activity during rest impairs spatial learning in the rat. Hippocampus 20, 1–10. doi: 10.1002/hipo.20707

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Eleuteri, A., Tagliaferri, R., and Milano, L. (2005). A novel information geometric approach to variable selection in MLP networks. Neural Netw. 18, 1309–1318. doi: 10.1016/j.neunet.2005.01.008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Euston, D. R., Tatsuno, M., and McNaughton, B. L. (2007). Fast-forward playback of recent memory sequences in prefrontal cortex during sleep. Science 318, 1147–1150. doi: 10.1126/science.1148979

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fellous, J. M., Tiesinga, P. H., Thomas, P. J., and Sejnowski, T. J. (2004). Discovering spike patterns in neuronal responses. J. Neurosci. 24, 2989–3001. doi: 10.1523/JNEUROSCI.4649-03.2004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gerstein, G. L., and Perkel, D. H. (1969). Simultaneously recorded trains of action potentials: analysis and functional interpretation. Science 164, 828–830. doi: 10.1126/science.164.3881.828

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gilestro, G. F., Tononi, G., and Cirelli, C. (2009). Widespread changes in synaptic markers as a function of sleep and wakefulness in Drosophila. Science 324, 109–112. doi: 10.1126/science.1166673

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ginzburg, I. I., and Sompolinsky, H. (1994). Theory of correlations in stochastic neural networks. Phys. Rev. E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics 50, 3171–3191.

Pubmed Abstract | Pubmed Full Text

Girardeau, G., Benchenane, K., Wiener, S. I., Buzsaki, G., and Zugaro, M. B. (2009). Selective suppression of hippocampal ripples impairs spatial memory. Nat. Neurosci. 12, 1222–1223. doi: 10.1038/nn.2384

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gray, C. M., Konig, P., Engel, A. K., and Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338, 334–337. doi: 10.1038/338334a0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Grun, S., Diesmann, M., and Aertsen, A. (2002a). Unitary events in multiple single-neuron spiking activity: I. Detection and significance. Neural Comput. 14, 43–80. doi: 10.1162/089976602753284455

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Grun, S., Diesmann, M., and Aertsen, A. (2002b). Unitary events in multiple single-neuron spiking activity: II. Nonstationary data. Neural Comput. 14, 81–119. doi: 10.1162/089976602753284464

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hansel, D., Mato, G., and Meunier, C. (1995). Synchrony in excitatory neural networks. Neural Comput. 7, 307–337. doi: 10.1162/neco.1995.7.2.307

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hoffman, K. L., and McNaughton, B. L. (2002). Coordinated reactivation of distributed memory traces in primate neocortex. Science 297, 2070–2073. doi: 10.1126/science.1073538

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Huber, R., Ghilardi, M. F., Massimini, M., and Tononi, G. (2004). Local sleep and learning. Nature 430, 78–81. doi: 10.1038/nature02663

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ikeda, K. (2005). Information geometry of interspike intervals in spiking neurons. Neural Comput. 17, 2719–2735. doi: 10.1162/089976605774320593

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ince, R. A., Senatore, R., Arabzadeh, E., Montani, F., Diamond, M. E., and Panzeri, S. (2010). Information-theoretic methods for studying population codes. Neural Netw. 23, 713–727. doi: 10.1016/j.neunet.2010.05.008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569–1572. doi: 10.1109/TNN.2003.820440

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Izhikevich, E. M. (2006). Polychronization: computation with spikes. Neural Comput. 18, 245–282. doi: 10.1162/089976606775093882

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kudrimoti, H. S., Barnes, C. A., and McNaughton, B. L. (1999). Reactivation of hippocampal cell assemblies: effects of behavioral state, experience, and EEG dynamics. J. Neurosci. 19, 4090–4101.

Pubmed Abstract | Pubmed Full Text

Laubach, M., Wessberg, J., and Nicolelis, M. A. (2000). Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task. Nature 405, 567–571. doi: 10.1038/35014604

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lee, A. K., and Wilson, M. A. (2002). Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 36, 1183–1194. doi: 10.1016/S0896-6273(02)01096-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lopes-Dos-Santos, V., Conde-Ocazionez, S., Nicolelis, M. A., Ribeiro, S. T., and Tort, A. B. (2011). Neuronal assembly detection and cell membership specification by principal component analysis. PLoS ONE 6:e20996. doi: 10.1371/journal.pone.0020996

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lovette, I. J., Arbogast, B. S., Curry, R. L., Zink, R. M., Botero, C. A., Sullivan, J. P., et al. (2011). Phylogenetic relationships of the mockingbirds and thrashers (Aves: Mimidae). Mol. Phylogenet. Evol. 63, 219–229. doi: 10.1016/j.ympev.2011.07.009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Luczak, A., Bartho, P., Marguet, S. L., Buzsaki, G., and Harris, K. D. (2007). Sequential structure of neocortical spontaneous activity in vivo. Proc. Natl. Acad. Sci. U.S.A. 104, 347–352. doi: 10.1073/pnas.0605643104

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meltzer, J. A., Zaveri, H. P., Goncharova, I. I., Distasio, M. M., Papademetris, X., Spencer, S. S., et al. (2008). Effects of working memory load on oscillatory power in human intracranial EEG. Cereb. Cortex 18, 1843–1855. doi: 10.1093/cercor/bhm213

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mirollo, R. E., and Strogatz, S. H. (1990). Synchronization of pulse-coupled biological oscillators. SIAM J. Appl. Math. 50, 1645–1662. doi: 10.1137/0150098

CrossRef Full Text

Miura, K., Okada, M., and Amari, S. (2006). Estimating spiking irregularities under changing environments. Neural Comput. 18, 2359–2386. doi: 10.1162/neco.2006.18.10.2359

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nakahara, H., and Amari, S. (2002). Information-geometric measure for neural spikes. Neural Comput. 14, 2269–2316. doi: 10.1162/08997660260293238

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nakahara, H., Amari, S., and Richmond, B. J. (2006). A comparison of descriptive models of a single spike train by information-geometric measure. Neural Comput. 18, 545–568. doi: 10.1162/neco.2006.18.3.545

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nie, Y., and Tatsuno, M. (2012). Information-geometric measures for estimation of connection weight under correlated inputs. Neural Comput. 24, 3213–3245. doi: 10.1162/NECO_a_00367

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Panzeri, S., and Schultz, S. R. (2001). A unified approach to the study of temporal, correlational, and rate coding. Neural Comput. 13, 1311–1349. doi: 10.1162/08997660152002870

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pavlides, C., and Winson, J. (1989). Influences of hippocampal place cell firing in the awake state on the activity of these cells during subsequent sleep episodes. J. Neurosci. 9, 2907–2918.

Pubmed Abstract | Pubmed Full Text

Peyrache, A., Benchenane, K., Khamassi, M., Wiener, S. I., and Battaglia, F. P. (2009a). Principal component analysis of ensemble recordings reveals cell assemblies at high temporal resolution. J. Comput. Neurosci. 29, 309–325. doi: 10.1007/s10827-009-0154-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S. I., and Battaglia, F. P. (2009b). Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat. Neurosci. 12, 919–926. doi: 10.1038/nn.2337

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Raghavachari, S., Kahana, M. J., Rizzuto, D. S., Caplan, J. B., Kirschen, M. P., Bourgeois, B., et al. (2001). Gating of human theta oscillations by a working memory task. J. Neurosci. 21, 3175–3183. doi: 10.1152/jn.00409.2005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schneidman, E., Berry, M. J. 2nd., Segev, R., and Bialek, W. (2006). Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440, 1007–1012. doi: 10.1038/nature04701

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shimazaki, H., and Shinomoto, S. (2007). A method for selecting the bin size of a time histogram. Neural Comput. 19, 1503–1527. doi: 10.1162/neco.2007.19.6.1503

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shimokawa, T., and Shinomoto, S. (2009). Estimating instantaneous irregularity of neuronal firing. Neural Comput. 21, 1931–1951. doi: 10.1162/neco.2009.08-08-841

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Singer, W., and Gray, C. M. (1995). Visual feature integration and the temporal correlation hypothesis. Annu. Rev. Neurosci. 18, 555–586. doi: 10.1146/annurev.ne.18.030195.003011

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Steriade, M., Nunez, A., and Amzica, F. (1993). A novel slow (< 1 Hz) oscillation of neocortical neurons in vivo: depolarizing and hyperpolarizing components. J. Neurosci. 13, 3252–3265.

Pubmed Abstract | Pubmed Full Text

Stickgold, R. (2005). Sleep-dependent memory consolidation. Nature 437, 1272–1278. doi: 10.1038/nature04286

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tang, A., Jackson, D., Hobbs, J., Chen, W., Smith, J. L., Patel, H., et al. (2008). A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. J. Neurosci. 28, 505–518. doi: 10.1523/JNEUROSCI.3359-07.2008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tatsuno, M., Fellous, J. M., and Amari, S. I. (2009). Information-geometric measures as robust estimators of connection strengths and external inputs. Neural Comput. 21, 2309–2335. doi: 10.1162/neco.2009.04-08-748

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tatsuno, M., Lipa, P., and McNaughton, B. L. (2006). Methodological considerations on the use of template matching to study long-lasting memory trace replay. J. Neurosci. 26, 10727–10742. doi: 10.1523/JNEUROSCI.3317-06.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tatsuno, M., and Okada, M. (2004). Investigation of possible neural architectures underlying information-geometric measures. Neural Comput. 16, 737–765. doi: 10.1162/089976604322860686

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tyler, A. L., Mahoney, J. M., Richard, G. R., Holmes, G. L., Lenck-Santini, P. P., and Scott, R. C. (2012). Functional network changes in hippocampal CA1 after status epilepticus predict spatial memory deficits in rats. J. Neurosci. 32, 11365–11376. doi: 10.1523/JNEUROSCI.1516-12.2012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Urban, N. N., Henze, D. A., and Barrionuevo, G. (2001). Revisiting the role of the hippocampal mossy fiber synapse. Hippocampus 11, 408–417. doi: 10.1002/hipo.1055

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Vanderwolf, C. H. (1969). Hippocampal electrical activity and voluntary movement in the rat. Electroencephalogr. Clin. Neurophysiol. 26, 407–418. doi: 10.1016/0013-4694(69)90092-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wilson, M. A., and McNaughton, B. L. (1993). Dynamics of the hippocampal ensemble code for space. Science 261, 1055–1058. doi: 10.1126/science.8351520

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wilson, M. A., and McNaughton, B. L. (1994). Reactivation of hippocampal ensemble memories during sleep. Science 265, 676–679. doi: 10.1126/science.8036517

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zhang, K., Ginzburg, I., McNaughton, B. L., and Sejnowski, T. J. (1998). Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. J. Neurophysiol. 79, 1017–1044.

Pubmed Abstract | Pubmed Full Text