Sahar Behpour
David J. Field
Mark V. Albert- 1Department of Information Science, University of North Texas, Denton, TX, United States
- 2Department of Psychology, Cornell University, Ithaca, NY, United States
- 3Department of Computer Science and Engineering, University of North Texas, Denton, TX, United States
- 4Department of Biomedical Engineering, University of North Texas, Denton, TX, United States
Correlated, spontaneous neural activity is known to play a necessary role in visual development, but the higher-order statistical structure of these coherent, amorphous patterns has only begun to emerge in the past decade. Several computational studies have demonstrated how this endogenous activity can be used to train a developing visual system. Models that generate spontaneous activity analogous to retinal waves have shown that these waves can serve as stimuli for efficient coding models of V1. This general strategy in development has one clear advantage: The same learning algorithm can be used both before and after eye-opening. This same insight can be applied to understanding LGN/V1 spontaneous activity. Although lateral geniculate nucleus (LGN) activity has been less discussed in the literature than retinal waves, here we argue that the waves found in the LGN have a number of properties that fill the role of a training pattern. We make the case that the role of “innate learning” with spontaneous activity is not only possible, but likely in later stages of visual development, and worth pursuing further using an efficient coding paradigm.
What Is the Purpose of Spontaneous Activity?
Highly organized patterns of spontaneous activity are observed across the developing sensory systems of a wide variety of species. These activity patterns are robust to perturbations (Turrigiano, 1999; Blankenship and Feller, 2010). Several studies suggest the patterns themselves carry information that guides nervous system development in areas as diverse as the retina, cochlea, spinal cord, cerebellum, hippocampus, and neocortex (Blankenship and Feller, 2010; Dehorter et al., 2012) and altering patterns of activity leads to poorer network refinement (Huberman et al., 2008). Therefore, these correlated patterns are crucial for neural circuit maturation (Kirkby et al., 2013). A collection of past and current discoveries about spontaneous activity in both developing and mature sensory systems is provided by (Imaizumi et al., 2018). Although there remains debate regarding the precise role the activity plays, it is necessary to establish critical neural circuits before eye-opening (Shatz and Stryker, 1988; Wong and Ghosh, 2002; Uesaka et al., 2006; Kim et al., 2009).
One fascinating form of spontaneous activity is found in the “retinal waves” of the developing retina. Retina is a layer of tissue in the back of the eye of most vertebrates and some mollusks that senses light and sends images to the brain. Retinal waves are spontaneous bursts of action potentials that occur roughly once per minute and propagate in a wave-like fashion across the developing retina. They have been demonstrated in a majority of common vertebrates, including chicks (Sernagor et al., 2001), turtles (Evelyne Sernagor and Grzywacz, 1999), cats (Meister et al., 1991), rats (Galli and Maffei, 1988; Maffei and Galli-Resta, 1990), ferrets (Feller et al., 1996a), primates (Warland et al., 2006), and mice (Maccione et al., 2014). This spontaneous retinal wave phenomenon that occurs before the eyes first open was found and described in the developing mammalian over two decades ago; for complete references, (see Zhou, 2001).
Using a multielectrode array (Meister et al., 1991; Wong et al., 1993) or fluorescence imaging of calcium indicators (Wong et al., 1995; Feller et al., 1996b), the spontaneous activity in the developing retina has been monitored across large areas. The role of cholinergic circuits in driving this activity bursts of action potentials was indicated by recording the spontaneous bursts in the neural rabbit retina and blocked by nicotinic acetylcholine receptor analogies. About 10years later, Vivo recordings from fetal rat pups showed that neighboring ganglion cells fire periodic correlated action potential bursts that propagate like waves across the ganglion cell layer of ferret retinas, according to multielectrode array experiments. In addition, using calcium imaging sensitive dyes, videos of these endogenous patterns were recorded and demonstrated that correlated calcium transients propagate across the ganglion cell layer of the retina in a wave-like manner (Feller et al., 1996a).
While calcium imaging experiments of the intact retina showed the qualitative nature of the retinal waves, the characteristics of these waves, such as wave shape, speed, size, and duration using physiological details like dendritic field size and measured cell spacing, have been explored by neural models of various studies (Burgi and Grzywacz, 1994; Butts et al., 1999; Godfrey and Swindale, 2007). The spontaneous activity of the retinal waves is far from random, with more structure than one finds in simple models of correlated noise. They spread as amorphous bursting neural activity patterns with relatively slow dynamics. For instance, the average time between retinal waves in a given location is a median of 115s in the ferret retina. Typically, the patterns form well-bounded shapes that expand, distort, and dissipate over time. In the ferret retina, the average time between retinal waves in a given location is a median of 115s (Feller et al., 1996a); however, a ganglion cell’s burst of activity lasts roughly 1s, and the average wave propagation speed is 177μm/s (Feller et al., 1997). Therefore, the spontaneous activity in retinal waves is so robust that the possibility of noise contamination from instruments is not a significant concern.
Spontaneous activity patterns propagate through many parts of the developing nervous system to shape the wiring of emerging circuits. Prior to eye-opening, these activity waves originate in the retina and propagate to the superior colliculus (Ackman et al., 2012) and through the LGN of the thalamus, a multilayered structure that receives input from both eyes to build a representation of the contralateral visual hemifield, to the V1. LGN waves can correlate between the eyes. However, retinal waves cannot. Training based on the retinal waves produces a single eye vision, monocular model, versus LGN, which provides a binocular model. Three distinct higher ganglion cells, magno, parvo, and konio, compress and transfer the generated information by cone photoreceptors to processing centers in the retina. These cells, organized in different layers of LGN and the V1, have distinct functional and structural characteristics. In this paper, for the simplicity of the model in addressing critical elements for the disparity-selective vision, we do not distinguish between the three cellular subcortical pathways.
Spontaneous neural activity is correlated with brain functionality and its anatomical structures (Tozzi et al., 2016). This activity plays a critical role in the functional organization of the nervous system’s standard architecture (Cole et al., 2014), is generated by brain circuits, and provides a window into their dynamic operations (Omer et al., 2019). A number of computational models have shown how different forms of spontaneous activity can produce a topographic map of neural responses as found in the primary visual cortex (V1) [(see Erwin et al., 1995; Miikkulainen et al., 2006; Goodhill, 2018) for reviews]. Previous studies demonstrate that ongoing activity in V1 is not noise, though it may be a form of information processing (Arieli et al., 1995, 1996; Tsodyks et al., 1999; Kenet et al., 2003; Gutnisky et al., 2017). Following this, multiple studies performed on human and other animals discovered that this activity has a major role in shaping perception and behavior (Wagner et al., 1998; Ress and Heeger, 2003; Dehaene and Changeux, 2005).
The Efficient Coding Paradigm
There is a wide variation in the nature of the spontaneous activity offered by different models. For example, von der Malsburg’s initial model (Malsburg, 1973) used simplistic bar-like stimuli, Linsker’s model (Linsker, 1986) begins with uncorrelated noise, Miller’s model (Miller, 1994) uses radially symmetric functions which represent the amount of correlation between units. Many such models rely on constraints in the neural connectivity (e.g., prespecified or adaptive lateral connectivity and dendritic field sizes) for receptive field formation with less emphasis on the precise nature of the activity. This can be contrasted with models of spontaneous neural activity that are primarily aimed at recreating the larger-scale statistical properties of the generated patterns (Butts et al., 1999; Godfrey and Swindale, 2007; Godfrey et al., 2009; Richter and Gjorgjieva, 2017). To find an excellent review of theoretical models of neural development, (see Goodhill, 2018). Here, we discuss this can be contrasted with computational approaches to map formation which use natural images as the input stimuli (Barrow et al., 1996; Shouval et al., 1996; Weber, 2001; Hyvärinen et al., 2009), In these models, cortical maps form due to the statistical properties of presented natural images.
There is a subset of developmental models which use natural images in addition to spontaneous activity; they apply the same learning method both before and after visual experience (Bednar and Miikkulainen, 1998; Burger and Lang, 1999; Bednar, 2002). These V1 cortical models have produced 2D maps which vary characteristically with orientation, ocular dominance, direction selectivity, spatial frequency, or a combination of these dimensions. The primary goal of these models was to produce a 2D cortical map and receptive fields in a physiologically plausible way. As with many models, the necessary constraints can either be in the network implementation or the statistics of the input; many of these models lie along this continuum, and consequently, there is a wide variety of network implementations. One simplification to this modeling approach is to focus at the computational, rather than algorithmic, level (Marr, 2010) evaluate a simple coding objective of V1, rather than an intricate network implementation, on natural and spontaneous inputs. Although the previous neural modeling techniques may produce a similar code, the particular learning approaches are less clear than simple statements of statistical independence or sparse coding.
We argue that an efficient coding approach would focus efforts on the sufficient statistical properties of spontaneous activity and natural scenes and less about the particular details of implementation. In practice, efficient coding as a computational approach explains how the brain represents and interprets information from the outside world and maximizes a particular metric of efficiency (Barlow, 1961). The efficient coding hypothesis proposes that early sensory processing aims to reduce redundancy (Barlow, 1961; Field, 1987). Sparse coding and independent component analysis (ICA) objectives are driven from the efficient coding hypothesis, each with a specific statistical method. Sparse coding is a type of unsupervised learning method of over-complete bases for efficiently representing the data. Sparse coding aims to represent information with as few simultaneously active neurons as possible in a large population.
The input vector is defined as a linear combination of basis vectors, which is the set of basis vectors. Independent codes can also be produced using ICA (Comon, 1994). Mathematically, ICA creates components through linear combinations of features with maximally and statistically independent responses under a specific set of assumptions. In other words, ICA transforms the observed data as a random vector into a vector of maximally independent components S measured by some function using the following linear transformation;
where is a hidden component, and is the observed data representation.
Unsupervised learning methods, such as ICA, with the goal of maximizing statistical independence, are successful in creating meaningful components. ICA was originally developed to address the blind source separation problem (Jutten and Herault, 1991) and has been particularly useful for problems with linear mixings, such as the classic cocktail party problem (Cherry, 1953; Haykin and Chen, 2005; Bronkhorst, 2015).
This efficient coding strategy has been successfully applied to various modalities, natural/non-natural images, and sounds (Urs et al., 2021). A self-contained Jupyter notebook of the efficient coding strategy is provided for the ease of experiments in computational neuroscience and computer science. However, in this study, our objective is to show that using the same strategy on LGN/V1 spontaneous activity patterns, it is possible to derive filters of those images similar to the Gabor filters. Therefore, we can use the extracted filters for future training task-oriented deep learning models. We follow a five-step strategy to apply efficient coding principles to V1/LGN spontaneous activity;
1. Data Collection: Collecting samples of spontaneously generated activity pattern images corresponding to the simulation of active ganglion cells in the retinal model, LGN, and V1.
2. Patch Extraction: Randomly extracting patches of these images, display them, and then reshape them accordingly.
3. Efficient Encoding: Applying encoding algorithms, that is, ICA using our self-contained Jupyter notebook (Urs et al., 2021).
4. Displaying Efficient coding Filters: Reducing the process used in patch extraction and flattening to display the resulting filters found through ICA.
5. Comparing to Experimental Neural Receptive Fields: Drawing comparisons between experimentally recorded receptive fields and the resulting filters from spontaneous neural activity.
The efficient coding approach, using sparse coding/ICA, has been successfully applied to activity patterns resembling retinal waves (Albert et al., 2008). This particular study investigated whether this activity could serve as a training pattern and produce the same type of V1 filters that are produced from natural scenes. The spontaneous activity model used was an abstraction of previous, more physiological models (Butts et al., 1999; Godfrey and Swindale, 2007; Godfrey et al., 2009). The three parameters of the model were directly related to known parameters in retinal wave physiology (fraction of recruitable amacrine cells, dendritic field sizes, and threshold number of neighboring amacrine cells needed to fire). The results showed this abstract, physiological spontaneous activity is capable of forming V1 receptive fields in the same way as an efficient coding of natural scenes. This model of development is not only straightforward conceptually, but elegantly simple given a neural implementation. This fits well with the recommendation for modeling approaches to V1 development in the (Swindale, 1996) review, “Remove as much detail as possible from your model, without reducing its descriptive scope.” Such simple innate learning strategies could be used as a bridge between molecular guidance cues forming crude receptive fields and adaptation using visual input after eye-opening. Most importantly, it is one more way animals that require a functioning visual system early in life may be able to adequately prepare the visual code in time for use.
It has been shown that spontaneous activity generation and propagation correspond to the percolation networks used in applied math and statistical physics (Hesse and Gross, 2014). As a typical example, the percolation theory involves networks with t=1, where t is the number of active points within distance r, often r=1. When “p,” a random fraction of the points on the grid square array of points, approaches a value known as the percolation threshold, “pc,” the pattern of activity is known to be fractal – the image statistics appear similar at all scales (Li et al., 2021). What we discuss in this paper is a system, an abstraction of hypothesized LGN and V1 developmental activity patterns, that can generate fractal patterns. To generate such large waves, we are approaching the percolation threshold. The whole reason for getting patterns at such a large scale is that there are advantages to getting self-similar patterns. That way, we can have systems trained at multiple scales. The method used to generate the patterns is based on a convergence of two general models: One is a percolation network that resembled retinal waves both visually and in how the patterns are generated (e.g., amacrine cells spontaneously firing and being recruitable) and the additional element is simply communication across eye layers (due to feedback to the LGN). We consider this model the minimal approach to create a binocular spontaneous activity pattern capable of producing disparity selective cells while still resembling retinal wave generation. The model can be set such that those spontaneous activity patterns have disparity distributions like in natural visual experience – which would have a beneficial effect in early learning, as evidenced by the disparity profiles of filters created through efficient coding. Similarly, other properties of the produced filters should approximately match natural scene statistics. This concept constrains the statistical properties of LGN waves to be ones in which they produce neural receptive fields on efficient coding.
There are significant principle advancements between this current position paper and the previous study by (Albert et al., 2008). First, this study proposed the role of V1/LGN spontaneous activity patterns, whereas, in the previous study, the focus has only been on V1. Second, the main goal of the previous paper was to show that an efficient coding algorithm can be applied for both natural and spontaneous activity inputs; however, in this perspective, this efficient coding strategy is introduced as a practical tool to create efficient codes of LGN/V1 activity patterns for further training roles.
Could the Measured Endogenous Activity Be a Potential Training Pattern For Lgn-V1 Neurons?
In the case of retinal waves, where the vast majority of experimentation on endogenous activity has focused, the extent of training is limited. One immediate objection is that the waves are not binocular. Days after birth, primates have an adult-like proportion of disparity selective cells (Chino et al., 1997) and retinal waves are not correlated between the eyes to promote such selectivity. Also, the retinal waves of the ON and OFF-center cells are more independent at later stages (Wong et al., 1993), but the responses of these cell classes are intimately related in V1 neurons. Despite the popularity of retinal spontaneous activity, such concerns have turned many researchers away from considering the possibility of an innate learning role for spontaneous activity.
Here, we draw attention to a form of spontaneous activity that does not suffer from the many deficiencies of retinal waves as a training pattern – activity originating in the LGN. Although much less is known about this activity, we believe that it may provide additional insights into the early development of neural properties in the visual cortex. The endogenous activity in the LGN significantly overlaps in time with V1 activation (Wong et al., 1993), thus being able to directly influence V1. The activity is also correlated between eye-specific layers, as well as ON and OFF-center cell layers (Weliky and Katz, 1999). Finally, the evidence from electrode arrays shows that the activity appears as a wavefront moving across these layers. By all accounts, LGN endogenous activity appears better suited as a potential source of innate training patterns.
There are potential confounds with such an assessment of LGN activity. First, it is more difficult to experimentally characterize LGN waves through electrode penetrations in comparison with the characterization of retinal waves using calcium imaging. Also, proper LGN activity requires V1 feedback; there is no correlated spontaneous activity between eye-specific layers in the LGN without feedback (Weliky and Katz, 1999). The alleged source of training patterns for V1 may more correctly be called dynamic LGN/V1 activity, making a label for the source of such activity more complicated than a simplistic, one-area label as in retinal waves. Ultimately, from a computational perspective, the specific source of the activity is not as critical as the statistical properties of the resulting activity. What is important for this paradigm is the existence of a functional separation between an innate training pattern (LGN/V1 waves) and efficient coding technique (e.g., sparse coding/ICA). These alternate roles need not be in physically separate locations. For example, this paradigm does not preclude the possibility of neural models with activity and adaptation occurring within V1 (Grabska-Barwinska and von der Malsburg, 2008), although here we discuss the combination of the LGN and V1 activity.
Our previous work pointed to this LGN/V1 activity as a potential source of visual training patterns. Endogenous activity is how the system could, in theory, bridge between molecular guidance cues and real-world experience for development. Beyond this, there are a number of parallels that can point toward a more complete interpretation of this activity. Activity model parameters have been directly related to similar parameters in physiological models. Receptive fields derived from these activity patterns are also qualitatively similar to experimental data. In addition to monocular and binocular properties of newly developed cells, the temporal component of this activity can be included in a more thorough analysis. A caveat to these models is that they are not meant to match adult coding performance – both in modeling and in animal development. For most animals, the goal in early visual development is to have a system that is not equivalent, but closer to the adult visual code than what can be produced by molecular guidance cues alone.
The nature of the explanation mentioned here is computational, although many other levels of understanding the role of this activity fit this framework (Freeman, 1989). On the level of implementation, endogenous activity is necessary for strengthening and guiding synaptic connections through synaptic pruning (Shatz and Stryker, 1988), axon branching (Hosokai et al., 2006), and dendritic patterning (Wong and Ghosh, 2002). This interpretation is not only compatible with a computational understanding, but a necessary part of a neural implementation. A number of neural models have implemented development of a topography and individual receptive fields. A subset of these neural models may be compatible with the computational, statistical interpretation presented here. The specific developmental algorithm may vary, but the goal of applying an efficient coding strategy (e.g., sparse coding/ICA) both before and after experience may be universal.
What Is the Role of Innate Learning with Spontaneous Activity in the Visual System?
Efficient coding/computational approaches have added to our understanding of adult V1 by providing an additional functional role. Neural response of simple cells in V1 can be understood by their neural connections in cortex (implementation, “how”), their visual receptive fields and generalized neural models (algorithm, “what”), and also by the goal of the neural processing (computation/efficient coding, “why”). Previous work (Olshausen and Field, 1996; Bell and Sejnowski, 1997) has shown that the unique, Gabor-like receptive fields of V1 simple cells can result from an efficient coding of natural scenes. The underlying hypothesis, that V1 receptive fields can be understood as the result of a general computational strategy, has led to an explosion of techniques trying to both capture the statistical properties of natural scenes and relate those properties to V1 cell responses.
In the same way that efficient coding applied to natural scenes has led to a deeper understanding of the role of V1, similar computational strategies can lead to a deeper understanding of spontaneous activity. The same efficient coding strategy learning both on endogenous activity and external, natural stimuli may help explain many of the statistical properties of spontaneous activity deeper than retinal waves. This has the potential to build bridges between development and experience-based learning in areas that are often treated separately. Much of the criticism of this approach leveled at retinal waves does not hold when considering combined LGN/V1 activity, permitting further work in this domain to continue. Current efficient coding “innate learning” models show promise and point toward further computational, theoretical, and experimental work.
The training role of LGN/V1 activity is demonstrated through early experiments and theoretical results, as is shown in Figure 1. According to (Albert et al., 2008), a simple model of spontaneous activity can produce receptive fields with adult-like receptive field properties. This has been done using the efficient coding strategy, mainly ICA applied on images of natural spontaneous activity patterns. In Figure 1A, an example of such a spontaneous activity pattern is given. After applying ICA on this set of data, the resulting filters and receptive fields are represented in Figure 1B. A visual comparison shows that Gabor filters, shown in Figure 1C, can properly fit these generated receptive fields by ICA. A histogram of orientation bandwidth, with a red line which indicates the average adult V1 primate orientation bandwidth, is used for the comparison and is shown in Figure 1D. Correlated LGN spontaneous activity across eye layers from (Weliky and Katz, 1999), reprinted with permission; electrodes 1–4 are in the contralateral eye layer while 5–8 are in the ipsilateral eye layer, indicated in Figure 1E,F. Ultimately, the goal is to answer the same question that is naturally posed when one is first exposed to the highly structured, spontaneous patterns, “What is the purpose of this activity?”
Figure 1. Early experimental and theoretical results suggesting the training role of LGN/V1 activity. (A–D) Efficient coding results demonstrating that a simple model of spontaneous activity can produce receptive fields with adult-like receptive field properties – details in (Albert et al., 2008). (A) An example spontaneous activity pattern, (B) the resulting receptive field after applying ICA, (C) Gabors fit to these receptive fields, (D) a histogram of orientation bandwidth with the red line indicating the average adult V1 primate orientation bandwidth for comparison. (E,F) Early experimental results indicating correlated LGN spontaneous activity across eye layers from (Weliky and Katz, 1999), reprinted with permission; electrodes 1–4 are in the contralateral eye layer, while −8 are in the ipsilateral eye layer.
Summary
Spontaneous neural activity plays a significant role in visual development. Although this activity in the retina is widely discussed, it is less investigated for LGN. In this study, we argue that LGN spontaneous activity may provide additional insights into the early development of neural properties in the visual cortex. We claim that the waves found in the LGN have several properties that fill a training pattern’s role. Thus, the role of “innate learning” with spontaneous activity in later stages of visual development is possible and worth pursuing further using an efficient coding paradigm.
Data Availability Statement
The original contributions presented in the study are included in the article/supplementary material, and further inquiries can be directed to the corresponding author.
Author Contributions
All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Ackman, J. B., Burbridge, T. J., and Crair, M. C. (2012). Retinal waves coordinate patterned activity throughout the developing visual system. Nature 490, 219–225.
Albert, M. V., Schnabel, A., and Field, D. J. (2008). Innate visual learning through spontaneous activity patterns. PLoS Comput. Biol. 4:e1000137. doi: 10.1371/journal.pcbi.1000137
Arieli, A., Shoham, D., Hildesheim, R., and Grinvald, A. (1995). Coherent spatiotemporal patterns of ongoing activity revealed by real-time optical imaging coupled with single-unit recording in the cat visual cortex. J. Neurophysiol. 73, 2072–2093. doi: 10.1152/jn.1995.73.5.2072
Arieli, A., Sterkin, A., Grinvald, A., and Aertsen, A. (1996). Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science 273, 1868–1871. doi: 10.1126/science.273.5283.1868
Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. Sens. Commun. 1, 217–234.
Barrow, H. G., Bray, A. J., and Budd, J. M. L. (1996). A self-organizing model of “color blob” formation. Neural Comput. 8, 1427–1448. doi: 10.1162/neco.1996.8.7.1427
Bednar, J. A. (2002). Learning to see: Genetic and environmental influences on visual development. The University of Texas at Austin.
Bednar, J. A., and Miikkulainen, R. (1998). “Pattern-generator-driven development in self-organizing models,” in Computational Neuroscience: Trends in Research. ed. J. M. Bower (Springer, Boston, MA: Springer), 317–323.
Bell, A. J., and Sejnowski, T. J. (1997). The “independent components” of natural scenes are edge filters. Vis. Res. 37, 3327–3338. doi: 10.1016/S0042-6989(97)00121-1
Blankenship, A. G., and Feller, M. B. (2010). Mechanisms underlying spontaneous patterned activity in developing neural circuits. Nature reviews. Neuroscience 11, 18–29. doi: 10.1038/nrn2759
Bronkhorst, A. W. (2015). The cocktail-party problem revisited: early processing and selection of multi-talker speech. Atten. Percept. Psychophys. 77, 1465–1487. doi: 10.3758/s13414-015-0882-9
Burger, T., and Lang, E. W. (1999). An incremental Hebbian learning model of the primary visual cortex with lateral plasticity and real input patterns. Zeitschrift Fur Naturforschung. C. J. Biosci. 54, 128–140.
Burgi, P. Y., and Grzywacz, N. M. (1994). Model for the pharmacological basis of spontaneous synchronous activity in developing retinas. J. Neurosci. 14, 7426–7439.
Butts, D. A., Feller, M. B., Shatz, C. J., and Rokhsar, D. S. (1999). Retinal waves are governed by collective network properties. J. Neurosci. 19, 3580–3593.
Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two ears. J. Acoust. Soc. Am. 25, 975–979. doi: 10.1121/1.1907229
Chino, Y. M., Smith, E. L. 3rd, Hatta, S., and Cheng, H. (1997). Postnatal development of binocular disparity sensitivity in neurons of the primate visual cortex. J. Neurosci. 17, 296–307.
Cole, M. W., Bassett, D. S., Power, J. D., Braver, T. S., and Petersen, S. E. (2014). Intrinsic and task-evoked network architectures of the human brain. Neuron 83, 238–251. doi: 10.1016/j.neuron.2014.05.014
Comon, P. (1994). Independent component analysis, A new concept? Signal Process. 36, 287–314. doi: 10.1016/0165-1684(94)90029-9
Dehaene, S., and Changeux, J.-P. (2005). Ongoing spontaneous activity controls access to consciousness: a neuronal model for inattentional blindness. PLoS Biol. 3:e141. doi: 10.1371/journal.pbio.0030141
Dehorter, N., Vinay, L., Hammond, C., and Ben‐Ari, Y. (2012). Timing of developmental sequences in different brain structures: physiological and pathological implications. Eur. J. Neurosci. 35, 1846–1856.
Erwin, E., Obermayer, K., and Schulten, K. (1995). Models of orientation and ocular dominance columns in the visual cortex: a critical comparison. Neural Comput. 7, 425–468. doi: 10.1162/neco.1995.7.3.425
Feller, M. B., Butts, D. A., Aaron, H. L., Rokhsar, D. S., and Shatz, C. J. (1997). Dynamic processes shape spatiotemporal properties of retinal waves. Neuron 19, 293–306. doi: 10.1016/s0896-6273(00)80940-x
Feller, S. E., Pastor, R. W., Rojnuckarin, A., Bogusz, S., and Brooks, B. R. (1996b). Effect of electrostatic force truncation on interfacial and transport properties of water. J. Phys. Chem. 100, 17011–17020. doi: 10.1021/jp9614658
Feller, M. B., Wellis, D. P., Stellwagen, D., Werblin, F. S., and Shatz, C. J. (1996a). Requirement for cholinergic synaptic transmission in the propagation of spontaneous retinal waves. Science 272, 1182–1187. doi: 10.1126/science.272.5265.1182
Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4, 2379–2394. doi: 10.1364/josaa.4.002379
Freeman, N. H. (1989). A computational approach to picture production and consumption is needed right here. Behav. Brain Sci. 12, 82–84. doi: 10.1017/S0140525X00024377
Galli, L., and Maffei, L. (1988). Spontaneous impulse activity of rat retinal ganglion cells in prenatal life. Science 242, 90–91. doi: 10.1126/science.3175637
Godfrey, K. B., Eglen, S. J., and Swindale, N. V. (2009). A multi-component model of the developing retinocollicular pathway incorporating axonal and synaptic growth. PLoS Comput. Biol. 5:e1000600. doi: 10.1371/journal.pcbi.1000600
Godfrey, K. B., and Swindale, N. V. (2007). Retinal wave behavior through activity-dependent refractory periods. PLoS Comput. Biol. 3:e245. doi: 10.1371/journal.pcbi.0030245
Goodhill, G. J. (2018). Theoretical models of neural development. iScience 8, 183–199. doi: 10.1016/j.isci.2018.09.017
Grabska-Barwinska, A., and von der Malsburg, C. (2008). Establishment of a scaffold for orientation maps in primary visual cortex of higher mammals. J. Neurosci. 28, 249–257. doi: 10.1523/jneurosci.5514-06.2008
Gutnisky, D. A., Beaman, C. B., Lew, S. E., and Dragoi, V. (2017). Spontaneous fluctuations in visual cortical responses influence population coding accuracy. Cereb. Cortex 27, 1409–1427. doi: 10.1093/cercor/bhv312
Haykin, S., and Chen, Z. (2005). The cocktail party problem. Neural Comput. 17, 1875–1902. doi: 10.1162/0899766054322964
Hesse, J., and Gross, T. (2014). Self-organized criticality as a fundamental property of neural systems. Front. Syst. Neurosci. 8:166. doi: 10.3389/fnsys.2014.00166
Hosokai, T., Kinoshita, K., Ohkubo, T., Maekawa, A., Uesaka, M., Zhidkov, A., et al. (2006). Observation of strong correlation between quasimonoenergetic electron beam generation by laser Wakefield and laser guiding inside a preplasma cavity. Phys. Rev. E 73:036407. doi: 10.1103/PhysRevE.73.036407
Huberman, A. D., Feller, M. B., and Chapman, B. (2008). Mechanisms underlying development of visual maps and receptive fields. Annu. Rev. Neurosci. 31, 479–509. doi: 10.1146/annurev.neuro.31.060407.125533
Hyvärinen, A., Hurri, J., and Hoyer, P. O. (2009). Natural Image Statistics: A Probabilistic Approach to Early Computational Vision. Berlin: Springer Science & Business Media.
Imaizumi, K., Ruthazer, E. S., MacLean, J. N., and Lee, C. C. (2018). Editorial: spontaneous activity in sensory systems. Front. Neural Circuits 12:27. doi: 10.3389/fncir.2018.00027
Jutten, C., and Herault, J. (1991). Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture. Signal Process. 24, 1–10. doi: 10.1016/0165-1684(91)90079-X
Kenet, T., Bibitchkov, D., Tsodyks, M., Grinvald, A., and Arieli, A. (2003). Spontaneously emerging cortical representations of visual attributes. Nature 425, 954–956. doi: 10.1038/nature02078
Kim, K. S., Zhao, Y., Jang, H., Lee, S. Y., Kim, J. M., Kim, K. S., et al. (2009). Large-scale pattern growth of graphene films for stretchable transparent electrodes. Nature 457, 706–710. doi: 10.1038/nature07719
Kirkby, L. A., Sack, G. S., Firl, A., and Feller, M. B. (2013). A role for correlated spontaneous activity in the assembly of neural circuits. Neuron 80, 1129–1144. doi: 10.1016/j.neuron.2013.10.030
Li, M., Liu, R.-R., Lü, L., Hu, M.-B., Xu, S., and Zhang, Y.-C. (2021). Percolation on complex networks: theory and application. Physics Reports. arXiv.
Linsker, R. (1986). From basic network principles to neural architecture: emergence of orientation columns. Proc. Natl. Acad. Sci. U. S. A. 83, 8779–8783. doi: 10.1073/pnas.83.22.8779
Maccione, A., Hennig, M. H., Gandolfo, M., Muthmann, O., van Coppenhagen, J., Eglen, S. J., et al. (2014). Following the ontogeny of retinal waves: pan-retinal recordings of population dynamics in the neonatal mouse. J. Physiol. 592, 1545–1563. doi: 10.1113/jphysiol.2013.262840
Maffei, L., and Galli-Resta, L. (1990). Correlation in the discharges of neighboring rat retinal ganglion cells during prenatal life. Proc. Natl. Acad. Sci. U. S. A. 87, 2861–2864. doi: 10.1073/pnas.87.7.2861
Malsburg, C. (1973). Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14, 85–100. doi: 10.1007/bf00288907
Marr, D. (2010). Vision: A Computational Investigation Into the Human Representation and Processing of Visual Information. Cambridge, MA: MIT Press.
Meister, M., Wong, R. O., Baylor, D. A., and Shatz, C. J. (1991). Synchronous bursts of action potentials in ganglion cells of the developing mammalian retina. Science 252, 939–943. doi: 10.1126/science.2035024
Miikkulainen, R., Bednar, J. A., Choe, Y., and Sirosh, J. (2006). Computational Maps in the Visual Cortex. New York: Springer Science & Business Media.
Miller, K. D. (1994). A model for the development of simple cell receptive fields and the ordered arrangement of orientation columns through activity-dependent competition between ON- and OFF-center inputs. J. Neurosci. 14, 409–441. doi: 10.1523/jneurosci.14-01-00409.1994
Olshausen, B. A., and Field, D. J. (1996). Natural image statistics and efficient coding. Network 7, 333–339. doi: 10.1088/0954-898X/7/2/014
Omer, D. B., Fekete, T., Ulchin, Y., Hildesheim, R., and Grinvald, A. (2019). Dynamic patterns of spontaneous ongoing activity in the visual cortex of anesthetized and awake monkeys are different. Cereb. Cortex 29, 1291–1304. doi: 10.1093/cercor/bhy099
Ress, D., and Heeger, D. J. (2003). Neuronal correlates of perception in early visual cortex. Nat. Neurosci. 6, 414–420. doi: 10.1038/nn1024
Richter, L. M., and Gjorgjieva, J. (2017). Understanding neural circuit development through theory and models. Curr. Opin. Neurobiol. 46, 39–47. doi: 10.1016/j.conb.2017.07.004
Sernagor, E., Eglen, S. J., and Wong, R. O. (2001). Development of retinal ganglion cell structure and function. Prog. Retin. Eye Res. 20, 139–174. doi: 10.1016/s1350-9462(00)00024-0
Sernagor, E., and Grzywacz, N. M. (1999). Spontaneous activity in developing turtle retinal ganglion cells: pharmacological studies. J. Neurosci. 19, 3874–3887. doi: 10.1523/jneurosci.19-10-03874.1999
Shatz, C. J., and Stryker, M. P. (1988). Prenatal tetrodotoxin infusion blocks segregation of retinogeniculate afferents. Science 242, 87–89. doi: 10.1126/science.3175636
Shouval, H., Intrator, N., and Cooper, L. N. (1996). BCM network develops orientation selectivity and ocular dominance in natural scene environment. 10.21236/ada316968
Swindale, N. V. (1996). The development of topography in the visual cortex: a review of models. Netw. Comput. Neural Syst. 7, 161–247. doi: 10.1088/0954-898X_7_2_002
Tozzi, A., Zare, M., and Benasich, A. A. (2016). New perspectives on spontaneous brain activity: dynamic networks and energy matter. Front. Hum. Neurosci. 10:247. doi: 10.3389/fnhum.2016.00247
Tsodyks, M., Kenet, T., Grinvald, A., and Arieli, A. (1999). Linking spontaneous activity of single cortical neurons and the underlying functional architecture. Science 286, 1943–1946. doi: 10.1126/science.286.5446.1943
Turrigiano, G. G. (1999). Homeostatic plasticity in neuronal networks: the more things change, the more they stay the same. Trends Neurosci. 22, 221–227. doi: 10.1016/s0166-2236(98)01341-1
Uesaka, N., Ruthazer, E. S., and Yamamoto, N. (2006). The role of neural activity in cortical axon branching. Neuroscientist 12, 102–106. doi: 10.1177/1073858405281673
Urs, N., Behpour, S., Georgaras, A., and Albert, M. V., (2021). Unsupervised learning in images and audio to produce neural receptive fields: a survey and accessible notebook. Artif. Intell. Rev. doi: 10.1007/s10462-021-10047-7
Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W., Maril, A., Dale, A. M., et al. (1998). Building memories: remembering and forgetting of verbal experiences as predicted by brain activity [review of building memories: remembering and forgetting of verbal experiences as predicted by brain activity]. Science 281, 1188–1191. doi: 10.1126/science.281.5380.1188
Warland, D. K., Huberman, A. D., and Chalupa, L. M. (2006). Dynamics of spontaneous activity in the fetal macaque retina during development of retinogeniculate pathways. J. Neurosci. 26, 5190–5197. doi: 10.1523/JNEUROSCI.0328-06.2006
Weber, C. (2001). Self-Organization of Orientation Maps, lateral connections, and dynamic receptive fields in the primary visual cortex. Artif. Neural Netw., 1147–1152. doi: 10.1007/3-540-44668-0_160
Weliky, M., and Katz, L. C. (1999). Correlational structure of spontaneous neuronal activity in the developing lateral geniculate nucleus in vivo. Science 285, 599–604. doi: 10.1126/science.285.5427.599
Wong, R. O., Chernjavsky, A., Smith, S. J., and Shatz, C. J. (1995). Early functional neural networks in the developing retina. Nature 374, 716–718. doi: 10.1038/374716a0
Wong, R. O. L., and Ghosh, A. (2002). Activity-dependent regulation of dendritic growth and patterning. Nat. Rev. Neurosci. 3, 803–812. doi: 10.1038/nrn941
Wong, R. O., Meister, M., and Shatz, C. J. (1993). Transient period of correlated bursting activity during development of the mammalian retina. Neuron 11, 923–938. doi: 10.1016/0896-6273(93)90122-8
Keywords: spontaneous activity, self-organization, efficient coding, innate learning, criticality
Citation: Behpour S, Field DJ and Albert MV (2021) On the Role of LGN/V1 Spontaneous Activity as an Innate Learning Pattern for Visual Development. Front. Physiol. 12:695431. doi: 10.3389/fphys.2021.695431
Edited by:
Herbert F. Jelinek, Khalifa University, United Arab EmiratesReviewed by:
Gholamreza Jafari, Shahid Beheshti University, IranJosé M. Medina, Valeo Lighting, Spain
Copyright © 2021 Behpour, Field, and Albert. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Sahar Behpour, c2FoYXIuYmVocG91ckB1bnQuZWR1