Dilara Gostolupce
Belinda P. P. Lay
Etienne J. P. Maes
Mihaela D. Iordanova- Center for Studies in Behavioral Neurobiology, Department of Psychology, Concordia University, Montreal, QC, Canada
Associative learning is often considered to require the physical presence of stimuli in the environment in order for them to be linked. This, however, is not a necessary condition for learning. Indeed, associative relationships can form between events that are never directly paired. That is, associative learning can occur by integrating information across different phases of training. Higher-order conditioning provides evidence for such learning through two deceptively similar designs – sensory preconditioning and second-order conditioning. In this review, we detail the procedures and factors that influence learning in these designs, describe the associative relationships that can be acquired, and argue for the importance of this knowledge in studying brain function.
Introduction
Understanding how stimuli occur relative to other stimuli in our environment is fundamental to making accurate predictions about the future and adapting behaviour accordingly. One way for such learning to ensue is to present stimuli together in time. For example, the painful experience of having been bitten by a dog can result in the development of fear of dogs causing one to avoid places whether dogs can be encountered. In the laboratory, this learning (i.e., first-order conditioning; Pavlov, 1927) is modeled using Pavlovian conditioning, which consists of pairings between a neutral sensory cue (or stimulus) such as a tone with an event of biological significance. While this form of learning accurately captures the formation of many associative relationships, it misses many others. Indeed, one need not directly experience event relationships in order to infer the likelihood of their occurrence in novel situations. To return to the example of the dog bite, one will likely not only avoid dogs (the stimulus directly associated with the aversive event) but also places where dogs frequent (e.g., parks, trails, your next-door neighbour’s yard) even though the bite had not occurred there. Here, the knowledge of where dogs can be encountered is integrated with the knowledge that dogs can cause painful bites. In other words, information acquired across different episodes or time points can be linked, thus offering an opportunity to infer unique event relationships and make novel predictions about the environment. Such integration is an example of the dynamic nature of memories, how memories become linked and how flexible behaviour is orchestrated [Holland, 1990; Gewirtz and Davis, 2000; Blaisdell, 2009; Seitz et al., 2021; for more on discontinuous events see Wallenstein et al. (1998) and Cai et al. (2016)].
Integration of distinct associative memories is elegantly captured in higher-order conditioning (e.g., Pavlov, 1927; Brogden, 1939). This learning consists of two conditioning episodes–one that leads to associative links between two neutral stimuli (i.e., S2→S1 where S2 could be an auditory cue such as a tone and S1 could be a visual cue such as a light) and another that links one of these stimuli (S1) with a biologically significant outcome (an appetitive or aversive unconditioned stimulus [US], i.e., S1→US). Subsequent presentations of S2 reveal its ability to invigorate conditioned responses (CRs) indicative of expectation of the US. This form of learning is termed higher-order because S2 is never directly paired with the US. Rather, it engages conditioned responding by virtue of its pairing with S1 which was directly associated with the US. That is, S2 acquires value through an intermediary. This learning requires integration of the sensory learning phase with the fear conditioning phase and provides a mechanism through which value (be it aversive or appetitive) can propagate across the memory network through higher-order associative links.
Procedures and Factors That Influence Higher-Order Conditioning
There are two classic designs of higher-order conditioning, namely, sensory preconditioning and second-order conditioning. While both types of designs consist of the same learning phases, that is, sensory training and appetitive or aversive conditioning (as outlined above), the order of the phases is reversed. In sensory preconditioning, S2→S1 pairings precede S1→US pairings, whereas in second-order conditioning S1→US pairings precede S2→S1 pairings. Although, the order of the learning phases may seem like a minor difference in experimental design, it is of tremendous importance because it governs what is learned during these distinct forms of higher-order conditioning (see below). Accounts of higher-order learning were originally reported by Pavlov (1927) where cues directly paired with an appetitive or aversive outcome could support the acquisition of secondary conditioned reflexes when paired with novel cues in the absence of the associated outcome (i.e., second-order conditioning). In both humans and animals, Prokofiev and Zeliony (1926) reported that sensory pairings between two cues followed by aversive conditioning of one of those cues led to fear of the other (indirectly paired) sensory cue (i.e., sensory preconditioning). This was subsequently investigated more thoroughly by Brogden (1939), coining the term “sensory preconditioning.” While these forms of higher-order learning have been replicated numerous times across species including drosophila, goldfish, pigeons, mice, rats, rabbits, monkeys, and humans (e.g., Reid, 1952; Rizley and Rescorla, 1972; Rashotte et al., 1977; Pfautz et al., 1978; Rescorla, 1979; Amiro and Bitterman, 1980; Cook and Mineka, 1987; Beauchamp and Gluck, 1988; Gibbs et al., 1991; Müller et al., 2000; Brembs and Heisenberg, 2001; Mead and Stephens, 2003; Tabone and de Belle, 2011; Lee and Livesey, 2012; Busquets-Garcia et al., 2017; Renaux et al., 2017; Craddock et al., 2018; Wong and Pittig, 2022), the precise design parameters employed can easily influence the strength and content of learning. Below, we enumerate a list of design factors that have been reported in the literature along with their associated influence on higher-order conditioning. This information has also been summarized in Table 1.
Table 1. Procedures and factors that influence higher-order conditioning.
Stimulus Type
Various stimuli have been used in higher-order conditioning experiments including colour (Rashotte et al., 1977), shape (Rescorla, 1980a), odour (Holland, 1981), flavour (Holland, 1981, 1983), auditory cues such as tone (Rizley and Rescorla, 1972), white noise (Holland and Ross, 1983), clicker (Ward-Robinson and Hall, 1998) and visual cues such as key light (Rashotte et al., 1977), flashing light (Parkes and Westbrook, 2010; Wong et al., 2019), and context (Archer and Sjöden, 1982; Helmstetter and Fanselow, 1989; Iordanova et al., 2008). The types of USs used in higher-order designs are similar to those used in first-order conditioning studies including footshocks, rewards such as food to a hungry rat, lithium chloride (LiCl)—induced illness (e.g., Rizley and Rescorla, 1972; Holland and Rescorla, 1975a; Archer and Sjöden, 1982; Ward-Robinson and Hall, 1998). Other aspects of stimulus type such as the intensity of the US with which S1 is paired, and the physical similarity between S2 and S1 (Garcia and Koelling, 1966; Rescorla and Furrow, 1977; Rescorla, 1980a) influence the strength of higher-order conditioning.
Stimulus Similarity
An important contributing factor to learning in higher-order conditioning is stimulus similarity. Specifically, when similar stimuli are used in the roles of S2 and S1, higher-order conditioning is facilitated compared to using dissimilar stimuli. Rescorla and Furrow (1977) showed that second-order conditioning proceeded more rapidly when S1 and S2 belonged to the same, compared to different, class of stimuli (e.g., colour: blue or green; orientation: horizontal or vertical lines). These effects were not due to stimulus generalization or pseudo-conditioning (Rescorla and Furrow, 1977). Cue similarity also facilitates second-order conditioning when the cues form a part-whole relationship. For example, in a pigeon autoshaping design, Rescorla (1980a) used achromatic shapes (triangle or square) as S2 and red shapes (triangle or square) as S1. Congruency in the shape, that is, when the achromatic shape was the same as the coloured shape, resulted in better second-order conditioning. Similar effects were reported in sensory preconditioning (Holland and Ross, 1983) and in appetitive second-order conditioning (Holland, 1977) using same cue modality or spatial similarity (Rescorla and Cunningham, 1979).
Stimulus Arrangement
The sensory cues used in higher order conditioning designs can be presented simultaneously or serially. Simultaneous presentations of S1 and S2 refer to instances when the cues are presented in compound such that they overlap. In serial presentations, S1 tends to follow S2 such that S2 offset often coincides with S1 onset. Although learning accrues to S2 in both scenarios, the temporal arrangement influences the association acquired by the higher-order S2. In sensory preconditioning, simultaneous presentation of stimuli during sensory training results in superior learning compared to serial S2→S1 pairings (Thompson, 1972; Rescorla, 1980b; Holland and Ross, 1983). This effect can be explained when considering the associations that form between the cues during sensory training. Simultaneous presentations facilitate associations between the sensory characteristics of S2 and S1 rather than a predictive relationship between them (i.e., S2 predicts S1 presentation). The latter is favored by a serial arrangement. Second-order conditioning is also achieved using both simultaneous and serial S2 and S1 presentations. Rescorla (1982) showed that both serial and simultaneous arrangements result in similar levels of second-order conditioning, but the arrangement has a differential effect on what is learned (see below).
Stimulus Order
In studies where the cues have been presented serially in sensory preconditioning or second-order conditioning, it is common for S2 to precede S1. However, instances of S1 preceding S2 (i.e., S1→S2) are also effective in supporting learning. In an aversive design, sensory preconditioning was successfully obtained using such a serial backward order (i.e., S1→S2; Ward-Robinson and Hall, 1998). Reversing the order during first-order conditioning (i.e., the US preceded S1) also resulted in robust sensory preconditioning and second-order conditioning, in a lick suppression preparation with rats (Barnet et al., 1997).
Trial Number
The number of trials used to establish higher-order conditioning depends on various factors including the nature of the design (e.g., fear, reward, taste aversion), cue modality, stimulus arrangement, the model organism (e.g., rat, pigeon, rabbit), and the response measure (e.g., magazine approach, freezing, conditioned suppression). Higher-order fear conditioning progresses fairly rapidly: four trials of serial S2→S1 pairings is sufficient to obtain second-order learning (Rizley and Rescorla, 1972; Parkes and Westbrook, 2010; Lay et al., 2018) and sensory preconditioning can be achieved in eight serial S2→S1 trials (Rizley and Rescorla, 1972; Parkes and Westbrook, 2011; Wong et al., 2019). Higher-order conditioning designs involving rewards require more extensive S2→S1 training. In particular, second-order conditioning is successful using 100 trials across 10 days in pigeons (Rashotte et al., 1977), or 40 trials across four days in rats (Holland and Rescorla, 1975a) whereas sensory preconditioning has been obtained with 200 trials across 10 days in pigeons (Reid, 1952), but with as few as 12 trials across two days in rats (Jones et al., 2012; Sadacca et al., 2018).
The large number of trials often required for second-order conditioning can have unintended effects. As the number of S2→S1 trials increase in second-order conditioning, responding to S2 decreases, which is in contrast with the increase in responding to S1 across S1→US pairings. When S2→S1 pairings are alternated with continued S1→US pairings, the S2 can become a signal for the absence of the US (Herendeen and Anderson, 1968; Rescorla et al., 1973; Holland and Rescorla, 1975b; Yin et al., 1994). That is, conditioned inhibition to S2 accrues, competing with its ability to exhibit second-order conditioning (Gewirtz and Davis, 2000; Parkes and Westbrook, 2010). In a lick suppression study in rats, 20 simultaneous S2→S1 pairings favored conditioned inhibition over second-order conditioning and a hundred such trials rendered S2 a conditioned inhibitor regardless of whether S2 and S1 were paired simultaneously or serially (Stout et al., 2004). The transition of S2 from a second-order excitor to a conditioned inhibitor was quicker when S2 and S1 were presented in compound (Stout et al., 2004). To limit the development of conditioned inhibition in second-order conditioning, fewer S2→S1 pairing should be employed. This is possible in conditioned taste aversion. Indeed, a single pairing between a gustatory S2 and a contextual S1 was sufficient to obtained sensory preconditioning and second-order conditioning provided the US used to conditioned S1 was very salient (i.e., LiCl; Archer and Sjöden, 1982). These data, among others, reveal the importance of the strength of S1→US association on higher-order conditioning (Bond and Harland, 1975; Bond and Di Giusto, 1976).
Reinforced Presentations
Some instances of second-order fear conditioning consist of reinforced serial S2→S1 pairings following S1 training [i.e., S2→S1→US; Williams-Spooner et al., 2019; see also Mahmud et al. (2019)]. This design, like the standard non-reinforced design, results in robust learning about the second-order stimulus relative to an unpaired control (Leidl et al., 2018; Williams-Spooner et al., 2019). In reward learning, reinforced serial S2→S1 presentations lead to higher level of responding during training compared to non-reinforced S2→S1 presentations (Holland, 1980). This effect, however, was likely due to the development of S2→US associations (Holland, 1980). To show this, Holland (1980) tested S2 under conditions that reveal the strength of second-order associations (i.e., under food satiation) and reported lower level of responding to S2 when trained in the reinforced serial case. Holland (1980) further showed that surprising food presentations or omissions were more detrimental to second-order conditioning than when such events were expected. This was taken as evidence for the role of outcome interference in the development S2→S1 associations, which was successfully alleviated by delaying outcome delivery (Holland, 1980).
Response Measures in Higher-Order Conditioning
In first-order conditioning, an aversive US (e.g., a mild electric shock) conditions species-specific defensive behaviours (e.g., freezing, Blanchard and Blanchard, 1969; Bolles, 1970; Fanselow, 1980) or conditioned suppression (e.g., Rescorla and Furrow, 1977; Bouton and Bolles, 1980), whereas an appetitive US (e.g., sucrose pellets) supports conditioned approach (e.g., Holland, 1977). The US, however, is not the only determinant of conditioned responses. Auditory and visual cues can support cue-based responses including rearing, head jerk, perambulation, and general activity (Holland and Rescorla, 1975a,b; Holland, 1977, 1984). While auditory stimuli elicit startle and head jerk, visual stimuli elicit rearing (Holland, 1977). Startle and rearing are considered orienting responses (OR) and are seen to novel but not familiar non-reinforced cues and maintained or augmented to cues that have undergone conditioning. Head jerk is specific to conditioned auditory cues. ORs and CRs are differentially distributed across the duration of a conditioned stimulus, with ORs occurring mostly during the beginning of visual cues and food-cup CRs following afterwards, while CRs and ORs elicited by auditory cues are more evenly distributed (Holland, 1977; Hatfield et al., 1996). In second-order conditioning, pairing an auditory S2 with either a visual or auditory S1 leads to similar proportions of CRs and ORs, with head jerk being the predominant response to S2 in both cases (Holland, 1977).
Content of Higher-Order Conditioning
The associative links that govern sensory preconditioning and second-order conditioning differ depending on the procedural details. As different designs are often used to study the neural substrates of higher-order learning, it is imperative that one is aware that procedural differences can lead to differences in associative content (i.e., what is learned; see also Gewirtz and Davis, 2000; Parkes and Westbrook, 2011; Gostolupce et al., 2021). We cover these below.
Extinction of S1
The first evidence to highlight the differences in learning between sensory preconditioning and second-order conditioning came from Rizley and Rescorla (1972). In a fear conditioning procedure with footshock as the US, the authors showed that reduction in responding to S1 via repeated presentations of this cue in the absence of the US (i.e., S1 extinction training) consequently reduced responding to S2 in sensory preconditioning but not in second-order conditioning [see also Parkes and Westbrook (2010), Holmes et al. (2014)]. Similar findings have been reported in higher-order reward conditioning by Holland and Rescorla (1975a) as well as in a conditioned taste aversion design (Archer and Sjöden, 1982). These data provide convincing evidence that the association between S2 and S1 is key to regulating sensory preconditioning but not second-order conditioning (Rizley and Rescorla, 1972; Holland and Rescorla, 1975a).
It turns out, however, that the arrangement of stimulus presentation or stimulus similarity can influence the nature of associative learning in second-order conditioning. Rescorla (1982) showed that simultaneous S2→S1 pairings produce second-order responding that is sensitive to S1-extinction. This may be because simultaneous presentations lead to within compound associations or the development of an S2S1 configural unit (Pearce, 1994, 2002), meaning that the sensory cues can activate representations that contain one another.
It is also possible to obtain second-order responding that is sensitive to extinction of S1 when S2 and S1 belong to the same cue modality (i.e., both S2 and S1 as auditory cues such as tones of different frequencies or as visual cues such as a flashing houselight or a jeweled signal light; Rescorla and Furrow, 1977). In an autoshaping procedure, pigeons were first trained to peck a white key light S1 by pairing it with grain delivery (Rashotte et al., 1977). A blue key light S2 was trained with S1 in a serial manner to achieve second-order conditioning to S2. Extinction of S1 resulted in disruption of second-order key peck responding. These findings demonstrate that second-order conditioning is sensitive to manipulations of S1 when S2 and S1 belong to the same cue modality (but see Experiment 4 using different modalities in which S1 is an operant discriminant).
Determining what is learned during second-order conditioning is further informed by the behaviours that are measured. While extinction of a visual S1 after second-order conditioning leaves intact food-cup approach and head jerk CRs to an auditory S2 (Holland and Rescorla, 1975a), rear ORs to S2 are abolished (Setlow et al., 2002; McDannald et al., 2013). Given that rearing is generally only evoked by either visual cues or S2s by virtue of them being paired with a visual S1, this OR is thought to represent the behavioural readout of a S2→S1 (i.e., stimulus→stimulus) association (Setlow et al., 2002). This suggests that an S2→S1 association may be formed, but such associations are unlikely to drive the conditioned responses normally measured in second-order conditioning.
Finally, in a series of clever studies that used the nature of responding to determine the nature of the associations between events in second-order conditioning, Holland (1977) revealed that S2 is likely linked to the affective or motivational state induced by the US. Specifically, he examined whether an auditory S2 would acquire auditory ORs or visual ORs when paired with either an auditory or visual S1 in a second-order design. The data confirmed that an auditory S2 elicits an auditory-specific response and does not become associated with the cue-based response elicited by S1, eliminating the likelihood of a S2→CR (i.e., stimulus→response) association. Holland’s interpretation was further confirmed by Winterbauer and Balleine (2005) who showed that second-order cues enter into associations with the specific motivational aspects of the US in water- and food-deprived rats and that this learning was not dependent on the motivational state at the time of training.
Devaluing the US
Although S2 is never directly paired with the US in higher-order designs, some evidence suggests that a S2→US association must not be discounted. Indeed, the development of associations between actual and associatively evoked stimuli are well-supported by the literature (e.g., Holland, 1981, 1983; Holland and Forbes, 1982; Iordanova et al., 2008; Lin and Honey, 2010, 2011, 2016; Wong et al., 2019). To account for such learning, Holland (1981, 1983) proposed a modification to Wagner’s Standard Operating Procedures (SOP; Wagner, 1981). Briefly, according to SOP, stimuli can be in three states of activation, a focal A1 state, a working memory A2 state and an inactive I state. Excitatory conditioning occurs when events are concurrently in an A1 state of activity whereas inhibitory conditioning occurs when a cue is in an A1 state and the US in an A2 state. In a series of conditioned taste aversion experiments, Holland (1981, 1983) provided evidence that excitatory learning also occurs between two events (e.g., food and LiCl) when the food is in A2 (i.e., associatively activated) and the LiCl in A1 (physically present). This proposal accounts for the development of S2→US associations in sensory preconditioning designs because S1 would place its associate S2 into an A2 state during S1→US training while the US is in an A1 state, thereby allowing for S2→US learning in this phase.
To determine whether the representation of the US is linked to S2, Holland and Rescorla (1975a) used a devaluation procedure. They showed that reducing the value of an appetitive US led to a corresponding reduction in responding to S2 in sensory preconditioning but not second-order conditioning. This was also confirmed by Rescorla (1973) who devalued an aversive US (loud noise) using habituation. In other words, S2 is not linked to the US in second-order conditioning, at least early on in S2→US training. The lack of devaluation effects in second-order conditioning is consistent with the original stipulation of SOP (Wagner, 1981), which holds that during S2→S1 pairings, S2 would be in an A1 state whereas the US would be associatively evoked by S1 and therefore in an A2 state, resulting in the development of S2 as a conditioned inhibitor for the US. As mentioned, conditioned inhibition can accrue to a second-order S2 when the number of S2-S1 pairings increases and these trials are alternated with continued S1-US pairings (Herendeen and Anderson, 1968; Rescorla et al., 1973; Holland and Rescorla, 1975b; Yin et al., 1994), lending support for the SOP proposal. Intriguingly, this inhibitory association can co-exist with the excitatory second-order association (Holland and Rescorla, 1975b) and is greater when stimuli are similar compared to dissimilar (Rescorla, 1980a).
In summary, the studies reviewed above show that higher-order conditioning can be obtained using a variety of stimuli under diverse conditions. While sensory preconditioning is supported by associations between S2 and S1 as well as between S2 and the associatively evoked US, those that drive second-order conditioning are parameter-dependent. In the somewhat classic serial design that uses cues of different modalities, second-order conditioning is dependent on S2→motivational state associations and not on S2→S1 (evidenced using S1 extinction), nor S2→US (evidenced using US devaluation), nor S2→CR (evidenced in the inability of S2 to acquire cue-specific responses indicative of S1 expectation) associations. Altering the cue arrangement or modality, shifts the content of what is learned.
Implications for Neuroscience
The quest for uncovering the neural mechanisms of higher-order learning is gaining momentum [e.g., Iordanova et al., 2009, 2011a,b; Horne et al., 2010; Jones et al., 2012; Wimmer and Shohamy, 2012; Holmes et al., 2013; Holland and Hsu, 2014; Holland, 2016; Lin and Honey, 2016; Sadacca et al., 2018; Wong et al., 2019; Hart et al., 2020; for reviews see Parkes and Westbrook (2011), Fournier et al. (2021), Gostolupce et al. (2021), and Holmes et al. (2021)]. Indeed, dissociations in the neural mechanisms of sensory preconditioning and second-order conditioning have been reported (Parkes and Westbrook, 2010; Holmes et al., 2013; Holland and Hsu, 2014; Holland, 2016) as have been nuances in the regulation of cue- vs. outcome-based responses elicited by higher-order stimuli (Gallagher et al., 1990; Hatfield et al., 1996; McDannald et al., 2013). These lines of evidence suggest that different neural areas regulate different types of associations despite the similarity in training, and that parallel systems drive subsets of behavioural responses established under the same training conditioning.
Our understanding of the functional role of distinct neural substrates can be greatly advanced by the study of higher-order learning. These preparations expand the conditions under which learning occurs, extending our study of how the brain learns. In addition, they provide important information into the associative structures that control behaviour, thereby offering particular insight into the function of brain areas that regulate this learning. This manuscript reviews the distinct procedures and parameters that supports higher-order learning and how this affects the corresponding associative architecture, which we hope will bolster the field’s analysis of the corresponding neural architecture.
Author Contributions
All authors contributed to the article by conceiving the idea for the manuscript and developing the narrative. All authors approved the submitted version.
Funding
This work was supported by a CIHR Project Grant (to MI Grant: PJT-173226), an excellence award by the Canada Research Chairs Program (to MI Grant: 950-230456), a FRQS post-doctoral fellowship (to BL #276908), and Concordia International Tuition award of excellence (to DG).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
We thank Alexandra Usypchuk for her insightful comments on the manuscript.
References
Amiro, T. W., and Bitterman, M. E. (1980). Second-order appetitive conditioning in goldfish. J. Exp. Psychol.Anim. Behav. Process. 6, 41–48. doi: 10.1037/0097-7403.6.1.41
Archer, T., and Sjöden, P. O. (1982). Higher-order conditioning and sensory preconditioning of a taste aversion with an exteroceptive CS. Q. J. Exp. Psychol. B 34, 1–17. doi: 10.1080/14640748208400886
Barnet, R. C., Cole, R. P., and Miller, R. R. (1997). Temporal integration in second-order conditioning and sensory preconditioning. Anim. Learn. Behav. 25, 221–233. doi: 10.3758/BF03199061
Beauchamp, A. J., and Gluck, J. P. (1988). Associative processes in differentially reared monkeys (Macaca mulatta): sensory preconditioning. Dev. Psychobiol. 21, 355–364. doi: 10.1002/dev.420210406
Blaisdell, A. P. (2009). Rational animals, irrational humans in The role of Associative Processes in Spatial, Temporal, and Causal Cognition, eds S. Watanabe, A. P. Blaisdell, L. Huber, and A. Young (Tokyo: Keio University). 153–172.
Blanchard, R. J., and Blanchard, D. C. (1969). Crouching as an index of fear. J. Comp. Physiol. Psychol. 67, 370–375. doi: 10.1037/h0026779
Bolles, R. C. (1970). Species-specific defense reactions and avoidance learning. Psychol. Rev. 77, 32–48. doi: 10.1037/h0028589
Bond, N., and Harland, W. (1975). Higher order conditioning of a taste aversion. Anim. Learn. Behav. 3, 295–296. doi: 10.3758/BF03213448
Bond, N. W., and Di Giusto, E. L. (1976). One trial higher-order conditioning of a taste aversion. Aust. J. Psychol. 28, 53–55. doi: 10.1080/00049537608255263
Bouton, M. E., and Bolles, R. C. (1980). Conditioned fear assessed by freezing and by the suppression of three different baselines. Anim. Learn. Behav. 8, 429–434. doi: 10.3758/BF03199629
Brembs, B., and Heisenberg, M. (2001). Conditioning with compound stimuli in Drosophila melanogaster in the flight simulator. J. Exp. Biol. 204, 2849–2859. doi: 10.1242/jeb.204.16.2849
Brogden, W. J. (1939). Sensory pre-conditioning. J. Exp. Psychol. 25, 323–332. doi: 10.1037/h0058944
Busquets-Garcia, A., Soria-Gómez, E., Redon, B., Mackenbach, Y., Vallée, M., Chaouloff, F., et al. (2017). Pregnenolone blocks cannabinoid-induced acute psychotic-like states in mice. Mol. Psychiatry. 22, 1594–1603. doi: 10.1038/mp.2017.4
Cai, D. J., Aharoni, D., Shuman, T., Shobe, J., Biane, J., Song, W., et al. (2016). A shared neural ensemble links distinct contextual memories encoded close in time. Nature 534, 115–118. doi: 10.1038/nature17955
Cook, M., and Mineka, S. (1987). Second-order conditioning and overshadowing in the observational conditioning of fear in mon- keys. Behav. Res. Ther. 25, 349–364. doi: 10.1016/0005-7967(87)90013-1
Craddock, P., Wasserman, J. S., Polack, C. W., Kosinski, T., Renaux, C., and Miller, R. R. (2018). Associative structure of second-order conditioning in humans. Learn. Behav. 46, 171–181. doi: 10.3758/s13420-017-0299-5
Fanselow, M. S. (1980). Conditional and unconditional components of post-shock freezing. Pavlov. J. Biol. Sci. 15, 177–182. doi: 10.1007/BF03001163
Fournier, D. I., Cheng, H. Y., Robinson, S., and Todd, T. P. (2021). Cortical contributions to higher-order conditioning: a review of retrosplenial cortex function. Front. Behav. Neurosci. 15:682426. doi: 10.3389/fnbeh.2021.682426
Gallagher, M., Graham, P. W., and Holland, P. C. (1990). The amygdala central nucleus and appetitive Pavlovian conditioning: lesions impair one class of conditioned behavior. J. Neurosci. 10, 1906–1911. doi: 10.1523/JNEUROSCI.10-06-01906.1990
Garcia, J., and Koelling, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychon. Sci. 4, 123–124. doi: 10.3758/bf03342209
Gewirtz, J. C., and Davis, M. (2000). Using Pavlovian higher-order conditioning paradigms to investigate the neural substrates of emotional learning and memory. Learn. Mem. 7, 257–266. doi: 10.1101/lm.35200
Gibbs, C. M., Cool, V., Land, T., Kehoe, E. J., and Gormezano, I. (1991). Second-order conditioning of the rabbit’s nictitating membrane response. Interstimulus interval and frequency of CS-CS pairings. Integr. Psychol. Behav. Neurosci. 26, 282–295. doi: 10.1007/BF02691064
Gostolupce, D., Iordanova, M. D., and Lay, B. P. P. (2021). Mechanisms of higher-order learning in the amygdala. Behav. Brain Res. 414:113435. doi: 10.1016/j.bbr.2021.113435
Hart, E. E., Sharpe, M. J., Gardner, M. P., and Schoenbaum, G. (2020). Responding to preconditioned cues is devaluation sensitive and requires orbitofrontal cortex during cue-cue learning. Elife 9:e59998. doi: 10.7554/eLife.59998
Hatfield, T., Han, J. S., Conley, M., Gallagher, M., and Holland, P. (1996). Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J. Neurosci. 16, 5256–5265. doi: 10.1523/JNEUROSCI.16-16-05256.1996
Helmstetter, F. J., and Fanselow, M. S. (1989). Differential second-order aversive conditioning using contextual stimuli. Anim. Learn. Behav. 17, 205–212. doi: 10.3758/BF03207636
Herendeen, D., and Anderson, D. C. (1968). Dual effects of second-order conditioned stimulus: excitation and inhibition. Psychon. Sci. 13, 15–16. doi: 10.3758/BF03342385
Holland, P. C. (1977). Conditioned stimulus as a determinant of the form of the Pavlovian conditioned response. J. Exp. Psychol.Anim. Behav. Process. 3, 77–104. doi: 10.1037/0097-7403.3.1.77
Holland, P. C. (1980). Second-order conditioning with and without unconditioned stimulus presentation. J. Exp. Psychol.Anim. Behav. Process. 6, 238–250. doi: 10.1037/0097-7403.6.3.238
Holland, P. C. (1981). Acquisition of representation-mediated conditioned food aversions. Learn. Motiv. 12, 1–18. doi: 10.1016/0023-9690(81)90022-9
Holland, P. C. (1983). Representation-mediated overshadowing and potentiation of conditioned aversions. J. Exp. Psychol. Anim. Behav. Process. 9, 1–13. doi: 10.1037/0097-7403.9.1.1
Holland, P. C. (1984). Unblocking in Pavlovian appetitive conditioning. J. Exp. Psychol.Anim. Behav. Process. 10, 476–497. doi: 10.1037/0097-7403.10.4.476
Holland, P. C. (1990). Event representation in Pavlovian conditioning: image and action. Cognition 37s, 105–131. doi: 10.1016/0010-0277(90)90020-k
Holland, P. C. (2016). Enhancing second-order conditioning with lesions of the basolateral amygdala. Behav. Neurosci. 130, 176–181. doi: 10.1037/bne0000129
Holland, P. C., and Forbes, D. T. (1982). Representation-mediated extinction of conditioned flavor aversions. Learn. Motiv. 13, 454–471. doi: 10.1016/0023-9690(82)90004-2
Holland, P. C., and Hsu, M. (2014). Role of amygdala central nucleus in the potentiation of consuming and instrumental lever-pressing for sucrose by cues for the presentation or interruption of sucrose delivery in rats. Behav. Neurosci. 128, 71–82. doi: 10.1037/a0035445
Holland, P. C., and Rescorla, R. A. (1975a). The effect of two ways of devaluing the unconditioned stimulus after first- and second-order appetitive conditioning. J. Exp. Psychol.Anim. Behav. Process. 1, 355–363. doi: 10.1037/0097-7403.1.4.355
Holland, P. C., and Rescorla, R. A. (1975b). Second-order conditioning with food unconditioned stimulus. J. Comp. Physiol. Psychol. 88, 459–467. doi: 10.1037/h0076219
Holland, P. C., and Ross, R. T. (1983). Savings test for associations between neutral stimuli. Anim. Learn. Behav. 11, 83–90. doi: 10.3758/BF03212312
Holmes, N. M., Cai, S. Y., Lay, B. P. P., Watts, N. R., and Westbrook, R. F. (2014). Extinguished second-order conditioned fear responses are renewed but not reinstated. J. Exp.l Psychol. Anim. Learn. Cogn. 40, 440–456. doi: 10.1037/xan0000036
Holmes, N. M., Parkes, S. L., Killcross, A. S., and Westbrook, R. F. (2013). The basolateral amygdala is critical for learning about neutral stimuli in the presence of danger, and the perirhinal cortex is critical in the absence of danger. J. Neurosci. 33, 13112–13125. doi: 10.1523/JNEUROSCI.1998-13.201
Holmes, N. M., Wong, F. S., Bouchekioua, Y., and Westbrook, R. F. (2021). Not “either-or” but “which-when”: a review of the evidence for integration in sensory preconditioning. Neurosci. Biobehav. Rev. 132, 1197–1204. doi: 10.1016/j.neubiorev.2021.10.032
Horne, M. R., Iordanova, M. D., Albasser, M. M., Aggleton, J. P., Honey, R. C., and Pearce, J. M. (2010). Lesions of the perirhinal cortex do not impair integration of visual and geometric information in rats. Behav. Neurosci. 124, 311–320. doi: 10.1037/a0019287
Iordanova, M. D., Burnett, D. J., Aggleton, J. P., Good, M., and Honey, R. C. (2009). The role of the hippocampus in mnemonic integration and retrieval: complementary evidence from lesion and inactivation studies. Eur. J. Neurosci. 30, 2177–2189. doi: 10.1111/j.1460-9568.2009.07010.x
Iordanova, M. D., Burnett, D. J., Good, M., and Honey, R. C. (2011a). Pattern memory involves both elemental and configural processes: evidence from the effects of hippocampal lesions. Behav. Neurosci. 125, 567–577. doi: 10.1037/a0023762
Iordanova, M. D., Good, M., and Honey, R. C. (2011b). Retrieval-mediated learning involving episodes requires synaptic plasticity in the hippocampus. J. Neurosci. 31, 7156–7162. doi: 10.1523/JNEUROSCI.0295-11.2011
Iordanova, M. D., Good, M. A., and Honey, R. C. (2008). Configural Learning without Reinforcement: integrated memories for correlates of what, where, and when. Q. J. Exp. Psychol. 61, 1785–1792. doi: 10.1080/17470210802194324
Jones, J. L., Esber, G. R., McDannald, M. A., Gruber, A. J., Hernandez, A., Mirenzi, A., et al. (2012). Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338, 953–956. doi: 10.1126/science.1227489
Lay, B. P. P., Westbrook, R. F., Glanzman, D. L., and Holmes, N. M. (2018). Commonalities and differences in the substrates underlying consolidation of first- and second-order conditioned fear. J. Neurosci. 38, 1926–1941. doi: 10.1523/JNEUROSCI.2966-17.2018
Lee, J. C., and Livesey, E. J. (2012). Second-order conditioning and conditioned inhibition: influences of speed versus accuracy on human causal learning. PLoS One 7:e49899. doi: 10.1371/journal.pone.0049899
Leidl, D. M., Lay, B. P. P., Chakouch, C., Frederick Westbrook, R., and Holmes, N. M. (2018). Protein synthesis in the basolateral amygdala complex is required for consolidation of a first-order fear memory, but not for consolidation of a higher-order fear memory. Neurobiol. Learn. Mem. 153, 153–165. doi: 10.1016/j.nlm.2018.04.001
Lin, T.-E., and Honey, R. C. (2010). Analysis of the content of configural representations: the role of associatively evoked and trace memories. J. Exp. Psychol. Anim. Behav. Process. 36, 501–505. doi: 10.1037/a0018348
Lin, T. E., and Honey, R. C. (2011). Encoding specific associative memory: evidence from behavioral and neural manipulations. J. Exp. Psychol. Anim. Behav. Process. 37, 317–329. doi: 10.1037/a0022497
Lin, T. E., and Honey, R. C. (2016). “Learning about stimuli that are present and those that are not: separable acquisition processes for direct and mediated learning,” in The Wiley Handbook on the Cognitive Neuroscience of Learning, eds R. A. Murphy and R. C. Honey (Oxford: Wiley-Blackwell). 69–85. doi: 10.1002/9781118650813.ch4
Mahmud, A., Petrov, P., Esber, G. R., and Iordanova, M. D. (2019). The serial blocking effect: a testbed for the neural mechanisms of temporal-difference learning. Sci. Rep. 9:5962. doi: 10.1038/s41598-019-42244-4,
McDannald, M. A., Setlow, B., and Holland, P. C. (2013). Effects of ventral striatal lesions on first- and second-order appetitive conditioning. Eur. J. Neurosci. 38, 2589–2599. doi: 10.1111/ejn.12255
Mead, A. N., and Stephens, D. N. (2003). Selective disruption of stimulus-reward learning in glutamate receptor andlt;emandgt;gria1andlt;/emandgt; knock-out mice. J. Neurosci. 23, 1041–1048. doi: 10.1523/JNEUROSCI.23-03-01041.2003
Müller, D., Gerber, B., Hellstern, F., Hammer, M., and Menzel, R. (2000). Sensory preconditioning in honeybees. J.Exp. Biol. 203, 1351–1364. doi: 10.1242/jeb.203.8.1351
Parkes, S. L., and Westbrook, R. F. (2010). The basolateral amygdala is critical for the acquisition and extinction of associations between a neutral stimulus and a learned danger signal but not between two neutral stimuli. J. Neurosci. 30, 12608–12618. doi: 10.1523/JNEUROSCI.2949-10.2010
Parkes, S. L., and Westbrook, R. F. (2011). Role of the basolateral amygdala and NMDA receptors in higher-order conditioned fear. Rev. Neurosci. 22, 317–333. doi: 10.1515/RNS.2011.025
Pavlov, I. P. (1927). Conditioned reflexes: An investigation of the physiological activity of the Ccerebral cortex (G.V. Anrep, trans.). New York, NY: Dover.
Pearce, J. M. (1994). Similarity and discrimination: a selective review and a connectionist model. Psychol. Rev. 101, 587–607. doi: 10.1037//0033-295x.101.4.587
Pearce, J. M. (2002). Evaluation and development of a connectionist theory of configural learning. Anim. Learn. Behav. 30, 73–95. doi: 10.3758/BF03192911
Pfautz, P. L., Donegan, N. H., and Wagner, A. R. (1978). Sensory preconditioning versus protection from habituation. journal of experimental psychology. Anim. Behav. Process. 4, 286–295. doi: 10.1037/0097-7403.4.3.286
Prokofiev, G., and Zeliony, G. (1926). Des modes d’associations cérébrales chez l’homme et chez les animaux. J. Psychol. 23, 1020–1028.
Rashotte, M. E., Griffin, R. W., and Cheryl, L. (1977). Second-order conditioning of the pigeon’s keypeck. Anim. Learn. Behav. 5, 25–38. doi: 10.3758/bf03209127
Reid, R. L. (1952). A test of sensory pre-conditioning in pigeons. Q. J.Exp. Psychol. 4, 49–56. doi: 10.1080/17470215208416603
Renaux, C., Rivière, V., Craddock, P., and Miller, R. R. (2017). Role of spatial contiguity in sensory preconditioning with humans. Behav. Process. 142, 141–145. doi: 10.1016/j.beproc.2017.07.005
Rescorla, R. A. (1973). Effects of US habituation following conditioning. J. Comp. Physiol. Psychol. 82, 137–143. doi: 10.1037/h0033815
Rescorla, R. A. (1979). Aspects of the reinforcer learned in second-order Pavlovian conditioning. J. Exp. Psychol. Anim. Behav. Process. 5, 79–95. doi: 10.1037/0097-7403.5.1.79
Rescorla, R. A. (1980b). Simultaneous and successive associations in sensory preconditioning. J. Exp. Psychol. Anim. Behav. Process. 6, 207–216. doi: 10.1037/0097-7403.6.3.207
Rescorla, R. A. (1982). Simultaneous second-order conditioning produces S-S learning in conditioned suppression. J. Exp. Psychol.Anim. Behav. Process. 8, 23–32. doi: 10.1037/0097-7403.8.1.23
Rescorla, R. A., and Cunningham, C. (1979). Spatial contiguity facilitates Pavlovian second-order conditioning. J. Exp. Psychol. Anim. Behav. Process. 5, 152–161. doi: 10.1037//0097-7403.5.2.152
Rescorla, R. A., and Furrow, D. R. (1977). Stimulus similarity as a determinant of Pavlovian conditioning. J. Exp. Psychol. Anim. Behav. Process. 3, 203–215. doi: 10.1037/0097-7403.3.3.203
Rescorla, R. A., McGuigan, F. J., and Lumsden, D. B. (eds) (1973). “Second-order conditioning: implications for theories of learning,” in Contemporary Approaches to Conditioning and Learning, (Oxford: V.H. Winston and Sons). 127–150.
Rizley, R. C., and Rescorla, R. A. (1972). Associations in second-order conditioning and sensory preconditioning. J. Comp. Physiol. Psychol. 81, 1–11. doi: 10.1037/h0033333
Sadacca, B. F., Wied, H. M., Lopatina, N., Saini, G. K., Nemirovsky, D., and Schoenbaum, G. (2018). Orbitofrontal neurons signal sensory associations underlying model-based inference in a sensory preconditioning task. Elife 7:e30373. doi: 10.7554/eLife.30373,
Seitz, B. M., Blaisdell, A. P., and Sharpe, M. J. (2021). Higher-order cognition and dopamine: charting a path forward. Front. Behav. Neurosci. 15:745388. doi: 10.3389/fnbeh.2021.745388
Setlow, B., Gallagher, M., and Holland, P. C. (2002). The basolateral complex of the amygdala is necessary for acquisition but not expression of CS motivational value in appetitive Pavlovian second-order conditioning. Eur. J. Neurosci. 15, 1841–1853. doi: 10.1046/j.1460-9568.2002.02010.x
Stout, S., Escobar, M., and Miller, R. R. (2004). Trial number and compound stimuli temporal relationship as joint determinants of second-order conditioning and conditioned inhibition. Learn. Behav. 32, 230–239. doi: 10.3758/bf03196024
Tabone, C. J., and de Belle, J. S. (2011). Second-order conditioning in Drosophila. Learn. Mem. 18, 250–253. doi: 10.1101/lm.2035411
Thompson, R. F. (1972). “Sensory preconditioning,” in Topics in Learning and Performance, eds R. F. Thompson and J. S. Voss (New York, N.Y: Academic Press). 105–129.
Wagner, A. R. (1981). “SOP: A model of automatic memory processing in animal behavior,” in Information Processing in Animals: Memory Mechanisms, eds N. E. Spear and R. R. Miller (Hillsdale: Erlbaum). 5–47.
Wallenstein, G. V., Hasselmo, M. E., and Eichenbaum, H. (1998). The hippocampus as an associator of discontiguous events. Trends Neurosci. 21, 317–323. doi: 10.1016/s0166-2236(97)01220-4
Ward-Robinson, J. W., and Hall, G. (1998). Backward sensory preconditioning when reinforcement is delayed. Q. J.Exp. Psychol. B 1, 349–362. doi: 10.1080/713932687
Williams-Spooner, M. J., Westbrook, R. F., and Holmes, N. M. (2019). The conditions under which consolidation of serial-order conditioned fear requires de novo protein synthesis in the basolateral amygdala complex. J.Neurosci. 39, 7357–7368. doi: 10.1523/JNEUROSCI.0768-19.2019
Wimmer, G. E., and Shohamy, D. (2012). Preference by association: how memory mechanisms in the hippocampus bias decisions. Science 338, 270–273. doi: 10.1126/science.1223252
Winterbauer, N. E., and Balleine, B. W. (2005). Motivational control of second-order conditioning. J.Exp. Psychol. Anim. Behav. Process. 31, 334–340. doi: 10.1037/0097-7403.31.3.334
Wong, A. H. K., and Pittig, A. (2022). Avoiding a feared stimulus: modelling costly avoidance of learnt fear in a sensory preconditioning paradigm. Biol. Psychol. 168:108249. doi: 10.1016/j.biopsycho.2021.108249
Wong, F. S., Westbrook, R. F., and Holmes, N. M. (2019). ‘Online’ integration of sensory and fear memories in the rat medial temporal lobe. Elife 8:e47085. doi: 10.7554/eLife.47085,
Keywords: associative learning, second-order conditioning, sensory preconditioning, memory integration, extinction
Citation: Gostolupce D, Lay BPP, Maes EJP and Iordanova MD (2022) Understanding Associative Learning Through Higher-Order Conditioning. Front. Behav. Neurosci. 16:845616. doi: 10.3389/fnbeh.2022.845616
Received: 30 December 2021; Accepted: 04 March 2022;
Published: 18 April 2022.
Edited by:
Nathan Holmes, University of New South Wales, AustraliaReviewed by:
Aaron P. Blaisdell, University of California, Los Angeles, United StatesCopyright © 2022 Gostolupce, Lay, Maes and Iordanova. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Mihaela D. Iordanova, bWloYWVsYS5pb3JkYW5vdmFAY29uY29yZGlhLmNh
†These authors share first authorship