David M. Lipton
Ben J. Gonzales
Ami Citri- 1Edmond and Lily Safra Center for Brain Sciences, Hebrew University of Jerusalem, Jerusalem, Israel
- 2Zuckerman Postdoctoral Scholar, Jerusalem, Israel
- 3Institute of Life Sciences, Edmond J. Safra Campus, Hebrew University of Jerusalem, Jerusalem, Israel
- 4Program in Child and Brain Development, MaRS Centre, Canadian Institute for Advanced Research, Toronto, ON, Canada
Here, we review the neural circuit bases of habits, compulsions, and addictions, behaviors which are all characterized by relatively automatic action performance. We discuss relevant studies, primarily from the rodent literature, and describe how major headway has been made in identifying the brain regions and neural cell types whose activity is modulated during the acquisition and performance of these automated behaviors. The dorsal striatum and cortical inputs to this structure have emerged as key players in the wider basal ganglia circuitry encoding behavioral automaticity, and changes in the activity of different neuronal cell-types in these brain regions have been shown to co-occur with the formation of automatic behaviors. We highlight how disordered functioning of these neural circuits can result in neuropsychiatric disorders, such as obsessive-compulsive disorder (OCD) and drug addiction. Finally, we discuss how the next phase of research in the field may benefit from integration of approaches for access to cells based on their genetic makeup, activity, connectivity and precise anatomical location.
Bundles of Habits
“When we look at living creatures from an outward point of view, one of the first things that strike us is that they are bundles of habits” (James, 1890). Behavioral automaticity, as eloquently expressed in William James’ treatise “Habit,” is a fundamental aspect of our existence, and is essential for freeing-up our cognitive capacities so they can be directed to engaging novel and complex experiences, as further elaborated by James: “The more of the details of our daily life we can hand over to the effortless custody of automatism, the more our higher powers of mind will be set free for their own proper work.” (James, 1890). However, James also was very clear that these very same attributes of habits are also responsible for the most severe restrictions on our liberty. “Habit is thus the enormous fly-wheel of society, its most precious conservative agent. It alone is what keeps us all within the bounds of ordinance…” The topic of habit formation and its role in adaptive and maladaptive behavior has been extensively reviewed, most comprehensively in a recent dedicated issue of Current Opinion in Behavioral Science (Knowlton and Diedrichsen, 2018). Here, we provide a concise synthesis of the literature on the neural circuit basis of habits and their more extreme counterparts, compulsions and addictions, focusing on striatal circuits, which have primarily been deciphered in rodents. We begin with an overview of the common circuitry utilized by automatic behaviors, highlighting the importance of the dorsal striatum and inputs to this structure. We subsequently describe behavioral models used to study habits, compulsions and addictions, and then examine the neural circuit bases of these behaviors at increasingly higher resolution of analysis. We illustrate the established roles of the dorsolateral and dorsomedial subregions of the striatum in behavioral automaticity, and then review the complex picture of the roles of different striatal input structures, as well as specific cellular and synaptic modifications. Finally, we propose a roadmap for future investigations, integrating emerging molecular and circuit analysis methodologies with increasingly detailed knowledge of the multidimensional diversity of striatal cell-types, in order to analyze the circuits underlying automatic behaviors.
What Are Habits, Compulsions, and Addictions and How Are They Related?
We intuitively use the term habit to describe behaviors that have become so ingrained that we perform them almost automatically, autonomously of the outcome (James, 1890; Dickinson, 1985; Graybiel, 2008; Robbins and Costa, 2017), and which, in extreme form, can become a compulsion or addiction. This is in contrast to goal-directed, purposeful behavior, in which an action is explicitly performed with the objective of obtaining a desired outcome (Valentin et al., 2007; Graybiel, 2008; Gremel and Costa, 2013; Robbins and Costa, 2017; Nonomura et al., 2018; Figures 1A,B). Goal-directed and habitual behaviors can be distinguished by their differential sensitivity to reward devaluation (i.e., reducing the value of the outcome; Figure 1C). Purposeful behavior will diminish if the outcome is no longer desired, while habitual performance will persist, since during the development of habitual behavior, the action becomes dissociated from the outcome, and performance is driven instead by antecedent stimuli and/or emotional states. Habitual behavior is therefore associated with behavioral automaticity, with diminished reliance on reinforcement. Thus, habits are shaped by past experience, and are characterized by computational efficiency and inflexibility, in contrast to goal-directed behavior, which is characterized by active deliberation of future consequences, high computational cost, and an adaptive flexibility to changing environments (Daw et al., 2005). Major benefits come from automaticity and independence from reinforcement, which allows the brain to free up rate-limiting attentional and decision-making resources. However, automaticity can also be detrimental, underlying the susceptibility to the development of maladaptive habits, which in the extreme can result in compulsions and addictions (Figures 1A,B). The central characteristic of compulsions and addictions is the continued pursuit of a previously rewarding stimulus, despite its clear current association with adverse consequences (Lüscher and Malenka, 2011; Volkow and Morales, 2015). This hallmark of addiction, action performance in spite of punishment, can be viewed as an extreme of habitual behavior (Figures 1A–C).
Figure 1. Characteristics of the shift from goal-directed to habitual behavior. (A) Left: Goal-directed and habitual behaviors are competitive processes that act in balance. Goal-directed behavior is characterized by a high requirement for attention, is highly contingent on present reward value, and demonstrates flexibility of responding. Habitual behavior is stimulus-driven, less dependent on present reward value, and governed by behavioral automaticity. Right: Addiction/compulsion represents an extreme state of habit. (B) The transition from goal-directed behavior to habitual behavior and then into compulsion, or addiction is graded. Shift from goal-directed to habitual behavior and then to compulsion/addiction corresponds to strengthened stimulus-response association and reduced action-outcome contingency. These processes are bidirectional, i.e., a behavior can shift on the spectrum from goal-directed to habitual performance, and back again—though in the extremes of addiction whether it is possible to return fully to habit/goal-directed states is less clear. (C) During instrumental training, rates of responding for a reward increase. Post-training reward devaluation reduces response rates more quickly for goal-directed behaviors than it does for habitual behaviors, which take many more extinction trials to fully dissipate. The extremes of addiction are characterized by compulsive responding that is resistant even to punishment. (D) The balance between goal-directed and habitual behavioral states corresponds to relative levels of neural activity in the dorsomedial (DMS) vs. dorsolateral (DLS) striatum. (E) Task-bracketing activity pattern emerges in the DLS as animals are over-trained on a rewarded behavioral sequence (e.g., running a T-maze for a tasty reward). Spiny Projection Neurons (SPNs) exhibit high activity at the beginning of a learned motor sequence and again at the end as the animal approaches the reward. Fast-spiking interneurons (FSIs) exhibit high activity during the middle stages of a behavioral sequence.
The intimate relationship of habits, compulsions and addictions is further made obvious by the coincident expression of behaviors of these categories. For instance, patients with obsessive-compulsive disorder (OCD) also demonstrate an enhanced tendency for dominance of habitual behavior (Gillan et al., 2011, 2016). Additionally, exposure to drugs of abuse, as well as binge-eating of palatable foods, enhance habit formation (Everitt and Robbins, 2016). Thus, cocaine addicts exhibit a higher tendency to form habits (Ersche et al., 2016), and alcohol exposure accelerates the emergence of habitual behavior (Corbit et al., 2012; Hogarth et al., 2012). These pathological states of behavioral automaticity have been shown to utilize overlapping circuitry.
Common Limbic Circuitry Underlying Reinforcement Learning and Behavioral Automaticity
The neural circuits involved in instrumental learning and the automation of behavior (habits, compulsions, and addictions) include the striatum, midbrain dopaminergic nuclei, and regions of cortex that project to the striatum. These circuits are the primary focus of this review article, although it should be noted that the amygdala, thalamus, pallidum, and other limbic regions that are part of the broader basal ganglia circuitry are also involved in these behaviors. It has long been known that the striatum and its associated circuitry play a pivotal role in reinforcement learning and the development of behavioral automaticity found in habits, compulsions and addictions. The circuit composed of the ventral tegmental area (VTA) midbrain neurons projecting to the ventral striatum is considered to be the main circuit mediating reward and reward prediction error in the brain. Drugs of abuse target this circuit by either directly (e.g., nicotine) or indirectly (e.g., opioids) increasing midbrain dopamine neuron activity, and therefore enhancing dopamine signaling at release sites in the ventral striatum, or by directly inhibiting dopamine’s reuptake upon its release (e.g., cocaine; Lüscher, 2016). Thus, many studies of drug addiction have focused on neuroplastic changes that are induced in the ventral striatum following consumption of drugs of abuse (Lüscher and Malenka, 2011; Wolf, 2016). At the same time, habit formation has mostly been studied in the context of changes that occur in the dorsal striatum, which receives dopaminergic input from the Substantia Nigra Pars Compacta (SNc), while genetic mouse models of compulsion have focused on abnormal corticostriatal circuitry, largely involving dorsal striatum (Graybiel and Grafton, 2015; Smith and Graybiel, 2016). Thus, there has historically been a divided focus within the striatum, with ventral-striatal circuitry primarily investigated in the context of drug addiction, and dorsal-striatal circuitry in goal-directed and habitual reinforcement learning.
Over a decade ago, it was proposed that all of these instrumental behaviors ranging from habits to compulsions/addictions involve a shift in activity from the ventral to the dorsal striatum as habit learning progresses, and from the dorsomedial striatum to dorsolateral striatum as behavioral automaticity becomes more ingrained (Everitt and Robbins, 2005, 2013, 2016; Graybiel, 2008). The anatomy of corticostriatal circuits is well-suited to support such a mechanism, as the striatum is composed of spiraling loops through dopaminergic-striatal circuitry, ascending from the ventromedial to dorsolateral striatum (Haber et al., 2000; Haber, 2016). Here, we review the evidence that habits, compulsions and addictions are linked not only by their phenotype of behavioral automaticity but also by the underlying neural circuitry and plasticity mechanisms that give rise to them. This review article will focus on the essential role of dorsal-striatal circuits in encoding behavioral automaticity in several of its diverse manifestations.
Experimental Paradigms Used to Model Habits, Compulsions and Addictions
Two major experimental paradigms have dominated the rodent literature on habits: (a) over-training (Jog et al., 1999; Graybiel, 2008; Smith and Graybiel, 2014); and (b) random interval (RI) training (Dickinson, 1985; Hilário et al., 2007; Rossi and Yin, 2012; Robbins and Costa, 2017). In both paradigms, animals are trained on an instrumental learning task, in which they learn to perform an action in order to obtain a reward. In over-training, an association between the stimulus and action (i.e., response) is formed and strengthened over the course of many more trials than are necessary for learning the task. During this overtraining, the stimulus-response association overwhelms the initially stronger relationship between the rewarding outcome and the contingent action (Graybiel, 2008; Smith and Graybiel, 2014). The strength of the stimulus-response association vs. that of the response-outcome is measured as the persistence in learned action performance during extinction trials following devaluation of the reward (Dickinson, 1985; Rossi and Yin, 2012). Thus, the rate of action performance following devaluation is used as a metric to assess the degree to which animals have become habit-entrained. Experimentally, such reward devaluation is often achieved by satiating the subject on the reward or pairing the reward with an aversive stimulus.
Though over-training is intuitive and advantageous in the simplicity of the experimental paradigm and framework, it is noteworthy that by definition, overtraining requires experimental subjects to perform many more trials than control subjects. This discrepancy in trial number forces an imbalance in experience between subjects and controls that may complicate analysis of the neural signatures of habit formation. An alternative approach to experimentally weaken the contingency between action and reward is RI training (Dickinson, 1985; Rossi and Yin, 2012; Robbins and Costa, 2017). In RI training, animals are trained to perform a specific action for a reward, which becomes available when the animal first successfully performs the required action after a random time interval has elapsed since the presentation of the previous reward. This paradigm promotes persistent, habitual behavior, as it is difficult for the subject to develop a clear association between action and outcome. A commonly used reference paradigm for RI training is random ratio (RR) training (Rossi and Yin, 2012), in which the contingency between the action and reward is more direct. RR training largely promotes similar behavioral output to RI training (similar rate of actions), while retaining goal-directed behavior, sensitive to devaluation (Figure 1C). In both overtraining and RI/RR paradigms, the contingency between action and outcome, or reward, is impacted, producing goal-directed behavior when response-outcome contingency is high, or habitual behavior when response-outcome contingency is low and stimulus-response contingency is high.
Drug addiction is modeled in animals in two principal ways: the first is non-contingent administration, where drugs are given to animals without being dependent on the animal’s response. The second is contingent drug self-administration, where the drug is delivered in response to an operant behavior, such as pressing a lever (Wolf, 2016). While non-contingent cocaine administration is advantageous in the experimental control over the parameters of cocaine exposure, self-administration more closely approximates the human experience of drug seeking, where individuals seek out drug-associated stimuli and perform responses that previously led to drug consumption (Wolf, 2016). Similar to habit learning, in drug self-administration, compulsive drug seeking can be studied during extinction trials, which are imposed after performance has passed a predefined criterion. Furthermore, drug self-administration also enables investigation of the impact of prolonged drug abstinence, during which it has been found that the degree of craving for the drug increases, a phenomenon termed “incubation of craving” (Wolf, 2016).
Rodent models of compulsive behaviors are largely based on tracking the performance of repetitive, stereotyped and seemingly purposeless behaviors, such as compulsive grooming (Ahmari, 2016). Importantly, OCD-like behaviors can emerge spontaneously, without a clear antecedent stimulus (Ahmari, 2016). These behaviors are primarily observed to develop naturally in genetically mutant rodents, rather than be induced by repeated instrumental learning.
The Dorsolateral Striatum Plays a Key Role in Habit Formation and the Development of Compulsions/Addictions
The dorsal striatum is classically segregated into a medial aspect, the dorso-medial striatum (DMS), and a lateral aspect, the dorso-lateral striatum (DLS), both of which receive substantial cortical inputs. While the sensorimotor DLS receives major inputs from somatosensory and motor-cortical regions, the associative DMS receives major inputs from associative frontal cortical areas, such as orbitofrontal cortex (OFC; Berendse et al., 1979, 1992; Hintiryan et al., 2016; Hunnicutt et al., 2016). Classic studies have shown that the DMS is associated with goal-directed actions (Yin and Knowlton, 2004; Yin et al., 2005; Yin and Knowlton, 2006), while the DLS is associated with habitual actions (Balleine and Dickinson, 1998; Yin et al., 2004; Yin and Knowlton, 2006; Graybiel, 2008; Amaya and Smith, 2018; Figure 1D). Thus, goal-directed behavior is maintained after lesions to DLS (Yin et al., 2004; Yin and Knowlton, 2004, 2006), even following extended training, while lesions to DMS result in an early emergence of habitual behavior (Yin et al., 2005; Yin and Knowlton, 2006). The DLS has long been implicated in the performance of action sequences (O’Hare et al., 2018), both innate sequences such as grooming (Aldridge and Berridge, 1998), as well as acquired skills like learning to balance on an accelerating rotarod (Yin et al., 2009). These lesion-based studies provide the conceptual scaffold for our current understanding of the roles of the DMS and DLS in regulating goal-directed and habitual behavior.
Subsequently, a series of several influential studies on the roles of DMS and DLS in habit formation used tetrodes to track the activity patterns of neurons in the dorsal striatum while rats over-trained on a specific learning task: running a T-maze to obtain a food reward (Figure 1E). This led to the observation of task-bracketing patterns of activity in the DLS, which emerged concurrently with the acquisition of habitual behavior. In task-bracketing activity, highly-active DLS neurons have been reported to fire at the initiation and termination of the behavioral routine, an activity pattern that becomes strengthened with over-training (Jog et al., 1999; Barnes et al., 2005; Thorn et al., 2010; Smith and Graybiel, 2013; Figure 1E). Importantly, such task-bracketing, or action-sequence related activity in the DLS has also been observed in rats (Martiros et al., 2018) and mice (Jin and Costa, 2010; Jin et al., 2014) during a sequential lever-pressing task. A contrasting phenomenon is observed in the DMS, where neural activity is elevated more consistently throughout the performance of a behavioral routine, especially during the initial phases of acquisition of a novel instrumental behavior (Yin et al., 2009; Thorn et al., 2010; Gremel and Costa, 2013). This DMS activity then subsides as animals become over-trained (Yin et al., 2009; Gremel and Costa, 2013), corresponding to the time frame when task-bracketing activity emerges in the DLS. It should be noted that the task bracketing activity in DLS was observed in a subset of the most highly active neurons in this sub-region (Barnes et al., 2005; Martiros et al., 2018). Indeed, the majority of neurons in the DLS exhibit activity throughout the execution of the entire habit routine: in mice that were well-trained to habitually accelerate running on a treadmill to obtain a reward, neural activity was engaged in the DLS throughout the routine, with different striatal neurons encoding different sensorimotor features of the task (Rueda-orozco and Robbe, 2015).
Notably, multiple sources of evidence suggest that DLS control of habitual behavior and DMS control of goal-directed behavior likely develop in parallel and can varyingly compete or cooperate for control over actions (Daw et al., 2005; Yin and Knowlton, 2006; Gremel and Costa, 2013; Smith and Graybiel, 2014; Kupferschmidt et al., 2017; Robbins and Costa, 2017). For instance, inactivation of the DLS after the establishment of habitual behavior can restore goal-directed responding (Yin and Knowlton, 2006). Furthermore, DLS lesions or optogenetic silencing can expedite learning early in training (Bradfield and Balleine, 2013; Bergstrom et al., 2018), possibly by shifting control to goal-directed systems. Thus, a key transition thought to occur during the formation of habits is the relative quieting of activity in DMS, coincident with generally elevated activity in DLS, including task-bracketing (Thorn et al., 2010; Gremel and Costa, 2013).
In compulsions, the dorsal striatum also plays a central role, as several studies of genetic models of OCD, notably the SAPAP3−/− model, have indicated that activity in striatal circuits is disrupted coincident with the expression of compulsive behavior. As will be subsequently discussed, these studies focused on the striatal regions to which the orbitofrontal/secondary motor cortical areas project, encompassing the ventromedial (Ahmari et al., 2013), centromedial (Burguière et al., 2013), and central subregions of the dorsal striatum (Corbit et al., 2019). Additionally, there is evidence that the dorsolateral striatum is functionally necessary for the sequencing of compulsive grooming, as rats with lesions of the DLS express disruptions in the stereotypy of grooming sequences (Cromwell and Berridge, 1996; Kalueff et al., 2016).
In contrast to studies on habit formation and compulsions, centering mostly on the dorsal striatum, the majority of studies on drug addiction have focused on the mesolimbic, ventral striatal “reward” pathway (Lüscher and Malenka, 2011; Volkow and Morales, 2015; Wolf, 2016; Francis et al., 2019). Studies of the dorsal striatum that have addressed drug-seeking behavior (primarily in the study of alcohol and cocaine) have shown it to associate with a medial-lateral transition in neural activity in this subregion (Corbit, 2018). Prolonged cocaine self-administration in rats results in a persistence of cocaine seeking, even in the presence of active punishment (Vanderschuren and Everitt, 2004). During this cued cocaine self-administration, dopamine release is detected in the dorsal striatum (Ito et al., 2002), and inactivating the DLS blocks punishment-resistant seeking of drug-predicting cues (Jonkman et al., 2012). Indeed, while activity in ventral striatal circuits is clearly essential for the development of compulsive cocaine seeking, after prolonged administration, dorsal-striatal circuits become increasingly engaged, to support drug seeking (Belin and Everitt, 2008; Belin et al., 2009). Furthermore, once the dorsal striatum is engaged, there is a further activity shift, from DMS-centric to DLS-centric. Initially, drug seeking is goal-directed, and depends on a network involving the DMS (Corbit et al., 2012; Murray et al., 2014). However, after prolonged exposure, drug seeking becomes habitual, depending on neural activity and dopamine action in the DLS. Indeed, rats trained to press a lever for cocaine reward will reduce their lever pressing due to perfusion of dopamine receptor antagonists in DMS early in training and in DLS following over-training (Vanderschuren et al., 2005; Murray et al., 2012). This reduction in drug seeking was also observed in rats as a consequence of lidocaine-induced DLS inactivation (Zapata et al., 2010). Additionally, alcohol exposure has been reported to disinhibit Spiny Projection Neurons (SPNs) in the DLS, providing a potential mechanism for the transition to automaticity (Wilcox et al., 2014; Patton et al., 2016). In addition, the DLS has been shown to be necessary in rats for the development of habitual heroin seeking (Hodebourg et al., 2018). Furthermore, long-term exposure to nicotine alters synaptic plasticity in the DLS of rats, perturbing endocannabinoid-mediated long-term depression (LTD; Adermark et al., 2019). Thus, the dorsal striatum, and particularly the DLS, is implicated in the development of habitual drug-seeking. However, it should be emphasized that the amount of evidence on the role of the dorsal striatum in drug-addiction still lags behind what is known for the ventral striatum. Further research will help clarify the role of the dorsal striatum in addictive behaviors.
Corticostriatal Circuitry and Other Limbic Circuits Underlying Behavioral Automaticity
The striatum receives inputs from multiple cortical regions (Webster, 1961; Beckstead, 1979; Hintiryan et al., 2016; Hunnicutt et al., 2016), and prefrontal inputs to the striatum have been shown to play significant roles in both goal-directed, as well as habitual behavior (Gourley and Taylor, 2016; Smith and Laiks, 2017; Amaya and Smith, 2018). The major frontal structures that have been implicated in instrumental and automatic behaviors are the prelimbic cortex (PL) and infralimbic cortex (IL) Amaya and Smith, 2018 in the medial prefrontal cortex (mPFC), as well as the OFC located in the ventral part of the PFC.
Interestingly, the two substructures of the mPFC, the IL and PL, seem to play opposing roles in balancing between goal and habit, with the IL supporting habitual behavior, and the PL supporting goal-directed behavior (Smith and Laiks, 2017; Amaya and Smith, 2018). The IL exhibits task-bracketing activity, similar to the activity observed in the DLS during habit learning (Smith and Graybiel, 2013). Furthermore, chronic perturbation of the IL disrupts both habit acquisition and expression (Smith et al., 2012; Smith and Graybiel, 2013), while its optogenetic inhibition disrupts habit expression (Smith et al., 2012).
Meanwhile, lesions to the PL of rats reduced their ability to act in a goal-directed manner, biasing the rats toward habitual behavior (Balleine and Dickinson, 1998; Corbit and Balleine, 2003; Killcross and Coutureau, 2003; Balleine and O’Doherty, 2010). Indeed, recent studies in rats have shown that PL inputs to the posterior DMS (pDMS) are necessary for goal-directed learning: in rats lacking this PL-pDMS connection, there is a failure to reduce instrumental responding after reward devaluation (Hart et al., 2018a,b). Thus, reducing the strength of the PL- input to the DMS might permit the development of automaticity, mediated through sensorimotor corticostriatal circuits converging on the DLS. Indeed, reduced activity of PL neurons was observed in rats that underwent extended training for cocaine self-administration; meanwhile, stimulating PL neurons reduced the extent of compulsive cocaine seeking in these compulsively self-administering rats (Chen et al., 2013). Together, these data make a strong case that activity in the IL is important for habitual behavior, while PL activity facilitates goal-directed behavior.
However, many reports complicate this simple IL = habit; PL = goal-directed view. For instance, the PL is reported to be involved in facilitating post-extinction reinstatement of drug-seeking. This reinstatement of drug-responding can be elicited by re-exposure to drug-associated cues, consumption of the drug itself, or a stressful experience (McFarland and Kalivas, 2001; McFarland et al., 2004; Gipson et al., 2013; Ma et al., 2014; Moorman et al., 2015; Gourley and Taylor, 2016; McGlinchey et al., 2016). At the same time, there is evidence supporting a role for the IL in driving drug-cue extinction learning (Peters et al., 2008; Ma et al., 2014; Moorman et al., 2015; Gourley and Taylor, 2016; Gutman et al., 2017), as opposed to habit-expression. Together, these results suggest that the PL, in general, mediates a “go” signal, driving drug-seeking responses, particularly during post-extinction reinstatement, whereas in contrast, the IL sends a “no-go” signal, necessary for extinction in drug-reward instrumental learning (Moorman et al., 2015; Gourley and Taylor, 2016). These results are potentially conflicting with the habit-literature, as IL promotes extinction of responding in the drug-reward paradigm, and seems to facilitate responding in habit learning paradigms, while PL also can play contrasting roles in each paradigm. One possible explanation for this discrepancy is that where specific projections from mPFC (PL and IL) to striatum are examined in drug-seeking, they are those to the ventral striatum (McFarland and Kalivas, 2001; Peters et al., 2008; Ma et al., 2014; Gourley and Taylor, 2016). Conversely, in habit formation, the projections from PL/IL to regions of dorsal striatum have been given more attention (Smith and Laiks, 2017; Hart et al., 2018a,b).
The OFC also plays an important role in instrumental behaviors, with evidence appearing to support the idea of the OFC promoting goal-directed behavior. However, the OFC is a large cortical structure, with multiple subregions, and its roles in instrumental behavior and economic choice appear to be varied and complex (Stalnaker et al., 2015; Gremel et al., 2016; Gardner et al., 2018; Panayi and Killcross, 2018; Zhou et al., 2019). The OFC receives multisensory input (Gourley and Taylor, 2016), projects to the anterior/intermediate DMS and central region of the striatum, and has been shown to exhibit activity that correlates with the reward assigned to a given stimulus (Zhou et al., 2019). The OFC exhibits greater activity during goal-directed behavior, and, similar to DMS neurons, is particularly active during random-ratio lever-pressing training, when action-reward contingency is high (Gremel and Costa, 2013; Gremel et al., 2016). OFC stimulation can increase the degree to which mice are goal-directed, and reduce the degree to which mice are habit-driven in lever-pressing (Gremel et al., 2016). Furthermore, endocannabinoid-dependent (eCB)-LTD of the OFC inputs to the DMS biases mice towards habitual behavior, providing further evidence for a competition between goal-directed and habitual behavior—such that if the activity of the OFC-DMS pathway is decreased (e.g., through eCB-LTD), then the DLS pathway prevails, promoting habitual behavior (Gremel et al., 2016).
Interestingly, OFC-striatal circuits are also implicated in compulsive behavioral automaticity. Abnormalities of the structure, connectivity and activity of the caudate (the human DMS) have been observed in OCD patients (Carmin et al., 2002; Guehl et al., 2008; Sakai et al., 2011; Fan et al., 2012). Furthermore, three genetic mouse models of OCD have been characterized (D1CT-7; SAPAP3−/− and Slitrk5−/−), and in each of them, the major circuit phenotype observed has been disruption of cortico-striatal synaptic transmission, particularly involving inputs from OFC (Nordstrom and Burton, 2002; Welch et al., 2007; Shmelkov et al., 2010; Burguière et al., 2013, 2015). Indeed, chronic activation of the medial OFC leads to the development of OCD-like grooming behavior in mice, and drives sustained activity of ventromedial striatal SPNs (Ahmari et al., 2013). In contrast, optogenetic stimulation of the lateral OFC (lOFC) has been reported to reduce the occurrence of grooming behaviors in genetically modified mice that compulsively over-groom, while activating feed-forward inhibition within the striatum (Burguière et al., 2013). Furthermore, a recent report compared lateral OFC-striatal circuit activity to the activity in projections from neighboring M2 cortex, in the SAPAP3−/− mouse model of OCD. They found that in the SAPAP3−/− mutant, lOFC input to striatal SPNs was reduced in strength, while M2 input to both SPNs and fast-spiking interneurons (FSIs) in striatum was increased 6-fold, suggesting that it is M2, and not lOFC inputs, that drive compulsive grooming (Corbit et al., 2019). Meanwhile, another study found that compulsive consumption of ethanol resulted in reduced OFC input to D1R-expressing DMS neurons during ethanol withdrawal, reducing goal-directed behavior, and resulting in habitual alcohol consumption (Renteria et al., 2018). Thus, many of these recent results suggest that OFC hypoactivity corresponds with automatic behavior and at least in some cases, activating OFC projections can counteract this automaticity, rather than drive it. However, in another recent article describing a mouse model of addiction (based on self-stimulation of VTA-dopamine neurons), potentiation of synapses from the lOFC to the central part of dorsal striatum was observed (Pascoli et al., 2018). Thus, while there is significant literature documenting the involvement of OFC projections to striatum in behavioral automaticity, the OFC appears to play varied roles in either facilitating or countering automaticity. Therefore, further research is required in order to clarify the principles of OFC-striatal connections and their role in driving and/or inhibiting automatic behavior.
As another main input source to the striatum, midbrain dopamine neurons are an essential component of the reward circuitry, and such neurons in both the VTA and SNc send collaterals to the striatum, PFC, and other forebrain targets (Volkow and Morales, 2015; Everitt and Robbins, 2016; Lüscher, 2016). Dopamine is a crucial modulator of striatal action and the transition from goal-directed to habitual behavior (Graybiel, 2008; Everitt and Robbins, 2016). It is well established that the cellular activity of midbrain dopamine neurons is increased upon exposure to rewarding drugs, in large part due to the strengthening of synaptic inputs onto these dopamine neurons (Ungless et al., 2001; Lammel et al., 2011; Creed et al., 2016; Francis et al., 2019). Plasticity mechanisms are also engaged within midbrain dopamine neurons during the formation of a naturally rewarded (i.e., food-reward) habit, as habitual responding after devaluation on a random-interval lever-press habit depends on this population’s expression of NMDA receptors (Wang et al., 2011).
Finally, an additional striatum-associated structure that has been implicated in habitual and addictive behavior is the amygdala (Lingawi and Balleine, 2012). Conceptually, the amygdalar connection is intriguing, as habit formation is exacerbated by stress (Dias-Ferreira et al., 2009), in a process that may be mediated by amygdalar-striatal circuits. One recent study demonstrated that both the basolateral and central amygdala (BLA and CeA) exert control over habitual behavior in rats; the BLA was found to be involved in habitual responding early in training, with the CeA playing a crucial role in generating habitual responding later in extended training (Murray et al., 2015). These amygdalar circuits, and the BLA in particular, play a key role in assigning valence, and have been shown to play a role in appetitive behaviors (Kim et al., 2017) while the CeA has been shown to play a role in alcohol addiction (de Guglielmo et al., 2019). Neither nucleus has direct connections to the DLS (Murray et al., 2015; Hunnicutt et al., 2016), and therefore the amygdala likely influences the DLS through multisynaptic connections. Given the direct projection of BLA neurons to the ventral striatum, these amygdalar circuits could influence dorsal striatal circuitry via ventral striatum (Murray et al., 2015).
Overall, we have focused on the brain regions that represent key nodes in the circuitry of habitual and compulsive behavior. Eventually however, continued and disordered performance of instrumental behaviors, particularly as occurs in chronic drug use, leads to alterations in reward and attentional related networks that likely involve changes to additional brain structures, such as the ventral hippocampus, and insular cortex (Everitt and Robbins, 2016). Other key structures involved in broader basal ganglia circuits also likely play important roles in encoding behavioral automaticity. For instance, thalamus sends a significant projection to striatum (Hunnicutt et al., 2016), and specific projections from thalamic nuclei to the DMS are necessary for goal-oriented behavioral flexibility (Bradfield et al., 2013; Díaz-Hernández et al., 2018).
Striatal Cell-Types, Microcircuits, and Their Specific Contributions to Habits and Compulsions
Within the striatum the vast majority of neurons (>90%) are SPNs, which are roughly evenly split between Dopamine D1 receptor (Drd1)-expressing direct pathway SPNs (dSPNs; projecting directly to the midbrain nucleus, Substantia Nigra reticulata, or SNr, as well as Globus Pallidus internus, or GPi) and Drd2-expressing indirect pathway SPNs (iSPNs; projecting to the Globus Pallidus externus, or GPe; Kreitzer and Malenka, 2008; Burke et al., 2017). The striatum also contains populations of interneurons, including Cholinergic (ChAT) and Parvalbumin-expressing Fast-Spiking Interneurons (PV+ FSIs) (Kreitzer and Malenka, 2008; Burke et al., 2017).
Over the past decade, progress has been made in deciphering the roles of dSPNs vs. iSPNs in motor behavior, action initiation, and reinforcement learning, all of which are combined to produce habitual and compulsive behaviors. A decade ago, a seminal study confirmed the prevalent assumption in the field that dSPNs in the direct pathway serve to promote actions/behaviors, while iSPNs in the indirect pathway inhibited behaviors (Kravitz et al., 2010; Bariselli et al., 2019). However, it is now apparent that dSPNs and iSPNs are concurrently activated during the initiation of actions (Cui et al., 2013; Tecuapetla et al., 2014, 2016), and thus the role of iSPNs seems to be more complex than simple broad behavioral inhibition (Tecuapetla et al., 2016; Vicente et al., 2016; Parker et al., 2018; Bariselli et al., 2019). Moreover, patterns of activity in locally concentrated clusters of both dSPNs and iSPNs have been recently observed to correspond to specific actions, like turning left or right (Barbera et al., 2016; Klaus et al., 2017; Markowitz et al., 2018; Parker et al., 2018). Still, several studies have found that dSPNs are activated with shorter latency than iSPNs during action initiation (Sippy et al., 2015; O’Hare et al., 2016). Meanwhile, other studies have demonstrated that dSPN activation reinforces the performance of specific action patterns (Sippy et al., 2015; Vicente et al., 2016), while iSPN activation might weakly reinforce actions more generally (Vicente et al., 2016) in some contexts, and inhibit action performance in others (Kravitz et al., 2010; Sippy et al., 2015). Thus, both dSPNs and iSPNs are likely to be engaged in both the learning and the execution of a habit, with dSPN activity likely to promote action performance, and iSPN activity likely to play an action-specific inhibitory and/or permissive role (Zalocusky et al., 2016; Parker et al., 2018; Bariselli et al., 2019). How exactly these SPN pathways coordinate and are modified during instrumental learning is currently still a topic of active research (Bariselli et al., 2019).
In addition to SPNs, recent studies in rodents have also implicated FSIs in the development of habits (Thorn and Graybiel, 2014; O’Hare et al., 2017; Martiros et al., 2018). For instance, FSIs are active during the middle phase of a lever-pressing motor sequence pattern, when the activity of task-bracketing SPNs is reduced (Martiros et al., 2018). In the context of compulsive behavior, in one of the OCD mouse models (SAPAP3−/−), a reduction in the number of striatal PV neurons was observed, leading to a reduction in feed-forward inhibition, potentially reducing inhibition of cortico-striatal inputs (Burguière et al., 2013). A reduction in striatal PV neurons has also been reported in patients suffering from Tourette’s syndrome (Kalanithi et al., 2005), a syndrome of ritualized, repetitive actions. Furthermore, selective ablation of striatal PV interneurons in mice has been reported to lead to increased stereotypic grooming, a measure of OCD-like behavior in rodents (Kalueff et al., 2016). In all of these examples, reduced activity of FSI interneurons leads to increased SPN activity, potentially leading to the promotion of automatic behaviors. In addition, striatal cholinergic interneurons also play a significant role in modulating SPN plasticity (Augustin et al., 2018), and are thought to mediate thalamic influence on striatal circuits involved in goal-directed behaviors (Bradfield et al., 2013; Peak et al., 2019).
Synaptic and Molecular Changes in Limbic Circuits for Behavioral Automaticity
In the context of addiction, significant progress has been made in determining how drugs of abuse affect synaptic plasticity in the mesolimbic ventral-striatal reward system, involving the VTA and ventral striatum, or Nucleus Accumbens (NAc). These mechanisms are extensively summarized elsewhere (Citri and Malenka, 2008; Lüscher and Malenka, 2011; Lüscher, 2016; Wolf, 2016; Francis et al., 2019). Yet, in the context of this review, there are several important principles to emerge that are worth mentioning. First, synaptic plasticity mechanisms in both the VTA and NAc involve dopamine and NMDAR-receptor dependent long-term plasticity (Ungless et al., 2001; Saal et al., 2003; Conrad et al., 2008; Lüscher and Malenka, 2011; Wolf, 2016). Second, these changes are input-specific, occurring at particular synaptic inputs onto VTA or NAc neurons (Lammel et al., 2011; Ma et al., 2014; MacAskill et al., 2014; Pascoli et al., 2014; Wolf, 2016; Barrientos et al., 2018). Finally, plasticity following exposure to drugs of abuse is dynamically regulated (Thomas et al., 2001; Kourrich et al., 2007; Lüscher and Malenka, 2011; Wolf, 2016). These rules of cellular and synaptic plasticity in the VTA-NAc circuit could provide a useful template for how mechanisms of plasticity in DLS circuitry might proceed.
Focusing on the dorsal striatum and natural reward habits, synaptic modulation has been observed in accordance with behavioral automaticity, principally at corticostriatal synapses. Indeed, the acquisition of goal-directed actions has been associated with synaptic plasticity at corticostriatal synapses within the DMS, enhancing transmission onto dSPNs, while weakening inputs onto iSPNs (Shan et al., 2014). Meanwhile, in mouse brain slices of habit-entrained mice, it was observed that inputs onto both dSPNs and iSPNs in dorsal striatum were strengthened, though inputs to dSPNs were activated with a shorter latency and moreover, habit suppression correlated with reduced activity of only dSPNs (O’Hare et al., 2016). Furthermore, glutamatergic synapses from secondary motor cortex onto DLS dSPNs (and not iSPNs) were observed to be strengthened with learning of simple sequences (Rothwell et al., 2015). All these studies suggest a selective modification of corticostriatal-dSPN synapses. However, during the learning of a rotorod-balancing skill, it was found that synaptic strength onto iSPNs in the DLS strengthened with training and was crucial for acquisition of skilled balancing (Yin et al., 2009), and so corticostriatal-iSPN synapses are likely important as well. In the studies mentioned thus far, synaptic changes recorded were post-synaptic. Yet, one elegant study, also examining striatal inputs in mice during rotorod balancing, found learning-induced activity differences in somata vs. pre-synaptic terminals from mPFC and M1 corticostriatal neurons, suggesting neuroplastic changes that were specific to pre-synaptic terminals during learning (Kupferschmidt et al., 2017). In the context of compulsions, in the Sapap3 mutant mice, which exhibit increased grooming, reduced synaptic transmission of corticostriatal synapses onto dSPNs (but not iSPNs) was observed, as measured by mESPC frequency (Wan et al., 2013). This finding is consistent with much of the learned skill/habit literature. To summarize, synaptic changes have been observed to occur in dorsal striatum during the learning of both goal-directed and habitual behaviors, mostly strengthening inputs onto DMS and DLS neurons, respectively. Clearly though, much more research remains to be done to decipher how habits and compulsions result from the modification of cell-type specific synapses within striatum, e.g., inputs to dSPNs, iSPNs, and local interneurons in striatum.
Facing Forward
In this review article, we have summarized the overlapping dorsal-striatal-centric circuitry responsible for learning habits, addictions, and compulsions, highlighting the transition from DMS to DLS as behaviors become more automatic. With this overarching framework in mind, we examine future directions concerning the mechanisms of behavioral automaticity and propose how our current understanding of different features of striatal circuit organization can be combined with novel molecular tools to provide insight into the central questions in the field. One crucial question is how dispersed is the representation of a given automatic behavior within the dorsal striatum? If the shift to automaticity involves the transition from DMS- to DLS-centric circuits, then is the same S-R behavior encoded simultaneously in medial and lateral locations, and furthermore what particular cells and synapses correspond to the storage of a given association?
A compelling hypothesis is that the long-range input/output connectivity (and local circuit structure) of a cluster of striatal neurons defines its recruitment to encoding a given S-R behavioral association (e.g., associating an auditory cue with a lever press response). Recently, it has been appreciated that unique patterns of dSPN and iSPN activity in locally concentrated clusters of SPNs correlate with the performance of specific actions (Barbera et al., 2016; Klaus et al., 2017; Markowitz et al., 2018), and that individual DLS neurons exhibit sensorimotor-relevant activity during habit performance (Rueda-orozco and Robbe, 2015). It is already known that different subregions of striatum are organized in overlapping topographic domains according to cortical input (Beckstead, 1979; Berendse et al., 1979; Hintiryan et al., 2016; Hunnicutt et al., 2016). Thus, there are multiple different dimensions along which striatal cells can be classified (depicted as dimensions, layers or “masks,” in Figure 2). One can define a striatal cell by its spatial location (Figure 2A), its neurotransmitter/cell-type identity (Figure 2B), its connectivity (Figure 2C) or its behavioral association (Figure 2D). The intersection of these dimensions is expected to define striatal ensembles encoding specific actions. Thus, a putative requirement for creating and strengthening a given behavioral S-R association might be the strengthening of specific connections between cortical neurons responsible for the representation of specific sensory inputs, and action-relevant cells in the striatum. The somatosensory organization of the striatum, which has recently been highlighted (Robbe, 2018), suggests that different actions utilize topographically dispersed ensembles of striatal neurons. Yet, these different ensembles very likely use common rules of local circuit organization and plasticity (Bamford et al., 2018; Bariselli et al., 2019) as dictated by the relatively uniform cell-type composition of the striatum.
Figure 2. Functional definitions of striatal neurons. (A–D) Different dimensions/layers/’masks’ describing striatal neurons. (A) Striatal subregion. (B) Molecular/genetic: principal striatal cell types include Drd1+ SPNs, Drd2+ SPNs, PV+ FSIs, ChAT+ cholinergic interneurons, and several other important subtypes of interneuron populations. (C) Homuncular: striatal cells preferentially receive inputs from different regions of cortex. Sensorimotor inputs corresponding to specific body parts map to specific regions of the striatum adapted from Robbe (2018). (D) Task-specific recruitment: segregated clusters of neurons recruited by specific behavioral sequences (Behavior A vs. Behavior B) are shown.
To comprehensively map the exact circuits encoding a given specific S-R association, implementation of large-scale mapping of immediate-early gene (IEG) expression (using FISH and single-cell RNA-seq) will be invaluable. To date, many studies have examined neural activity in single brain regions, using tetrode recordings or calcium imaging, where at most hundreds of cells can be monitored. The unbiased identification of neuronal activity in basal-ganglia relevant neuronal populations and their genetic identity will be accelerated with scRNAseq, smFISH, and similar molecular techniques, followed by approaches using targeted recording of neuronal activity in defined neuronal populations (Jun et al., 2017). Such experiments will facilitate progress in localizing a specific behavior within basal ganglia circuitry. It would be especially exciting to find a specific serial path of connectivity: i.e., from a distinct cortical input through the relevant subset of striatal cells and finally to a unique output in downstream brain areas.
This achievement will enable investigators to ask crucial questions about cellular and synaptic plasticity in behavioral automaticity. Since the striatum is composed of repeating microcircuit elements, common rules are likely to prevail for the encoding of diverse actions within the striatum. Some major questions are: during the encoding of a habit, compulsion, or addiction, is the activity of dSPNs or iSPNs modulated to a greater degree? Do dSPNs and iSPNs representing the same behavior sit adjacent, in the same locally concentrated cluster? If so, do they vie for control over the same behavior, or do iSPNs primarily function to inhibit competing behaviors (Tecuapetla et al., 2016; Vicente et al., 2016; Bariselli et al., 2019)?
Once the ensemble representation of a defined S-R trace has been clearly demarcated, it will accelerate the investigation into the rules governing microcircuit organization and plasticity, as has been partially achieved recently by isolating the trace of a particular auditory stimulus within the striatum (Xiong et al., 2015; Chen et al., 2019). With some notable exceptions (e.g., Gremel and Costa, 2013), most studies have primarily examined differences in circuit properties between animals that are habit-trained vs. control animals. Ideally, one would be able to target, record and manipulate specific subsets of behaviorally relevant (Figure 2D; Markowitz et al., 2018; Bariselli et al., 2019) striatal cells according to their anatomical/“humuncular” projection patterns (Figures 2A,B; Hintiryan et al., 2016; Hunnicutt et al., 2016) and compare them to adjacent (task-irrelevant) neurons in the same animal.
In order to realize this goal, one can gain genetic access to cells participating in a given S-R association, by utilizing activity-dependent, cell-specific targeting approaches such as TRAP mice (Guenthner et al., 2013; Luo et al., 2018; Figure 2D). Similarly, connectivity-based cellular targeting (Schwarz et al., 2015; Luo et al., 2018), will enable genetic access to striatal neurons that exhibit specific input/output architecture (Figure 2C). Intersectional genetic techniques will then allow the targeting of the overlap of these two dimensions, with sub-region and cell-type resolution. Adoption of these genetic techniques will permit investigators to identify cell-specific intrinsic and synaptic plasticity within the striatum induced by a particular S-R.
Next, it will be important to test the necessity of activity patterns in genetically targeted neurons for the encoding and actuating of particular behaviors. For instance, during the development of habitual cued lever-pressing, how necessary are the striatal cells active during lever-pressing for expression of this behavior? Using optogenetic and chemogenetic approaches in combination with cell-specific targeting tools, it can be tested whether the activity of a particular ensemble or synapse-type is indispensable for a given automatic behavior and whether activation of the ensemble can induce it.
Finally, a rapidly increasing body of evidence acquired from humans with genetic mutations (Hancock et al., 2018) and adverse life experiences (Corbit, 2018; Wirz et al., 2018) that are predisposing to compulsive and addictive disorders provide further opportunities for understanding the mechanisms underlying behavioral automaticity. Here, the use of CRISPR to simulate human disease in model organisms could facilitate substantial progress in modeling and potentially reversing the pathological disorders of habitual behavior. We anticipate that increased neural circuit insight into automatic behaviors will advance treatments for human disease. Recent progress in the study of drug addiction can serve as a guiding light in this regard, as recent therapeutic approaches have been developed based on the circuit-level understanding of the plasticity induced by exposure to drugs of abuse (Creed et al., 2015; Lüscher et al., 2015; Terraneo et al., 2016).
Habit formation, expression, and related disorders are among the most fundamental topics in behavioral neuroscience, and significant progress has been made in this field. We anticipate that the next decade of research into the roles of cortico-basal ganglia circuits in supporting behavioral automaticity will involve integrating innovative molecular techniques and overlaying the different anatomical and functional representations of striatal organization. Such combined high-resolution approaches will be instrumental in pinpointing specific circuits and synapses, as well as defining basic rules of microcircuit function within the vast cortico-basal ganglia circuitry driving the development and expression of habits, compulsions, and addictions.
Author Contributions
DL, BG and AC wrote the manuscript.
Funding
The Citri lab is supported by funding from the H2020 European Research Council (ERC-CoG-770951); The Israel Science Foundation (393/12; 1796/12; 1062/18); The Canadian Institute for Advanced Research, The European Union FP7 People: Marie Curie Fellowship (PCIG13-GA-2013-618201); The Israeli Ministry of Public Security; The National Institute for Psychobiology in Israel, Hebrew University of Jerusalem, generous donations of the Resnick and Cohen families, and start-up funds provided by the Edmond and Lily Safra Center for Brain Sciences. DL is supported by a Zuckerman postdoctoral scholarship.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Adermark, L., Morud, J., Lotfi, A., Ericson, M., and Söderpalm, B. (2019). Acute and chronic modulation of striatal endocannabinoid-mediated plasticity by nicotine. Addict. Biol. 24, 355–363. doi: 10.1111/adb.12598
Ahmari, S. E. (2016). Using mice to model obsessive compulsive disorder: from genes to circuits. Neuroscience 321, 121–137. doi: 10.1016/j.neuroscience.2015.11.009
Ahmari, S. E., Spellman, T., Douglass, N. L., Kheirbek, M. A., Simpson, H. B., Deisseroth, K., et al. (2013). Repeated cortico-striatal stimulation generates persistent OCD-like behavior. Science 340, 1234–1239. doi: 10.1126/science.1234733
Aldridge, J. W., and Berridge, K. C. (1998). Coding of serial order by neostriatal neurons: a “natural action” approach to movement sequence. J. Neurosci. 18, 2777–2787. doi: 10.1523/JNEUROSCI.18-07-02777.1998
Amaya, K. A., and Smith, K. S. (2018). Neurobiology of habit formation. Curr. Opin. Behav. Sci. 20, 145–152. doi: 10.1016/j.cobeha.2018.01.003
Augustin, S. M., Chancey, J. H., and Lovinger, D. M. (2018). Dual dopaminergic regulation of corticostriatal plasticity by cholinergic interneurons and indirect pathway medium spiny neurons. Cell Rep. 24, 2883–2893. doi: 10.1016/j.celrep.2018.08.042
Balleine, B. W., and Dickinson, A. (1998). Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37, 407–419. doi: 10.1016/s0028-3908(98)00033-1
Balleine, B. W., and O’Doherty, J. P. (2010). Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35, 48–69. doi: 10.1038/npp.2009.131
Bamford, N. S., Wightman, R. M., and Sulzer, D. (2018). Review Dopamine’s effects on corticostriatal synapses during reward-based behaviors. Neuron 97, 494–510. doi: 10.1016/j.neuron.2018.01.006
Barbera, G., Liang, B., Zhang, L., Gerfen, C. R. R., Culurciello, E., Chen, R., et al. (2016). Spatially compact neural clusters in the dorsal striatum encode locomotion relevant information. Neuron 92, 202–213. doi: 10.1016/j.neuron.2016.08.037
Bariselli, S., Fobbs, W. C., Creed, M. C., and Kravitz, A. V. (2019). A competitive model for striatal action selection. Brain Res. 1713, 70–79. doi: 10.1016/j.brainres.2018.10.009
Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z., and Graybiel, A. M. (2005). Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158–1161. doi: 10.1038/nature04053
Barrientos, C., Knowland, D., Wu, M. M. J., Lilascharoen, V., Huang, K. W., Malenka, R. C., et al. (2018). Cocaine-induced structural plasticity in input regions to distinct cell types in nucleus accumbens. Biol. Psychiatry 84, 893–904. doi: 10.1016/j.biopsych.2018.04.019
Beckstead, R. M. (1979). Convergent prefrontal and nigral projections to the striatum of the rat. Neurosci. Lett. 12, 59–64. doi: 10.1016/0304-3940(79)91480-0
Belin, D., and Everitt, B. J. (2008). Cocaine seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron 57, 432–441. doi: 10.1016/j.neuron.2007.12.019
Belin, D., Jonkman, S., Dickinson, A., Robbins, T. W., and Everitt, B. J. (2009). Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav. Brain Res. 199, 89–102. doi: 10.1016/j.bbr.2008.09.027
Berendse, H. W., Graaf, Y. G., and Groenewegen, H. J. (1992). Topographical organization and relationship with ventral striatal compartments of prefrontal corticostriatal projections in the rat. J. Comp. Neurol. 316, 314–347. doi: 10.1002/cne.903160305
Berendse, H. W., Graaf, Y. G., Groenewegen, H. J., Sesack, S. R., Deutch, A. Y., Roth, R. H., et al. (1979). Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: an anterograde tract-tracing study with Phaseolus vulgaris leucoagglutinin. J. Comp. Neurol. 316, 213–242.
Bergstrom, H. C., Lipkin, A. M., Lieberman, A. G., Pinard, C. R., Gunduz-Cinar, O., Brockway, E. T., et al. (2018). Dorsolateral striatum engagement interferes with early discrimination learning. Cell Rep. 23, 2264–2272. doi: 10.1016/j.celrep.2018.04.081
Bradfield, L. A., and Balleine, B. W. (2013). Hierarchical and binary associations compete for behavioral control during instrumental biconditional discrimination. J. Exp. Psychol. Anim. Behav. Process. 39, 2–13. doi: 10.1037/a0030941
Bradfield, L. A., Bertran-Gonzalez, J., Chieng, B., and Balleine, B. W. (2013). The thalamostriatal pathway and cholinergic control of goal-directed action: interlacing new with existing learning in the striatum. Neuron 79, 153–166. doi: 10.1016/j.neuron.2013.04.039
Burguière, E., Monteiro, P., Feng, G., Graybiel, A. M., Burguière, E., Monteiro, P., et al. (2013). Optogenetic stimulation of lateral orbitofronto-striatal pathway suppresses compulsive behaviors. Science 340, 1243–1246. doi: 10.1126/science.1232380
Burguière, E., Monteiro, P., Mallet, L., Feng, G., and Graybiel, A. M. (2015). Striatal circuits, habits, and implications for obsessive-compulsive disorder. Curr. Opin. Neurobiol. 30, 59–65. doi: 10.1016/j.conb.2014.08.008
Burke, D. A., Rotstein, H. G., and Alvarez, V. A. (2017). Striatal local circuitry: a new framework for lateral inhibition. Neuron 96, 267–284. doi: 10.1016/j.neuron.2017.09.019
Carmin, C. N., Wiegartz, P. S., Yunus, U., and Gillock, K. L. (2002). Treatment of late-onset OCD following basal ganglia infarct. Depress. Anxiety 15, 87–90. doi: 10.1002/da.10024
Chen, B. T., Yau, H.-J., Hatch, C., Kusumoto-Yoshida, I., Cho, S. L., Hopf, F. W., et al. (2013). Rescuing cocaine-induced prefrontal cortex hypoactivity prevents compulsive cocaine seeking. Nature 496, 359–362. doi: 10.1038/nature12024
Chen, L., Wang, X., Ge, S., and Xiong, Q. (2019). Medial geniculate body and primary auditory cortex differentially contribute to striatal sound representations. Nat. Commun. 10:418. doi: 10.1038/s41467-019-08350-7
Citri, A., and Malenka, R. C. (2008). Synaptic plasticity: multiple forms, functions, and mechanisms. Neuropsychopharmacology 33, 18–41. doi: 10.1038/sj.npp.1301559
Conrad, K. L., Tseng, K. Y., Uejima, J. L., Reimers, J. M., Heng, L.-J., Shaham, Y., et al. (2008). Formation of accumbens GluR2-lacking AMPA receptors mediates incubation of cocaine craving. Nature 454, 118–121. doi: 10.1038/nature06995
Corbit, L. H. (2018). Understanding the balance between goal-directed and habitual behavioral control. Curr. Opin. Behav. Sci. 20, 161–168. doi: 10.1016/j.cobeha.2018.01.010
Corbit, L. H., and Balleine, B. W. (2003). The role of prelimbic cortex in instrumental conditioning. Behav. Brain Res. 146, 145–157. doi: 10.1016/j.bbr.2003.09.023
Corbit, V. L., Manning, E. E., Gittis, A. H., and Ahmari, S. E. (2019). Strengthened inputs from secondary motor cortex to striatum in a mouse model of compulsive behavior. J. Neurosci. 39, 2965–2975. doi: 10.1523/JNEUROSCI.1728-18.2018
Corbit, L. H., Nie, H., and Janak, P. H. (2012). Habitual alcohol seeking: time course and the contribution of subregions of the dorsal striatum. Biol. Psychiatry 72, 389–395. doi: 10.1016/j.biopsych.2012.02.024
Creed, M., Kaufling, J., Fois, G. R., Jalabert, M., Yuan, T., Lüscher, X. C., et al. (2016). Cellular/molecular cocaine exposure enhances the activity of ventral tegmental area dopamine neurons via calcium-impermeable NMDARs. J. Neurosci. 36, 10759–10768. doi: 10.1523/JNEUROSCI.1703-16.2016
Creed, M., Pascoli, V. J., and Lüscher, C. (2015). Refining deep brain stimulation to emulate optogenetic treatment of synaptic pathology. Science 347, 659–664. doi: 10.1126/science.1260776
Cromwell, H. C., and Berridge, K. C. (1996). Implementation of action sequences by a neostriatal site: a lesion mapping study of grooming syntax. J. Neurosci. 16, 3444–3458.
Cui, G., Jun, S. B., Jin, X., Pham, M. D., Vogel, S. S., Lovinger, D. M., et al. (2013). Concurrent activation of striatal direct and indirect pathways during action initiation. Nature 494, 238–242. doi: 10.1038/nature11846
Daw, N. D., Niv, Y., and Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711. doi: 10.1038/nn1560
de Guglielmo, G., Kallupi, M., Pomrenze, M. B., Crawford, E., Simpson, S., Schweitzer, P., et al. (2019). Inactivation of a CRF-dependent amygdalofugal pathway reverses addiction-like behaviors in alcohol-dependent rats. Nat. Commun. 10:1238. doi: 10.1038/s41467-019-09183-0
Dias-Ferreira, E., Sousa, J. C., Melo, I., Morgado, P., Mesquita, A. R., Cerqueira, J. J., et al. (2009). Chronic stress causes. Science 325, 621–625. doi: 10.1126/science.1171203
Díaz-Hernández, E., Contreras-López, R., Sánchez-Fuentes, A., Rodríguez-Sibrían, L., Ramírez-Jarquín, J. O., and Tecuapetla, F. (2018). The thalamostriatal projections contribute to the initiation and execution of a sequence of movements. Neuron 100, 739.e5–752.e5. doi: 10.1016/j.neuron.2018.09.052
Dickinson, A. (1985). Actions and habits: the development of behavioural autonomy. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 308, 67–78. doi: 10.1098/rstb.1985.0010
Ersche, K. D., Gillan, C. M., Jones, P. S., Williams, G. B., Ward, L. H. E., Luijten, M., et al. (2016). Carrots and sticks fail to change behavior in cocaine addiction. Science 352, 1468–1471. doi: 10.1126/science.aaf3700
Everitt, B. J., and Robbins, T. W. (2005). Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 8, 1481–1489. doi: 10.1038/nn1579
Everitt, B. J., and Robbins, T. W. (2013). From the ventral to the dorsal striatum: devolving views of their roles in drug addiction. Neurosci. Biobehav. Rev. 37, 1946–1954. doi: 10.1016/j.neubiorev.2013.02.010
Everitt, B. J., and Robbins, T. W. (2016). Drug addiction: updating actions to habits to compulsions ten years on. Annu. Rev. Psychol. 67, 23–50. doi: 10.1146/annurev-psych-122414-033457
Fan, Q., Yan, X., Wang, J., Chen, Y., Wang, X., Li, C., et al. (2012). Abnormalities of white matter microstructure in unmedicated obsessive-compulsive disorder and changes after medication. PLoS One 7:e35889. doi: 10.1371/journal.pone.0035889
Francis, T. C., Gantz, S. C., Moussawi, K., and Bonci, A. (2019). Synaptic and intrinsic plasticity in the ventral tegmental area after chronic cocaine. Curr. Opin. Neurobiol. 54, 66–72. doi: 10.1016/j.conb.2018.08.013
Gardner, M. P., Conroy, J. C., Styer, C. V., Huynh, T., Whitaker, L. R., and Schoenbaum, G. (2018). Medial orbitofrontal inactivation does not affect economic choice. Elife 7:e38963. doi: 10.7554/elife.38963
Gillan, C. M., Papmeyer, M., Morein-Zamir, S., Sahakian, B. J., Fineberg, N. A., Robbins, T. W., et al. (2011). Disruption in the balance between goal-directed behavior and habit learning in obsessive-compulsive disorder. Am. J. Psychiatry 168, 718–726. doi: 10.1176/appi.ajp.2011.10071062
Gillan, C. M., Robbins, T. W., Sahakian, B. J., van den Heuvel, O. A., and van Wingen, G. (2016). The role of habit in compulsivity. Eur. Neuropsychopharmacol. 26, 828–840. doi: 10.1016/j.euroneuro.2015.12.033
Gipson, C. D., Kupchik, Y. M., Shen, H., Reissner, K. J., Thomas, C. A., and Kalivas, P. W. (2013). Relapse induced by cues predicting cocaine depends on rapid, transient synaptic potentiation. Neuron 77, 867–872. doi: 10.1016/j.neuron.2013.01.005
Gourley, S. L., and Taylor, J. R. (2016). Going and stopping: dichotomies in behavioral control by the prefrontal cortex. Nat. Neurosci. 19, 656–664. doi: 10.1038/nn.4275
Graybiel, A. M. (2008). Habits, rituals and the evaluative brain. Annu. Rev. Neurosci. 31, 359–387. doi: 10.1146/annurev.neuro.29.051605.112851
Graybiel, A. M., and Grafton, S. T. (2015). The striatum: where skills and habits meet. Cold Spring Harb. Perspect. Biol. 7:a021691. doi: 10.1101/cshperspect.a021691
Gremel, C. M., Chancey, J. H., Atwood, B. K., Luo, G., Neve, R., Ramakrishnan, C., et al. (2016). Endocannabinoid modulation of orbitostriatal circuits gates habit formation. Neuron 90, 1312–1324. doi: 10.1016/j.neuron.2016.04.043
Gremel, C. M., and Costa, R. M. (2013). Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat. Commun. 4:2264. doi: 10.1038/ncomms3264
Guehl, D., Benazzouz, A., Aouizerate, B., Cuny, E., Rotgé, J. Y., Rougier, A., et al. (2008). Neuronal correlates of obsessions in the caudate nucleus. Biol. Psychiatry 63, 557–562. doi: 10.1016/j.biopsych.2007.06.023
Guenthner, C. J., Miyamichi, K., Yang, H. H., Heller, H. C., and Luo, L. (2013). Permanent genetic access to transiently active neurons via TRAP: targeted recombination in active populations. Neuron 79:1257. doi: 10.1016/j.neuron.2013.08.031
Gutman, A. L., Nett, K. E., Cosme, C. V., Worth, W. R., Gupta, S. C., Wemmie, J. A., et al. (2017). Extinction of cocaine seeking requires a window of infralimbic pyramidal neuron activity after unreinforced lever presses. J. Neurosci. 37, 6075–6086. doi: 10.1523/JNEUROSCI.3821-16.2017
Haber, S. N. (2016). “Corticostriatal circuitry,” in Neuroscience in the 21st Century: From Basic to Clinical, 2nd Edition, eds D. Pfaff and N. Volkow (New York, NY: Springer), 1721–1741.
Haber, S. N., Fudge, J. L., and McFarland, N. R. (2000). Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382. doi: 10.1523/JNEUROSCI.20-06-02369.2000
Hancock, D. B., Markunas, C. A., Bierut, L. J., and Johnson, E. O. (2018). Human genetics of addiction: new insights and future directions. Curr. Psychiatry Rep. 20:8. doi: 10.1007/s11920-018-0873-3
Hart, G., Bradfield, L. A., Bernard, X., and Balleine, W. (2018a). Prefrontal corticostriatal disconnection blocks the acquisition of goal-directed action. J. Neurosci. 38, 1311–1322. doi: 10.1523/JNEUROSCI.2850-17.2017
Hart, G., Bradfield, L. A., Fok, S. Y., Chieng, B., and Balleine, B. W. (2018b). The bilateral prefronto-striatal pathway is necessary for learning new goal-directed actions. Curr. Biol. 28, 2218.e7–2229.e7. doi: 10.1016/j.cub.2018.05.028
Hilário, M. R. F., Clouse, E., Yin, H. H., and Costa, R. M. (2007). Endocannabinoid signaling is critical for habit formation. Front. Integr. Neurosci. 1:6. doi: 10.3389/neuro.07.006.2007
Hintiryan, H., Foster, N. N., Bowman, I., Bay, M., Song, M. Y., Gou, L., et al. (2016). The mouse cortico-striatal projectome. Nat. Neurosci. 19, 1100–1114. doi: 10.1038/nn.4332
Hodebourg, R., Murray, J. E., Fouyssac, M., Puaud, M., Everitt, B. J., and Belin, D. (2018). Heroin seeking becomes dependent on dorsal striatal dopaminergic mechanisms and can be decreased by N-acetylcysteine. Eur. J. Neurosci. doi: 10.1111/ejn.13894 [Epub ahead of print].
Hogarth, L., Attwood, A. S., Bate, H. A., and Munafò, M. R. (2012). Acute alcohol impairs human goal-directed action. Biol. Psychol. 90, 154–160. doi: 10.1016/j.biopsycho.2012.02.016
Hunnicutt, B. J., Jongbloets, B. C., Birdsong, W. T., Gertz, K. J., Zhong, H., and Mao, T. (2016). A comprehensive excitatory input map of the striatum reveals novel functional organization. Elife 5:e19103. doi: 10.7554/elife.19103
Ito, R., Dalley, J. W., Robbins, T. W., and Everitt, B. J. (2002). Dopamine release in the dorsal striatum during cocaine-seeking behavior under the control of a drug-associated cue. J. Neurosci. 22, 6247–6253. doi: 10.1523/JNEUROSCI.22-14-06247.2002
James, W. (1890). The Principles of Psychology, Volume 1. New York, NY: Henry Holt and Company. doi: 10.2307/2107586
Jin, X., and Costa, R. M. (2010). Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 466, 457–462. doi: 10.1038/nature09263
Jin, X., Tecuapetla, F., and Costa, R. M. (2014). Basal ganglia subcircuits distinctively encode the parsing and concatenation of action sequences. Nat. Neurosci. 17, 423–430. doi: 10.1038/nn.3632
Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V., and Graybiel, A. M. (1999). Building neural representations of habits. Science 286, 1745–1749. doi: 10.1126/science.286.5445.1745
Jonkman, S., Pelloux, Y., and Everitt, B. J. (2012). Differential roles of the dorsolateral and midlateral striatum in punished cocaine seeking. J. Neurosci. 32, 4645–4650. doi: 10.1523/JNEUROSCI.0348-12.2012
Jun, J. J., Steinmetz, N. A., Siegle, J. H., Denman, D. J., Bauza, M., Barbarits, B., et al. (2017). Fully integrated silicon probes for high-density recording of neural activity. Nature 551, 232–236. doi: 10.1038/nature24636
Kalanithi, P. S. A., Zheng, W., Kataoka, Y., DiFiglia, M., Grantz, H., Saper, C. B., et al. (2005). Altered parvalbumin-positive neuron distribution in basal ganglia of individuals with Tourette syndrome. Proc. Natl. Acad. Sci. U S A 102, 13307–13312. doi: 10.1073/pnas.0502624102
Kalueff, A. V., Stewart, A. M., Song, C., Berridge, K. C., Graybiel, A. M., and Fentress, J. C. (2016). Neurobiology of rodent self-grooming and its value for translational neuroscience. Nat. Rev. Neurosci. 17, 45–59. doi: 10.1038/nrn.2015.8
Killcross, S., and Coutureau, E. (2003). Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb. Cortex 2, 400–408. doi: 10.1093/cercor/13.4.400
Kim, J., Zhang, X., Muralidhar, S., LeBlanc, S. A., and Tonegawa, S. (2017). Basolateral to central amygdala neural circuits for appetitive behaviors in brief. Neuron 93, 1464.e5–1479.e5. doi: 10.1016/j.neuron.2017.02.034
Klaus, A., Martins, G. J., Paixao, V. B., Zhou, P., Paninski, L., and Costa, R. M. (2017). The spatiotemporal organization of the striatum encodes action space. Neuron 95, 1171.e7–1180.e7. doi: 10.1016/j.neuron.2017.08.015
Knowlton, B. J., and Diedrichsen, J. (2018). Editorial overview: habits and skills. Curr. Opin. Behav. Sci. 20, iv–vi. doi: 10.1016/j.cobeha.2018.02.009
Kourrich, S. S., Rothwell, P. E., Klug, J. R., and Thomas, M. J. (2007). Cocaine experience controls bidirectional synaptic plasticity in the nucleus accumbens. J. Neurosci. 27, 7921–7928. doi: 10.1523/JNEUROSCI.1859-07.2007
Kravitz, A. V., Freeze, B. S., Parker, P. R. L., Kay, K., Thwin, M. T., Deisseroth, K., et al. (2010). Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622–626. doi: 10.1038/nature09159
Kreitzer, A. C., and Malenka, R. C. (2008). Striatal plasticity and basal ganglia circuit function. Neuron 60, 543–554. doi: 10.1016/j.neuron.2008.11.005
Kupferschmidt, D. A., Juczewski, K., Cui, G., Johnson, K. A., and Lovinger, D. M. (2017). Parallel, but dissociable, processing in discrete corticostriatal inputs encodes skill learning. Neuron 96, 476.e5–489.e5. doi: 10.1016/j.neuron.2017.09.040
Lammel, S., Ion, D. I., Roeper, J., and Malenka, R. C. (2011). Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70, 855–862. doi: 10.1016/j.neuron.2011.03.025
Lingawi, N. W., and Balleine, B. W. (2012). Amygdala central nucleus interacts with dorsolateral striatum to regulate the acquisition of habits. J. Neurosci. 32, 1073–1081. doi: 10.1523/JNEUROSCI.4806-11.2012
Luo, L., Callaway, E. M., and Svoboda, K. (2018). Genetic dissection of neural circuits: a decade of progress. Neuron 98:865. doi: 10.1016/j.neuron.2018.05.004
Lüscher, C. (2016). The emergence of a circuit model for addiction. Annu. Rev. Neurosci. 39, 257–276. doi: 10.1146/annurev-neuro-070815-013920
Lüscher, C., and Malenka, R. C. (2011). Drug-evoked synaptic plasticity in addiction: from molecular changes to circuit remodeling. Neuron 69, 650–663. doi: 10.1016/j.neuron.2011.01.017
Lüscher, C., Pascoli, V., and Creed, M. (2015). Optogenetic dissection of neural circuitry: from synaptic causalities to blue prints for novel treatments of behavioral diseases. Curr. Opin. Neurobiol. 35, 95–100. doi: 10.1016/j.conb.2015.07.005
Ma, Y.-Y., Lee, B. R., Wang, X., Guo, C., Liu, L., Cui, R., et al. (2014). Bidirectional modulation of incubation of cocaine craving by silent synapse-based remodeling of prefrontal cortex to accumbens projections. Neuron 83, 1453–1467. doi: 10.1016/j.neuron.2014.08.023
MacAskill, A. F., Cassel, J. M., and Carter, A. G. (2014). Cocaine exposure reorganizes cell type- and input-specific connectivity in the nucleus accumbens. Nat. Neurosci. 17, 1198–1207. doi: 10.1038/nn.3783
Markowitz, J. E., Gillis, W. F., Beron, C. C., Neufeld, S. Q., Robertson, K., Bhagat, N. D., et al. (2018). The striatum organizes 3d behavior via moment-to-moment action selection. Cell 174, 44.e17–58.e17. doi: 10.1016/j.cell.2018.04.019
Martiros, N., Burgess, A. A., and Graybiel, A. M. (2018). Inversely active striatal projection neurons and interneurons selectively delimit useful behavioral sequences. Curr. Biol. 28, 560.e5–573.e5. doi: 10.1016/j.cub.2018.01.031
McFarland, K., Davidge, S., Lapish, C., and Kalivas, P. W. (2004). Limbic and motor circuitry underlying footshock-induced reinstatement of cocaine-seeking behavior. J. Neurosci. 24, 1551–1560. doi: 10.1523/jneurosci.4177-03.2004
McFarland, K., and Kalivas, P. W. (2001). The circuitry mediating cocaine-induced reinstatement of drug-seeking behavior. J. Neurosci. 21, 8655–8663. doi: 10.1523/jneurosci.21-21-08655.2001
McGlinchey, E. M., James, M. H., Mahler, S. V., Pantazis, C., and Aston-Jones, G. (2016). Prelimbic to accumbens core pathway is recruited in a dopamine-dependent manner to drive cued reinstatement of cocaine seeking. J. Neurosci. 36, 8700–8711. doi: 10.1523/jneurosci.1291-15.2016
Moorman, D. E., James, M. H., McGlinchey, E. M., and Aston-Jones, G. (2015). Differential roles of medial prefrontal subregions in the regulation of drug seeking. Brain Res. 1628, 130–146. doi: 10.1016/j.brainres.2014.12.024
Murray, J. E., Belin, D., and Everitt, B. J. (2012). Double dissociation of the dorsomedial and dorsolateral striatal control over the acquisition and performance of cocaine seeking. Neuropsychopharmacology 37, 2456–2466. doi: 10.1038/npp.2012.104
Murray, J. E., Belin-Rauscent, A., Simon, M., Giuliano, C., Benoit-Marand, M., Everitt, B. J., et al. (2015). Basolateral and central amygdala differentially recruit and maintain dorsolateral striatumdependent cocaine-seeking habits. Nat. Commun. 6:10088. doi: 10.1038/ncomms10088
Murray, J. E., Dilleen, R., Pelloux, Y., Economidou, D., Dalley, J. W., Belin, D., et al. (2014). Increased impulsivity retards the transition to dorsolateral striatal dopamine control of cocaine seeking. Biol. Psychiatry 76, 15–22. doi: 10.1016/j.biopsych.2013.09.011
Nonomura, S., Nishizawa, K., Sakai, Y., Kawaguchi, Y., Kato, S., Uchigashima, M., et al. (2018). Monitoring and updating of action selection for goal-directed behavior through the striatal direct and indirect pathways. Neuron 99, 1302.e5–1314.e5. doi: 10.1016/j.neuron.2018.08.002
Nordstrom, E. J., and Burton, F. H. (2002). A transgenic model of comorbid Tourette’s syndrome and obsessive-compulsive disorder circuitry. Mol. Psychiatry 7, 617–625. doi: 10.1038/sj.mp.4001144
O’Hare, J. K., Ade, K. K., Sukharnikova, T., Van Hooser, S. D., Palmeri, M. L., Yin, H. H., et al. (2016). Pathway-specific striatal substrates for habitual behavior. Neuron 89, 472–479. doi: 10.1016/j.neuron.2015.12.032
O’Hare, J., Calakos, N., and Yin, H. H. (2018). Recent insights into corticostriatal circuit mechanisms underlying habits. Curr. Opin. Behav. Sci. 20, 40–46. doi: 10.1016/j.cobeha.2017.10.001
O’Hare, J. K., Li, H., Kim, N., Gaidis, E., Ade, K., Beck, J., et al. (2017). Striatal fast-spiking interneurons selectively modulate circuit output and are required for habitual behavior. Elife 6:e26231. doi: 10.7554/elife.26231
Panayi, M. C., and Killcross, S. (2018). Functional heterogeneity within the rodent lateral orbitofrontal cortex dissociates outcome devaluation and reversal learning deficits. Elife 7:e37357. doi: 10.7554/elife.37357
Parker, J. G., Marshall, J. D., Ahanonu, B., Wu, Y. W., Kim, T. H., Grewe, B. F., et al. (2018). Diametric neural ensemble dynamics in parkinsonian and dyskinetic states. Nature 557, 177–182. doi: 10.1038/s41586-018-0090-6
Pascoli, V., Hiver, A., Van Zessen, R., Loureiro, M., Achargui, R., Harada, M., et al. (2018). Stochastic synaptic plasticity underlying compulsion in a model of addiction. Nature 564, 366–371. doi: 10.1038/s41586-018-0789-4
Pascoli, V., Terrier, J., Espallergues, J., Valjent, E., Cornelius O’connor, E., and Lüscher, C. (2014). Contrasting forms of cocaine-evoked plasticity control components of relapse. Nature 509, 459–464. doi: 10.1038/nature13257
Patton, M. H., Roberts, B. M., Lovinger, D. M., and Mathur, B. N. (2016). Ethanol disinhibits dorsolateral striatal medium spiny neurons through activation of a presynaptic delta opioid receptor. Neuropsychopharmacology 41, 1831–1840. doi: 10.1038/npp.2015.353
Peak, J., Hart, G., and Balleine, B. W. (2019). From learning to action: the integration of dorsal striatal input and output pathways in instrumental conditioning. Eur. J. Neurosci. 49, 658–671. doi: 10.1111/ejn.13964
Peters, J., LaLumiere, R. T., and Kalivas, P. W. (2008). Infralimbic prefrontal cortex is responsible for inhibiting cocaine seeking in extinguished rats. J. Neurosci. 28, 6046–6053. doi: 10.1523/JNEUROSCI.1045-08.2008
Renteria, R., Baltz, E. T., and Gremel, C. M. (2018). Chronic alcohol exposure disrupts top-down control over basal ganglia action selection to produce habits. Nat. Commun. 9:211. doi: 10.1038/s41467-017-02615-9
Robbe, D. (2018). To move or to sense? Incorporating somatosensory representation into striatal functions. Curr. Opin. Neurobiol. 52, 123–130. doi: 10.1016/j.conb.2018.04.009
Robbins, T. W., and Costa, R. M. (2017). Habits. Curr. Biol. 27, R1200–R1206. doi: 10.1016/j.cub.2017.09.060
Rossi, M. A., and Yin, H. H. (2012). Methods for studying habitual behavior in mice. Curr. Protoc. Neurosci. 60, 8.29.1–8.29.9. doi: 10.1002/0471142301.ns0829s60
Rothwell, P. E., Hayton, S. J., Sun, G. L., Fuccillo, M. V., Lim, B. K., and Malenka, R. C. (2015). Input- and output-specific regulation of serial order performance by corticostriatal circuits. Neuron 88, 345–356. doi: 10.1016/j.neuron.2015.09.035
Rueda-orozco, P. E., and Robbe, D. (2015). The striatum multiplexes contextual and kinematic information to constrain motor habits execution. Nat. Neurosci. 18, 453–460. doi: 10.1038/nn.3924
Saal, D., Dong, Y., Bonci, A., and Malenka, R. C. (2003). Report drugs of abuse and stress trigger a common synaptic adaptation in dopamine neurons.
Sakai, Y., Narumoto, J., Nishida, S., Nakamae, T., Yamada, K., Nishimura, T., et al. (2011). Corticostriatal functional connectivity in non-medicated patients with obsessive-compulsive disorder. Eur. Psychiatry 26, 463–469. doi: 10.1016/j.eurpsy.2010.09.005
Schwarz, L. A., Miyamichi, K., Gao, X. J., Beier, K. T., Weissbourd, B., Deloach, K. E., et al. (2015). Viral-genetic tracing of the input-output organization of a central noradrenaline circuit. Nature 524, 88–92. doi: 10.1038/nature14600
Shan, Q., Ge, M., Christie, M. J., and Balleine, B. W. (2014). The acquisition of goal-directed actions generates opposing plasticity in direct and indirect pathways in dorsomedial striatum. J. Neurosci. 34, 9196–9201. doi: 10.1523/jneurosci.0313-14.2014
Shmelkov, S. V., Hormigo, A., Jing, D., Proenca, C. C., Bath, K. G., Milde, T., et al. (2010). Slitrk5 deficiency impairs corticostriatal circuitry and leads to obsessive-compulsive-like behaviors in mice. Nat. Med. 16, 598–602. doi: 10.1038/nm.2125
Sippy, T., Lapray, D., Crochet, S., and Petersen, C. C. H. (2015). Cell-type-specific sensorimotor processing in striatal projection neurons during goal-directed behavior. Neuron 88, 298–305. doi: 10.1016/j.neuron.2015.08.039
Smith, K., and Graybiel, A. (2013). A dual operator view of habitual behavior reflecting cortical and striatal dynamics. Neuron 79, 361–374. doi: 10.1016/j.neuron.2013.05.038
Smith, K. S., and Graybiel, A. M. (2014). Investigating habits: strategies, technologies and models. Front. Behav. Neurosci. 8:39. doi: 10.3389/fnbeh.2014.00039
Smith, K. S., and Graybiel, A. M. (2016). Habit formation. Dialogues Clin. Neurosci. 18, 33–43. doi: 10.1111/clr.12458.111
Smith, R. J., and Laiks, L. S. (2017). Behavioral and neural mechanisms underlying habitual and compulsive drug seeking. Prog. Neuropsychopharmacol. Biol. Psychiatry 87, 11–21. doi: 10.1016/j.pnpbp.2017.09.003
Smith, K. S., Virkud, A., Deisseroth, K., and Graybiel, A. M. (2012). Reversible online control of habitual behavior by optogenetic perturbation of medial prefrontal cortex. Proc. Natl. Acad. Sci. U S A 109, 18932–18937. doi: 10.1073/pnas.1216264109
Stalnaker, T. A., Cooch, N. K., and Schoenbaum, G. (2015). What the orbitofrontal cortex does not do. Nat. Neurosci. 18, 620–627. doi: 10.1038/nn.3982
Tecuapetla, F., Jin, X., Lima, S. Q., and Costa, R. M. (2016). Complementary contributions of striatal projection pathways to action initiation and execution. Cell 166, 703–715. doi: 10.1016/j.cell.2016.06.032
Tecuapetla, F., Matias, S., Dugue, G. P., Mainen, Z. F., and Costa, R. M. (2014). Balanced activity in basal ganglia projection pathways is critical for contraversive movements. Nat. Commun. 5:4315. doi: 10.1038/ncomms5315
Terraneo, A., Leggio, L., Saladini, M., Ermani, M., Bonci, A., and Gallimberti, L. (2016). Transcranial magnetic stimulation of dorsolateral prefrontal cortex reduces cocaine use: a pilot study. Eur. Neuropsychopharmacol. 26, 37–44. doi: 10.1016/j.euroneuro.2015.11.011
Thomas, M. J., Beurrier, C., Bonci, A., and Malenka, R. C. (2001). Long-term depression in the nucleus accumbens: a neural correlate of behavioral sensitization to cocaine. Nat. Neurosci. 4, 1217–1223. doi: 10.1038/nn757
Thorn, C. A., Atallah, H., Howe, M., and Graybiel, A. M. (2010). Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66, 781–795. doi: 10.1016/j.neuron.2010.04.036
Thorn, C. A., and Graybiel, A. M. (2014). Differential entrainment and learning-related dynamics of spike and local field potential activity in the sensorimotor and associative striatum. J. Neurosci. 34, 2845–2859. doi: 10.1523/jneurosci.1782-13.2014
Ungless, M. A., Whistler, J. L., Malenka, R. C., and Bonci, A. (2001). Single cocaine exposure in vivo induces long-term potentiation in dopamine neurons. Nature 411, 583–587. doi: 10.1038/35079077
Valentin, V. V., Dickinson, A., and O’Doherty, J. P. (2007). Determining the neural substrates of goal-directed learning in the human brain. J. Neurosci. 27, 4019–4026. doi: 10.1523/JNEUROSCI.0564-07.2007
Vanderschuren, L. J. M. J., Di Ciano, P., and Everitt, B. J. (2005). Involvement of the dorsal striatum in cue-controlled cocaine seeking. J. Neurosci. 25, 8665–8670. doi: 10.1523/jneurosci.0925-05.2005
Vanderschuren, L. M. J. J., and Everitt, B. J. (2004). Drug seeking becomes compulsive after prolonged cocaine self-administration. Science 305, 1017–1019. doi: 10.1126/science.1098975
Vicente, A. M., Galvão-Ferreira, P., Tecuapetla, F., and Costa, R. M. (2016). Direct and indirect dorsolateral striatum pathways reinforce different action strategies. Curr. Biol. 26, R267–R269. doi: 10.1016/j.cub.2016.02.036
Volkow, N. D., and Morales, M. (2015). The brain on drugs: from reward to addiction. Cell 162, 715–725. doi: 10.1016/j.cell.2015.07.046
Wan, Y., Ade, K., Caffall, Z., Ozlu, M. I., Eroglu, C., Feng, G., et al. (2013). Circuit-selective striatal synaptic dysfunction in the Sapap3 knockout mouse model of obsessive-compulsive disorder. Biol. Psychiatry 75, 623–630. doi: 10.1016/j.biopsych.2013.01.008
Wang, L. P., Li, F., Wang, D., Xie, K., Wang, D. D. D. D., Shen, X., et al. (2011). NMDA receptors in dopaminergic neurons are crucial for habit learning. Neuron 72, 1055–1066. doi: 10.1016/j.neuron.2011.10.019
Welch, J. M., Lu, J., Rodriguiz, R. M., Trotta, N. C., Peca, J., Ding, J.-D., et al. (2007). Cortico-striatal synaptic defects and OCD-like behaviours in Sapap3-mutant mice. Nature 448, 894–900. doi: 10.1038/nature06104
Wilcox, M. V., Carlson, V. C. C., Sherazee, N., Sprow, G. M., Bock, R., Thiele, T. E., et al. (2014). Repeated Binge-like ethanol drinking alters ethanol drinking patterns and depresses striatal GABAergic transmission. Neuropsychopharmacology 39, 579–594. doi: 10.1038/npp.2013.230
Wirz, L., Bogdanov, M., and Schwabe, L. (2018). Habits under stress: mechanistic insights across different types of learning. Curr. Opin. Behav. Sci. 20, 9–16. doi: 10.1016/j.cobeha.2017.08.009
Wolf, M. E. (2016). Synaptic mechanisms underlying persistent cocaine craving. Nat. Rev. Neurosci. 17, 351–365. doi: 10.1038/nrn.2016.39.
Xiong, Q., Znamenskiy, P., and Zador, A. M. (2015). Selective corticostriatal plasticity during acquisition of an auditory discrimination task. Nature 521, 348–351. doi: 10.1038/nature14225
Yin, H. H., and Knowlton, B. J. (2004). Contributions of striatal subregions to place and response learning. Learn. Mem. 11, 459–463. doi: 10.1101/lm.81004
Yin, H. H., and Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7, 464–476. doi: 10.1038/nrn1919
Yin, H. H., Knowlton, B. J., and Balleine, B. W. (2004). Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189. doi: 10.1111/j.1460-9568.2004.03095.x
Yin, H. H., Knowlton, B. J., and Balleine, B. W. (2005). Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur. J. Neurosci. 22, 505–512. doi: 10.1111/j.1460-9568.2005.04219.x
Yin, H. H., Mulcare, S. P., Hilário, M. R. F., Clouse, E., Holloway, T., Davis, M. I., et al. (2009). Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat. Neurosci. 12, 333–341. doi: 10.1038/nn.2261
Zalocusky, K. A., Ramakrishnan, C., Lerner, T. N., Davidson, T. J., Knutson, B., and Deisseroth, K. (2016). Nucleus accumbens D2R cells signal prior outcomes and control risky decision-making. Nature 531, 642–646. doi: 10.1038/nature17400
Zapata, A., Minney, V. L., and Shippenberg, T. S. (2010). Shift from goal-directed to habitual cocaine seeking after prolonged experience in rats. J. Neurosci. 30, 15457–15463. doi: 10.1523/jneurosci.4072-10.2010
Zhou, J., Gardner, M. P. H., Stalnaker, T. A., Ramus, S. J., Wikenheiser, A. M., Niv, Y., et al. (2019). Rat orbitofrontal ensemble activity contains multiplexed but dissociable representations of value and task structure in an odor sequence task. Curr. Biol. 29, 897.e3–907.e3. doi: 10.3410/f.735226042.793558592
Keywords: habits, goal-directed behavior, striatum, prefrontal cortex, dorsomedial striatum, dorsolateral striatum
Citation: Lipton DM, Gonzales BJ and Citri A (2019) Dorsal Striatal Circuits for Habits, Compulsions and Addictions. Front. Syst. Neurosci. 13:28. doi: 10.3389/fnsys.2019.00028
Received: 17 January 2019; Accepted: 27 June 2019;
Published: 18 July 2019.
Edited by:
Jimena Andersen, Stanford University, United StatesReviewed by:
Sarah Heilbronner, University of Minnesota Twin Cities, United StatesEric Burguière, Centre National de la Recherche Scientifique (CNRS), France
Copyright © 2019 Lipton, Gonzales and Citri. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ami Citri ami.citri@mail.huji.ac.il orcid.org/0000-0002-9914-0278
† These authors have contributed equally to this work