Joman Y. Natsheh
Michael W. Shiflett- 1Center for Molecular and Behavioral Neuroscience, Rutgers, The State University of New Jersey, Newark, NJ, USA
- 2Palestinian Neuroscience Initiative, Faculty of Medicine, Al-Quds University, Jerusalem, State of Palestine
- 3Department of Psychology, Rutgers, The State University of New Jersey, Newark, NJ, USA
Although attentional and motor alterations in Attention Deficit Hyperactivity Disorder (ADHD) have been well characterized, less is known about how this disorder impacts goal-directed behavior. To investigate whether there is a misbalance between goal-directed and habitual behaviors in an animal model of ADHD, we tested adult [P75–P105] Spontaneously Hypertensive Rats (SHR; ADHD rat model) and Wistar–Kyoto rats (WKY), the normotensive control strain, on an instrumental conditioning paradigm with two phases: a free-operant training phase in which rats separately acquired two distinct action–outcome contingencies, and a choice test conducted in extinction prior to which one of the food outcomes was devalued through specific satiety. To assess the effects of Methylphenidate (MPH), a commonly used ADHD medication, on goal-directed behavior, we injected rats with either MPH or saline prior to the choice test. Both rat strains acquired an instrumental response, with SHR responding at greater rates over the course of training. During the choice test WKY demonstrated goal-directed behavior, responding more frequently on the lever that delivered, during training, the still-valued outcome. In contrast, SHR showed no goal-directed behavior, responding equally on both levers. However, MPH administration prior to the choice test restored goal-directed behavior in SHR, and disrupted this behavior in WKY rats. This study provides the first experimental evidence for selective impairment in goal-directed behavior in rat models of ADHD, and how MPH acts differently on SHR and WKY animals to restore or impair this behavior, respectively.
Introduction
Attention Deficit Hyperactivity Disorder (ADHD) is one of the most prevalent psychiatric disorders, which is typically diagnosed in childhood and can continue to adolescence and adulthood. It is characterized by developmentally inappropriate symptoms of inattention, impulsivity, and hyperactivity (Castellanos and Tannock, 2002; Barkley, 2005). The neurobiological underpinnings of ADHD are not well established; however, dopaminergic hypofunction is thought to play an important role in the etiology of this disorder (Hynd et al., 1993; Gill et al., 1997; Waldman et al., 1998; Russell, 2003; Bush et al., 2005; Sagvolden et al., 2005b). Consistent with this notion, ADHD symptoms are reduced in response to drugs that increase dopamine signaling, such as Methylphenidate (MPH) (Ritalin©), a psychostimulant that preferentially blocks the reuptake of catecholamines, including dopamine (DA) and norepinephrine (NE), in both the striatum and prefrontal cortex (Mehta et al., 2000, 2004; Schiffer et al., 2006; Heal et al., 2009).
The spontaneously hypertensive rat strain (SHR), a rat model bred from progenitor Wistar–Kyoto rats (WKY; Okamoto and Aoki, 1963), is the most widely accepted rodent model of ADHD (Sagvolden et al., 1993; Sagvolden, 2000; Davids et al., 2003). SHR rats show deficits in sustained attention, motor impulsiveness, and hyperactivity in a novel environment (Knardahl and Sagvolden, 1979; Sagvolden et al., 1992, 2005b; Wultz and Sagvolden, 1992; Sagvolden, 2000). Further, SHR rats display reduced DA signaling and increased DA transporter (DAT) expression, similar to ADHD patients (Russell et al., 1995; Russell, 2003; Heal et al., 2008; Roessner et al., 2010). Likewise, MPH corrects attentional and motor impairments in SHR rats, lending further support for SHR rats as a model of human ADHD (Sagvolden et al., 2005a,b; Kantak et al., 2008).
Although attentional and motor alterations in ADHD have been well characterized, less is known about how this disorder impacts goal-directed behavior. Maze performance of SHR rats suggests they preferentially use response strategies to guide behavior in spatial tasks (Clements and Wainwright, 2006; Clements et al., 2007; Kantak et al., 2008). Here we employ operant conditioning procedures that allow us to more precisely distinguish goal-directed from stimulus–response (habitual) action control. Contemporary theories of action control suggest that two processes guide action selection: (1) a goal-directed system based on current knowledge of action–outcome contingencies and (2) a habit system based on acquired stimulus–response associations (Dickinson, 1985). Operant paradigms, such as pressing a lever to receive a food reward, provide means of assessing goal-directed and habitual action control. For example, changes in operant behavior in response to changes in either outcome value or the action–outcome contingency reflect goal-directed action control, whereas lack of sensitivity to these changes likely reflects habitual action control. Behavior of SHR rats has yet to be assessed using these paradigms.
Studying goal-directed behavior in ADHD will advance our understanding of the brain networks involved in reward processing and contingency learning in ADHD, thereby revealing new mechanisms and potential treatments for this disorder (Griffiths et al., 2014). In the context of reinforcement mechanisms, Tripp and Wickens (2008, 2009) propose that one factor underlying ADHD is diminished anticipatory DA cell firing. They suggest that, in ADHD, the transfer of DA signals to cued rewards fails to develop normally, especially late in learning, which leads to more rapid extinction of the behavioral response and loss of behavioral control (Tripp and Wickens, 2008, 2009; Furukawa et al., 2014). These studies, while not directly examining goal-directed and habitual action control, suggest that these processes may be altered in ADHD. On the other hand, some studies suggest that DA levels in the prefrontal cortex and the dorsomedial striatum are critical for action control as measured by contingency degradation, but not outcome devaluation test (Naneix et al., 2009; Lex and Hauber, 2010).
In the present study we examined goal-directed action control in SHR rats using an instrumental learning paradigm. Although previous research has shown overactive instrumental responding in SHR rats that was corrected by MPH, it is not known whether animals performed responses in a goal-directed or habitual manner (Sagvolden et al., 1993). We used outcome devaluation and contingency degradation paradigms to probe goal-directed behavior in adult SHR and WKY rats. We also examined the effects of an acute dose of MPH on choice behavior following outcome devaluation in SHR and WKY rats. Our findings illustrate that SHR rats are predominated by habitual action control; however, MPH can restore goal-directed control in these rats. In contrast, MPH impaired goal-directed behavior in control rats that previously showed intact behavior following saline injection.
Materials And Methods
Subjects and Apparatus
Twenty nine male adult (P75–P105) rats were used in this study; 12 of which were SHR (ADHD model) from Charles River Laboratories (Wilmington, MA, USA), and 17 were WKY, the normotensive control strain, from Harlan Laboratories (Indianapolis, IN, USA). Rats weighed approximately 175–250 g at the time of testing. Rats were housed in pairs in 47.6 × 20.3 × 26 cm (w × h × d) polycarbonate containers with Alpha Chip bedding material (Northeastern Products Corp., Warrensburg, NY, USA) and had free access to water. One week after arrival, all rats were placed on a restricted food diet of approximately 20 g of standard rat pellets per day (Purina, St. Louis, MO, USA). Rats were fed after their daily behavioral training session. Food restriction continued for the duration of the experiment. All procedures were approved by the Rutgers University Institutional Animal Care and Use Committee.
Behavioral training and testing took place in 12 identical rat operant conditioning chambers (Med Associates, St. Albans, VT, USA). Each operant conditioning chamber measured 30.5 × 24.1 × 21 cm (w × h × d) and was constructed of stainless steel and clear plastic walls and a stainless steel grid floor. A food cup with infrared detectors was centered on one wall of the operant conditioning chamber. Retractable levers were situated to the left and right of the food cup. Responses on these levers delivered one food pellet from a pellet dispenser mounted outside the operant conditioning chamber. Two types of pellets were used in the experimental procedures: 45-mg grain-based pellets and chocolate-flavored purified pellets (Bio-serv, Frenchtown, NJ, USA). Each operant conditioning chamber was housed in a sound attenuating shell and equipped with a ventilation fan that was activated during behavioral procedures. Control over the operant conditioning chambers was enabled by a personal computer operating through an interface. Operant conditioning chamber operation and data collection were carried out with Med Associates proprietary software (Med-PC).
Behavioral Procedures
General Procedures
A timeline of behavioral procedures is depicted in Table 1. Behavioral procedures commenced after 1 week of food restriction. Rats were provided with one 15-min session to habituate to the testing chamber, after which they began behavioral training. MPH or saline injections were carried out before devaluation test sessions to examine their effects on choice performance.
TABLE 1. Timeline of behavioral procedures.
Instrumental Conditioning
Rats underwent two training sessions per day; in one session, responses on one lever were associated with delivery of grain pellets and in the other session responses on a different lever were associated with chocolate pellet delivery. For each training session, one lever was inserted into the chamber and responses the rats made on the lever delivered a single food pellet associated with that lever. The session terminated when rats earned 20 pellets or 25 min had elapsed. Rats were trained daily on each lever in separate sessions with a 30-min interval between sessions. Training lasted for 10 days (see Table 1); on days 1–3, each response on the lever resulted in pellet delivery (continuous reinforcement). On days 4–5, pellets were delivered according to a variable-ratio (VR) five schedule, which required, on average, five responses to earn a pellet reward. On days 6–8, pellets were delivered according to a VR-15 schedule. On days 9–10, pellets were delivered according to a VR-20 schedule.
Outcome Devaluation Test and MPH Injection
Rats were placed in individual cages identical to their home cage and provided with 25 g of chocolate-flavored pellets. After 30 min, rats were given an intraperitoneal injection of MPH hydrochloride (Sigma Aldrich, St. Louis, MO, USA) dissolved in 0.8% saline or, for the control condition, an equal volume of 0.8% saline. MPH dosage was 2 mg/kg body weight diluted to 2 mg/ml. Rats were returned to the cages containing chocolate pellets for an additional 30 min. They were then placed in the operant conditioning chamber and both levers were inserted. Rats had the opportunity to respond on either lever for 10 min. No outcomes were presented in this session. The test was repeated the following day with the same outcome devalued. Rats that received MPH on the first test received saline on the second test, while rats that received saline on the first test received MPH on the second test. Since our results showed that rats had no preference for chocolate or grain pellets (see Results section: “Instrumental Training”), we only devalued one of the two outcomes to control for this variable across medication status (MPH and saline injections). Further, this will prevent overtraining that could result from devaluing each outcome for each medication status (four devaluation sessions for each rat).
Contingency Degradation Training
After the devaluation test, rats received two sessions of retraining (one session with each lever) on a VR-20 schedule before the selective degradation of one of the instrumental contingencies (Hammond, 1980). During contingency degradation, responses on each lever continued to deliver the same outcomes as during training. However, one of the two outcomes was also delivered non-contingently; for every second in, which rats made no lever response, there was a 5% probability of dispensing one pellet. For half the animals, the degraded outcome was chocolate pellets, and for the remaining animals it was grain pellets. Non-contingent outcome delivery occurred during all training sessions. Thus, for one lever-training session, the non-contingent outcome was the same as that earned by a response on the lever, whereas for the other lever-training session the non-contingent outcome differed from the contingent outcome. The rats were given two 20-min training sessions each day, one on each lever with a break of approximately one hour between sessions. Training continued for 4–5 days.
Contingency Degradation Test
On the day after the final day of contingency training, rats in both groups received a choice extinction test. The test was identical to the choice test following outcome devaluation; it began with the insertion of both levers and the onset of the house light and ended 10 min later with the retraction of the levers and the offset of the house light. No outcomes were presented during this session. We made no injections prior to this test.
Locomotor Activity Assay
Rats were individually placed in an activity monitoring arena equipped with an automated locomotor activity detection system (Accuscan, Columbus, OH, USA). Rats were placed in the arena for a 60-min habituation session. Immediately after habituation, rats were injected with saline and returned to the arena for 60 min, followed by a 60-min session with 2 mg/kg MPH injections. A measure of locomotor activity (HACTV: horizontal activity) was collected based on the number of photobeam breaks that occurred as animals moved through the arena.
Statistics and Data Analysis
For instrumental conditioning tests, the rate of responses was calculated as the number of lever presses per minute during each session. Reinforcer type (chocolate or grain pellet) was collapsed across training sessions, as no effect of reinforcer type was observed on measures of response rate. Responses on the two levers were categorized as devalued or valued for the outcome devaluation test, and degraded and non-degraded for the contingency degradation test. The lever that delivers grain pellets was labeled as valued, and the one that delivers chocolate pellets was labeled as devalued. Similarly, the lever associated with the contingent outcome was labeled as non-degraded, and the lever associated with the non-contingent outcome was labeled as degraded. Data were normalized by dividing responses on the valued or devalued lever by total (valued plus devalued) responses. Normalization was carried out because of strain differences in overall response rates during the tests. MPH and saline injections were intermixed for all experiments; therefore, there was no injection-order effect to influence outcome devaluation responding. For the contingency degradation test, we lost data for one SHR rat due to a technical error. Additionally, food consumption and locomotor activity tests were conducted on 12 WKY rats. Data analysis was conducted using SPSS Statistics Version 20.0. The normality of data distribution was checked using Kolmogorov–Smirnov tests. All data were normally distributed (p > 0.1). To analyze instrumental performance we used 2-factor ANOVA and planned comparisons using two-tailed t-tests. The level of significance was set at α = 0.05.
Results
Instrumental Training
All rats acquired an instrumental response; however, SHR rats exhibited greater response rates across training sessions compared to WKY rats. Figure 1 represents the lever-pressing rate in SHR and WKY rats. A repeated measures ANOVA confirmed (1) a significant effect of training Block [F(3,81) = 259.3, p < 0.001], (2) a significant effect of Strain [F(1,27) = 6.7, p = 0.015], and (3) a significant Block ∗ Strain interaction [F(3,81) = 3.2, p = 0.028]. SHR responses were significantly higher than WKY responses over blocks two and four (p = 0.001, p = 0.026, respectively, independent-sample t-test).
FIGURE 1. Performance of SHR and WKY rats on instrumental training blocks (each block represents the average of two training sessions). The mean number of presses per min on the four blocks of instrumental training in SHR (N = 12) and WKY (N = 17) rats (error bars = ±SEM) (∗p < 0.05).
To investigate wither reward type (chocolate vs. grain pellets) had any influence on lever presses, we examined each strain’s responses with the reward type as a factor. A repeated-measures ANOVA revealed that there is no effect of reward type in both SHR [F(1,11) = 2.88, p = 0.12] and WKY [F(1,16) = 0.36, p = 0.56] rats.
Outcome Devaluation Test
Because of variability in overall response rates during the choice test, responses on the valuated and devaluated levers were normalized as a percentage of total responses during the test. Figure 2 illustrates the percentage of responses on the valuated versus the devaluated lever in SHR and WKY rats after saline or MPH injections. Repeated-measures ANOVAs were conducted for each of the two groups tested using outcome value and type of injection as within-subject factors. These analyses revealed significant Outcome value ∗ Injection interactions among WKY rats [F(1,16) = 4.83, p = 0.043] (Figure 2A) as well as SHR [F(1,11) = 12.52, p = 0.005] (Figure 2B) rats. Following saline injections, WKY rats showed significant goal-directed behavior by responding more on the valuated versus the devaluated lever [Figure 2A paired-sample t-test: t(16) = 2.6, p = 0.02]. In contrast, MPH disrupted goal-directed behavior in these rats, as their responses did not differ significantly between valuated and devaluated levers following MPH injection [paired-samples t-test: t(16) = 0.24, p = 0.82].
FIGURE 2. Normalized performance of SHR and WKY rats during the 10-min devaluation test. The percentage of responses on the valued and devalued levers for (A) WKY (N = 17) and (B) SHR (N = 12) rats (error bars = ±SEM) (∗p < 0.05).
The reverse pattern was observed in SHR rats. Following saline injections, SHR rats showed no goal-directed behavior, responding equally on the valuated and devaluated levers [paired-sample t-test: t(11) = 0.2, p = 0.84] (Figure 2B). MPH restored goal-directed behavior in these rats, as shown by significantly greater responding on the valuated lever compared to the devaluated lever after MPH injection [paired-sample t-test: t(11) = 4.65, p = 0.001].
We additionally examined whether MPH injection influenced overall response rates during the devaluation test. Figure 3 shows the effect of MPH on overall response rates (the average of the response rates on both levers) of both rat strains during the devaluation test after receiving MPH or saline injections. MPH administration suppressed overall response rates. A repeated measures ANOVA showed a significant effect of injection [F(1,27) = 4.3, p < 0.05], but no strain effect [F(1,27) = 3.67, p = 0.07], or Injection ∗ Strain interaction [F(1,27) = 0.002, p > 0.05]. Although there was a significant main effect of injection, group comparisons were not significant [paired sample t-test- WKY: t(16) = 1.88, p = 0.079; SHR t(11) = 1.15, p > 0.05, respectively]. Overall, these data indicate that MPH caused rats to modestly suppress instrumental activity during the choice test.
FIGURE 3. Overall response rate during the 10-min devaluation test, after receiving methylphenidate (MPH) or saline (Control) injections. The average of the response rates on both levers for SHR (N = 12) and WKY (N = 17) rats (error bars = ±SEM) (∗p < 0.05).
Contingency Degradation Test
Following outcome devaluation, rats underwent contingency degradation training and choice test. Figure 4 shows response rates, normalized as a percentage of total responses, during the choice test conducted in extinction after contingency degradation training for SHR and WKY rats. A repeated-measures ANOVA revealed a significant main effect of degradation [F(1,26) = 11.53, p = 0.002] and Degradation ∗ Strain interaction [F(1,26) = 5.12, p = 0.03]. A paired-sample t-test showed that WKY responses on the non-degraded lever were significantly higher than their responses on the degraded lever [t(16) = 3.98, p = 0.001]. However, SHR rats did not show any difference between their responses on the non-degraded versus the degraded lever [t(10) = 0.98, p = 0.35].
FIGURE 4. Normalized response rates during the extinction test after contingency degradation training, shown separately for the action that had the action–outcome contingency degraded and that for which the contingency was not degraded. The percentage of responses on the degraded and non-degraded levers for WKY (N = 17) and SHR (N = 11) rats (error bars = ±SEM) (∗p < 0.05).
Food Consumption
To determine whether MPH or rat strain influenced food consumption during the devaluation procedure, we examined the amount of food rats consumed during the first 30 min of the devaluation test prior to injections as well as in the 30 min after injections. All rats reached satiety; however, the amount of food required to reach satiety differed by rat strain (Table 2). In the 30 min prior to injection, SHR rats consumed a significantly greater amount of food than WKY rats [independent-sample t-test: t(22) = 3.69, p < 0.001].
TABLE 2. Number of food pellets consumed during satiety-induced devaluation.
The majority of food consumption occurred in the first 30 min prior to MPH injection (79%). However, MPH altered food consumption in SHR and WKY rats in the remaining 30 min. An ANOVA, using type of injection as a within subject factor, confirmed a significant effect of injection [F(1,22) = 25.52, p = 0.018]. MPH significantly reduced food consumption in WKY rats [paired-sample t-test: t(11) = 2.97, p = 0.013] but not SHR rats [paired-sample t-test, t(11) = 0.923, p = 0.38]. Overall, these data indicate that SHR rats consumed more food before reaching satiety and that MPH suppressed food consumption selectively in WKY rats.
Locomotor Activity Test
We examined locomotor activity to determine whether strain and MPH injection influenced this behavior. We found SHR rats traveled a greater distance as measured by HACTV (see Materials and Methods) and that MPH increased locomotor activity in both strains. HACTV was averaged across 5-min blocks for 1 h after saline and MPH injections. A repeated measures ANOVA on HACTV revealed a significant effect of strain [F(1,11) = 26.4, p < 0.001] and injection [F(2,22) = 13.6, p < 0.01]. However, there was no Phase ∗ Strain interaction [F(2,22) = 2.36, p > 0.05; Figure 5B]. MPH injections did significantly increase HACTV of both strains as compared to saline injection [Figure 5A, paired-sample t-test- SHR: t(22) = 6.45, p < 0.001; WKY: t(22) = 7.77, p < 0.001]. Moreover, HACTV was significantly greater in SHR rats as compared to WKY rats after both saline and MPH injections [Figure 5B, independent-sample t-test- saline injection: t(22) = 11.62, p < 0.001; MPH injection: t(22) = 3.85, p = 0.001].
FIGURE 5. Locomotor activity in SHR and WKY rats after saline and MPH injections. (A) Locomotor activity expressed as average horizontal activity (HACTV) for one hour after saline and MPH injections SHR (N = 12) and WKY (N = 12). (B) Locomotor activity in SHR and WKY rats in 5-min blocks over (1) habituation phase, block 1–12, (2) saline injection phase, block 12–24, and (3) MPH injection phase, block 24–36 (error bars = ± SEM) (∗significant at p < 0.05).
Discussion
In this study, we show using outcome devaluation and contingency degradation paradigms that SHR rats have a deficit in goal-directed behavior compared to normotensive WKY rats. Furthermore, we found that deficits in goal-directed action control following outcome devaluation were remediated in SHR rats with MPH administration. In contrast, while goal-directed behavior is well displayed in WKY rats, MPH administration disrupts this behavior. These results suggest that the behavioral phenotype of SHR rats, along with MPH treatment, play complementary roles in determining goal-directed action control.
Different Patterns of Goal-directed Behavior in WKY vs. SHR Rats and Distinct Effects of MPH
Using an instrumental conditioning paradigm, we present the first experimental evidence of disrupted goal-directed behavior using instrumental procedures in a rat model of ADHD. Both control WKY rats and SHR rats were successful at acquiring an instrumental response, with SHR rats showing a significantly greater response rate during training. Further, using an open field test to evaluate rat locomotor activity, SHR rats showed enhanced locomotor activity as compared to WKY rats. These findings of SHR operant and motor hyperactivity replicate previous research evaluating the use of the SHR strain as a model of ADHD (Wultz et al., 1990; Hill et al., 2012).
Our results suggest a fundamental impairment in goal-directed action control in SHR rats that is remediated by MPH. We used outcome devaluation and contingency degradation paradigms to assess whether animals had formed action–outcome (goal-directed) or stimulus-response (habit) associations. While WKY rats showed goal-directed action control in both paradigms, SHR rats demonstrated a marked deficit in sensitivity to changes in outcome value and to changes in the action–outcome contingency. Lack of sensitivity to these manipulations may reflect either (1) an inability to use action–outcome information to guide choice behavior (i.e., performance deficit) or (2) an inability of SHR rats to learn and/or retrieve action–outcome associations (i.e., learning/memory deficit). We found that treating SHR rats with MPH prior to the choice test following outcome devaluation revealed value-sensitive responding in these animals. SHR rats did encode action–outcome associations during instrumental learning; however, they were only able to use these associations to guide behavior when tested under the effects of MPH. Therefore the deficits we observed in tests of goal-directed behavior in non-medicated SHR rats likely reflects a deficit in performance and not learning of goal-directed actions.
The performance of SHR rats following outcome devaluation is not likely mediated by strain differences in the selective satiety process itself. We did find that SHR rats consumed more pellets compared to WKY animals during the satiety process. Nevertheless, many pellets remained after the 1-h session, by which time all animals had significantly curtailed food consumption, suggesting that all animals had reached a state of satiety. Likewise, although MPH reduced food consumption, it is unlikely that the anorexic effect of MPH altered the devaluation process itself since the majority of food consumption occurred in the 30 min prior to MPH injection. Based on these considerations, we are confident that the selective satiety procedure was equally effective in both SHR and WKY rats, and that the effects of MPH on choice performance were not a consequence of changing the satiety procedure.
To further strengthen our finding of impaired goal-directed behavior in SHR rats, and to exclude any effect of outcome devaluation on this behavior, we used a contingency degradation test after training the rats on a selective degradation of the instrumental contingency. Like the outcome devaluation test, SHR rats were impaired on this test. They performed significantly fewer responses on the lever for which the contingency had been degraded. This result further supports the notion of impaired goal-directed action control in SHR rats.
Overall, our data suggest that SHR rats’ impaired goal-directed behavior is not due to a lack of knowledge of causal consequences or to a failure of the devaluation process prior to the extinction test. This impairment is likely due to a predominance of habitual action control in SHR rats. In contrast, others have reported impaired habit formation and preserved goal-directed behavior in SHR rats (Gauthier et al., 2014). Differences in experimental design and interpretation may explain some of these discrepant results. Finally, one limitation of our study was injecting all rats with the same dose of MPH, while many studies have reported significant variations in the therapeutic doses of MPH. The use of one drug dose limits the conclusions we can draw from this study. A future study exploring the dose-response relationship between MPH and action control is required to fully address this issue.
Different Neuronal Mechanisms Underlie the Effect of MPH on the Behavior of WKY vs. SHR Rats
Cortico-striatal circuits that include the PFC and striatum mediate goal-directed behavior and habitual learning. Electrophysiological studies using primates (Matsumoto et al., 2003; Matsumoto and Tanaka, 2004) and rats (Mulder et al., 2003) have found neural activity in the PFC related to engagement of specific action–outcome associations. Likewise, lesions of the medial prefrontal cortex result in behavior that is insensitive to changes in outcome value with a stimulus-elicited, rather than goal-anticipated, instrumental responding (Hitchcott et al., 2007). Furthermore, Yin et al. have shown that the dorsomedial striatum plays a critical role in the acquisition and performance of goal-directed actions (Yin et al., 2005), and that the dorsolateral striatum mediates habitual instrumental performance (Yin et al., 2004). Further, they have shown that lesions of the dorsolateral striatum brought normal habitual actions under the control of the goal-directed system. Accordingly, the insensitivity to outcome devaluation in SHR rats might suggest that the neural circuits required for goal-directed actions are dysfunctional in these animals and thus they rely on the habit system instead to control responding.
The effects of MPH on goal-directed behavior likely occur through its modulation of catecholamine availability in the prefrontal cortex and striatum. The therapeutic dose of MPH works primarily via (1) increasing DA signaling through multiple actions, including DAT blockade in the PFC, and significantly enhancing extracellular DA release in the striatum (Volkow et al., 2002; Wilens, 2008) and (2) indirectly increasing NE actions through blockade of its transporters (Berridge et al., 2006). Increasing DA signaling in the PFC enhances goal-directed behavior (Hitchcott et al., 2007). Further, recent physiological studies have shown that NE strengthens network connectivity of the PFC and maintains persistent firing of PFC neurons during working memory tasks through stimulation of postsynaptic α2-adrenoceptors (Arnsten and Dudley, 2005). Thus, the increase in dopaminergic and noradrenergic availability in the PFC of SHR rats might be critical in restoring goal-directed actions by enhancing attentional mechanisms necessary for carrying out goal-directed behavior (Ostlund and Balleine, 2005; Ostlund et al., 2009). However, excessive DA stimulation might cause PFC dysfunction leading to impaired inhibition of undesirable behavior, and a deficit in sustaining attention, as these two behaviors are highly regulated by dopaminergic release in the PFC (Russell, 2003; Brennan and Arnsten, 2008; Arnsten, 2011).
In addition to modulating neurotransmitter release in the PFC, MPH may restore goal-directed behavior through reinforcement-based mechanisms. In the context of DA transfer deficit theory of ADHD, one can argue that administration of MPH restored goal-directed behavior by increasing the magnitude of the anticipatory DA cell firing to predictive cues (pressing the lever) (Tripp and Wickens, 2009, 2012). This is also consistent with the hypothesis that MPH selectively increases the efficacy of conditioned reinforcers (Hill, 1970; Robbins, 1975, 1978). Further studies examining the neural circuits activated in SHR rats during instrumental performance and the site of action of MPH will help to better understand the neural mechanisms underlying altered goal-directed action in ADHD.
Conclusion
We show for the first time that goal-directed behavior is impaired in a rat model of ADHD and can be remediated in SHR rats with MPH treatment. These results suggest that clinical symptoms exhibited by ADHD patients may reflect impaired goal-directed action control and that MPH may activate this system to re-establish goal-directed behavior.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This research was supported by grant R15DA029544 to MS from the National Institute on Drug Abuse. We would like to thank Joan Morrell for the use of her locomotor activity test chambers, and Jose Torres, RoseMarie DiMatteo, and Farah Al-Shammary for their assistance with data collection.
References
Arnsten, A. F. (2011). Catecholamine influences on dorsolateral prefrontal cortical networks. Biological Psychiatry 15, e89–e99. doi: 10.1016/j.biopsych.2011.01.027
Arnsten, A. F., and Dudley, A. G. (2005). Methylphenidate improves prefrontal cortical cognitive function through alpha2 adrenoceptor and dopamine D1 receptor actions: relevance to therapeutic effects in Attention Deficit Hyperactivity Disorder. Behav. Brain Funct. 22, 2. doi: 10.1186/1744-9081-1-2
Barkley, R. A. (2005). “Attention-Deficit Hyperactivity Disorder,” in A Handbook for Diagnosis and Treatment, 3rd Edn, ed. R. A. Barkley (New York: The Guilford Press), 770.
Berridge, C. W., Devilbiss, D. M., Andrzejewski, M. E., Arnsten, A. F., Kelley, A. E., Schmeichel, B., et al. (2006). Methylphenidate preferentially increases catecholamine neurotransmission within the prefrontal cortex at low doses that enhance cognitive function. Biol. Psychiatry 15, 1111–1120. doi: 10.1016/j.biopsych.2006.04.022
Brennan, A. R., and Arnsten, A. F. (2008). Neuronal mechanisms underlying attention deficit hyperactivity disorder: the influence of arousal on prefrontal cortical function. Ann. N. Y. Acad. Sci. 1129, 236–245. doi: 10.1196/annals.1417.007
Bush, G., Valera, E. M., and Seidman, L. J. (2005). Functional neuroimaging of attention-deficit/hyperactivity disorder: a review and suggested future directions. Biol. Psychiatry 1, 1273–1284. doi: 10.1016/j.biopsych.2005.01.034
Castellanos, F. X., and Tannock, R. (2002). Neuroscience of attention-deficit/hyperactivity disorder: the search for endophenotypes. Nat. Rev. Neurosci. 3, 617–628. doi: 10.1038/nrn896
Clements, K. M., Saunders, A. J., Robertson, B. A., and Wainwright, P. E. (2007). Spontaneously hypertensive, Wistar Kyoto and Sprague-Dawley rats differ in their use of place and response strategies in the water radial arm maze. Neurobiol. Learn. Mem. 87, 285–294. doi: 10.1016/j.nlm.2006.09.003
Clements, K. M., and Wainwright, P. E. (2006). Spontaneously hypertensive, Wistar-Kyoto and Sprague-Dawley rats differ in performance on a win-shift task in the water radial arm maze. Behav. Brain Res. 28, 295–304. doi: 10.1016/j.bbr.2005.09.016
Davids, E., Zhang, K., Tarazi, F. I., and Baldessarini, R. J. (2003). Animal models of attention-deficit hyperactivity disorder. Brain Res. Brain Res. Rev. 42, 1–21. doi: 10.1016/S0165-0173(02)00274-6
Dickinson, A. (1985). Actions and habits: the development of behavioural autonomy. Phil. Trans. R. Soc. Lond. B. 308, 67–78. doi: 10.1098/rstb.1985.0010
Furukawa, E., Bado, P., Tripp, G., Mattos, P., Wickens, J. R., Bramati, I. E., et al. (2014). Abnormal striatal BOLD responses to reward anticipation and reward delivery in ADHD. PLoS ONE 9:e89129. doi: 10.1371/journal.pone.0089129
Gauthier, J. M., Tassin, D. H., Dwoskin, L. P., and Kantak, K. M. (2014). Effects of dopamine D1 receptor blockade in the prelimbic prefrontal cortex or lateral dorsal striatum on frontostriatal function in Wistar and Spontaneously Hypertensive Rats. Behav. Brain Res. 15, 229–238. doi: 10.1016/j.bbr.2014.04.018
Gill, M., Daly, G., Heron, S., Hawi, Z., and Fitzgerald, M. (1997). Confirmation of association between attention deficit hyperactivity disorder and a dopamine transporter polymorphism. Mol. Psychiatry 2, 311–313. doi: 10.1038/sj.mp.4000290
Griffiths, K. R., Morris, R. W., and Balleine, B. W. (2014). Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders. Front. Syst. Neurosci. 8:101. doi: 10.3389/fnsys.2014.00101
Hammond, L. J. (1980). The effect of contingency upon the appetitive conditioning of free-operant behavior. J. Exp. Anal. Behav. 34, 297–304. doi: 10.1901/jeab.1980.34-297
Heal, D. J., Cheetham, S. C., and Smith, S. L. (2009). The neuropharmacology of ADHD drugs in vivo: insights on efficacy and safety. Neuropharmacology. 57, 608–618. doi: 10.1016/j.neuropharm.2009.08.020
Heal, D. J., Smith, S. L., Kulkarni, R. S., and Rowley, H. L. (2008). New perspectives from microdialysis studies in freely-moving, spontaneously hypertensive rats on the pharmacology of drugs for the treatment of ADHD. Pharmacol. Biochem. Behav. 90, 184–197. doi: 10.1016/j.pbb.2008.03.016
Hill, J., Herbst, K., and Sanabria, F. (2012). Characterizing operant hyperactivity in the Spontaneously Hypertensive Rat. Behav. Brain Funct. 8, 5. doi: 10.1186/1744-9081-8-5
Hill, R. (1970). “Facilitation of conditioned reinforcement as a mechanism of psychomotor stimulation,” in Amphetamines and Related Compounds, eds E. Costa and S. Garattini (New York: Raven Press), 781–795.
Hitchcott, P. K., Quinn, J. J., and Taylor, J. R. (2007). Bidirectional modulation of goal-directed actions by prefrontal cortical dopamine. Cereb. Cortex 17, 2820–2827. doi: 10.1093/cercor/bhm010
Hynd, G. W., Hern, K. L., Novey, E. S., Eliopulos, D., Marshall, R., Gonzalez, J. J., et al. (1993). Attention deficit-hyperactivity disorder and asymmetry of the caudate nucleus. J. Child Neurol. 8, 339–347. doi: 10.1177/088307389300800409
Kantak, K. M., Singh, T., Kerstetter, K. A., Dembro, K. A., Mutebi, M. M., Harvey, R. C., et al. (2008). Advancing the spontaneous hypertensive rat model of attention deficit/hyperactivity disorder. Behav. Neurosci. 122, 340–357. doi: 10.1037/0735-7044.122.2.340
Knardahl, S., and Sagvolden, T. (1979). Open-field behavior of spontaneously hypertensive rats. Behav. Neural Biol. 27, 187–200. doi: 10.1016/S0163-1047(79)91801-6
Lex, B., and Hauber, W. (2010). The role of dopamine in the prelimbic cortex and the dorsomedial striatum in instrumental conditioning. Cereb. Cortex 20, 873–883. doi: 10.1093/cercor/bhp151
Matsumoto, K., Suzuki, W., and Tanaka, K. (2003). Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 11, 229–232. doi: 10.1126/science.1084204
Matsumoto, K., and Tanaka, K. (2004). The role of the medial prefrontal cortex in achieving goals. Curr. Opin. Neurobiol. 14, 178–185. doi: 10.1016/j.conb.2004.03.005
Mehta, M. A., Calloway, P., and Sahakian, B. J. (2000). Amelioration of specific working memory deficits by methylphenidate in a case of adult attention deficit/hyperactivity disorder. J. Psychopharmacol. 14, 299–302. doi: 10.1177/026988110001400314
Mehta, M. A., Goodyer, I. M., and Sahakian, B. J. (2004). Methylphenidate improves working memory and set-shifting in AD/HD: relationships to baseline memory capacity. J. Child Psychol. Psychiatry Allied Discipl.. 45, 293–305. doi: 10.1111/j.1469-7610.2004.00221.x
Mulder, A. B., Nordquist, R. E., Orgut, O., and Pennartz, C. M. (2003). Learning-related changes in response patterns of prefrontal neurons during instrumental conditioning. Behav. Brain Res. 30, 77–88. doi: 10.1016/j.bbr.2003.09.016
Naneix, F., Marchand, A. R., Di Scala, G., Pape, J. R., and Coutureau, E. (2009). A role for medial prefrontal dopaminergic innervation in instrumental conditioning. J. Neurosci. 20, 6599–6606. doi: 10.1523/JNEUROSCI.1234-09.2009
Okamoto, K., and Aoki, K. (1963). Development of a strain of spontaneously hypertensive rats. Jpn. Circul. J. 27, 282–293. doi: 10.1253/jcj.27.282
Ostlund, S. B., and Balleine, B. W. (2005). Lesions of medial prefrontal cortex disrupt the acquisition but not the expression of goal-directed learning. J. Neurosci. 24, 7763–7770. doi: 10.1523/JNEUROSCI.1921-05.2005
Ostlund, S. B., Winterbauer, N. E., and Balleine, B. W. (2009). Evidence of action sequence chunking in goal-directed instrumental conditioning and its dependence on the dorsomedial prefrontal cortex. J. Neurosci. 24, 8280–8287. doi: 10.1523/JNEUROSCI.1176-09.2009
Robbins, T. W. (1975). The potentiation of conditioned reinforcement by psychomotor stimulant drugs. A test of Hill’s hypothesis. Psychopharmacology 45, 103–114.
Robbins, T. W. (1978). The acquisition of responding with conditioned reinforcement: effects of pipradol, methylphenidate, d-amphetamine, and nomifensine. Psychopharmacology (Berl.) 58, 79–87. doi: 10.1007/BF00426794
Roessner, V., Sagvolden, T., Dasbanerjee, T., Middleton, F. A., Faraone, S. V., Walaas, S. I., et al. (2010). Methylphenidate normalizes elevated dopamine transporter densities in an animal model of the attention-deficit/hyperactivity disorder combined type, but not to the same extent in one of the attention-deficit/hyperactivity disorder inattentive type. Neuroscience 2, 1183–1191. doi: 10.1016/j.neuroscience.2010.02.073
Russell, V. (2003). Dopamine hypofunction possibly results from a defect in glutamate-stimulated release of dopamine in the nucleus accumbens shell of a rat model for attention deficit hyperactivity disorder–the spontaneously hypertensive rat. Neurosci. Biobehav. Rev. 27, 671–682. doi: 10.1016/j.neubiorev.2003.08.010
Russell, V., de Villiers, A., Sagvolden, T., Lamm, M., and Taljaard, J. (1995). Altered dopaminergic function in the prefrontal cortex, nucleus accumbens and caudate-putamen of an animal model of attention-deficit hyperactivity disorder–the spontaneously hypertensive rat. Brain Res. 10, 343–351. doi: 10.1016/0006-8993(95)00135-D
Sagvolden, T. (2000). Behavioral validation of the spontaneously hypertensive rat (SHR) as an animal model of attention-deficit/hyperactivity disorder (AD/HD). Neurosci. Biobehav. Rev. 24, 31–39. doi: 10.1016/S0149-7634(99)00058-5
Sagvolden, T., Hendley, E. D., and Knardahl, S. (1992)). Behavior of hypertensive and hyperactive rat strains: hyperactivity is not unitarily determined. Physiol. Behav. 52, 49–57. doi: 10.1016/0031-9384(92)90432-2
Sagvolden, T., Johansen, E. B., Aase, H., and Russell, V. A. (2005a). A dynamic developmental theory of attention-deficit/hyperactivity disorder (ADHD) predominantly hyperactive/impulsive and combined subtypes. Behav. Brain Sci. 28, 397–419. doi: 10.1017/S0140525X05000075 discussion 419–368,
Sagvolden, T., Russell, V. A., Aase, H., Johansen, E. B., and Farshbaf, M. (2005b). Rodent models of attention-deficit/hyperactivity disorder. Biol. Psychiatry 1, 1239–1247. doi: 10.1016/j.biopsych.2005.02.002
Sagvolden, T., Pettersen, M. B., and Larsen, M. C. (1993). Spontaneously hypertensive rats (SHR) as a putative animal model of childhood hyperkinesis: SHR behavior compared to four other rat strains. Physiol. Behav. 54, 1047–1055. doi: 10.1016/0031-9384(93)90323-8
Schiffer, W. K., Volkow, N. D., Fowler, J. S., Alexoff, D. L., Logan, J., and Dewey, S. L. (2006). Therapeutic doses of amphetamine or methylphenidate differentially increase synaptic and extracellular dopamine. Synapse 15, 243–251. doi: 10.1002/syn.20235
Tripp, G., and Wickens, J. (2012). Reinforcement, dopamine and rodent models in drug development for ADHD. Neurotherapeutics 9, 622–634. doi: 10.1007/s13311-012-0132-y
Tripp, G., and Wickens, J. R. (2008). Research review: dopamine transfer deficit: a neurobiological theory of altered reinforcement mechanisms in ADHD. J. Child Psychol. Psychiatry Allied Discipl. 49, 691–704. doi: 10.1111/j.1469-7610.2007.01851.x
Tripp, G., and Wickens, J. R. (2009). Neurobiology of ADHD. Neuropharmacology 57, 579–589. doi: 10.1016/j.neuropharm.2009.07.026
Volkow, N. D., Fowler, J. S., Wang, G., Ding, Y., and Gatley, S. J. (2002). Mechanism of action of methylphenidate: insights from PET imaging studies. J. Atten. Disord. 6(Suppl. 1), S31–S43.
Waldman, I. D., Rowe, D. C., Abramowitz, A., Kozel, S. T., Mohr, J. H., Sherman, S. L., et al. (1998). Association and linkage of the dopamine transporter gene and attention-deficit hyperactivity disorder in children: heterogeneity owing to diagnostic subtype and severity. Am. J. Hum. Genet. 63, 1767–1776. doi: 10.1086/302132
Wilens, T. E. (2008). Effects of methylphenidate on the catecholaminergic system in attention-deficit/hyperactivity disorder. J. Clin. Psychopharmacol. 28, S46–S53. doi: 10.1097/JCP.0b013e318173312f
Wultz, B., and Sagvolden, T. (1992). The hyperactive spontaneously hypertensive rat learns to sit still, but not to stop bursts of responses with short interresponse times. Behav. Genet. 22, 415–433. doi: 10.1007/BF01066613
Wultz, B., Sagvolden, T., Moser, E. I., and Moser, M. B. (1990). The spontaneously hypertensive rat as an animal model of attention-deficit hyperactivity disorder: effects of methylphenidate on exploratory behavior. Behav. Neural Biol. 53, 88–102. doi: 10.1016/0163-1047(90)90848-Z
Yin, H. H., Knowlton, B. J., and Balleine, B. W. (2004). Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189. doi: 10.1111/j.1460-9568.2004.03095.x
Keywords: attention deficit hyperactivity disorder, spontaneously hypertensive rats, Wistar–Kyoto rats, goal-directed behavior, methylphenidate
Citation: Natsheh JY and Shiflett MW (2015) The Effects of Methylphenidate on Goal-directed Behavior in a Rat Model of ADHD. Front. Behav. Neurosci. 9:326. doi: 10.3389/fnbeh.2015.00326
Received: 11 July 2015; Accepted: 13 November 2015;
Published: 25 November 2015.
Edited by:
Allan V. Kalueff, ZENEREI Institute, USA; Guangdong Ocean University, China; St Petersburg State University, RussiaReviewed by:
Matthew O. Parker, University of Portsmouth, UKSandeep Sharma, University of Calgary, Canada
Copyright © 2015 Natsheh and Shiflett. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Joman Y. Natsheh, jomannatsheh@gmail.com