Amy R. Bland1
Jonathan P. Roiser2
Mitul A. Mehta3
Thea Schei4
Heather Boland1
Daniel K. Campbell-Meiklejohn5Richard A. Emsley6
Marcus R. Munafo7
Ian S. Penton-Voak7
Ana Seara-Cardoso8,9
Essi Viding8Valerie Voon10,11
Barbara J. Sahakian10,11
Trevor W. Robbins4,11
Rebecca Elliott1*- 1Neuroscience and Psychiatry Unit, University of Manchester, Manchester, UK
- 2Institute of Cognitive Neuroscience, University College London, London, UK
- 3Institute of Psychiatry, Psychology and Neuroscience, Kings College London, London, UK
- 4Department of Psychology, University of Cambridge, Cambridge, UK
- 5School of Psychology, University of Sussex, Brighton, UK
- 6Institute of Population Health, University of Manchester, Manchester, UK
- 7School of Experimental Psychology, University of Bristol, Bristol, UK
- 8Psychology and Language Sciences, University College London, London, UK
- 9School of Psychology, University of Minho, Guimaraes, Portugal
- 10Department of Psychiatry, University of Cambridge, Cambridge, UK
- 11Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, UK
In mental health practice, both pharmacological and non-pharmacological treatments are aimed at improving neuropsychological symptoms, including cognitive and emotional impairments. However, at present there is no established neuropsychological test battery that comprehensively covers multiple affective domains relevant in a range of disorders. Our objective was to generate a standardized test battery, comprised of existing, adapted and novel tasks, to assess four core domains of affective cognition (emotion processing, motivation, impulsivity and social cognition) in order to facilitate and enhance treatment development and evaluation in a broad range of neuropsychiatric disorders. The battery was administered to 200 participants aged 18–50 years (50% female), 42 of whom were retested in order to assess reliability. An exploratory factor analysis identified 11 factors with eigenvalues greater than 1, which accounted for over 70% of the variance. Tasks showed moderate to excellent test-retest reliability and were not strongly correlated with demographic factors such as age or IQ. The EMOTICOM test battery is therefore a promising tool for the assessment of affective cognitive function in a range of contexts.
Introduction
Mental health problems represent an extremely significant health burden, with global costs estimated at $2.5 trillion, projected to increase to $6.5 trillion by 2030, more than any other form of disease (Bloom et al., 2012; Fineberg et al., 2013). Impairments of emotional, motivational and social function are increasingly thought to be fundamental to the neurobehavioral pathology of psychiatric disorders and are becoming important targets for therapeutic intervention (Roiser et al., 2012). Major advances in treatment development will therefore be facilitated by well-designed, carefully validated measures of a comprehensive range of emotional, motivational, and social functions. This is critically important in clinical trials, of both of pharmacological and psychological interventions, which specifically aim to target emotional, motivational, and social processes. Currently, the outcome measures used in trials of such interventions are typically changes in clinical symptoms, and there is a pressing need for new outcome measures that quantitatively measure the effects of these treatments. A validated affective battery would also have important implications in other research contexts; for example, investigating cognitive profiles relevant to the NIMH Research Domain Criteria (RDoC) initiative that aims to create a new framework for mental health research (Insel et al., 2010; Sanislow et al., 2010) focusing on dimensions that cut across DSM diagnostic categories, investigating endophenotypes for genetic studies' or identifying biomarkers for high-risk individuals. However, at present there is no established neuropsychological test battery that offers a comprehensive assessment of “hot” cognitive functions.
Various individual tests have been developed and validated to test specific cognitive hypotheses. However, without standardization, it is difficult to make progress, replicate results, or identify gaps that need to be addressed (Elliott et al., 2011). Multi-center studies and clinical trials would benefit from a comprehensive, validated battery probing emotional, motivational, and social functions. The success of existing, standardized cognitive batteries highlights their recognized importance for assessing cognitive function. For example, the Cambridge Automated Neuropsychological Test Battery (CANTAB; Cambridge Cognition Ltd) has become a widely used battery in both academic research and clinical trials (Robbins et al., 1994, 1998; Cambridge Cognition Ltd). However, the focus is primarily on “cold” cognitive functions (Roiser and Sahakian, 2013) such as executive function, visuospatial memory and various types of attention. Here we generate normative data for a battery of neuropsychological tasks, which assesses a comprehensive range of processes relevant to affective cognition.
Affective Cognition
Affective cognition is a term used to describe aspects of cognitive function where stimuli have affective salience; the term “hot” cognition has been coined to distinguish these aspects of cognition from non-emotive “cold” cognitions (Roiser and Sahakian, 2013). Affective cognition, can be defined as reflecting an interface at which emotional and cognitive processes are integrated to generate behavior (Elliott et al., 2011).
Disrupted affective cognition is a core feature of many mental health disorders and cuts across DSM diagnostic categories. For example, biases in processing emotional stimuli have been observed in depression (Surguladze et al., 2004), anxiety (Mogg and Bradley, 2002), schizophrenia (Pomarol-Clotet et al., 2010), substance abuse (Ersche and Sahakian, 2007), eating disorders (Lovell et al., 1997), ADHD (Seymour et al., 2015), and phobic anxiety (Watts et al., 1986). Reward learning and motivation have been shown to be impaired in schizophrenia (Murray et al., 2007; Waltz et al., 2010), Parkinson's Disease (Voon et al., 2010), substance abuse (Park et al., 2010), affective disorders (Murphy et al., 2003), and ADHD (Thomas et al., 2015). Impulsivity has been in described in substance abuse (Voon et al., 2014), eating disorders (Mobbs et al., 2011), and ADHD (Malloy-Diniz et al., 2007). Finally, social cognition impairments have been demonstrated in autism (Happé and Frith, 1996), depression (Zahn et al., 2015), and schizophrenia (Fett et al., 2011). Social, emotional, and motivational aspects of cognition are thought to be key predictors of functional outcomes. Therefore, novel interventions targeting affective cognition may be effective for improving functional outcomes, as well as reducing symptoms. Examples include cognitive bias modification in depression (Baert et al., 2010; Roiser et al., 2012), social cognition training in schizophrenia (Combs et al., 2007), or pharmacological agents to promote social function such as oxytocin (Feifel et al., 2012).
A number of studies have explored the potential factor structure of affective cognition in mental health disorders. For example, the MATRICS project identified five sub-processes relevant to schizophrenia including, theory of mind, social perception, social knowledge, attributional bias, and emotional processing (Green et al., 2008). Others have identified four factors including perceiving emotions, facilitating thought, understanding emotions, and managing emotions (Mayer and Salovey, 1997; Mayer et al., 2003); or two factors including an emotional perception and understanding factor and an emotional facilitation and management factor (Eack et al., 2010).
However, despite a clear consensus that “hot” cognitive function is a multidimensional construct with many underlying sub-processes, no comprehensive battery assessing affective function is currently available. There are a number of batteries that include a limited number of affective tasks in predominately “cold” cognitive test batteries (e.g., CANTAB; www.cambridgecognition.com, MATRICS; www.matricsinc.org, CogState; www.cogstate.com, WebNeuro; www.brainresource.com). A recently developed explicit hot cognition battery, the Emotional Test Battery (ETB; www.p1vital.com), focuses on emotion processing tasks of particular relevance to depression.
In developing the test battery described here we chose to focus on four distinct domains of affective cognition: emotion processing, the ability to process and respond to affective stimuli, including emotional faces; motivation, the ability to learn, apply effort and make decisions driven by incentives; impulsivity, premature or risky responding; and social cognition, the ability to process information about situations involving interpersonal interactions. For each of these domains we piloted in 30 individuals a combination of novel, adapted and existing tasks designed to probe key underlying affective functions. We selected for inclusion in the final battery those tasks that were feasible in brief versions, readily understood and well-tolerated by participants and (for existing or adapted tasks) that elicited robust replication of previously observed effects. Further details of excluded tasks are available from the authors on request.
Emotional Processing
Emotion recognition/categorization
Recognition of facial expressions is a widely-used paradigm in neuropsychiatry, particularly in studies of depressed patients who tend to rate ambiguous expressions as more negative (Bouhuys et al., 1999; Surguladze et al., 2004). Harmer et al. (2011) argues that emotional face recognition may be a sensitive biomarker for effective antidepressant treatment. We therefore aimed to develop a task that effectively probed emotional facial recognition. The Emotion Recognition Task (ERT) included in the CANTAB battery (Cambridge Cognition Ltd) has proven to be a promising task examining emotion recognition in clinical populations. However, in order to include an ERT in EMOTICOM with limited time available, we opted to focus on basic emotions; happy, sad, anger and fear and chose to exclude more complex emotions such as surprise and disgust. We also adapted the task to include two versions; one that assessed facial recognition, similarly to the original CANTAB ERT and one that more specifically assessed eye recognition. Including emotional eyes recognition was motivated by evidence supporting the “reading the mind from the eyes” test (Baron-Cohen et al., 2001) as an effective assessment of the ability to recognize the emotional state of others using just the expressions around the eyes. We further adapted the task to include control conditions, i.e., identifying the age of a face and eyes, to provide baseline measurement in neuroimaging investigations.
Attentional bias
Biased emotional attention can be effectively measured using the affective go/no-go test (Cambridge Cognition Ltd). Attentional biases have been observed in depression (Murphy et al., 1999; Erickson et al., 2005), mania (Murphy et al., 1999), anxiety disorders (Watts et al., 1986; Mogg et al., 1995), substance abuse (Ersche and Sahakian, 2007), and eating disorders (Lovell et al., 1997). Negative biases in processing emotional stimuli have been suggested as an important biomarker for antidepressant efficacy and may predict responses to both psychological and pharmacological interventions (Harmer et al., 2009; Roiser et al., 2012). We therefore adapted two versions of the Affective Go No-Go task: one similar to the CANTAB with word stimuli and one with face stimuli. The motivation for adapting the AGN to include faces was to potentially improve any cross-cultural, educational, and age influences on the word version. For example, emotionally salient word stimuli may require a minimum reading level that may not be suitable for use in children. Indeed, a facial version of the AGN has shown to be a promising tool in pediatric anxiety and depression (Ladouceur et al., 2006). Additionally, an emotionally cued Posner task (Posner, 1980) using eye gaze in emotional facial expressions was piloted as part of the development of the EMOTICOM battery but did not show significant condition effects.
Emotional memory
Biased emotional memory for personal experiences has been suggested as an important trait marker for depression (Brittlebank et al., 1993). Depressed patients also show a more general bias toward remembering negative information (Hamilton and Gotlib, 2008) and patients with schizophrenia show deficits in remembering positive stimuli (Herbener et al., 2008) suggesting a possible double dissociation between the two disorders. We therefore developed an emotional memory task that required an encoding phase presented at the start of the EMOTICOM battery and a retrieval phase presented at the end in order to assess biases in emotional memory. We also piloted an emotional working memory task using a spatial n-back (for review see Owen et al., 2005) with emotional faces, however this did not produce sufficient significant condition effects.
Motivation and Reward
Reinforcement learning
Behavioral tests assessing reinforcement learning (RL) in humans are directly comparable to operant conditioning tasks used in animals (Roberts et al., 1988; Birrell and Brown, 2000). Human reinforcement learning tests typically involve learning which abstract stimuli predict winning or losing points or money (Owen et al., 1991; Pessiglione et al., 2006). Reinforcement learning, and corresponding responses in the brain's reward system, are reliably disrupted in several neuropsychiatric diseases, including schizophrenia (Murray et al., 2007; Waltz et al., 2010), Parkinson's Disease (Voon et al., 2010), alcohol dependence (Park et al., 2010), and depression (Murphy et al., 2003). One weakness of several tests is the conflation of reward and punishment learning (Cools et al., 2002). This is important, since reward and punishment may be subserved by separable, opponent processes in the brain (Daw et al., 2006). We therefore aimed to develop a novel reinforcement learning task that separated reward and punishment feedback in order to assess sensitivity to these independently.
Incentive motivation
Tests of incentive motivation measure how much effort an individual is prepared to exert to gain reward. The monetary incentive delay (MID) functional neuroimaging task features a speeded response to obtain a reward or avoid a loss (Knutson et al., 2001). However, the behavioral measure arising from this paradigm has seldom been shown to be altered by diagnosis or pharmacological manipulation (Knutson et al., 2004; Scheres et al., 2007). Indeed, the MID continually updates the threshold for success, which might reduce behavioral differences between conditions. Hence we aimed to develop an incentive motivation task that produced reliable behavioral differences that have the potential to provide important biomarkers for assessment and treatment interventions. We adapted the Salience Attribution Task (Roiser et al., 2009a) which has previously shown robust behavioral markers of adaptive motivational salience in Schizophrenia and developed a version that specifically evaluated motivation relating to reward and punishment separately.
Value-based choice
Tests of value-based choice investigate how subjects use different types of information (e.g., probability, reward, punishment) in order to guide economic decision-making. In contrast to tests of reinforcement learning, there is typically no learning component in tests of value-based choice. As such, the widely-used Iowa Gambling Task (Bechara et al., 1994) is not a specific test of value-based choice, since it also involves learning. The Cambridge Gamble Task (CGT: Rogers et al., 1999), part of the CANTAB suite of tests, asks subjects to decide on which of two options to bet, and to stake a certain percentage of their points on this bet. The CGT is sensitive to unipolar (Murphy, et al., 2001) and bipolar depression (Roiser et al., 2009b), schizophrenia (Hutton et al., 2002), and psychopharmacological manipulation (Rogers et al., 1999). However, it cannot determine whether decision-making is influenced by reward seeking or punishment avoidance. A later development (Rogers et al., 2003) can distinguish between these and is sensitive to several neuropsychiatric conditions (Roiser et al., 2006; Chandler et al., 2009) and psychopharmacological manipulations (Scarna et al., 2005), but includes a very restricted set of probabilities. We therefore adapted the CANTAB CGT (Cambridge Cognition Ltd) to investigate reward seeking and punishment avoidance separately.
Impulsivity
Waiting impulsivity
Coordination between initiation and inhibition of actions is required for successful behavior. Patients with ADHD (Aron and Poldrack, 2005), obsessive compulsive disorder (Malloy-Diniz et al., 2007), and schizophrenia (Kaladjian et al., 2007) show impairments in impulsivity. The four choice serial reaction time task (4-CSRTT) is a novel translation from the widely used 5-choice serial reaction time rodent task (5-CSRTT; Robbins, 2002). It has demonstrated clear deficits in substance abuse (Voon et al., 2014) and is sensitive to effects of dietary tryptophan depletion which is thought to reduce central 5-HT (Worbe et al., 2014). We therefore decided to incorporate the 4CSRTT (Voon et al., 2014) into the EMOTICOM battery which measures incentive motivation to rewards and premature responses elicited by anticipated reward.
Delay and probability discounting
Another aspect of impulsivity is the preference for immediate gratification, even when waiting longer might lead to higher absolute gain. Delay discounting is the progressive reduction in subjective value of a reinforcer with time. It can be assessed using two types of task—hypothetical or experiential. Hypothetical discounting tasks require choices between immediate (e.g., £1 now) and delayed (e.g., £5 in 1 month) rewards (Mazur, 1987; Green et al., 1996; Kirby, 2009). The experiential discounting tasks differs from hypothetical in that respondents directly experience the delay and receive the reward during the task (Reynolds and Schiffbauer, 2004). Patients with ADHD and substance use disorders show steeper discounting rates in such tasks, which also show good temporal stability similar to personality traits (Ohmura et al., 2006; Kirby, 2009). We therefore developed a computerized delay discounting task based on Richards et al.'s (1999) adjustment procedure.
Social Cognition
Moral emotion
Moral emotions can be experimentally induced either in response to verbal descriptions or pictures of specific interpersonal behavior (Moll and de Oliveira-Souza, 2007) or behavior contravening normal social values (Zahn et al., 2009). Patients with ventromedial prefrontal (VMPFC) lesions show abnormal responses to hypothetical moral dilemmas (Ciaramelli et al., 2007; Koenigs et al., 2007) and patients with antisocial personality disorder (Blair, 1995) and Autism (Moran et al., 2011) show deficits in moral judgment. We developed a novel computerized Moral Emotions task that comprising of cartoon scenarios rather than lengthy vignettes that are more likely to be affected by reading ability, intelligence and age.
Theory of mind
Theory of Mind (TOM) refers to the ability to infer the mental states of others (Frith and Frith, 2003). A number of paradigms have been proposed to probe this function including false belief tasks (Frith and Corcoran, 1996), “faux pas” tests, visual jokes, understanding irony and the “Reading the Mind in the Eyes” test (Baron-Cohen et al., 2001; although note that this is most similar to an emotional recognition task—see above). Patients with autism typically show impaired TOM (Happé and Frith, 1996) and it is also sensitive to schizophrenia (Frith and Corcoran, 1996; Bora et al., 2009; Fett et al., 2011). While valuable in populations with overt impairment, existing TOM tasks are typically insensitive to variation in normal adult performance as most participants perform at ceiling. Therefore, we developed a complementary task that depicted ambiguous social situations with no right or wrong answer, thus allowing greater variation of responses in healthy volunteers. Rather than assessing whether participants have TOM ability, this task assesses the extent to which people choose to use TOM information.
Social economic exchange games
Economic games, such as the Ultimatum Game and Prisoners' Dilemma are popular tasks for exploring the neurobiology of social decision-making (King-Casas et al., 2005; Miller, 2005; Fehr and Camerer, 2007; Crockett, 2009). A number of patient groups have been studied using these games, including psychopathy (Koenigs et al., 2010; Rilling et al., 2015), schizophrenia (Agay et al., 2008), autism (Andari et al., 2010), depression (Pulcu et al., 2015), and borderline personality disorder (King-Casas et al., 2008; Seres et al., 2009; Unoka et al., 2009). It has been argued that these games may provide specific and sensitive biomarkers for social pathologies (Kishida et al., 2010). Traditionally, these games are long and involve a complex set-up with multiple players, which are unsuitable for neuropsychological testing. Therefore, we developed a simple one-player game of the Ultimatum Game and Prisoners' Dilemma, which probe social interaction within the context of a test battery.
Social decision making
Optimal decision-making in social contexts recruits a combination of associative and inferential computations. For example, one may have first-hand experience, one may observe choices of other people (or receive a recommendation), and one may infer the knowledge or intentions of the others to weight the influence of their decision. Therefore, we included the Inference Task which approximates the contribution of each of these processes to decision-making. Specifically, it employs the useful heuristic of confidence which can be used to infer the certainty of an agent's information and weight the influence of his/her endorsement on privately held beliefs (Thomas and McFadyen, 1995). The effects of such inferences on value computation are hypothesized to underlie the reassuring influence of another's confidence and generate distinct representations of value in the subject. Successful task performance requires cue combination, the integration of value computations, and theoretically, social inferences of other people's knowledge.
Aims and Objectives
The specific aims of the project were to: (a) generate a computerized test battery assessing multiple aspects of “hot” cognition; (b) demonstrate ease of administration, feasibility, and tolerability; (c) standardize the test battery in a large cohort of healthy volunteers, including an exploratory factor analysis to identify important, independent constructs; and (d) establish measurement stability in a smaller sample of healthy volunteers.
Hypotheses
We hypothesized a factor analysis would reveal that the tasks would probe affective function best explained by a four factor model mapping onto emotion processing, motivation, impulsivity, and social cognition. We further hypothesized that tasks without a learning component would show at least moderate test-retest reliability.
Materials and Methods
Participants
Two hundred healthy volunteers were assessed (see Table 1 for demographic characteristics), 42 of whom were re-tested within 5–10 days in order to assess test re-test reliability. This will furnish sufficient power to detect test-retest reliability of >0.35 (p = 0.05, 80% power). Potential participants were recruited via advertisements in the local community and on social media. Following telephone screening, participants were included if they met the following criteria: 18–50 years old; no self-reported previous or current psychiatric disorders, including depression, anxiety, eating disorders, and drug/alcohol dependence; no neurological disorders; no significant head injury resulting in unconsciousness; no current use of medication known to affect mood or cognition; no first-degree relatives suffering from any psychiatric disorders; smoked fewer than five cigarettes per day; drank less than the government guidelines for weekly alcohol intake (www.drinkaware.co.uk); and fluent in English. Participants completed the Brief Symptom Inventory (Derogatis and Melisaratos, 1983), meeting the criteria for adult non-patients across nine symptom dimensions; somatisation, obsessive compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism. Participants were further interviewed using the Mini International Neuropsychiatric Interview (Sheehan et al., 1998) to exclude any psychopathology.
Table 1. Demographic characteristics of sample (N = 200), stratified by age, IQ, gender, and ethnicity for the standardization of the EMOTICOM neuropsychological test battery.
Eligible participants were invited to attend a 3.5-h appointment at the Neuroscience and Psychiatry Unit, University of Manchester or the Behavioral and Clinical Neuroscience Institute, University of Cambridge. Participants provided written informed consent after the study procedures were explained, and their IQ was estimated using the WTAR (Wechsler, 2008). This study was approved by the University of Manchester and the University of Cambridge Research Ethics Committees.
Design
Participants completed 16 neuropsychological tests programmed in PsychoPy (Peirce, 2007) on a touchscreen laptop (Dell XT3). The tasks were administered in a quiet testing room over 3 h. Some participants chose to complete the tasks over two sessions no longer than 1 week apart. The tests were administered in a randomized sequence to eliminate systematic effects of fatigue. Participants were reimbursed for their time and travel expenses, they also received an additional bonus of up to £10, calculated on the basis of the average money won on tasks that involved a monetary incentive.
Neuropsychological Tasks
Analysis
All analyses were performed with SPSS statistical software (IBM SPSS Statistics Version 20.0).
Factor Analysis
The measures thought to be most reflective of the constructs investigated were standardized using z-scores (after transformation if appropriate) and entered into a factor analysis to determine the underlying latent variable structure of the data. Here, we conducted an exploratory factor analysis to identify the number of factors needed to maximize the amount of variance explained. An eigenvalue cut-off of 1 was used to determine whether a factor explained sufficient variability in the data. The method employed utilized varimax rotation with Kaiser normalization.
Reliability Analysis
The reliability and stability of the tasks was assessed by comparing performance in 42 volunteers who competed the battery on two occasions, 5–10 days apart. Test–retest was assessed by calculating the average-measures intraclass correlation coefficient using a two-way mixed effects model, which controls for overall changes in performance between sessions (i.e., repetition effects). Different guidelines exist for the interpretation of the ICC. Here we take an ICC value of less than 0.40 to be poor, 0.41–0.59 as fair, 0.60–0.74 as good and values exceeding 0.75 as excellent (Fleis et al., 2003). These terms should be interpreted with caution as they do not take into account the confidence intervals of the ICC measure.
Correlation Analysis
In an exploratory supplemental analysis, two tailed Pearson's correlations were used to correlate task performance with demographic measures such as age, IQ and years of education. Gender differences were examined using independent samples t-tests. The statistical significance of all correlations were corrected for multiple comparisons (0.05/n; n = number of task variables).
Task Variables
For each task there are a number of possible outcome measures. For the factor analysis, test-retest analysis and correlations with demographic variables, which are the focus of the present publication, we chose the primary outcome measures outlined in Table 2.
Table 2. Full list of neuropsychological tasks with outcome measures which are included in the EMOTICOM neuropsychological test battery.
Results
Standardization
A summary of the means and standard deviations can be found in Table 3.
Table 3. Summary of the means and standard deviations.
Factor Analysis
Data from all participants were entered into the factor analysis. The results of the varimax rotation for the tasks are shown in Table 4. An eleven-factor solution was derived based on eigenvalues greater than 1, which cumulatively accounted for 70% of the variance (see Figure 1). Only factor loadings greater than 0.40 are shown. Data were assessed for the adequacy of factor analytic methods. Bartlett's test was highly significant [χ.72, p < 0.001], suggesting that variable correlations did not form an identity matrix. Measures of sampling adequacy were also sufficient (KMO = 0.54).
Table 4. Summary of the factor loadings for EMOTICOM tests on factors 1–11.
Figure 1. Eleven-factor solution derived from an exploratory factor analysis.
Factor 1 represents affective biases in emotional recognition whereas Factors 2 and 3 capture affective biases in reaction times. Factor 4 contains tasks that have an element of value adjustment. Bias in emotional memory and probability discounting load onto Factor 5. Factor 6 represents measures of probabilistic decision making. Factor 7 represents latency measures of incentive motivation. Factor 8 represents social cognition, specifically moral emotions. Factor 9 represents social information preference and Factor 10 captures cooperation in social exchange games. Finally, Factor 11 loads onto social decision making. The 4CSRTT was omitted from the analysis in order to retain the full sample of participants; however, when running the factor analysis with the motivational index included, this variable loads onto factors 2 and 3 (affective biases in RTs).
Test-Retest Reliability
Test-retest reliability results are summarized in Table 5.
Table 5. Test re-test reliability.
Associations with Demographic Factors
Demographic factors associated with test performance are listed in Table 6.
Table 6. Association between tasks and demographic characteristics.
Discussion
Neuropsychological test batteries are vital tools for assessing the efficacy of treatment in neuropsychiatric disorders. In order to provide valid assessments of cognitive function, a neuropsychological test battery must possess good test retest reliability and examine a variety of cognitive functions with little redundancy. A further requirement of a test battery specifically assessing emotional and social function is that it should be (at least to some extent) independent of cognitive ability or IQ. In this paper we have presented data from 200 participants' performance to demonstrate that these requirements are met by the EMOTICOM neuropsychological test battery. This battery draws upon adaptations of pre-existing tasks as well as novel tasks in order to provide a comprehensive assessment of emotion processing, rewards and motivation, impulsivity and social cognition.
An exploratory factor analysis identified 11 factors, many of them loading onto a single task. Not all the factors are readily explicable and factors including variables with poor reliability should be viewed with considerable caution. We therefore do not attempt to draw conclusions about the meaning of individual factors. Rather we suggest that the central conclusion is simply that the tasks measure multiple constructs and therefore the battery has little redundancy. Our hypothesis of a four factor solution was categorially disproved suggesting that our prior operational concept of four domains was an over-simplification. This highlights the importance of administering multiple tests in order to assess these “hot” cognitive processes. Various reviews and meta-analyses have identified multiple domains of social cognition (Green and Leitman, 2008; Savla et al., 2012), however existing standardized batteries such as the MATRICS Consensus Cognitive Battery (MCCB; www.matricsinc.org) and CANTAB contain only one task targeting social cognition. The results presented here clearly indicate that there are different components of “hot” cognition that load onto multiple factors and therefore cannot be captured by a single test. Therefore, the EMOTICOM test battery provides a more comprehensive assessment of performance in a variety of affective processes and represents a significant advance over batteries including only a single test.
The majority of EMOTICOM tasks also showed moderate to excellent test-retest reliability. This is extremely important for assessing the efficacy of treatments and interventions, where it is important that differences in task performance can be attributed to effects of the interventions rather than methodological issues or random fluctuations. Furthermore, we demonstrate that our “hot” cognitive tasks have comparable retest reliability to traditional “cold” cognitive tasks (e.g., Lowe and Rabbitt, 1998). However, reliability of the reinforcement learning outcome variable was poor, consistent with previous observations that learning and memory tasks often do not exhibit good re-test reliability (Lowe and Rabbitt, 1998; Dikmen et al., 1999). Learning on these tasks transfers from the first session to the second. Such learning transfer results in significantly improved scores and lower variability at session 2, as we observed here. Given this poor reliability, the EMOTICOM reinforcement learning task could potentially be improved by creating parallel versions using different stimuli, although participants are still likely to be able to generalize rule-learning from the first session. Reliability of bias measures in the Affective Go No Go and Monetary Incentive Reward tasks were also poor. Bias reliability scores in reaction times are often reported to be much lower than mean RTs from each condition (Eide et al., 2002; Strauss et al., 2005; Brown et al., 2014) and our results are therefore comparable with previous studies. Poor test-retest reliability on specific tasks suggests caution in using these measures in longitudinal contexts with healthy volunteers, however it does not preclude the use of these tasks in between-group studies with patient populations.
The majority of EMOTICOM tasks were not strongly correlated with demographic factors such as age, years in education or IQ suggesting that performance of these tasks is not dependent upon general intellectual function. There are a few exceptions: the risk adjustment measure from the loss condition in the adapted Cambridge Gambling Task and the delay discounting measures were correlated with IQ, with delay discounting also being correlated with years in education. Previous studies have also suggested that gambling (Demaree et al., 2010; Webb et al., 2014) and delay discounting (Shamosh and Gray, 2008) correlate with intelligence. Therefore, it is recommended that studies using these measures take particular care to control for IQ and years of education. Interestingly we observed emotional bias measures in the face and eyes emotional recognition task to be significantly correlated with age, such that biases became more positive with increasing age. This finding supports a line of research that has recently gathered momentum, with many recent studies demonstrating that people attend to and remember positive information more as they get older (e.g., Mather and Carstensen, 2003; Reed and Carstensen, 2012). In spite of a prevailing view that hot cognitive tests are dependent on gender, we only observed a significant effect of gender in the Moral Emotions Task, whereby females show greater guilt ratings compared to males. This is in line with existing meta-analyses showing that women tend to experience negative emotions, such as guilt, more intensely than men (Else-Quest et al., 2012). This task may therefore be useful in understanding gender differences in treatment outcomes, particularly in terms of self-blame biases and their suggested link to a vulnerability to depression (Green et al., 2013).
Limitations
The ethnic characteristics of our sample of 200 participants was representative of the UK demographic (Office for National Statistics, 2011). Nevertheless, caution is recommended in generalizing these findings across cultures. Evidence suggests that cultural variations are evident in affective cognition. For example cultural variations have been observed in emotional facial recognition (Prado et al., 2014) economical games such as the Ultimatum Game and Prisoners' Dilemma (Oosterbeek et al., 2004; Wong and Hong, 2005) and arguably moral judgment (Gibbs et al., 2007). Such differences observed in performance across cultures suggest care in generalizing performance on UK validated and standardized tasks to other cultures. Another limitation is that we were not able to enter all the task variables into the factor analysis due to the reduced number of participants who completed some of the tasks. For instance, the progressive ratio parameters were improved part way through the study and so data were only available from 78 participants. Similarly, only a subset of participants completed the 4CSRTT. Therefore, in order to increase power and retain the full participant sample, the decision was made to omit these measures from factor analysis. A limitation of the test-retest reliability component was that we only assessed reliability over a short duration; in future it will important to assess longer durations to determine the potential value of the tasks in different intervention contexts.
In summary, we have demonstrated the potential power of the EMOTICOM test battery for the assessment of affective cognitive function. We have shown that affective cognition is far from a unitary construct, implying that assessment of multiple aspects of affective cognition is required. Our 16 task battery has little redundancy from the 11 factor underlying structure. We have also demonstrated that the majority of tasks have moderate to excellent test-retest reliability and are not strongly correlated with demographic factors such as IQ. We therefore conclude that the EMOTICOM test battery meets certain key criteria for a useful and valid tool with potential utility in clinical trials and studies investigating psychiatric disorders and relevant treatment interventions.
Important future directions include validation in patients and validation in intervention studies in both healthy controls and patients in order to further investigate the utility of EMOTICOM test battery, and diagnosis-appropriate subsets of tasks, as an investigative tool in mental health research. This will enable us assess which tasks are most valid, sensitive and reliable for use in particular patient populations and which can be used as outcome measures in intervention trials.
Funding
This work was supported by the MRC under Grant MR/J011894/1.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The reviewer YW and the handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.
Acknowledgments
We thank our colleagues at the University of Manchester, University of Cambridge, University College London and Kings College London for their contribution to discussions. In particular we thank Dr Vincent Valton (UCL) and Dr Sanja Abbott (University of Cambridge) for assistance with analyses. We also thank Dr Jonathan Peirce (University of Nottingham) and Dr Martyn McFarquar (University of Manchester) for their guidance and expertise in PsychoPy.
References
Agay, N., Kron, S., Carmel, Z., Mendlovic, S., and Levkovitz, Y. (2008). Ultimatum bargaining behavior of people affected by schizophrenia. Psychiatry Res. 157, 39–46. doi: 10.1016/j.psychres.2006.03.026
Andari, E., Duhamel, J.-R., Zalla, T., Herbrecht, E., Leboyer, M., and Sirigu, A. (2010). Promoting social behavior with oxytocin in high-functioning autism spectrum disorders. Proc. Natl. Acad. Sci. U.S.A. 107, 4389–4394. doi: 10.1073/pnas.0910249107
Aron, A. R., and Poldrack, R. A. (2005). The cognitive neuroscience of response inhibition: relevance for genetic research in attention-deficit/hyperactivity disorder. Biol. Psychiatry 57, 1285–1292. doi: 10.1016/j.biopsych.2004.10.026
Baert, S., De Raedt Schacht, R. R., and Koster, E. H. W. (2010). Attentional bias training in depression: therapeutic effects depend on depression severity. J. Behav. Ther. Exp. Psychiatry 41, 265–274. doi: 10.1016/j.jbtep.2010.02.004
Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., and Plumb, I. (2001). The ‘reading the mind in the eyes’ test revised version: a study with normal adults, and adults with asperger syndrome or high-functioning autism. J. Child Psychol. Psychiatry 42, 241–251. doi: 10.1111/1469-7610.00715
Bechara, A., Damasio, A. R., Damasio, H., and Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50, 7–15. doi: 10.1016/0010-0277(94)90018-3
Birrell, J. M., and Brown, V. J. (2000). Medial frontal cortex mediates perceptual attentional set shifting in the rat. J. Neurosci. 20, 4320–4324.
Blair, R. J. (1995). A cognitive developmental approach to mortality: investigating the psychopath. Cognition 57, 1–29. doi: 10.1016/0010-0277(95)00676-P
Bloom, D. E., Cafiero, E., Jané-Llopis, E., Abrahams-Gessel, S., Bloom, L. R., Fathima, S., et al. (2012). The Global Economic Burden of Noncommunicable Diseases (No. 8712). Program on the Global Demography of Aging. Available online at: https://ideas.repec.org/p/gdm/wpaper/8712.html
Bora, E., Yucel, M., and Pantelis, C. (2009). Theory of mind impairment in schizophrenia: meta-analysis. Schizophr. Res. 109, 1–9. doi: 10.1016/j.schres.2008.12.020
Bouhuys, A. L., Geerts, E., and Gordijn, M. (1999). Gender-specific mechanisms associated with outcome of depression: perception of emotions, coping and interpersonal functioning. Psychiatry Res. 85, 247–261. doi: 10.1016/S0165-1781(99)00003-7
Bradshaw, C. M., and Killeen, P. R. (2012). A theory of behaviour on progressive ratio schedules, with applications in behavioural pharmacology. Psychopharmacology 222, 549–564. doi: 10.1007/s00213-012-2771-4
Brittlebank, A. D., Scott, J., Williams, J. M., and Ferrier, I. N. (1993). Autobiographical memory in depression: state or trait marker? Br. J. Psychiatry 162, 118–121. doi: 10.1192/bjp.162.1.118
Brown, H. M., Eley, T. C., Broeren, S., MacLeod, C., Rinck, M., Hadwin, J. A., et al. (2014). Psychometric properties of reaction time based experimental paradigms measuring anxiety-related information-processing biases in children. J. Anxiety Disord. 28, 97–107. doi: 10.1016/j.janxdis.2013.11.004
Chandler, R. A., Wakeley, J., Goodwin, G. M., and Rogers, R. D. (2009). Altered risk-aversion and risk-seeking behavior in bipolar disorder. Biol. Psychiatry 66, 840–846. doi: 10.1016/j.biopsych.2009.05.011
Ciaramelli, E., Muccioli, M., Ladavas, E., and di Pellegrino, G. (2007). Selective deficit in personal moral judgment following damage to ventromedial prefrontal cortex. Soc. Cogn. Affect. Neurosci. 2, 84–92. doi: 10.1093/scan/nsm001
Combs, D. R., Adams, S. D., Penn, D. L., Roberts, D., Tiegreen, J., and Stem, P. (2007). Social cognition and interaction training (SCIT) for inpatients with schizophrenia spectrum disorders. Prelim. Find. 91, 112–116. doi: 10.1016/j.schres.2006.12.010
Cools, R., Clark, L., Owen, A. M., and Robbins, T. W. (2002). Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J. Neurosci. 22, 4563–4567.
Crockett, M. J. (2009). The neurochemistry of fairness. Ann. N.Y. Acad. Sci. 1167, 76–86. doi: 10.1111/j.1749-6632.2009.04506.x
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., and Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature 441, 876–879. doi: 10.1038/nature04766
Demaree, H. A., Burns, K. J., and DeDonno, M. A. (2010). Intelligence, but not emotional intelligence, predicts iowa gambling task performance. Intelligence 38, 249–254. doi: 10.1016/j.intell.2009.12.004
Derogatis, L. R., and Melisaratos, N. (1983). The brief symptom inventory: an introductory report. Psychol. Med. 13, 595–605. doi: 10.1017/S0033291700048017
Dikmen, S. S., Heaton, R. K., Grant, I., and Temkin, N. R. (1999). TestReitan neuropsychological test battery. J. Inter. Neuropsychol. Soc. 5, 346–356.
Eack, S. M., Greeno, C. G., Pogue-Geile, M. F., Newhill, C. E., Hogarty, G. E., and Keshavan, M. S. (2010). Assessing social-cognitive deficits in schizophrenia with the mayer-salovey-caruso emotional intelligence test. Schizophr. Bull. 36, 370–380. doi: 10.1093/schbul/sbn091
Eide, P., Kemp, A., Silberstein, R. B., Nathan, P. J., and Stough, C. (2002). Test-Retest reliability of the emotional stroop task: examining the paradox of measurement change. J. Psychol. 136, 514–520. doi: 10.1080/00223980209605547
Elliott, R., Zahn, R., William Deakin, J. F., and Anderson, I. M. (2011). Affective cognition and its disruption in mood disorders. Neuropsychopharmacology 36, 153–182. doi: 10.1038/npp.2010.77
Else-Quest, N. M., Higgins, A., Allison, C., and Morton, L. C. (2012). Gender differences in self-conscious emotional experience: a meta-analysis. Psychol. Bull. 138, 947–981. doi: 10.1037/a0027930
Erickson, K., Drevets, W. C., Clark, L., Cannon, D. M., Bain, E. E., Zarate, C. A., et al. (2005). Mood-congruent bias in affective go/no-go performance of unmedicated patients with major depressive disorder. Am. J. Psychiatry 162, 2171–2173. doi: 10.1176/appi.ajp.162.11.2171
Ersche, K. D., and Sahakian, B. J. (2007). The neuropsychology of amphetamine and opiate dependence: implications for treatment. Neuropsychol. Rev. 17, 317–336. doi: 10.1007/s11065-007-9033-y
Fehr, E., and Camerer, C. F. (2007). Social neuroeconomics: the neural circuitry of social preferences. Trends Cogn. Sci. 11, 419–427. doi: 10.1016/j.tics.2007.09.002
Feifel, D., Macdonald, K., Cobb, P., and Minassian, A. (2012). adjunctive intranasal oxytocin improves verbal memory in people with schizophrenia. Schizophr. Res. 139, 207–210. doi: 10.1016/j.schres.2012.05.018
Fett, A.-K. J., Viechtbauer, W., Dominguez, M. G., Penn, D. L., van Os, J., and Krabbendam, L. (2011). The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci. Biobehav. Rev. 35, 573–588. doi: 10.1016/j.neubiorev.2010.07.001
Fineberg, N. A., Haddad, P. M., Carpenter, L., Gannon, B., Sharpe, R., Young, A. H., et al. (2013). The size, burden and cost of disorders of the brain in the UK. J. Psychopharmacol. (Oxford). 27, 761–770. doi: 10.1177/0269881113495118
Fleis, J. L., Levin, B., and Paik, M. C. (2003). Statistical Methods for Rates and Proportions. Hoboken, NJ: John Wiley & Sons.
Frith, C. D., and Corcoran, R. (1996). Exploring'theory of mind'in people with schizophrenia. Psychol. Med. 26, 521–530. doi: 10.1017/S0033291700035601
Frith, U., and Frith, C. D. (2003). Development and neurophysiology of mentalizing. Philos. Trans. R. Soc. Lond. Ser. B 358, 459–473. doi: 10.1098/rstb.2002.1218
Gibbs, J. C., Basinger, K. S., Grime, R. L., and Snarey, J. R. (2007). Moral judgment development across cultures: revisiting kohlberg's universality claims. Dev. Rev. 27, 443–500. doi: 10.1016/j.dr.2007.04.001
Green, L., Myerson, J., Lichtman, D., Rosen, S., and Fry, A. (1996). Temporal discounting in choice between delayed rewards: the role of age and income. Psychol. Aging 11, 79–84. doi: 10.1037/0882-7974.11.1.79
Green, M. F., and Leitman, D. I. (2008). Social cognition in schizophrenia. Schizophr. Bull. 34, 670–672. doi: 10.1093/schbul/sbn045
Green, M. F., Penn, D. L., Bentall, R., Carpenter, W. T., Gaebel, W., Gur, R. C., et al. (2008). Social cognition in schizophrenia: an nimh workshop on definitions, assessment, and research opportunities. Schizophr. Bull. 34, 1211–1220. doi: 10.1093/schbul/sbm145
Green, S., Lambon Ralph, M. A., Moll, J., Zakrzewski, J., William Deakin, J. F., Grafman, J., et al. (2013). The neural basis of conceptual-emotional integration and its role in major depressive disorder. Soc. Neurosci. 8, 417–433. doi: 10.1080/17470919.2013.810171
Hamilton, J. P., and Gotlib, I. H. (2008). Neural substrates of increased memory sensitivity for negative stimuli in major depression. Biol. Psychiatry 63, 1155–1162. doi: 10.1016/j.biopsych.2007.12.015
Happé, F., and Frith, U. (1996). The neuropsychology of autism. Brain 119, 1377–1400. doi: 10.1093/brain/119.4.1377
Harmer, C. J., Goodwin, G. M., and Cowen, P. J. (2009). Why do antidepressants take so long to work? a cognitive neuropsychological model of antidepressant drug action. Br. J. Psychiatry 195, 102–108. doi: 10.1192/bjp.bp.108.051193
Harmer, C. J., Cowen, P. J., and Goodwin, G. M. (2011). Efficacy markers in depression. J. Psychopharmacol. 25, 1148–1158. doi: 10.1177/0269881110367722
Herbener, E. S., Song, W., Khine, T. T., and Sweeney, J. A. (2008). What aspects of emotional functioning are impaired in schizophrenia? Schizophr. Res. 98, 239–246. doi: 10.1016/j.schres.2007.06.025
Hodos, W. (1961). Progressive ratio as a measure of reward strength. Science 134, 943–944. doi: 10.1126/science.134.3483.943
Hutton, S. B., Murphy, F. C., Joyce, E. M., Rogers, R. D., Cuthbert, I. T., Robbins, T. W., et al. (2002). Decision making deficits in patients with first-episode and chronic schizophrenia. Schizophr. Res. 55, 249–257. doi: 10.1016/S0920-9964(01)00216-X
Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., et al. (2010). Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am. J. Psychiatry 167, 748–751. doi: 10.1176/appi.ajp.2010.09091379
Kaladjian, A., Jeanningros, R., Azorin, J.-M., Grimault, S., Anton, J.-L., and Mazzola-Pomietto, P. (2007). Blunted activation in right ventrolateral prefrontal cortex during motor response inhibition in schizophrenia. Schizophr. Res. 97, 184–193. doi: 10.1016/j.schres.2007.07.033
King-Casas, B., Sharp, C., Lomax-Bream, L., Lohrenz, T., Fonagy, P., and Montague, P. R. (2008). The rupture and repair of cooperation in borderline personality disorder. Science (N.Y.) 321, 806–810. doi: 10.1126/science.1156902
King-Casas, B., Tomlin, D., Anen, C., Camerer, C. F., Quartz, S. R., and Montague, P. R. (2005). Getting to know you: reputation and trust in a two-person economic exchange. Science 308, 78–83. doi: 10.1126/science.1108062
Kirby, K. N. (2009). One-year temporal stability of delay-discount rates. Psychon. Bull. Rev. 16, 457–462. doi: 10.3758/PBR.16.3.457
Kishida, K. T., King-Casas, B., and Montague, P. R. (2010). Neuroeconomic approaches to mental disorders. Neuron 67, 543–554. doi: 10.1016/j.neuron.2010.07.021
Knutson, B., Adams, C. M., Fong, G. W., and Hommer, D. (2001). Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 21, RC159.
Knutson, B., Bjork, J. M., Fong, G. W., Hommer, D., Mattay, V. S., and Weinberger, D. R. (2004). Amphetamine modulates human incentive processing. Neuron 43, 261–269. doi: 10.1016/j.neuron.2004.06.030
Koenigs, M., Kruepke, M., and Newman, J. P. (2010). Economic decision-making in psychopathy: a comparison with ventromedial prefrontal lesion patients. Neuropsychologia 48, 2198–2204. doi: 10.1016/j.neuropsychologia.2010.04.012
Koenigs, M., Young, L., Adolphs, R., Tranel, D., Cushman, F., Hauser, M., et al. (2007). Damage to the prefrontal cortex increases utilitarian moral judgements. Nature 446, 908–911. doi: 10.1038/nature05631
Ladouceur, C. D., Dahl, R. E., Williamson, D. E., Birmaher, B., Axelson, D. A., Ryan, N. D., et al. (2006). Processing emotional facial expressions influences performance on a Go/NoGo task in pediatric anxiety and depression. J. Child Psychol. Psychiatry 47, 1107–1115. doi: 10.1111/j.1469-7610.2006.01640.x
Lovell, D. M., Mark, J., Williams, G., and Hill, A. B. (1997). Selective processing of shape−related words in women with eating disorders, and those who have recovered. Br. J. Clin. Psychol. 36, 421–432. doi: 10.1111/j.2044-8260.1997.tb01249.x
Lowe, C., and Rabbitt, P. (1998). Test\re-Test reliability of the CANTAB and ISPOCD neuropsychological batteries: theoretical and practical issues. Neuropsychologia 36, 915–923. doi: 10.1016/S0028-3932(98)00036-0
Malloy-Diniz, L., Fuentes, D., Borges Leite, W., Correa, H., and Bechara, A. (2007). Impulsive behavior in adults with attention deficit/hyperactivity disorder: characterization of attentional, motor and cognitive impulsiveness. J. Inter. Neuropsychol. Soc. 13, 693–698. doi: 10.1017/s1355617707070889
Mather, M., and Carstensen, L. L. (2003). Aging and attentional biases for emotional faces. Psychol. Sci. 14, 409–415. doi: 10.1111/1467-9280.01455
Mayer, J. D., and Salovey, P. (1997). “What is emotional intelligence?,” in Emotional Development and Emotional Intelligence: Educational Implications, eds P. Salovey and D. J. Sluyter (New York, NY: Harper Collins), 3–34.
Mayer, J. D., Salovey, P., Caruso, D. R., and Sitarenios, G. (2003). Measuring emotional intelligence with the MSCEIT V2.0. Emotion 3, 97–105.
Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. Qt. Analys. Behav. 5, 55–73.
Miller, G. (2005). Economic game shows how the brain builds trust. Science 308:36. doi: 10.1126/science.308.5718.36a
Mobbs, O., Iglesias, K., Golay, A., and Van der Linden, M. (2011). Cognitive deficits in obese persons with and without binge eating disorder. Investigation using a mental flexibility task. Appetite 57, 263–271. doi: 10.1016/j.appet.2011.04.023
Mogg, K., and Bradley, B. P. (2002). Selective orienting of attention to masked threat faces in social anxiety. Behav. Res. Ther. 40, 1403–1414. doi: 10.1016/S0005-7967(02)00017-7
Mogg, K., Bradley, B. P., and Williams, R. (1995). Attentional bias in anxiety and depression: the role of awareness. Br. J. Clin. Psychol. 34, 17–36. doi: 10.1111/j.2044-8260.1995.tb01434.x
Moll, J., and de Oliveira-Souza, R. (2007). Moral judgments, emotions and the utilitarian brain. Trends Cogn. Sci. 11, 319–321. doi: 10.1016/j.tics.2007.06.001
Moran, J. M., Young, L. L., Saxe, R., Lee, S. M., O'Young, D., Mavros, P. L., et al. (2011). Impaired theory of mind for moral judgment in high-functioning autism. Proc. Natl. Acad. Sci. U.S.A. 108, 2688–2692. doi: 10.1073/pnas.1011734108
Murphy, F. C., Michael, A., Robbins, T. W., and Sahakian, B. J. (2003). Neuropsychological impairment in patients with major depressive disorder: the effects of feedback on task performance. Psychol. Med. 33, 455–467. doi: 10.1017/S0033291702007018
Murphy, F. C., Sahakian, B. J., Rubinsztein, J. S., Michael, A., Rogers, R. D., Robbins, T. W., et al. (1999). Emotional bias and inhibitory control processes in mania and depression. Psychol. Med. 29, 1307–1321. doi: 10.1017/S0033291799001233
Murphy, F. C., Rubinsztein, J. S., Michael, A., Rogers, R. D., Robbins, T. W., Paykel, E. S., et al. (2001). Decision-making cognition in mania and depression. Psychol. Med. 31, 679–693. doi: 10.1017/S0033291701003804
Murray, G. K., Corlett, P. R., Clark, L., Pessiglione, M., Blackwell, A. D., Honey, G., et al. (2007). Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol. Psychiatry 13, 267–276. doi: 10.1038/sj.mp.4002058
Office for National Statistics (2011). Census: Ethnic Group, Local Authorities in England and Wales.
Ohmura, Y., Takahashi, T., Kitamura, N., and Wehr, P. (2006). Three-month stability of delay and probability discounting measures. Exp. Clin. Psychopharmacol. 14, 318. doi: 10.1037/1064-1297.14.3.318
Oosterbeek, H., Sloof, R., and van de Kuilen, G. (2004). Cultural differences in ultimatum game experiments: evidence from a meta-analysis. Exp. Econ. 7, 171–188. doi: 10.1023/B:EXEC.0000026978.14316.74
Owen, A. M., McMillan, K. M., Laird, A. R., and Bullmore, E. D. (2005). N-Back working memory paradigm: a meta-analysis of normative functional neuroimaging studies. Hum. Brain Mapp. 25, 46–59. doi: 10.1002/hbm.20131
Owen, A. M., Roberts, A. C., Polkey, C. E., Sahakian, B. J., and Robbins, T. W. (1991). Extra-dimensional versus intra-dimensional set shifting performance following frontal lobe excisions, temporal lobe excisions or amygdalo-hippocampectomy in man. Neuropsychologia 29, 993–1006. doi: 10.1016/0028-3932(91)90063-E
Park, S. Q., Kahnt, T., Beck, A., Cohen, M. X., Dolan, R. J., Wrase, J., et al. (2010). Prefrontal cortex fails to learn from reward prediction errors in alcohol dependence. J. Neurosci. 30, 7749–7753. doi: 10.1523/JNEUROSCI.5587-09.2010
Peirce, J. W. (2007). PsychoPy—psychophysics software in python. J. Neurosci. Methods 162, 8–13. doi: 10.1016/j.jneumeth.2006.11.017
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., and Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442, 1042–1045. doi: 10.1038/nature05051
Pomarol-Clotet, E., Hynes, F., Ashwin, C., Bullmore, E. T., McKenna, P. J., and Laws, K. R. (2010). Facial emotion processing in schizophrenia: a non-specific neuropsychological deficit? Psychol. Med. 40, 911–919. doi: 10.1017/S0033291709991309
Posner, M. I. (1980). Orienting of attention. Q. J. Exp. Psychol. 32, 3–25. doi: 10.1080/00335558008248231
Prado, C., Mellor, D., Linda Byrne, K., Wilson, C., Xu, X., and Liu, H. (2014). Facial emotion recognition: a cross-cultural comparison of chinese, chinese living in Australia, and Anglo-Australians. Motiv. Emot. 38, 420–428. doi: 10.1007/s11031-013-9383-0
Pulcu, E., Thomas, E. J., Trotter, P. D., McFarquhar, M., Juhasz, G., Sahakian, B. J., et al. (2015). Social-economical decision making in current and remitted major depression. Psychol. Med. 45, 1301–1313. doi: 10.1017/S0033291714002414
Reed, A. E., and Carstensen, L. L. (2012). The Theory behind the age-related positivity effect. Front. Psychol. 3:339. doi: 10.3389/fpsyg.2012.00339
Reynolds, B., and Schiffbauer, R. (2004). Measuring state changes in human delay discounting: an experiential discounting task. Behav. Processes 67, 343–356. doi: 10.1016/S0376-6357(04)00140-8
Richards, J. B., Zhang, L., Mitchell, S. H., and de Wit, H. (1999). Delay or probability discounting in a model of impulsive behavior: effect of alcohol. J. Exp. Anal. Behav. 71, 121–143. doi: 10.1901/jeab.1999.71-121
Rilling, J. K., Glenn, A. L., Jairam, M. R., Pagnoni, G., Goldsmith, D. R., Elfenbein, H. A., et al. (2015). Neural correlates of social cooperation and non-cooperation as a function of psychopathy. Biol. Psychiatry 61, 1260–1271. doi: 10.1016/j.biopsych.2006.07.021
Roane, H. S. (2008). On the applied use of progressive-ratio schedules of reinforcement. J. Appl. Behav. Anal. 41, 155–161. doi: 10.1901/jaba.2008.41-155
Robbins, T. W. (2002). The 5-choice serial reaction time task: behavioural pharmacology and functional neurochemistry. Psychopharmacology (Berl). 163, 362–380. doi: 10.1007/s00213-002-1154-7
Robbins, T. W., James, M., Owen, A. M., Sahakian, B. J., Lawrence, A. D., McInnes, L., et al. (1998). A study of performance on tests from the CANTAB battery sensitive to frontal lobe dysfunction in a large sample of normal volunteers: implications for theories of executive functioning and cognitive aging. Cambridge neuropsychological test automated batte. J. Int. Neuropsychol. Soc. 4, 474–490. doi: 10.1017/S1355617798455073
Robbins, T. W., James, M., Owen, A. M., Sahakian, B. J., McInnes, L., and Rabbitt, P. (1994). Cambridge neuropsychological test automated battery (CANTAB): a factor analytic study of a large sample of normal elderly volunteers. PG - 266-81. Dementia 5, 266–81. doi: 10.1159/000106735
Roberts, A. C., Robbins, T. W., and Everitt, B. J. (1988). The effects of intradimensional and extradimensional shifts on visual discrimination learning in humans and non-human primates. Q. J. Exp. Psychol. 40, 321–341.
Rogers, R. D., Blackshaw, A. J., Middleton, H. C., Matthews, K., Hawtin, K., Crowley, C., et al. (1999). Tryptophan Depletion impairs stimulus-reward learning while methylphenidate disrupts attentional control in healthy young adults: implications for the monoaminergic basis of impulsive behaviour. Psychopharmacology (Berl). 146, 482–491. doi: 10.1007/PL00005494
Rogers, R. D., Tunbridge, E. M., Bhagwagar, Z., Drevets, W. C., Sahakian, B. J., and Carter, C. S. (2003). Tryptophan depletion alters the decision-making of healthy volunteers through altered processing of reward cues. Neuropsychopharmacology 28, 153–162. doi: 10.1038/sj.npp.1300001
Roiser, J. P., Elliott, R., and Sahakian, B. J. (2012). Cognitive mechanisms of treatment in depression. Neuropsychopharmacology 37, 117–136. doi: 10.1038/npp.2011.183
Roiser, J. P., Stephan, K. E., Den Ouden, H. E. M., Barnes, T. R. E., Friston, K. J., and Joyce, E. M. (2009a). Do patients with schizophrenia exhibit aberrant salience? Psychol. Med. 39, 199. doi: 10.1017/S0033291708003863
Roiser, J. P., Blackwell, A. D., Cools, R., Clark, L., Rubinsztein, D. C., Robbins, T. W., et al. (2006). Serotonin transporter polymorphism mediates vulnerability to loss of incentive motivation following acute tryptophan depletion. Neuropsychopharmacology 31, 2264–2272. doi: 10.1038/sj.npp.1301084
Roiser, J. P., Cannon, D. M., Gandhi, S. K., Tavares, J. T., Erickson, K., Wood, S., et al. (2009b). Hot and cold cognition in unmedicated depressed subjects with bipolar disorder. Bipolar Disord. 11, 178–189. doi: 10.1111/j.1399-5618.2009.00669.x
Roiser, J. P., and Sahakian, B. J. (2013). Hot and cold cognition in depression. CNS Spectr. 18, 139–149. doi: 10.1017/S1092852913000072
Sanislow, C. A., Pine, D. S., Quinn, K. J., Kozak, M. J., Garvey, M. A., Heinssen, R. K., et al. (2010). Developing constructs for psychopathology research: research domain criteria. J. Abnorm. Psychol. 119, 631–639. doi: 10.1037/a0020909
Savla, G. N., Vella, L., Armstrong, C. C., Penn, D. L., and Twamley, E. W. (2012). Deficits in domains of social cognition in schizophrenia: a meta-analysis of the empirical evidence. Schizophr. Bull. 39, 979–992. doi: 10.1093/schbul/sbs080
Scarna, A. S., McTavish, F. B., Cowen, P. J., Goodwin, G. M., and Rogers, R. D. (2005). The effects of a branched chain amino acid mixture supplemented with tryptophan on biochemical indices of neurotransmitter function and decision-making. Psychopharmacology (Berl). 179, 761–768. doi: 10.1007/s00213-004-2105-2
Scheres, A., Milham, M. P., Knutson, B., and Castellanos, F. X. (2007). Ventral striatal hyporesponsiveness during reward anticipation in attention-deficit/hyperactivity disorder. Biol. Psychiatry 61, 720–724. doi: 10.1016/j.biopsych.2006.04.042
Seres, I., Unoka, Z., and Keri, S. (2009). The broken trust and cooperation in borderline personality disorder. Neuroreport 20, 388–392. doi: 10.1097/WNR.0b013e328324eb4d
Seymour, K. E., Reinblatt, S. P., Benson, L., and Carnell, S. (2015). Overlapping neurobehavioral circuits in adhd, obesity, and binge eating: evidence from neuroimaging research. CNS Spectr. 20, 401–411. doi: 10.1017/S1092852915000383
Shamosh, N. A., and Gray, J. R. (2008). Delay discounting and intelligence: a meta-analysis. Intelligence 36, 289–305. doi: 10.1016/j.intell.2007.09.004
Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., et al. (1998). The mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J. Clin. Psychiatry 20, 22–33.
Strauss, G. P., Allen, D. N., Jorgensen, M. L., and Cramer, S. L. (2005). Test-retest reliability of standard and emotional stroop tasks: an investigation of color-word and picture-word versions. Assessment 12, 330–337. doi: 10.1177/1073191105276375
Surguladze, S. A., Young, A. W., Senior, C., Brébion, G., Travis, M. J., and Phillips, M. L. (2004). Recognition accuracy and response bias to happy and sad facial expressions in patients with major depression. Neuropsychology 18:212. doi: 10.1037/0894-4105.18.2.212
Thomas, J. P., and McFadyen, R. G. (1995). The confidence heuristic: a game-theoretic analysis. J. Econ. Psychol. 16, 97–113. doi: 10.1016/0167-4870(94)00032-6
Thomas, R., Sanders, S., Doust, J., Beller, E., and Glasziou, P. (2015). Prevalence of attention-deficit/hyperactivity disorder: a systematic review and meta-analysis. Pediatrics 135, 994–1001. doi: 10.1542/peds.2014-3482
Unoka, Z., Seres, I., Aspan, N., Bodi, N., and Keri, S. (2009). Trust game reveals restricted interpersonal transactions in patients with borderline personality disorder. J. Pers. Disord. 23, 399–409. doi: 10.1521/pedi.2009.23.4.399
Voon, V., Irvine, M. A., Derbyshire, K., Worbe, Y., Lange, I., Abbott, S., et al. (2014). Measuring ‘waiting’ impulsivity in substance addictions and binge eating disorder in a novel analogue of rodent serial reaction time task. Biol. Psychiatry 75, 148–155. doi: 10.1016/j.biopsych.2013.05.013
Voon, V., Pessiglione, M., Brezing, C., Gallea, C., Fernandez, H. H., Dolan, R. J., et al. (2010). Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron 65, 135–142. doi: 10.1016/j.neuron.2009.12.027
Waltz, J. A., Schweitzer, J. B., Ross, T. J., Kurup, P. K., Salmeron, B. J., Rose, E. J., et al. (2010). Abnormal responses to monetary outcomes in cortex, but not in the basal ganglia, in schizophrenia. Neuropsychopharmacology 35, 2427–2439. doi: 10.1038/npp.2010.126
Watts, F. N., McKenna, F. P., Sharrock, R., and Trezise, L. (1986). Colour naming of phobia-related words. Br. J. Psychol. 77, 97–108. doi: 10.1111/j.2044-8295.1986.tb01985.x
Webb, C. A., DelDonno, S., and Killgore, W. D. S. (2014). The role of cognitive versus emotional intelligence in iowa gambling task performance: what's emotion got to do with it? Intelligence 44, 112–119. doi: 10.1016/j.intell.2014.03.008
Wechsler, D. (2008). Wechsler Adult Intelligence scale–Fourth Edition (WAIS–IV). San Antonio, TX: NCS Pearson.
Wong, R. Y.-M., and Hong, Y. (2005). Dynamic influences of culture on cooperation in the prisoner's dilemma. Psychol. Sci. 16, 429–434. doi: 10.1111/j.0956-7976.2005.01552.x
Worbe, Y., Savulich, G., Voon, V., Fernandez-Egea, E., and Robbins, T. W. (2014). Serotonin depletion induces ‘waiting impulsivity’ on the human four-choice serial reaction time task: cross-species translational significance. Neuropsychopharmacology 39, 1519–1526. doi: 10.1038/npp.2013.351
Zahn, R., Lythe, K. E., Gethin, J. A., Green, S., Deakin, J. F. W., Workman, C., et al. (2015). Negative emotions towards others are diminished in remitted major depression. Eur. Psychiatry 30, 448–453. doi: 10.1016/j.eurpsy.2015.02.005
Keywords: EMOTICOM, neuropsychological tests, social cognition, motivation and emotion, implusivity, neuropsychiatry, mental health
Citation: Bland AR, Roiser JP, Mehta MA, Schei T, Boland H, Campbell-Meiklejohn DK, Emsley RA, Munafo MR, Penton-Voak IS, Seara-Cardoso A, Viding E, Voon V, Sahakian BJ, Robbins TW and Elliott R (2016) EMOTICOM: A Neuropsychological Test Battery to Evaluate Emotion, Motivation, Impulsivity, and Social Cognition. Front. Behav. Neurosci. 10:25. doi: 10.3389/fnbeh.2016.00025
Received: 28 August 2015; Accepted: 04 February 2016;
Published: 24 February 2016.
Edited by:
Raymond C. K. Chan, Chinese Academy of Sciences, ChinaCopyright © 2016 Bland, Roiser, Mehta, Schei, Boland, Campbell-Meiklejohn, Emsley, Munafo, Penton-Voak, Seara-Cardoso, Viding, Voon, Sahakian, Robbins and Elliott. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Rebecca Elliott, rebecca.elliott@manchester.ac.uk