Judgment of musical emotions after cochlear implantation in adults with progressive deafness

While cochlear implantation is rather successful in restoring speech comprehension in quiet environments (Nimmons et al., 2008), other auditory tasks, such as music perception, can remain challenging for implant users. Here, we tested how patients who had received a cochlear implant (CI) after post-lingual progressive deafness perceive emotions in music. Thirteen adult CI recipients with good verbal comprehension (dissyllabic words ≥70%) and 13 normal hearing participants matched for age, gender, and education listened to 40 short musical excerpts that selectively expressed fear, happiness, sadness, and peacefulness ( Vieillard et al., 2008). The participants were asked to rate (on a 0–100 scale) how much the musical stimuli expressed these four cardinal emotions, and to judge their emotional valence (unpleasant–pleasant) and arousal (relaxing–stimulating). Although CI users performed above chance level, their emotional judgments (mean correctness scores) were generally impaired for happy, scary, and sad, but not for peaceful excerpts. CI users also demonstrated deficits in perceiving arousal of musical excerpts, whereas rating of valence remained unaffected. The current findings indicate that judgments of emotional categories and dimensions of musical excerpts are not uniformly impaired after cochlear implantation. These results are discussed in relation to the relatively spared abilities of CI users in perceiving temporal (rhythm and metric) as compared to spectral (pitch and timbre) musical dimensions, which might benefit the processing of musical emotions (Cooper et al., 2008).

Introduction

Most hearing people consider music to be a source of pleasure, which certainly constitutes the main motivation for listening and playing music. In post-lingual progressive deafness, however, not only is oral communication compromised, but also musical pleasure and the social enjoyment of music (Gfeller et al., 2002b,c). When hearing loss becomes too profound to be alleviated by conventional hearing aids, cochlear implantation is often proposed to restore hearing. Cochlear implants (CIs) consist of an array of electrodes that are inserted inside the cochlea, and which directly stimulate the auditory nerve fibers by converting the acoustic information from environmental sounds into electrical pulses. The temporally and spectrally coded information contained in the CI signal is then transmitted to the auditory cortex.

While CIs are rather successful with respect to restoring speech comprehension in quiet environments (Nimmons et al., 2008), music perception and enjoyment remains challenging for cochlear implantees (Mirza et al., 2003; Migirov et al., 2009; Brockmeier et al., 2011). CI users generally tend to minimize their exposure to music (Lassaletta et al., 2008), either because their relation to music has been altered during the period of profound deafness, or following implantation, whereby the electric nature of the signal delivered by CI dramatically changes musical sensations (Smith et al., 2002; Kong and Zeng, 2006; Gfeller et al., 2007; Filipo et al., 2008; Nardo et al., 2008). Most CI recipients exhibit poor performance in musical processing (Leal et al., 2003; Looi et al., 2008). One notable exception in the literature depicts the case of a 30-year-old non-professional musician who displayed excellent musical perception abilities after bilateral cochlear implantation following 3 years of deafness that was caused by inner ear autoimmune disease (Maarefvand et al., 2013). Surprisingly, this person presented no deficit in pitch direction discrimination with a threshold to half of a semitone, and no impairment in melody and timbre recognition, which was tested using the Clinical Assessment of Music Perception tests (CAMP, Kang et al., 2009). Most post-lingually deaf adult CI users, however, display severe difficulties in pitch discrimination and in musical timbre or familiar melody recognition (Gfeller et al., 2002a; Kong et al., 2004; McDermott, 2004; Looi et al., 2008; Kang et al., 2009) relative to normal hearing (NH) persons. When exposed to the Montreal Battery for Evaluation of Amusia (Peretz et al., 2003), CI users have been found to perform at near chance level on pitch–based tests (Scale, Contour, and Interval), yet obtained relatively higher scores on temporal-based tests (Rhythm and Meter), which suggests that they have no deficit in rhythm-based perception (Gfeller et al., 2000; Looi et al., 2008) or tempo discrimination (Kong et al., 2004). Temporal processing capacity is determined by the sampling rate of the implants, which is generally very high and thus yields minimal impairment. For pitch perception, however, the limitations are both more drastic and more complex, as they result from the combination of constraints on the place- and rate-coding of pitch. One fundamental limitation of pitch place-coding in CI users is that the spread of excitation produced by electrical stimulation is less focused than in the normal ear (Macherey and Carlyon, 2014). There is an upper rate pitch limit of around 300 Hz on most (Shannon, 1983; Zeng, 2002) but not all subjects (Kong et al., 2009; Penninger et al., 2014). Several attempts have been made so far to improve this temporal pitch limit. The two most promising attempts to overcome this limitation are the use of asymmetric pulses (Macherey et al., 2011), and stimulating on multiple electrodes (Venter and Hanekom, 2014; Penninger et al., 2015).

Paradoxically, deaf persons specify music enjoyment as a major motivation for getting an implant, beyond the expected benefit on speech perception and communication (Gfeller et al., 2002b). One third of CI candidates resort to cochlear implantation for the mere purpose of being able to listen again to music (Kohlberg et al., 2013) and many of them consider that improving music perception abilities will enhance their quality of life (Drennan and Rubinstein, 2008). Results from two surveys (Lassaletta et al., 2008; Migirov et al., 2009) completed by 65 and 53 individuals, respectively, indicate that post-lingually deaf CI users continue to listen to music, albeit at a generally diminished exposure duration. Although classical Western music heavily relies on accurate pitch perception, listening to music remains a pleasant experience for some CI listeners (Kohlberg et al., 2013). This finding suggests that CI users might recognize emotions conveyed by music despite their impaired pitch perception.

However, there are still very few objective data on the capacity of CI users to perceive musically evoked emotions. Music appreciation in CI patients has essentially been explored using self-report questionnaires that characterize listening habits and the enjoyment of music before and after cochlear implantation. A single study reports both emotional judgments and music perception abilities in six post-lingual CI adult users and six NH participants (Rosslau et al., 2012). CI users were able to perceive various degrees of arousal in famous movie soundtracks, and did not show the typical difficulties in pitch, rhythm, and harmony perception, as measured by the standard MuSIC perception test battery (Brockmeier et al., 2011). Although these findings suggest that CI users are still able to perceive arousal conveyed by music, one should remain cautious about the interpretation of these results given the small number of participants and the familiarity of the tested musical material.

In order to address the ability for CI users to perceive emotions conveyed by music, we examined how post-lingually deaf CI users evaluate musical excerpts composed with the intention of evoking fear, peacefulness, happiness, and sadness, and compared their results with those of NH subjects. Participants performed three different judgment tasks. The first is an emotional categorization judgment that requires the listener to assess the degree of threat, peacefulness, happiness, and sadness in musical excerpts. The second and third judgments involve the rating of emotional dimensions: arousal and valence. Participants evaluated the quality of music excerpts in terms of arousal (from ‘relaxing’ to ‘stimulating’), and in terms of valence [from ‘pleasant’ (positive) to ‘unpleasant’ (negative)]. Therefore, musical emotion perception was assessed using both a categorical and a dimensional approach. Admittedly, while music can induce basic emotions such as happiness, sadness, anger, or fear, the categorical approach is insufficient to accurately portray the richness of emotional feelings related to musical experience (Bigand et al., 2005). A combined approach accomodates Russell’s (1980) model of affect, which stipulates that emotions can be represented in a two-dimensions emotional space defined by arousal and valence (Eerola and Vuoskoski, 2011).

The objective of this study was to assess the abilities of CI users to perceive and judge emotions in music, as well as the preservation of emotional skills in the case of progressive deafness. Given the music perception disturbances in CI users (e.g., Cooper et al., 2008; Kang et al., 2009), we predicted that these persons would show deficits in recognizing the four cardinal emotions (happiness, peacefulness, fear, and sadness) of the musical excerpts. In particular, we hypothesized that emotional labels with disctinctive rapid tempos such as happiness will be better recognized in musical excerpts due to the specific use of temporal code for pitch processing (Macherey and Carlyon, 2010). In turn, peacefulness, fear, and sadness were hypothesized to be more poorly recognized because of their similar and characteristically slower tempo with melodic lines lying in the medium pitch range. Since it has been shown in NH individuals that arousal judgments rely mainly on spectral cues, which are badly transmitted by CIs, we predicted that CI users would be also impaired in judging emotional arousal in musical excerpts. Finally, considering that CI patients continue to listen to music, we conjectured that they would still experience pleasure and appreciate music, and therefore that they might be less affected in rating emotional valence than they would rating arousal.

Materials and Methods

Participants

Thirteen French-speaking patients with severe to profound post-lingual progressive sensorineural hearing loss were tested at least 1 year after unilateral cochlear implantation. Most patients had a controlateral hearing aid (HA; n = 11) and the remaining participants (n = 2) had no controlateral HA. All participants used oral communication rather than sign language and had good speech perception performances (dissyllabic word recognition ≥70%) after cochlear implantation, at the time of testing. None had a history of neurological or psychiatric illness. All participants were tested at the ENT department (Beaujon hospital, Paris). A group of 13 normal-hearing (NH: 9 female/4 male; mean age = 57.0 years, SD = 11.1; mean duration of education = 17.1 years, SD = 3.0) controls matched to the CI patients in terms of sex, age and education (CI: 9 female/4 male; mean age = 57.8, SD = 11.5; mean duration of education = 15.2 years, SD = 1.8) were also tested. None of the participants were musicians except for one CI patient (CI 13). Demographic, clinical, and language data of the CI users are presented in Table 1. All participants gave their written informed consent before being tested in accordance with the Declaration of Helsinki.

TABLE 1

TABLE 1. Demographic, clinical and language data for each patient of the CI group (CI, cochlear implant; HA, hearing aid; S/N, signal to noise ratio).

Preliminary Mood Assessment

In order to ensure that the evaluation of emotions conveyed by music was not biased by transient or chronic mood disturbances, which are often observed in deaf patients, subjects completed two questionnaires. The State and Trait Anxiety Inventory (STAI: Spielberger, 1983) and the Profile of Mood States (POMS: McNair et al., 1992) were administrated before the emotional rating task. The STAI is composed of two scales: the trait STAI to assess the general level of anxiety, which is presumed to be stable over time, and the state STAI to assess the present level of anxiety. The POMS is composed of 30 emotional adjectives and the participants had to rate how they describe their current mood (from 0 not at all to 4 extremely). Six emotional scales can be derived from these responses: Tension, Depression, Anger, Vigor, Fatigue, and Confusion. As report in Table 2, there was no difference between the two groups of participants in either of these scores.

TABLE 2

TABLE 2. Results for mood questionnaire for CI and normal hearing (NH) participants.

Stimuli

The musical material consisted of 40 excerpts that conveyed four intended emotions (happiness, n = 10; sadness, n = 10; threat, n = 10, and peacefulness, n = 10). The intended emotion, valence, and arousal of each selected musical excerpt have been validated by previous studies (Gosselin et al., 2005; Vieillard et al., 2008). Musical excerpts were composed following the rules of the Western tonal system, and were based on a melody with an accompaniment. The stimuli had a regular temporal structure with the exception of a few fearful excerpts. The happy excerpts were written in a major mode at an average rapid tempo, the melodic line lying in the medium–high pitch range, and the pedal was not used. In contrast, the sad excerpts were written in a minor mode at an average slow tempo, and the pedal was used. The peaceful music was composed in a major mode, had an intermediate tempo, and was played with pedal and arpeggio accompaniment. Most fearful excerpts were regular and consonant with various tempos that ranged from slow to rapid, and were composed of minor chords on the third and sixth degrees, which implies the use of accidentals. Only few excerpts had irregular rhythms and were dissonant. Among the 10 fearful musical excerpts, ratings of overall dissonance (from 1 for “not dissonant at all” to 5 for “very dissonant”) indicated that all the stimuli were not considered as dissonant, with scores ranging from 1 to 3. For the ratings of rhythmic regularity of these fearful excepts (from 1 for “fairly regular” to 5 for “irregular”), six stimuli were judged to be irregular, with scores ranging from 3 to 5, while four of them were judged to be more regular, with scores from 1 to 2. All excerpts were computer-generated and recorded in a piano timbre with a mean duration of 12 s. Examples can be heard at http://www.brams.umontreal.ca/peretz.

Procedure

The musical excerpts were presented in a pseudo-randomized order using Presentation software (Neurobehavioural Systems Inc., San Pablo, CA, USA) and at a comfortable listening level of 70 dB SPL. The stimuli were delivered via two loudspeakers (Logitech X-140, RMS total power of 5 W, bandwidth 80 Hz–18 kHz) that were positioned on each side of the video monitor, which allowed us to test CI subjects under optimal listening conditions, i.e., binaural hearing if they had a contralateral HA, or with their CI only.

After the presentation of each musical excerpt, participants performed three different judgments. In the first, they rated to what extent the music expressed threat, peacefulness, happiness, and sadness on four rating scales, whereby 0 corresponded to ‘absent’ and 100 to ‘present.’ The second and third judgments required participants to rate the arousal (from ‘relaxing’ to ‘stimulating’) and valence (from ‘unpleasant’ or ‘negative’ to ‘pleasant’ or ‘positive’) of each musical excerpt.

Participants were explicitly asked not to judge their own felt emotion but to rate the emotion conveyed by the musical excerpt. The stimuli were presented only once and no feedback was given. Ratings along the visual analog scale were displayed on a computer screen with identical unmarked horizontal lines and numerical labels at each extremity (0–100).

Prior to the experiment, a training session was carried out in order to ensure that each participant recognized the targeted emotional category (fear, sadness, happiness, and peacefulness) by associating short sentences to the most appropriate emotional label (e.g., for happiness: ‘Eric had just won the lottery’). The comprehension of the terms ‘valence’ and ‘arousal’, was also verified by rating four situations (e.g., ‘Sophie has just received a wonderful travel for her wedding’ was supposed to be highly positive and arousing). Feedback was presented to ensure the appropriate use of the two scales.

Results

Categorical Judgments

Since participants were allowed to indicate a combination of perceived emotions (happiness, fear, sadness, and peacefulness) by providing a graded judgment for each, we first derived, for each participant, the dominant emotion attributed to each musical excerpt by selecting the rating scale that had received the maximal rating. When the highest rating was given for more than one emotion, e.g., when a participant judged a musical excerpt to express both surprise and fear to the same degree, it was considered ambivalent and was not included in the analysis.

To compare emotional labels assigned by CI users and by NH participants, we calculated correctness scores for each participant using a proportional approach as proposed by Heberlein et al. (2004): each response was given a correctness score based on the proportion of subjects in the NH group that gave that response (minimum of the correctness score = 0; maximum = 1). Higher correctness scores corresponded to answers that were chosen a large number of times by NH participants, whereas lower correctness scores corresponded to answers that were less frequently chosen by NH participants. However, NH participants used both ears to make their judgments.

A correctness score of ‘zero’ corresponded to responses that had never been given by NH participants. Correctness scores are useful because any given emotional musical excerpt can be perceived as expressing more than one single emotion, inducing variability in the labeling by normal listeners. In other words, such a proportional method takes into account the type of errors, considering that some errors are more acceptable than others. For example, a peaceful musical excerpt can also express some sadness, and it is less inaccurate to judge this excerpt as expressing sadness than happiness even if sadness is not the intended emotion. Considering such a graduation among the type of errors, using correctness scores permitted a refinement of the analysis.

As can be seen in Figure 1, CI patients’ judgments differed from those of NH participants, especially for happiness, fear, and sadness. Happy, scary, and sad music were indeed less well recognized by CI patients (happy = 0.73; scary = 0.53; sad = 0.58) than by NH participants (happy = 0.99; scary = 0.78; sad = 0.78). Since the criteria for variance homogeneity were not met, we used non-parametric analyses to compare average correctness scores in CI and NH groups as a function of the intended emotion. Mann–Whitney U-tests revealed that CI participants were less accurate in recognizing happiness than were NH participants (z = -2.54; P < 0.05), fear (z = -3.36; P < 0.001) and sadness (z = -2.59; P < 0.01), but there was no difference for peacefulness (p > 0.05). However, all CI participants performed well above chance level (25%).

FIGURE 1

FIGURE 1. Mean correct scores of the two groups of participants (NH, normal hearing controls; CI, cochlear implanted users) as a function of the four intended emotions (bars represent the SE of the mean). Asterisk means significant difference.

Judgments of Valence and Arousal

The ratings of valence and arousal obtained for each emotional category by the CI and NH groups are presented respectively in Figures 2 and 3. Mann–Whitney U-tests revealed there to be no group difference between the valence scores for the different emotions (happy: z = -1.31; P > 0.05; peaceful: z = -1.72; P > 0.05; fearful: z = -0.44; P > 0.05; sad: z = -0.90; P > 0.05). However, we found group differences between the ratings of arousal for happy (z = -2.07; P < 0.05), peaceful (z = -2.28; P < 0.05), fearful (z = -3.31; P < 0.001), and sad musical excerpts (z = -3.00; P < 0.01), whereby CI users judged musical excerpts to be less arousing than did NH participants.

FIGURE 2

FIGURE 2. Mean ratings of the two groups of participants in judging valence of musical excerpts as a function of the intended emotions and groups (NH, normal hearing controls; CI, cochlear implanted users; bars represent the SE of the mean).

FIGURE 3

FIGURE 3. Mean ratings of the two groups of participants in judging arousal of musical excerpts as a function of the intended emotions and groups (NH, normal hearing controls; CI, cochlear implanted users; bars represent the SE of the mean).

Individual Analyses

To further explore the results for musical emotions, we compared individual correctness scores between CI and NH participants (Crawford and Garthwaite, 2007), using the mean score of the four intended musical emotions. The results showed that seven CI participants (CI 1, 6, 7, 8, 9, 11, and 12) out of 13 were significantly impaired in categorical judgments of musical emotions (all ps < 0.05). These impaired patients were on average 57 years old [50–68] and the mean duration of CI use was 28 months [12–57]. By contrast, the six CI participants that did not show any deficit (CI 2, 3, 4, 5, 10, and 13), who had an average age of 58 years [39–76], had a mean duration of CI use of 44 months [12–120].

Emotional Judgments of Music and Language Performances

Spearman’s correlation between language performance in noise with CI (global score of sentence recognition with a signal to noise ratio of 10 dB) and emotional judgments of musical excerpts was computed for CI users. Language performance in noise was correlated with the global mean correctness score of the four intended musical emotions (ρ = 0.60, p < 0.05) but not with the perception of arousal (ρ = -0.29, p > 0.05).

Discussion

The aim of our study was to test how CI users with acquired post-lingual progressive deafness perceive emotions conveyed by music. We used an emotional rating task in which participants evaluated the amount of the four cardinal emotions (peacefulness, happiness, sadness, and fear) present in musical excerpts on visual analog scales, using the same stimuli as Gosselin et al. (2005) and Vieillard et al. (2008). We also assessed the perception of arousal and valence of the same musical excerpts in order to specify the ability CI users to experience emotional dimensions.

Our results show that CI participants performed less accurately than NH participants in recognizing happy, sad, and scary music, with scores that were, however, well above chance level. By contrast, they performed just as well as NH controls on recognizing peaceful musical excerpts. Altogether, these results suggest that emotional judgments of musical excerpts were not uniformly affected by artificial hearing with a CI. We also found there to be a deficit in perceiving emotional arousal, CI users being less excited by music than NH listeners. Conversely, rating of emotional valence was not impaired in CI users. Given that none of the participants were able to perform such judgments before cochlear implantation due to profound deafness, these results demonstrate the benefit of CIs for perceiving musical emotions and for discriminating between pleasantness and unpleasantness in music. These findings give experimental support to the observation that CI users can still be attracted to music and can enjoy music listening (Kohlberg et al., 2013), even if they cannot access the whole spectrum of musical emotions.

The impairments we observed in musical emotion ratings were not accounted for by depressive disorders that often affect profoundly deaf subjects (Hallam et al., 2006; Garnefski and Kraaij, 2012). The STAI (Spielberger, 1983) and POMS (McNair et al., 1992) questionnaires revealed there to be no difference between CI and NH participants, which indicates that depression or anxiety disturbances did not interfere with judgments of musical emotion. Moreover, as all patients were able to understand the emotional labels used in the tasks, the deficits in recognizing happy, sad, and scary music seen here cannot be explained by difficulties in language comprehension or by a general emotional deficit, as recognition of peaceful music and perception of valence remained intact.

The deficits in emotional judgments shown here can presumably be attributed to the poor transmission of pitch cues and particularly fundamental frequency (F0) by the device. Most current CIs less efficiently transmit the fine structural key aspects of musical sounds than does a physiological cochlea, and convey coarse spectral cues. One reason for this is that a CI device has a limited number of wide filter bands with fixed center frequencies that may hinder complete resolution of the lower harmonics of complex musical sounds. However, pitch may also be perceived through sound rate (periodicity pitch), although this mechanism is limited to rates up to 300–400 Hz. The deficit in place coding cues, with reduced excitation in the most apical part of the cochlea, the mismatch between the rate of stimulation and the cochlear tonotopy (Looi et al., 2012), and the insufficiency of fine temporal variations processing, all concur to limit pitch perception in CI users (Carlyon et al., 2014) and prevent a full harmonic processing. Limitations of CI transmission that produce impairments in pitch, timbre, and melody perception (i.e., Gfeller et al., 2002a, 2008; McDermott, 2004) can specifically affect the recognition of happy, sad, and scary music by CI users. Yet, as pitch discrimination remains fairly good when stimuli are far apart in the spectral domain (Gfeller et al., 2002a; Kong et al., 2004; Looi et al., 2004; McDermott, 2004), CI users may retain the ability to distinguish emotions in music, such as telling apart happiness from peacefulness or sadness, whose melodic lines are in the medium–high pitch and medium–low pitch ranges, respectively.

The preserved ability to recognize some musical emotions can also be explained by the use of timing and rhythmic cues that CI users remain able to process. Perception of rhythm in music is related to the perception of the duration of sounds and intervals between sounds that CI users perceive (McDermott, 2004). Rhythmic differences are hence encoded as temporal gaps or amplitude modulations, or both (Shannon, 1989, 1992), and CI listeners have temporal processing abilities with performances for periodic pulse trains’ rate discrimination tasks based on perceived pitch up to 300 pps (Moore and Carlyon, 2005). Therefore, CI recipients perform as well as NH adults in rhythm discrimination tasks (Gfeller et al., 2000; Kong et al., 2004; Looi et al., 2008) and tempo perception (Kong et al., 2004).

Given that a rapid tempo is often associated with a positive but also arousing emotion such as happiness, while a slower tempo often conveys a more negative and less arousing emotion such as sadness, temporal cues could also play a role in perceiving emotional valence and arousal. However, it remains difficult to explain why CI listeners can rate emotional valence while at the same time being less sensitive to arousing stimuli than NH participants. Gingras et al. (2013) proposed that emotions in music are conveyed by two types of underlying cues: universal acoustic cues (i.e., intensity, pitch, and tempo) and culturally determined cues that are associated with a specific musical system such as the Western common-practice tonality. Before their hearing loss, CI users have experienced music listening in their everyday life, and have thus been exposed to the use of both types of cues. Due to the preceding period of deafness, CI listeners might continue to rely on cultural cues to compensate for difficulties in processing acoustic cues, being therefore able to perceive emotional valence in music. It is also possible that this ability could be partly explained by their strong motivation to feel pleasantness in music. Preservation of valence judgment could be explained by the search of emotional reward in music, and by the strong motivation of CI users to retrieve normal everyday emotional life, including the social sharing of emotions. Considering that CI users continue to listen to music, we may conjecture that they are still experiencing musical pleasure and appreciation, and that they might be able to feel musical pleasantness through emotional valence, even though the exact cues they use remain to be clarified.

In agreement with our predictions, but in contradiction with Rosslau et al.’s (2012), we found that CI listeners were impaired in arousal judgments. The apparent discrepancy between the studies might be explained by methodological differences. Although the authors used a simple task in which subjects rated three levels of arousal in familiar musical excerpts that all had positive valence, the lack of significant results could be attributed to the small number of participants included in their study (six CI users). To interpret the arousal perception deficit, some authors have invoked the crucial role of intensity cues as musical emotions vectors, positive association between sound intensity and arousal induction in subjective ratings being well documented (Scherer, 1989; Ilie and Thompson, 2006). However, recent evidence suggests that spectral features are more important than intensity cues in arousal rating (Gingras et al., 2013), as the manipulation of these acoustic cues does not have the same effect on music-induced arousal and pleasantness feelings. To validate their hypothesis, non-musician participants were asked to rate subjective arousal and valence (pleasantness) of original vs. amplitude-normalized loudness-matched musical excerpts. Although the manipulation of intensity did not affect the subjective rating of arousal and valence, spectral features (spectral flux and entropy) modulated the perception of arousal in both original and amplitude-normalized loudness matched musical excerpts. This suggests that arousal judgment depends on spectral properties rather than on intensity features. In addition, the authors found that the proportion of variance explained by basic acoustic features is higher for arousal than for valence ratings of musical excerpts. Based on these findings, they concluded that arousal induction could correspond to a bottom–up process related to the physical characteristics of the stimulus (spectral information) whereas valence induction may be more culturally determined in the context of musical emotions. We may therefore propose that difficulties in processing spectral properties of sounds observed after cochlear implantation in post-lingually deaf persons should have a more deleterious impact on arousal than on valence ratings. In addition, according to Brunswick’s (1957) concept assuming that a listener can infer arousal levels from the available inter-correlated cues (rhythmic, tonal, and spectral features), when a more efficient cue is not available, the poor transmission of spectro-temporal redundancy cues by the CI could also be responsible for the impairments of CI users in rating arousal of musical excerpts.

Impairments in arousal judgments in CI subjects may also be considered by opposing the emotivist approach, which focuses focusing on the emotional feelings to the cognitivist approach centered on the perception of emotions in which recognition can operate without association with a subjective feeling (Krumhansl, 1997). This distinction may explain, at least in part, the relative preservation of valence judgment abilities along the cognitive’s view, which presupposes that the pleasant or unpleasant character of a piece of music could be perceived without being felt. Conversely, arousal judgment could be based on the subjective experience of emotional intensity, which could be limited due to the progressive limitation of social interactions in case of progressive deafness. Another possibility is that our results may have been contaminated by the existence of a methodological bias, since we did not counterbalance the order of ratings performed after the presentation of each musical excerpts (categorical judgment was always followed by arousal and valence ratings). However, it remains difficult to understand why this order would have only benefited the last emotional rating of each musical excerpt considering that we found no correlation between the three types of ratings. Moreover, each of them involved very different emotional judgments.

Finally, we found that emotional recognition of musical excerpts, but not perception of arousal, was correlated with sentence recognition with a signal to noise ratio of 10 dB. This finding suggests that recognition of emotional categories and the rating of arousal do not depend on the same variables. Similarly to the perception of emotional features in music, speech perception in noise requires the ability to use pitch cues to separate information from the background, as shown by Qin and Oxenham (2003). Poor pitch perception abilities in CI patients with small benefit for speech perception in noise could partly explain their impairment in recognizing emotional categories of musical excerpts. This impairment could in turn be explained by the lack of pitch coding in CI speech processing strategies conveying more efficiently envelope information than fine (spectral and temporal) structure information (Cullington and Zeng, 2011). Inspection of individual data also showed CI users that present difficulties in recognizing emotion also had shorter post-CI duration and less practice with the device, (except for one CI user who was a former musician) than non-impaired CI users. While these results confirm the impact of the electric nature of the CI signal on musical emotional judgments, they also show that there is some plasticity in the recovery of these auditory functions that are involved in the recognition of musical emotions. In summary, emotional judgment in music could be seen as a complex task that requires good pitch perception, as for speech-in-noise perception with concurrent talkers, and time to adapt to the device.

Conclusion

This study examined for the first time emotional judgments for music in CI adult recipients with progressive sensorineural hearing loss. We tested emotion recognition and the perception of arousal and valence in music, and showed that CI listeners were impaired in judging emotions conveyed by music, even though their performance remained above chance. This finding suggests that cochlear implantation does partially allow for musical emotion processing, and that despite their sensory impairment, CI users can appreciate listening to music, and perceive its valence. The relatively spared abilities of CI listeners to judge emotional valence, as compared to arousal, and to recognize peacefulness in musical excerpts confirm previous observations indicating that CI users can use temporal acoustic cues to process music (Kong et al., 2004). We therefore propose that the larger improvement in processing temporal (rhythm and metric) than spectral (pitch and timbre) cues that follows cochlear implantation (Cooper et al., 2008) contributes to the regain the processing of musical emotions. Taken together, these results demonstrate the benefit of cochlear implantation for emotional perception in music. Although CI users remain impaired relative to NH persons in judging emotional categories and in perceiving arousal in music, they may retrieve the ability to enjoy music listening through the ability to judge emotional valence.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The project was supported by grants from l’Agence Nationale de la Recherche (ANR-09-BLAN-0310- 02) and l’Institut Universitaire de France to SS.

References

Bigand, E., Vieillard, S., Madurell, F., Marozeau, J., and Dacquet, A. (2005). Multidimensional scaling of emotional responses to music: the effect of musical expertise, and of the duration of the excerpts. Cogn. Emot. 19, 1113–1139. doi: 10.1080/02699930500204250

CrossRef Full Text | Google Scholar

Brockmeier, S. J., Fitzgerald, D., Searle, O., Fitzgerald, H., Grasmeder, M., Hilbig, S.,et al. (2011). The MuSIC perception test: a novel battery for testing music perception of cochlear implant users. Cochlear Implants Int. 12, 10–20. doi: 10.1179/146701010X12677899497236

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Brunswick, E. (1957). “Scope and aspects of the cognitive problem,” in Contemporary Approaches to Cognition, eds H. Gruber, K. R. Hammond, and R. Jessor (Cambridge, MA: Harvard University Press), 5–31.

Carlyon, R. P., Monstrey, J., Deeks, J. M., and Macherey, O. (2014). Evaluation of a cochlear-implant processing strategy incorporating phantom stimulation and asymmetric pulses. Int. J. Audiol. 53, 871–879. doi: 10.3109/14992027.2014.932024

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Cooper, W. B., Tobey, E., and Loizou, P. C. (2008). Music perception by cochlear implant and normal hearing listeners as measured by the Montreal battery for evaluation of Amusia. Ear Hear. 29, 618–626. doi: 10.1097/AUD.0b013e318174e787

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Crawford, J. R., and Garthwaite, P. H. (2007). Comparison of a single case to a control or normative sample in neuropsychology: development of a Bayesian approach. Cogn. Neuropsychol. 24, 343–372. doi: 10.1080/02643290701290146

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Cullington, H. E., and Zeng, F. G. (2011). Comparison of bimodal and bilateral cochlear implant users on speech recognition with competing talker, music perception, affective prosody discrimination, and talker identification. Ear Hear. 32, 16–30. doi: 10.1097/AUD.0b013e3181edfbd2

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Drennan, W. R., and Rubinstein, J. T. (2008). Music perception in cochlear implant users and its relationship with psychophysical capabilities. J. Rehabil. Res. Dev. 45, 779–789. doi: 10.1682/JRRD.2007.08.0118

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Eerola, T., and Vuoskoski, J. K. (2011). A comparison of the discrete and dimensional models of emotion in music. Psychol. Music 39, 18–49. doi: 10.1177/0305735610362821

CrossRef Full Text | Google Scholar

Filipo, R., Ballantyne, D., Mancini, P., and D’Elia, C. (2008). Music perception in cochlear implant recipients: comparison of findings between HiRes90 and HiRes120. Acta Otolaryngol. 128, 378–381. doi: 10.1080/00016480701796951

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Garnefski, N., and Kraaij, V. (2012). Effects of a cognitive behavioral self-help program on emotional problems for people with acquired hearing loss: a randomized controlled trial. J. Deaf. Stud. Deaf. Educ. 17, 75–84. doi: 10.3109/14992027.2012.675628

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gfeller, K., Oleson, J., Knutson, J. F., Breheny, P., Driscoll, V., and Olszewski, C. (2008). Multivariate predictors of music perception and appraisal by adult cochlear implant users. J. Am. Acad. Audiol. 19, 120–34. doi: 10.3766/jaaa.19.2.3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gfeller, K., Turner, C., Oleson, J., Zhang, X., Gantz, B., Froman, R.,et al. (2007). Accuracy of cochlear implant recipients on pitch perception, melody recognition, and speech reception in noise. Ear Hear. 28, 412–423. doi: 10.1097/AUD.0b013e3180479318

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gfeller, K., Turner, C., Woodworth, G., Mehr, M., Fearn, R., Witt, S.,et al. (2002a). Recognition of familiar melodies by adult cochlear implant recipients and normal-hearing adults. Cochlear Implants Int. 3, 31–55. doi: 10.1002/cii.50

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gfeller, K., Witt, S., Adamek, M., Mehr, M., Rogers, J., and Stordahl, J. (2002b). Effects of training on timbre recognition and appraisal by postlingually deafened cochlear implant recipients. J. Am. Acad. Audio 13, 132–145 doi: 10.1055/s-0032-1329230

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gfeller, K., Witt, S., Woodworth, G., Mehr, M., and Knutson, J. F. (2002c). Effects of frequency, instrumental family, and cochlear implant type on timbre recognition and appraisal. Ann. Otol. Rhinol. Laryngol. 111, 349–356. doi: 10.1177/000348940211100412

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gfeller, K., Witt, S., Stordahl, J., Mehr, M., and Woodworth, G. (2000). The effects of training on melody recognition and appraisal by adult cochlear implant recipients. J. Am. Acad. Rehab. Audiol. 33, 115–138. doi: 10.1055/s-0032-1329230

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gingras, B., Marin, M. M., and Fitch, W. T. (2013). Beyond intensity: spectral features effectively predict music-induced subjective arousal. Q. J. Exp. Psychol. (Hove) 67, 1428–1446. doi: 10.1080/17470218.2013.863954

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gosselin, N., Peretz, I., Noulhiane, M., Hasboun, D., Beckett, C., Baulac, M.,et al. (2005). Impaired recognition of scary music following unilateral temporal lobe excision. Brain 128, 628–640. doi: 10.1093/brain/awh420

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Hallam, R., Ashton, P., Sherbourne, K., and Gailey, L. (2006). Acquired profound hearing loss: mental health and other characteristics of a large sample. Int. J. Audiol. 45, 715–723. doi: 10.1080/14992020600957335

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Heberlein, A. S., Adolphs, R., Tranel, D., and Damasio, H. (2004). Cortical regions for judgments of emotions and personality traits from point-light walkers. J. Cogn. Neurosci. 16, 1143–1158. doi: 10.1162/0898929041920423

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Ilie, G., and Thompson, W. F. (2006). A comparison of acoustic cues in music and speech for three dimensions of affect. Music Percep. 23, 319–330. doi: 10.1525/mp.2006.23.4.319

CrossRef Full Text | Google Scholar

Kang, R., Nimmons, G. L., Drennan, W., Longnion, J., Ruffin, C., Nie, K.,et al. (2009). Development and validation of the University of Washington Clinical Assessment of Music Perception test. Ear Hear. 30, 411–418. doi: 10.1097/AUD.0b013e3181a61bc0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Kohlberg, G., Spitzer, J. B., Mancuso, D., and Lalwani, A. K. (2013). Does cochlear implantation restore music appreciation?. Laryngoscope 124, 587–588. doi: 10.1002/lary.24171

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Kong, Y. Y., Cruz, R., Jones, J. A., and Zeng, F. G. (2004). Music perception with temporal cues in acoustic and electric hearing. Ear Hear. 25, 173–185. doi: 10.1097/01.AUD.0000120365.97792.2F

CrossRef Full Text | Google Scholar

Kong, Y. Y., Deeks, J. M., Axon, P. R., and Carlyon, R. P. (2009). Limits of temporal pitch in cochlear implants. J. Acoust. Soc. Am. 125, 1649–1657. doi: 10.1121/1.3068457

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Kong, Y. Y., and Zeng, F. G. (2006). Temporal and spectral cues in mandarin tone recognition. J. Acoust. Soc. Am. 120(5 Pt 1), 2830–2840. doi: 10.1121/1.2346009

CrossRef Full Text | Google Scholar

Krumhansl, C. L. (1997). An exploratory study of musical emotions and psychophysiology. Can. J. Exp. Psychol. 51, 336–353. doi: 10.1037/1196-1961.51.4.336

CrossRef Full Text | Google Scholar

Lassaletta, L., Castro, A., Bastarrica, M., Pérez-Mora, R., Herrán, B., Sanz, L.,et al. (2008). Changes in listening habits and quality of musical sound after cochlear implantation. Otolaryngol. Head Neck Surg. 138, 363–367. doi: 10.1016/j.otohns.2007.11.028

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Leal, M. C., Shin, Y. J., Laborde, M. L., Calmels, M. N., Verges, S., Lugardon, S.,et al. (2003). Music perception in adult cochlear implant recipients. Acta Otolaryngol. 123, 826–835. doi: 10.1080/00016480310000386

CrossRef Full Text | Google Scholar

Looi, V., Gfeller, K., and Driscoll, V. (2012). Music appreciation and training for cochlear implant recipients: a review. Semin. Hear. 33, 307–334. doi: 10.1055/s-0032-1329222

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Looi, V., McDermott, H., McKay, C., and Hickson, L. (2004). “Pitch discrimination and melody recognition by cochlear implant users,” in Cochlear Implants: Proceedings of the 8th International Cochlear Implant Conference, ed. R. T. Miyamoto (Amsterdam: Elsevier), 197–200.

Google Scholar

Looi, V., McDermott, H., McKay, C., and Hickson, L. (2008). The effect of cochlear implantation on music perception by adults with usable pre-operative acoustic hearing. Int. J. Audiol. 47, 257–268. doi: 10.1080/14992020801955237

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Maarefvand, M., Marozeau, J., and Blamey, P. J. (2013). A cochlear implant user with exceptional musical hearing ability. Int. J. Audiol. 52, 424–432. doi: 10.3109/14992027.2012.762606

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Macherey, O., and Carlyon, R. P. (2010). Temporal pitch percepts elicited by dual-channel stimulation of a cochlear implant. J. Acoust. Soc. Am. 127, 339–349. doi: 10.1121/1.3269042

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Macherey, O., and Carlyon, R. P. (2014). Cochlear implants. Curr. Biol. 24, R878–R884. doi: 10.1016/j.cub.2014.06.053

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Macherey, O., Deeks, J. M., and Carlyon, R. P. (2011). Extending the limits of place and temporal pitch perception in cochlear implant users. J. Assoc. Res. Otolaryngol. 12, 233–251. doi: 10.1007/s10162-010-0248-x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

McDermott, H. J. (2004). Music perception with cochlear implants: a review. Trends Amplif. 8, 49–82. doi: 10.1177/108471380400800203

CrossRef Full Text | Google Scholar

McNair, D. M., Lorr, M., and Droppleman, L. F. (1992). Profile of Mood States (Revised). San Diego, CA: Edits: Educational and Industrial Testing Service.

Migirov, L., Kronenberg, J., and Henkin, Y. (2009). Self-reported listening habits and enjoyment of music among adult cochlear implant recipients. Ann. Otol. Rhinol. Laryngol. 118, 350–355. doi: 10.1177/000348940911800506

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Mirza, S., Douglas, S. A., Lindsey, P., Hildreth, T., and Hawthorne, M. (2003). Appreciation of music in adult patients with cochlear implants: A patient questionnaire. Cochlear Implants Int. 4, 85–95. doi: 10.1002/cii.68

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Moore, B. C. J., and Carlyon, R. P. (2005). “Perception of pitch by people with cochlear hearing loss and by cochlear implant users,” in Pitch: Neural Coding and Perception, eds C. J. Plack, A. J. Oxenham, R. Fay, and A. N. Popper (New York: Springer), 234–277.

Google Scholar

Nardo, W. D., Cantore, I., Marchese, M. R., Cianfrone, F., Scorpecci, A., Giannantonio, S.,et al. (2008). Electric to acoustic pitch matching: a possible way to improve individual cochlear implant fitting. Eur. Arch. Otorhinolaryngol. 265, 1321–1328. doi: 10.1007/s00405-008-0655-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Nimmons, G. L., Kang, R. S., Drennan, W. R., Longnion, J., Ruffin, C., Worman, T.,et al. (2008). Clinical assessment of music perception in cochlear implant listeners. Otol. Neurotol. 29, 149–155. doi: 10.1097/mao.0b013e31812f7244

CrossRef Full Text | Google Scholar

Penninger, R. T., Kludt, E., Buchner, A., and Nogueira, W. (2015). Stimulating on multiple electrodes can improve temporal pitch perception. Int. J. Audiol. 29, 1–8. doi: 10.3109/14992027.2014.997313

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Penninger, R. T., Limb, C. J., Dhooge, I., and Buechner, A. (2014). Effect of dynamic range and location on temporal pitch perception of cochlear implant users. Biomed. Tech. 59, 782–785.

Google Scholar

Peretz, I., Champod, S., and Hyde, K. (2003). Varieties of musical disorders: the Montreal battery of Evaluation of Amusia. Ann. N. Y. Acad. Sci. 999, 58–75. doi: 10.1196/annals.1284.006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Qin, M. K., and Oxenham, A. J. (2003). Effects of simulated cochlear implant processing on speech reception in fluctuating maskers. J. Acoust. Soc. Am. 114, 446–454. doi: 10.1121/1.1579009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Rosslau, K., Spreckelmeyer, K. N., Saalfeld, H., and Westhofen, M. (2012). Emotional and analytic music perception in cochlear implant users after optimizing the speech processor. Acta Otolaryngol. 132, 64–71. doi: 10.3109/00016489.2011.619569

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Russell, J. A. (1980). A circumplex model of affect. J. Pers. Soc. Psychol. 39, 1161–1178. doi: 10.1037/h0077714

CrossRef Full Text | Google Scholar

Scherer, K. R. (1989). “Vocal correlates of emotional arousal and affective disturbance,” in Handbook of Social Psychophysiology: Emotion and Social Behavior, eds H. Wagner and A. Manstead (New York: Wiley), 165–197.

Google Scholar

Shannon, R. V. (1983). Multichannel electrical stimulation of the auditory nerve in man. Hear. Res. 11, 157–189. doi: 10.1016/0378-5955(83)90077-1

CrossRef Full Text | Google Scholar

Shannon, R. V. (1989). Detection of gaps in sinusoids and pulse trains by patients with cochlear implants. J. Acoust. Soc. Am. 85, 2587–2592. doi: 10.1121/1.397753

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Shannon, R. V. (1992). Temporal modulation transfer functions in patients with cochlear implants. J. Acoust. Soc. Am. 91, 2156–2164. doi: 10.1121/1.403807

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Smith, Z. M., Delgutte, B., and Oxenham, A. J. (2002). Chimaeric sounds reveal dichotomies in auditory perception. Nature 416, 87–90. doi: 10.1038/416087a

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Spielberger, C. D. (1983). Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press Inc.

Venter, P. J., and Hanekom, J. J. (2014). Is there a fundamental 300 Hz limit to pulse rate discrimination in Cochlear Implants? J. Assoc. Res. Otolaryngol. 15, 849–866. doi: 10.1007/s10162-014-0468-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Vieillard, S., Peretz, I., Gosselin, N., Khalfa, S., Gagnon, L., and Bouchard, B. (2008). Happy, sad, scary and peaceful musical excerpts for research on emotions. Cogn. Emot. 22, 720–752. doi: 10.1080/02699930701503567

CrossRef Full Text | Google Scholar

Zeng, F. G. (2002). Temporal pitch in electric hearing. Hear. Res. 174, 101–106. doi: 10.1016/S0378-5955(02)00644-5

CrossRef Full Text | Google Scholar