- 1Department of Psychology, Faculty of Education, Hubei University, Wuhan, China
- 2Brain and Cognition Research Center (BCRC), Faculty of Education, Hubei University, Wuhan, China
- 3Department of Psychology, College of Humanities and Management, Guizhou University of Traditional Chinese Medicine, Guiyang, China
- 4Applied Brain Science Lab, Faculty of Interdisciplinary Science and Engineering in Health Systems, Okayama University, Okayama, Japan
- 5Research Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, Guangdong, China
Introduction: In comparison to the audiovisual integration of younger adults, the same process appears more complex and unstable in older adults. Previous research has found that stimulus intensity is one of the most important factors influencing audiovisual integration.
Methods: The present study compared differences in audiovisual integration between older and younger adults using dynamic hand-held tool stimuli, such as holding a hammer hitting the floor. Meanwhile, the effects of stimulus intensity on audiovisual integration were compared. The intensity of the visual and auditory stimuli was regulated by modulating the contrast level and sound pressure level.
Results: Behavioral results showed that both older and younger adults responded faster and with higher hit rates to audiovisual stimuli than to visual and auditory stimuli. Further results of event-related potentials (ERPs) revealed that during the early stage of 60–100 ms, in the low-intensity condition, audiovisual integration of the anterior brain region was greater in older adults than in younger adults; however, in the high-intensity condition, audiovisual integration of the right hemisphere region was greater in younger adults than in older adults. Moreover, audiovisual integration was greater in the low-intensity condition than in the high-intensity condition in older adults during the 60–100 ms, 120–160 ms, and 220–260 ms periods, showing inverse effectiveness. However, there was no difference in the audiovisual integration of younger adults across different intensity conditions.
Discussion: The results suggested that there was an age-related dissociation between high- and low-intensity conditions with audiovisual integration of the dynamic hand-held tool stimulus. Older adults showed greater audiovisual integration in the lower intensity condition, which may be due to the activation of compensatory mechanisms.
Introduction
With the extension of life expectancy, the number of older adults continues to grow. Aging impairs sensory abilities, which makes it difficult for older adults to perceive stimuli in their surroundings (Chou et al., 2016; Jayakody et al., 2018). However, the effective use of multisensory information can help people better perceive external stimuli. Compared with single-sensory information, multisensory information can improve the probability and speed of the correct recognition of stimuli (Diederich and Colonius, 2004; Mercier and Cappe, 2020). The visual and auditory modalities are two important sources to obtain information from the outside world. The process of combining visual and auditory signals into a unified and stable percept is called audiovisual integration (Fendrich, 1993; Stein and Stanford, 2008). In comparison to the audiovisual integration of younger adults, the same process appears more complex and unstable in older adults (de Dieuleveult et al., 2017; Jones and Noppeney, 2021).
Many studies have compared audiovisual integration between older and younger adults under different stimulus types. Peiffer et al. (2007) used light emitting diodes and white noise as stimuli and asked participants to complete an audiovisual detection task. The results showed that under visual and auditory stimuli, there was no difference in the response time between the older adults and the younger adults and that under the audiovisual stimulus, the older participants responded faster than the younger participants. Then, the race model further found greater audiovisual integration in older adults than in younger adults (Peiffer et al., 2007). Later, researchers used event-related potential (ERP) technology to explore the facilitation of audiovisual speech information in older adults. The results showed that the amplitudes of P1 and N1 evoked by the audiovisual stimulus in the older adults were smaller than the sum of the amplitudes evoked by visual and auditory stimuli, and the observed P1 amplitude reduction was larger in older adults than younger adults. The reduced amplitudes of these components suggested that older adults could perform behavioral tasks well with fewer neural resources when audiovisual stimuli were presented simultaneously (Winneke and Phillips, 2011). Further research using magnetoencephalography (MEG) showed that both younger and older adults presented increased brain activity in response to audiovisual stimuli compared to visual or auditory stimuli presented alone after 100 ms stimulus onset. However, only older adults preferentially responded to audiovisual stimuli in the posterior parietal and medial prefrontal regions between 150 and 300 ms (Diaconescu et al., 2013). The results support the view that although the sensory processing ability decreases with age, audiovisual integration may provide a strategy that can ameliorate sensory deficits caused by a decline in single sensory processing and that brain activation is greater or more widespread in older adults than in younger adults, which might reflect compensation mechanisms with aging (Laurienti et al., 2006; Winneke and Phillips, 2011; Cabeza et al., 2018). Previous studies on audiovisual integration in older adults mostly used static simple meaningless stimuli or dynamic complex speech stimuli. However, few studies have explored the impact of dynamic hand-held tools on audiovisual integration in older adults. The complexity of dynamic hand-held tools, such as holding a hammer hitting the floor, is between static simple stimuli and dynamic speech stimuli, including the simultaneous presentation of object motion and body parts, which could reflect the motion-related process (Stevenson et al., 2006; Stevenson and James, 2009; Werner and Noppeney, 2010). A recent study found that the theta-band functional connectivity of older adults to the audiovisual dynamic hand-held tool was higher than that of younger adults (Ren et al., 2020). Dynamic hand tools, including the continuous process of early tool representation and late tool usage, are important for studying how older adults dynamically integrate visual and auditory information during this process. Therefore, this study used a dynamic hand-held tool as stimulation material to further explore the dynamic processing mechanism of audiovisual integration in older adults.
Previous research has found that stimulus intensity is one of the most important factors influencing audiovisual integration. Many behavioral studies involving younger adults have shown that low-intensity audiovisual stimuli can lead to more robust behavioral facilitation than high-intensity audiovisual stimuli (Corneil et al., 2002; Otto et al., 2013). Subsequent ERP studies using meaningless stimuli investigated the neurophysiological basis of the relationship between stimulus intensity and audiovisual integration. The results showed that after 40–60 ms of stimulus onset, audiovisual integration occurred only with a low-intensity stimulus in the left posterior and right anterior regions, but no audiovisual integration was observed in the medium- and high-intensity conditions (Senkowski et al., 2011). This phenomenon of enhanced audiovisual integration in low-intensity conditions is known as inverse effectiveness. However, a recent study using the same meaningless stimulus found that there was no difference in the audiovisual integration of older adults under different intensity conditions, indicating that older adults did not exhibit inverse effectiveness (Yang et al., 2021). For the complex speech stimulus, some studies found that at the whole-word level, the greater audiovisual integration in older adults than in younger adults under the intermediate stimulus intensity, but older adults reduced benefit at low stimulus intensity. In contrast, at the phoneme level, older adults showed similar audiovisual enhancement as younger adults when the stimulus intensity decreased (Stevenson et al., 2015). These results indicated that the inverse effectiveness was different between the younger and older adults across the different stimulus types. Some researchers used dynamic natural scenes as stimuli and found that dynamic high-intensity audiovisual stimuli are integrated through the direct exchange of information between the visual and auditory sensory cortex, while dynamic low-intensity audiovisual stimuli rely on the integration of input from the intraparietal sulcus (IPS), which may be related to the executive cortex network (Regenbogen et al., 2018). The dynamic hand-held tool was not only associated with motor execution but also associated with episodic memory. However, the effect of different intensities of the dynamic hand-held tool on audiovisual integration in older adults was unclear. Due to the instability of the inverse effectiveness in older adults, as well as the decline in executive function and episodic memory (Cepeda et al., 2001; Isingrini and Taconnat, 2008), it is necessary to further explore the effect of dynamic hand-held tools in different stimulus intensities on audiovisual integration in older adults.
The present study used dynamic hand-held tools to compare audiovisual integration between high- and low-intensity conditions to explore the dynamic processing mechanism of the audiovisual integration of motion-related tools in older and younger adults. We divided dynamic hand-held tools stimuli into high- and low-intensity stimuli and had the subjects perform a discrimination task in a random presentation of visual, auditory, and audiovisual stimuli. At the same time, we used ERPs to record the brain activity of subjects as they completed the task. Based on the above literature, it was hypothesized that stimulus intensity moderated and played a key role in the audiovisual integration of older and younger adults.
Materials and Methods
Participants
Twenty older adults (6 males, 14 females; mean age = 63.7 years, SD = 2.9 years) and 24 younger adults (11 males, 13 females; mean age = 21.4 years, SD = 2.6 years) participated in this experiment. All subjects had normal or corrected-to-normal vision and hearing and were free of chronic or neurological diseases through self-report. All subjects were right-handed. In addition, to exclude cognitive impairments that might affect the experimental results, older subjects were screened using the Simple Mental State Examination (MMSE). Subjects whose score was higher than 26 were considered normal (Bravo and Hébert, 1997). Due to recording EEG data, subjects were asked to keep their head and body as stable as possible during the experiment and to reduce blinking. One older participant withdrew from the experiment due to physical fatigue during the low-intensity block, and one older subject and four younger subjects were excluded due to excessive EEG artifacts resulting in fewer than 30 remaining segments. Finally, 18 older adults (6 males, 12 females; mean age = 63.0 years, SD = 2.9 years) and 20 younger adults (10 males, 10 females; mean age = 21.6 years, SD = 2.7 years) completed the entire experiment, and data were collected for further analysis. The average number of remaining epochs in older adults was 111.49 ± 21.83 (high-intensity condition: visual = 115.72 ± 25.76, auditory = 110.78 ± 29.74, audiovisual = 114.06 ± 27.31; low-intensity condition: visual = 111.44 ± 11.53, auditory = 107.88 ± 15.63, audiovisual = 108.31 ± 13.31). The average number of remaining epochs in younger adults was 96.91 ± 23.23 (high-intensity condition: visual = 101.28 ± 21.87, auditory = 94.89 ± 24.56, audiovisual = 97.67 ± 25.83; low-intensity condition: visual = 102.11 ± 20.35, auditory = 94.72 ± 23.60, audiovisual = 96.11 ± 23.13). All subjects were blinded to the purpose of this study and gave informed consent for the experiment. Each subject was paid a monetary reward after the experiment. The experiment had previously been approved by the ethics committee of Hubei University.
Stimuli
Stimuli consisted of two dynamic hand-held tools (hammer and stick) that were recorded with a MiniDV Digital Camcorder (Sony, DCR-PC55). The video time is 800 ms, including the complete movement cycle and percussive sound of the hand-held tool. To separate the visual and auditory stimuli, Adobe Premiere CS6 was used to extract visual and auditory files from the original video. The visual stimuli were the videos acquired from the original video of 966 × 544 pixels and converted from color to grayscale. The auditory stimuli were the sound acquired at the original video of 32-bit audio with a sampling rate of 48 kHz and converted from stereo to mono. The intensity of the visual stimuli was regulated by modulating the contrast level, (max − min)/(max + min), with the high-intensity contrast being 90% and the low-intensity contrast being 10%. The intensity of the auditory stimuli was regulated by modulating the sound pressure level (SPL), with the high-intensity stimuli at 70 dB and the low-intensity stimuli at 50 dB. The selection of these methods was based on previous studies to ensure their effectiveness (Ren et al., 2020; Yang et al., 2021).
During the experiment, the stimuli were divided into target stimuli and standard stimuli. The visual target stimulus was the dynamic video of holding a wooden stick to hit the floor, and the visual standard stimulus was the dynamic video of holding a hammer hitting the floor. The auditory target stimulus was the sound of a wooden stick hitting the marble floor, and the auditory standard stimulus was the sound of the hammer hitting the marble floor. The audiovisual target stimulus was the combination of the dynamic video of holding a wooden stick to hit the floor and the sound of a wooden stick hitting the marble floor. The audiovisual standard stimulus was the combination of the dynamic video of holding a hammer hitting the floor and the sound of the hammer hitting the marble floor.
Procedure and task
The experimental procedure was carried out using E-prime 2.0 software. As shown in Figure 1, a 2,000 ms fixation point was presented at the beginning of each block. After the fixation point, target stimuli (visual, auditory, audiovisual) and standard stimuli (visual, auditory, audiovisual) were randomly presented for 800 ms. Then, a random trial interstimulus interval (ISI) was between 1,200 ms and 1,500 ms. Visual stimuli were presented using a Dell computer monitor at approximately 60 cm from the monitor. The visual stimulus had a vertical visual angle of 5° and a horizontal visual angle of 12°. The auditory stimuli were presented via speakers located on the left and right sides of the computer monitor.
The experiment included a high-intensity stimulus block and a low-intensity stimulus block. Each block contained 450 trials: 150 visual trials, 150 auditory trials, and 150 audiovisual trials, and each modality included 30 target trials and 120 standard stimulus trials. During the experiment, the laboratory room was dark and quiet. The participants were asked to fix their eyes on the screen and respond quickly and accurately to the target stimuli (left mouse button) but not to the standard stimuli. In the process of EEG recording, the participants were told to minimize blinking and keep their bodies and heads stable to avoid excessive artifacts. The duration of each block was approximately 16 min, and the subjects were allowed to rest every 4 min. High-intensity and low-intensity stimuli were administered in two blocks and randomly assigned between subjects.
EEG recording and preprocessing
The EEG data were recorded with a 32-channel cap using the BrainAmp MR plus system and Brain recorder software. The resistance of reference and the other interelectrode impedances were set below 5 kΩ. Vertical and horizontal electrooculograms (EOGs) were recorded by two pairs of electrodes to monitor eye movements and eye blinks. The raw signals were digitized with a sampling frequency of 250 Hz.
All the data were stored digitally for offline analysis. EEG data preprocessing was conducted in Brain analysis software, including rereferencing to the bilateral mastoid electrodes (TP9 and TP10). The ERP data elicited by the standard stimuli were analyzed. Epochs were cut from 100 ms pre-stimuli to 400 ms post-stimulus onset for each trial. The original data were bandpass filtered from 0.01 to 60 Hz, and the time window between 100 ms and 0 ms before stimulus presentation was used as baseline correction. In addition to a ±100 μV artifact rejection criterion, EEG epochs containing eye blinks or other noise transients were removed based on a trial-by-trial inspection of the data. Then, ERP data were grand-averaged for each stimuli modality, followed by digital filtering with a bandpass filter of 0.1–30 Hz.
Data Analysis
Behavioral data analysis
The hit rate (HT) was the percentage of correct responses relative to the total number of target stimuli. The false alarm (FA) was the percentage of incorrect responses relative to the total number of standard stimuli. The response times (RTs) were calculated based on the responses that fell within the average period ±3 SD. The number of trails remaining under each condition accounted for more than 97.5% of the total number of trials. Then, a three-mixed factor analysis of variance (ANOVA) of age (younger, older), stimulus intensity (high, low), and modality (visual, auditory, audiovisual) was conducted. If ANOVA revealed a significant interaction effect, subsequent simple effect analysis was conducted separately for each factor.
ERP analysis
The multisensory integration effect was measured by the different amplitudes between the AV and A + V. The effectiveness of this method is attested by a statistical hypothesis that the amplitudes evoked by the neural activities by audiovisual stimuli (AV) should be equivalent to the summation of the amplitudes evoked by the neural activities by unimodal stimuli plus (A + V). However, if the amplitude of ERP(AV) − [ERP(A) + ERP(V)] was significantly greater or less than zero, it can be inferred that audiovisual integration occurred (Giard and Peronnet, 1999; Senkowski et al., 2007).
To measure the audiovisual integration between the different situations, the statistical analysis was conducted in the following steps. First, pointwise running t-tests were used. The mean amplitudes of ERP(AV) − [ERP(A) + ERP(V)] were compared with 0 at each time point from 0 to 400 ms in each electrode. If 12 (24 ms) or more consecutive points were significant (criterion p < 0.050), we defined that audiovisual integration occurred. Based on the t-test analysis, the time course showed four audiovisual integration time windows (60–100 ms, 120–160 ms, 220–260 ms, 340–380 ms). Furthermore, the time course showed possible lateralization of audiovisual integration. In previous research, audiovisual integration might appear in the hemisphere of the brain (Molholm et al., 2002; Diaconescu et al., 2013). Therefore, based on the results of previous and present studies, the five regions of interest (ROIs) were set as left anterior (F3, FC5, FC1), right anterior (F4, FC6, FC2), central (C3, Cz, C4), left posterior (P3, CP5, CP1), and right posterior (P4, CP6, CP2). Subsequently, the wave amplitudes of the electrodes in each region were averaged for inclusion in subsequent data analysis. Second, a three-factor mixed-design ANOVA 2 (age: younger, older) × 2 (stimulus intensity: high, low) × 5 (ROI: left anterior, right anterior, central, left posterior, right posterior) was conducted for the selected time interval by using the amplitudes of ERP(AV) − [ERP(A) + ERP(V)]. Age was a between-subject factor, and stimuli intensity and ROI were within-subject factors. The SPSS version 26.0 software package was used for all statistical analyses and Greenhouse-Geisser corrections with corrected degrees of freedom.
Results
Behavioral results
The hit rate was analyzed with a 2 (age: younger adults, older adults) × 2 (stimulus intensity: high, low) × 3 (modality: visual, auditory, audiovisual) mixed-factors ANOVA. The analysis only revealed that the modality main effect was significant, F(2,72) = 7.86, p = 0.002, = 0.18, with the hit rate of the audiovisual modality being higher than that of the visual, p = 0.013, and auditory modality, p < 0.001. The hit rate of the visual modality and auditory modality was not significantly different, p = 0.058. No significant interaction effect was observed for the hit rate, ps > 0.050. The detailed hit rates are shown in Table 1.
Table 1. Response times and hit rates for both the younger and older adults under the high-intensity and low-intensity conditions.
The false alarm was analyzed using a 2 (age: younger, older) × 2 (stimulus intensity: high, low) × 3 (modality: visual, auditory, audiovisual) mixed-factors ANOVA. The analysis revealed that the main effect of age was not significant, F(1,36) = 2.82, p = 0.102, = 0.07. The main effect of stimulus intensity was significant, F(1,36) = 4.29, p = 0.046, = 0.11, with the false alarm to the high-intensity stimulus being greater than that to the low-intensity stimulus. The modality main effect was significant, F(2,72) = 10.63, = 0.38, p < 0.001, with the false alarm of the auditory modality being greater than that of the audiovisual modality, p < 0.001, and visual modality, p < 0.001. The false alarm of the audiovisual modality and visual modality were not significantly different, p = 0.303. No other significant interaction effect was observed for false alarm, ps > 0.050. The detailed false alarms are shown in Table 1.
The response time was analyzed using a 2 (age: younger, older) × 2 (stimulus intensity: high, low) × 3 (modality: visual, auditory, audiovisual) mixed-factors ANOVA. The analysis revealed that the main effect of age was significant, F(1,36) = 12.96, p = 0.001, = 0.27, with the response time of older adults being longer than that of younger adults. The main effect of stimulus intensity was significant, F(1,36) = 6.03, p = 0.019, = 0.14, with the response time to the high-intensity stimulus being shorter than that to the low-intensity stimulus. The modality main effect was significant, F(2,72) = 99.31, = 0.73, p < 0.001, with the response time of the audiovisual modality being shorter than that of the visual modality, p < 0.001, and auditory modality, p < 0.001. Moreover, the response time of the visual modality was shorter than that of the auditory modality, p = 0.001 (AV < V < A). Additionally, there was a significant interaction between the group and modality, F(2,72) = 4.14, = 0.10, p = 0.021. Further simple effect analysis showed that for all modalities, the response time of younger adults was shorter than that of older adults, ps ≤ 0.008. For both younger and older adults, the response to the audiovisual modality was the fastest, ps ≤ 0.016. No other significant interaction effect was observed for response time, ps > 0.050. The detailed response times are shown in Table 1.
ERP results
Audiovisual integration at 60–100 ms
A three mixed-factor ANOVA revealed that the main effect of age was not significant, F(1,36) = 0.35, p = 0.560, = 0.01. The main effect of stimulus intensity was marginally significant, F(1,36) = 3.92, p = 0.055, = 0.10, with the amplitudes of the low-intensity stimulus being more negative than those of the high-intensity stimulus. The ROI main effect was also significant, F(4,144) = 3.37, p = 0.029, = 0.09, with the amplitudes of the right anterior being more negative than those of the left anterior, central, and left posterior, ps < 0.040. The amplitudes of the central and right posterior regions were more negative than those of the left posterior region, ps < 0.033. A three-factor interaction effect was significant, F(4,144) = 9.86, p < 0.001, = 0.22, and the stimulus intensity and age interaction effect were also significant, F(1,36) = 11.92, p = 0.001, = 0.25.
Further simple effect analysis indicated that in the high-intensity condition, the amplitudes of the younger adults were more negative than those of older adults in the left anterior, central, and right anterior directions, ps < 0.047. In the low-intensity condition, the amplitudes of the older adults were more negative than those of younger adults in the left and right anterior directions, ps < 0.017. For the younger adults, in the high-intensity condition, the amplitudes of the right anterior were more negative than those of the left anterior, central, and left posterior, ps < 0.035, and the amplitudes of the right posterior were marginally more negative than those of the left posterior, p = 0.056. In the low-intensity condition, the amplitudes of the right posterior were more negative than those of the left posterior, p = 0.028. However, the amplitudes of the high-intensity condition and low-intensity condition were not significantly different in all regions, ps > 0.050. For the older adults, in the high-intensity condition, the amplitudes of all regions were not significant, ps > 0.050. However, in the low-intensity condition, the amplitudes of the left and right anterior were more negative than those of the left and right posterior, ps < 0.036, and the amplitudes of the center were more negative than those of the left anterior, p = 0.014. Furthermore, the amplitudes of the low-intensity condition were more negative than those of the high-intensity condition in the left anterior, right anterior, central, and right posterior, ps < 0.005. Grand-averaged event-related potentials and topography maps of audiovisual integration in the time window of 60–100 ms are shown in Figure 2.
Figure 2. Grand-averaged event-related potentials and topography map of audiovisual integration in the time window of 60–100 ms. Grand-averaged event-related potentials included the left anterior (F3, FC5, FC1), left posterior (P3, CP5, CP1), right anterior (F4, FC6, FC2), and right posterior (P4, CP6, CP2) regions. The solid line is the event-related potentials of the audiovisual stimulus (AV), and the dotted line is the sum of the event-related potentials of visual and auditory stimuli (A + V). The areas where audiovisual integration increases are marked with gray squares. The topographic map shows the difference between AV and A + V. (A) Under the high-intensity condition, audiovisual integration was greater in the right anterior and right posterior regions in young adults than in older adults. (B) Under the low-intensity condition, audiovisual integration in the left anterior and right anterior regions was greater in older adults than in younger adults.
Audiovisual integration at 120–160 ms
A three mixed-factor ANOVA revealed that the main effect of age was not significant, F(1,36) = 0.01, p = 0.945, = 0.01. The stimulus intensity main effect was significant, F(1,36) = 4.97, p = 0.032, = 0.12, with the amplitudes of the low-intensity condition being more negative than those of the high-intensity condition. However, the main effect of ROI was not significant, F(4,144) = 3.36, p = 0.052, = 0.09. A three-factor interaction effect was significant, F(4,144) = 3.25, p = 0.005, = 0.08, and the age and intensity interaction effect were also significant, F(1,36) = 4.33, p = 0.045, = 0.11.
Further simple effect analysis indicated that for the older adults, the amplitudes of the low-intensity condition were more negative than those of the high-intensity condition, p = 0.005. Furthermore, in the low-intensity condition, the amplitudes of the left and right anterior were more negative than those of the central, left posterior, and right posterior in older adults, ps < 0.044. No significant effect was observed for other conditions, ps > 0.050.
Audiovisual integration at 220–260 ms
A three mixed-factor ANOVA revealed that the main effect of age was not significant, F(1,36) = 0.06, p = 0.805, = 0.01. The main effect of stimulus intensity was not significant, F(1,36) = 1.90, p = 0.177, = 0.05. The main effect of ROI was also not significant, F(4,144) = 3.03, p = 0.064, = 0.08. However, a three-way interaction effect was significant, F(4,144) = 2.88, p = 0.025, = 0.07, and other interaction effects were not significant, ps > 0.050.
Further simple effect analysis indicated that for the older adults, the amplitudes of the low-intensity condition were more negative than those of the high-intensity condition, p = 0.019. In the low-intensity condition, the amplitudes of the right anterior were more negative than those of the left anterior and posterior, ps < 0.044. No significant effect was observed for other conditions, ps > 0.050.
Audiovisual integration at 340–380 ms
A three mixed-factor ANOVA revealed that the main effect of age was not significant, F(1,36) = 0.01, p = 0.943, = 0.01. The main effect of stimulus intensity was not significant, F(1,36) = 3.37, p = 0.075, = 0.09. The main effect of ROI was also not significant, F(4,144) = 2.59, p = 0.074, = 0.07. However, the age and ROI interaction effect was significant, F(4,144) = 5.69, p = 0.003, = 0.14. The other interaction effects were not significant, all ps > 0.050.
Further analysis shows that for the younger adults, the amplitude of the central region was more negative than that of the left anterior, p = 0.049. For older adults, the amplitudes of the left anterior, right anterior, and central regions were more negative than those of the left and right posterior regions, ps < 0.047. No significant effect was observed for other conditions, ps > 0.050. Grand-averaged event-related potentials and topography maps of audiovisual integration of older adults in the high- and low-intensity conditions are shown in Figure 3.
Figure 3. Grand-averaged event-related potentials and topography map of audiovisual integration in the time window of 60–100 ms, 120–160 ms, 220–260 ms, and 340–380 ms. The wave amplitude is the average of the wave amplitudes of the three electrodes C3, Cz, and C4. The solid line is the event-related potentials of the audiovisual stimulus (AV), and the dotted line is the sum of the event-related potentials of visual and auditory stimuli (A + V). The areas where audiovisual integration increases are marked with gray squares. The topographic map shows the difference between AV and A + V. (A) The amplitude of waves in older adults under high- and low-intensity conditions. (B) The amplitude of waves in the younger adults under high- and low-intensity conditions.
Discussion
This study found that both older and younger adults responded faster and at higher hit rates to audiovisual stimuli than to visual and auditory stimuli. The response time of the older adults was significantly longer than that of the younger adults in each condition. ERPs revealed that during the early integration stage of 60–100 ms, in the low-intensity condition, audiovisual integration of the left anterior and right anterior regions was greater in older adults than in younger adults; however, in the high-intensity condition, audiovisual integration of the right anterior and right posterior regions was greater in younger adults than in older adults. In addition, audiovisual integration was greater in the low-intensity condition than in the high-intensity condition in older adults during the 60–100 ms, 120–160 ms, and 220–260 ms periods, showing inverse effectiveness.
Aging effect of audiovisual integration
During the early integration stage of 60–100 ms, audiovisual integration of the left anterior and right anterior regions was greater in older adults than in younger adults under low-intensity conditions. Some studies on younger adults found that in the early stage of processing, audiovisual integration generally appears in the primary processing cortex, such as the occipital cortex (Yang et al., 2015). In contrast, the study found audiovisual integration in the posterior parietal and medial prefrontal cortices in older adults during the early stage of processing. Increased activity in these regions in older adults predicts faster detection of cross-modality stimuli, suggesting that the posterior parietal and medial prefrontal cortices may be the compensatory regions of audiovisual integration in older adults (Diaconescu et al., 2013). Furthermore, according to the hemisphere asymmetric reduction hypothesis of older adults (HAROLD), older adults tend to show more symmetrical activation than younger adults (Cabeza, 2002; Cabeza et al., 2018). In aging studies, activation of the bilateral prefrontal cortex is generally considered to be an adaptive and compensatory neural mechanism for older adults, which is likely to compensate for the functional impairment of other brain areas. For example, some studies have found that in attention and memory tasks, brain activity in the bilateral prefrontal and precuneus regions increases in older adults (Velanova et al., 2007; Patrick et al., 2009). This compensation mechanism is adapted to the aging of brain structure and function, suggesting that compensatory brain activation in older adults could help maintain or improve cognitive performance. Therefore, the enhancement of audiovisual integration for older adults could be interpreted due to the compensatory mechanism of the brain recruitment strategy. In addition, the brain recruitment strategy in older adults may reflect the involvement of higher cognitive functions during the early stage of integrated processing. According to the cognitive permeation hypothesis, when older adults cannot measure stimulus information effectively, they might use higher cognitive functions to aid in sensory processing (Lindenberger et al., 2000). Under low-intensity conditions, due to the decrease in the contrast of visual stimuli and the loudness of sound stimuli, older adults may allocate more cognitive resources to participate in the early integration stage, resulting in greater audiovisual integration. On the other hand, compared with young adults, older adults are less able to suppress cross-modal information and are more susceptible to information from different sensory modalities (de Dieuleveult et al., 2017). Hugenschmidt et al. (2009) found that under the audiovisual condition, even if older adults were required to selectively pay attention to the information of a certain visual or auditory modality, the facilitation effect of audiovisual integration would also appear. However, under the condition of selective attention, the facilitation effect of audiovisual integration did not appear in younger adults (Hugenschmidt et al., 2009). Thus, the decreased ability to suppress cross-modality information could also explain the greater audiovisual integration in older adults than younger adults.
In addition, the current results showed that audiovisual integration in the right anterior and right posterior regions was greater in young adults than in older adults under high-intensity conditions. Studies have found that younger adults usually activate the right hemisphere when integrating motor-related stimuli (Senkowski et al., 2007). Moreover, the ERP results revealed that the audiovisual integration of younger adults appeared in the right parietal-occipital cortex 46 ms after stimuli presentation (Molholm et al., 2002). However, older adults did not exhibit a lateralization effect, which may be due to the dedifferentiation of the brain activation patterns caused by aging. Some studies have found that the differentiation of unilateral specific processing of the brain disappears with age, resulting in a decrease in the accuracy of information transmission and the clarity of mental presentation (Li et al., 2001). Older adults experience a decline in sensory function, including decreased visual acuity and increased auditory thresholds (Brooks et al., 2018; Jayakody et al., 2018). The declines in sensory function can further affect cognitive abilities, including deterioration in motor speed and executive function in older adults (Roberts and Allen, 2016). Considering that the effectiveness of multisensory integration depends on the functions of the sensory organs and cognitive processes (Talsma et al., 2010), it seems reasonable to observe a decline in audiovisual integration in older adults.
Inverse effectiveness of audiovisual integration for older adults
The results showed that the audiovisual integration of older adults was greater in low-intensity conditions than in high-intensity conditions during the 60–100 ms, 120–160 ms, and 220–260 ms periods. The results might be due to attention acting on audiovisual integration at early and late stages. The study found that audiovisual integration was greatest when the stimulus intensity was close to the sensory threshold (Cappe et al., 2010). It is possible that low-intensity stimuli might drive bottom-up attention, thereby enhancing audiovisual integration in the early stage (Van Zoest et al., 2004). Considering that stimulus intensity is reduced to approach the perceptual threshold of older adults, low-intensity stimuli might drive more attention in older adults during the early integration stage than in younger adults. Furthermore, subjects also actively allocated top-down attention to complete the task in the late stage (Talsma et al., 2010; Talsma, 2015). Considering the decline in executive function and reaction flexibility in older adults (Cepeda et al., 2001; Grady, 2012), older adults might allocate more attentional resources in the late integration stage to maintain performance. The parallel integration framework suggests that multisensory integration can occur in multiple stages and is dynamically modulated by attentional resources (Calvert and Thesen, 2004). For older adults, increasing attention to cross-modality information was a better strategy when one or more sensory sources become unreliable, which can be effective in helping older adults complete the task. Thus, the audiovisual integration of older adults under low-intensity stimuli might be modulated by the dynamics of attention. It can be observed at different stages that the audiovisual integration of older adults is greater in the low-intensity condition than in the high-intensity condition.
In addition, the dynamic hand-held tool used as the experimental material in this study has a certain special characteristic. The dynamic hand-held tool not only contains images of the movement of the arm and tool but also contains the experience when using the tool. Some studies have found that the brain regions that are active in observing a specific behavior closely match the brain regions that are active in performing the same behavior (Beauchamp and Martin, 2007; James et al., 2011). This mirror system reflects the actual experience of the individual in observing tool use. According to the sensory/motor system, individuals match the visual and auditory stimuli during tool use and store them in the perceptual system and in the motor system responsible for grasping and manipulating tools (Martin et al., 2000; Barsalou et al., 2003). Due to long-term life experience and practice, older adults have a strong connection between visual and auditory aspects when using tools. Furthermore, older adults maintain a better level of crystal intelligence related to the accumulation of experience. When the stimulus intensity decreases, the crystal intelligence and the richness of experience can compensate for the lack of information when perceiving the stimuli. Some studies further found that the connection between vision and auditory function obtained through practice influences perceptual judgment. Furthermore, the connection between vision and auditory stimuli is enhanced when stimulus uncertainty increases (Kafaligonul, 2018). Therefore, when the intensity of the stimuli intensity decreases, the robust connection between the visual and auditory senses might facilitate audiovisual integration in older adults.
In summary, there was an age-related dissociation between high- and low-intensity conditions with audiovisual integration of the dynamic hand-held tool stimulus. Older adults showed greater audiovisual integration in the lower intensity condition, which may be explained by the activation of compensatory mechanisms. However, the research still had some limitations. The presentation of sound stimuli only used a speaker, not the headphone, which might not completely shield outside sounds. Furthermore, we presented high-intensity and low-intensity stimuli between blocks in the current study, as sequential presentations might bring about fatigue effect and practice effect. If the subjects completed the high-intensity stimulation block first, the fatigue effect would make it more difficult for them to complete the low-intensity stimulation block. On the other hand, if subjects completed the low-intensity stimulation block first, there was a practice effect that made it easier for them to complete the high-intensity stimulation block. The stimulation block completed first would affect the stimulation block completed later, due to the fatigue effect and the practice effect, which might further affect the behavior response and neural activity of the subjects. Although the study carried out a balance among subjects, the effect of sequence effects could not be completely removed. Therefore, in subsequent studies, we would consider presenting the high- and low-intensity stimulus intermixed to reduce the influence of additional variables on the experimental results.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by Hubei University. The patients/participants provided their written informed consent to participate in this study.
Author Contributions
All authors contributed to the study conception and design. Material preparation and data collection were performed by AG, SL, and ZL. Data analysis was performed by XY, YR, and JL. The first draft of the manuscript was written by XY and WY. All authors commented on previous versions of the manuscript. All authors contributed to the article and approved the submitted version.
Funding
This study was funded by the National Natural Science Foundation of China (31700973, 31800932, 62103404, and 32260198), the Humanity and Social Science Youth Foundation of Ministry of Education of China (16YJC190025 and 18XJC190003), Science and Technology Planning Project of Guizhou Province [QianKeHeJiChu-ZK (2021) General 120], Shenzhen Overseas Innovation Team Project (KQTD20180413181834876), Shenzhen Basic Research Program (JCYJ20210324101402008 and JCYJ20210324115810030), and the CAS President’s International Fellowship Initiative (2022VBA0031).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Barsalou, L. W., Simmons, W. K., Barbey, A. K., and Wilson, C. D. (2003). Grounding conceptual knowledge in modality-specific systems. Trends Cogn. Sci. 7, 84–91. doi: 10.1016/s1364-6613(02)00029-3
Beauchamp, M. S., and Martin, A. (2007). Grounding object concepts in perception and action: evidence from FMRI studies of tools. Cortex 43, 461–468. doi: 10.1016/s0010-9452(08)70470-2
Bravo, G., and Hébert, R. (1997). Age- and education-specific reference values for the Mini-Mental and modified Mini-Mental State Examinations derived from a non-demented elderly population. Int. J. Geriatr. Psychiatry 12, 1008–1018. doi: 10.1002/(sici)1099-1166(199710)12:10<1008::aid-gps676>3.0.co;2-a
Brooks, C. J., Man, C. Y., Anderson, A. J., and Mckendrick, A. M. (2018). Audiovisual temporal perception in aging: the role of multisensory integration and age-related sensory loss. Front. Hum. Neurosci. 12:192. doi: 10.3389/fnhum.2018.00192
Cabeza, R. (2002). Hemispheric asymmetry reduction in older adults: the HAROLD model. Psychol. Aging 17, 85–100. doi: 10.1037//0882-7974.17.1.85
Cabeza, R., Albert, M., Belleville, S., Craik, F. I. M., Duarte, A., Grady, C. L., et al. (2018). Maintenance, reserve and compensation: the cognitive neuroscience of healthy ageing. Nat. Rev. Neurosci. 19, 701–710. doi: 10.1038/s41583-018-0068-2
Calvert, G. A., and Thesen, T. (2004). Multisensory integration: methodological approaches and emerging principles in the human brain. J. Physiol. Paris 98, 191–205. doi: 10.1016/j.jphysparis.2004.03.018
Cappe, C., Murray, M. M., Barone, P., and Rouiller, E. M. (2010). Multisensory facilitation of behavior in monkeys: effects of stimulus intensity. J. Cogn. Neurosci. 22, 2850–2863. doi: 10.1162/jocn.2010.21423
Cepeda, N. J., Kramer, A. F., and Sather, J. (2001). Changes in executive control across the life span: examination of task-switching performance. Dev. Psychol. 37, 715–730. doi: 10.1037/0012-1649.37.5.715
Chou, R., Dana, T., Bougatsos, C., Grusing, S., and Blazina, I. (2016). Screening for impaired visual acuity in older adults: updated evidence report and systematic review for the US preventive services task force. JAMA 315, 915–933. doi: 10.1001/jama.2016.0783
Corneil, B. D., Wanrooij, M. V., Munoz, D. P., and Opstal, A. V. (2002). Auditory-visual interactions subserving goal-directed saccades in a complex scene. J. Neurophysiol. 88, 438–454. doi: 10.1152/jn.2002.88.1.438
de Dieuleveult, A. L., Siemonsma, P. C., van Erp, J. B., and Brouwer, A. M. (2017). Effects of aging in multisensory integration: a systematic review. Front. Aging Neurosci. 9:80. doi: 10.3389/fnagi.2017.00080
Diaconescu, A. O., Hasher, L., and McIntosh, A. R. (2013). Visual dominance and multisensory integration changes with age. Neuroimage 65, 152–166. doi: 10.1016/j.neuroimage.2012.09.057
Diederich, A., and Colonius, H. (2004). Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept. Psychophys. 66, 1388–1404. doi: 10.3758/bf03195006
Fendrich, R. (1993). The merging of the senses. J. Cogn. Neurosci. 5, 373–374. doi: 10.1162/jocn.1993.5.3.373
Giard, M. H., and Peronnet, F. (1999). Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 11, 473–490. doi: 10.1162/089892999563544
Grady, C. (2012). The cognitive neuroscience of ageing. Nat. Rev. Neurosci. 13, 491–505. doi: 10.1038/nrn3256
Hugenschmidt, C. E., Mozolic, J. L., and Laurienti, P. J. (2009). Suppression of multisensory integration by modality-specific attention in aging. Neuroreport 20, 349–353. doi: 10.1097/WNR.0b013e328323ab07
Isingrini, M., and Taconnat, L. (2008). Episodic memory, frontal functioning and aging. Revue Neurologique 164, S91–S95. doi: 10.1016/S0035-3787(08)73297-1
James, T. W., VanDerKlok, R. M., Stevenson, R. A., and James, K. H. (2011). Multisensory perception of action in posterior temporal and parietal cortices. Neuropsychologia 49, 108–114. doi: 10.1016/j.neuropsychologia.2010.10.030
Jayakody, D. M. P., Friedland, P. L., Martins, R. N., and Sohrabi, H. R. (2018). Impact of aging on the auditory system and related cognitive functions: a narrative review. Front. Neurosci. 12:125. doi: 10.3389/fnins.2018.00125
Jones, S. A., and Noppeney, U. (2021). Ageing and multisensory integration: a review of the evidence and a computational perspective. Cortex 138, 1–23. doi: 10.1016/j.cortex.2021.02.001
Kafaligonul, H. (2018). Examining the effects of audiovisual associations on motion perception through task-based fMRI. Dusunen Adam J. Psychiatry Neurol. Sci. 138, 125–134. doi: 10.5350/DAJPN2018310201
Laurienti, P. J., Burdette, J. H., Maldjian, J. A., and Wallace, M. T. (2006). Enhanced multisensory integration in older adults. Neurobiol. Aging 27, 1155–1163. doi: 10.1016/j.neurobiolaging.2005.05.024
Li, S. C., Lindenberger, U., and Sikström, S. (2001). Aging cognition: from neuromodulation to representation. Trends Cogn. Sci. 5, 479–486. doi: 10.1016/s1364-6613(00)01769-1
Lindenberger, U., Marsiske, M., and Baltes, P. B. (2000). Memorizing while walking: increase in dual-task costs from young adulthood to old age. Psychol. Aging 15, 417–436. doi: 10.1037//0882-7974.15.3.417
Martin, A., Ungerleider, L. G., and Haxby, J. V. (2000). “Category specificity and the brain: the sensory/motor model of semantic representations of objects,” in The New Cognitive Neurosciences, 2nd Edn (Cambridge MA: MIT Press), 1023–1036.
Mercier, M. R., and Cappe, C. (2020). The interplay between multisensory integration and perceptual decision making. Neuroimage 222:116970. doi: 10.1016/j.neuroimage.2020.116970
Molholm, S., Ritter, W., Murray, M. M., Javitt, D. C., Schroeder, C. E., Foxe, J. J., et al. (2002). Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Cogn. Brain Res. 14, 115–128. doi: 10.1016/s0926-6410(02)00066-6
Otto, T. U., Dassy, B., and Mamassian, P. (2013). Principles of multisensory behavior. J. Neurosci. 33, 7463–7474. doi: 10.1523/JNEUROSCI.4678-12.2013
Patrick, C. M. W., Jin, X. J., Gunasekera, G. M., Abel, R., Lee, E. R., Dhar, S., et al. (2009). Aging and cortical mechanisms of speech perception in noise. Neuropsychologia 47, 693–703. doi: 10.1016/j.neuropsychologia.2008.11.032
Peiffer, A. M., Mozolic, J. L., Hugenschmidt, C. E., and Laurienti, P. J. (2007). Age-related multisensory enhancement in a simple audiovisual detection task. Neuroreport 18, 1077–1081. doi: 10.1097/WNR.0b013e3281e72ae7
Regenbogen, C., Seubert, J., Johansson, E., Finkelmeyer, A., Andersson, P., and Lundstrom, J. N. (2018). The intraparietal sulcus governs multisensory integration of audiovisual information based on task difficulty. Hum. Brain Mapp. 39, 1313–1326. doi: 10.1002/hbm.23918
Ren, Y. N., Guo, A., Xu, Z. H., Wang, T., Wu, R., Yang, W. P., et al. (2020). Age-related functional brain connectivity during audio-visual hand-held tool recognition. Brain Behav. 10:e01759. doi: 10.1002/brb3.1759
Roberts, K. L., and Allen, H. A. (2016). Perception and cognition in the ageing brain: a brief review of the short- and long-term links between perceptual and cognitive decline. Front. Aging Neurosci. 8:39. doi: 10.3389/fnagi.2016.00039
Senkowski, D., Saint-Amour, D., Hofle, M., and Foxe, J. J. (2011). Multisensory interactions in early evoked brain activity follow the principle of inverse effectiveness. Neuroimage 56, 2200–2208. doi: 10.1016/j.neuroimage.2011.03.075
Senkowski, D., Saint-Amour, D., Kelly, S. P., and Foxe, J. J. (2007). Multisensory processing of naturalistic objects in motion: a high-density electrical mapping and source estimation study. Neuroimage 36, 877–888. doi: 10.1016/j.neuroimage.2007.01.053
Stein, B. E., and Stanford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9, 255–266. doi: 10.1038/nrn2331
Stevenson, R. A., and James, T. W. (2009). Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44, 1210–1223. doi: 10.1016/j.neuroimage.2008.09.034
Stevenson, R. A., Geoghegan, M. L., and James, T. W. (2006). Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects. Exp. Brain Res. 179, 85–95. doi: 10.1007/s00221-006-0770-6
Stevenson, R. A., Nelms, C. E., Baum, S. H., Zurkovsky, L., Barense, M. D., Newhouse, P. A., et al. (2015). Deficits in audiovisual speech perception in normal aging emerge at the level of whole-word recognition. Neurobiol. Aging 36, 283–291. doi: 10.1016/j.neurobiolaging.2014.08.003
Talsma, D. (2015). Predictive coding and multisensory integration: an attentional account of the multisensory mind. Front. Integr. Neurosci. 9:19. doi: 10.3389/fnint.2015.00019
Talsma, D., Senkowski, D., Soto-Faraco, S., and Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends Cogn. Sci. 14, 400–410. doi: 10.1016/j.tics.2010.06.008
Van Zoest, W., Donk, M., and Theeuwes, J. (2004). The role of stimulus-driven and goal-driven control in saccadic visual selection. J. Exp. Psychol. Hum. Percept. Perform. 30:746. doi: 10.1037/0096-1523.30.4.749
Velanova, K., Lustig, C., Jacoby, L. L., and Buckner, R. L. (2007). Evidence for frontally mediated controlled processing differences in older adults. Cereb. Cortex 17, 1033–1046. doi: 10.1093/cercor/bhl013
Werner, S., and Noppeney, U. (2010). Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb. Cortex 20, 1829–1842. doi: 10.1093/cercor/bhp248
Winneke, A. H., and Phillips, N. A. (2011). Does audiovisual speech offer a fountain of youth for old ears? An event-related brain potential study of age differences in audiovisual speech perception. Psychol. Aging 26, 427–438. doi: 10.1037/a0021683
Yang, W. P., Li, Z. M., Guo, A., Li, S. N., Yang, X. F., Ren, Y., et al. (2021). Effects of stimulus intensity on audiovisual integration in aging across the temporal dynamics of processing. Int. J. Psychophysiol. 162, 95–103. doi: 10.1016/j.ijpsycho.2021.01.017
Keywords: audiovisual integration, stimulus intensity, older adults, dynamic hand-held tool, ERPs
Citation: Yang W, Yang X, Guo A, Li S, Li Z, Lin J, Ren Y, Yang J, Wu J and Zhang Z (2022) Audiovisual integration of the dynamic hand-held tool at different stimulus intensities in aging. Front. Hum. Neurosci. 16:968987. doi: 10.3389/fnhum.2022.968987
Received: 14 June 2022; Accepted: 15 November 2022;
Published: 14 December 2022.
Edited by:
Bernadette Ann Murphy, Ontario Tech University, CanadaReviewed by:
Phakkharawat Sittiprapaporn, Mae Fah Luang University, ThailandZhaohua Lu, Shandong University of Finance and Economics, China
Copyright © 2022 Yang, Yang, Guo, Li, Li, Lin, Ren, Yang, Wu and Zhang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yanna Ren, eWFubmEwNTIyNjcyMTNAMTYzLmNvbQ==; Zhilin Zhang, emhhbmd6aGlsaW5Ac2lhdC5hYy5jbg==