Ole Jensen1* Ali Bahramisharif1,2 Robert Oostenveld1 Stefan Klanke1 Avgis Hadjipapas1 Yuka O. Okazaki1 Marcel A. J. van Gerven1,2
- 1 Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Netherlands
- 2 Institute for Computing and Information Sciences, Radboud University Nijmegen, Netherlands
Large efforts are currently being made to develop and improve online analysis of brain activity which can be used, e.g., for brain–computer interfacing (BCI). A BCI allows a subject to control a device by willfully changing his/her own brain activity. BCI therefore holds the promise as a tool for aiding the disabled and for augmenting human performance. While technical developments obviously are important, we will here argue that new insight gained from cognitive neuroscience can be used to identify signatures of neural activation which reliably can be modulated by the subject at will. This review will focus mainly on oscillatory activity in the alpha band which is strongly modulated by changes in covert attention. Besides developing BCIs for their traditional purpose, they might also be used as a research tool for cognitive neuroscience. There is currently a strong interest in how brain-state fluctuations impact cognition. These state fluctuations are partly reflected by ongoing oscillatory activity. The functional role of the brain state can be investigated by introducing stimuli in real-time to subjects depending on the actual state of the brain. This principle of brain-state dependent stimulation may also be used as a practical tool for augmenting human behavior. In conclusion, new approaches based on online analysis of ongoing brain activity are currently in rapid development. These approaches are amongst others informed by new insight gained from electroencephalography/magnetoencephalography studies in cognitive neuroscience and hold the promise of providing new ways for investigating the brain at work.
Introduction
In recent years there has been a strong increase in the interest in characterizing brain activity online, for instance in the context of brain–computer interfaces (BCIs). A typical BCI setup allows for the online characterization of ongoing brain activity recorded by electroencephalography (EEG), magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), or intracranial recordings. The ongoing brain activity is used for controlling various devices through computer interfaces. Examples of applications are computer gaming, communication devices for highly impaired patients, rehabilitation, control of artificial limbs, and neuro-feedback (Lebedev and Nicolelis, 2006; Pfurtscheller and Neuper, 2006; Allison et al., 2007; Daly and Wolpaw, 2008; Van Gerven et al., 2009; Zander and Kothe, 2011).
To construct a robust BCI it is essential to make use of signatures of brain activity that can be modulated at will and quantified using short segments of data. In recent years there has been a rapid development in the application of brain imaging techniques to address cognitive questions. From these applications we have gained insight into which signal modulations are robust given a certain task. In particular it is becoming clear that oscillatory brain activity in the lower frequency range (<30 Hz) is reliably modulated in various cognitive tasks and can be detected with a reasonable signal-to-noise ratio using EEG and MEG (Hari and Salmelin, 1997; Klimesch et al., 2006; Jensen and Mazaheri, 2010).
While cognitive neuroscience can inform the development of BCI, the reverse might also hold (Blankertz et al., 2010). First of all, BCI forces researchers to focus on the strongest and most robust task-dependent modulations of brain signals. This could serve to ensure that an empirical investigation does not “get stuck” on a working hypothesis pertaining to aspects of the data being relatively weak. Also when developing a BCI it is essential to control for various confounds such as task difficulty in order to get reliable signals. During this process one often stumbles on new experimental questions pertaining to fundamental aspects that might not have been addressed before. See for instance (Bahramisharif et al., 2011) in which modulations of brain activity by covert attention prompted a question on how these changes were modulated by eccentricity. Second, online detection of brain activity can be used to train subjects’ ability to modulate their own ongoing brain activity and investigate the consequences for cognitive processing. Third, online detection of brain activity allows for presenting stimuli to the subject timed by the online measured brain activity. This is here referred to as brain-state dependent stimulation (BSDS) and provides a new avenue for investigating the neuronal substrate of cognition. In this paper we will give some examples in which this approach has been applied to gain new insight into human cognition. Furthermore we will propose ideas on how BSDS can be used in future settings to provide new neuroscience knowledge and augment human behavior.
The Functional Role of Oscillatory Brain Activity in the Alpha Band
The human brain produces oscillatory activity in various frequency bands both during rest and while performing cognitive tasks. What is the functional role of oscillatory brain activity? We will first focus on activity in the alpha band (8–13 Hz) since it has a strong signal-to-noise ratio and is often used as a control signal in BCI setups (Pfurtscheller and Neuper, 2006; Lou et al., 2008; Van Gerven et al., 2009; Boord et al., 2010). The alpha rhythm was first reported by Hans Berger in the late 1920s (Berger, 1929). When subjects are resting, it is by far the strongest electrophysiological signal that can be recorded from the human scalp. MEG research using source modeling has localized the sources of the alpha activity to parieto-occipital regions and primary sensorimotor regions around the central sulcus (Hari and Salmelin, 1997; Jensen and Vanni, 2002). Given that alpha activity is strongest when people rest and close their eyes, it has long been thought that it is related to brain states where few mental operations are occurring and thus labeled the “idling rhythm” (Pfurtscheller et al., 1996). Over the last decade, the idling hypothesis has been challenged (Klimesch et al., 2006; Palva and Palva, 2007; Thut and Miniussi, 2009; Jensen and Mazaheri, 2010). In particular it has been demonstrated that alpha activity actually can increase with cognitive load (Jensen et al., 2002; Scheeringa et al., 2009; Haegens et al., 2010; Khader et al., 2010; Meeuwissen et al., 2010). For instance, during working memory retention when no visual input are introduced, the alpha activity increases with the number of items that the subject has to retain (Jensen et al., 2002; Jensen, 2006; Tuladhar et al., 2007; Scheeringa et al., 2009). Other studies have used cognitive tasks which directly manipulate the engagement and disengagement of certain regions (Jokisch and Jensen, 2007; Rihs et al., 2007; Van Der Werf et al., 2008; Sauseng et al., 2009; Haegens et al., 2010; Romei et al., 2010; Freunberger et al., 2011). For instance, it is well established that alpha activity is depressed when covert attention is directed toward visual stimuli. However, when attention is directed toward auditory stimuli, the alpha activity over occipital areas increases (Fu et al., 2001). Other studies in which attention was directed toward the left or right visual hemifield have established that alpha activity is depressed in the hemisphere contralateral to the attended hemifield, while it often increases ipsilaterally (Worden et al., 2000; Thut et al., 2006; Rihs et al., 2007; Van Gerven et al., 2009; Cosmelli et al., 2011; Gould et al., 2011). These findings support the notion that the alpha activity reflects inhibition of regions not involved in a given task (Klimesch et al., 2006). This inhibition will serve to allocate resources to areas involved in the actual processing. Indeed it has been suggested that oscillatory activity subserves a gating function (Lopes Da Silva, 1991). In the light of recent empirical findings, this view has been pushed further in the “gating by inhibition hypothesis” (Jensen and Mazaheri, 2010). According to this hypothesis, task-irrelevant areas are inhibited by the alpha activity in order to actively gate information to the task-relevant areas. By this principle the alpha activity can shape the functional architecture of the working brain. This idea is supported by several recent findings which have demonstrated that if the alpha activity is not strong enough in the task-irrelevant areas, then performance is suboptimal. For instance, in a recent MEG study, oscillatory brain activity was investigated in a somatosensory working memory task (Haegens et al., 2010). Stimuli were delivered to the right hand and thus engaged the left hemisphere. Interestingly, behavioral performance correlated with the magnitude of widespread right hemispheric alpha activity from the retention interval (between sample and probe) including right somatosensory areas (Figure 1). Support for the notion that the alpha activity serves to causally inhibit was found using paradigms engaging respectively the left or the right hemisphere together with TMS and EEG combined. Entraining the task-irrelevant hemisphere by TMS pulses at alpha frequency serves to increase performance (Sauseng et al., 2009; Romei et al., 2010).
Figure 1. Alpha activity in right sensorimotor areas predicts performance in a somatosensory working memory task engaging left hemisphere areas. This implies that functional inhibition of the right somatosensory cortex by alpha activity is predictive of successful performance. (A) Patterns of electrical stimuli were presented to the right hand and probed 2 s later. (B) When comparing the 8- to 13-Hz alpha activity in the retention interval between sample and probe (correct–incorrect), the right hemisphere alpha activity was stronger for successful performance. (C) Time–frequency representations of power showed that the task-dependent effect was constrained to the alpha band in the retention interval. (D) The alpha sources predicting successful performance were localized in areas including right sensorimotor and parieto-occipital regions. Reproduced with permission from Haegens et al. (2010).
A recent long-term memory study investigated the encoding of word sequences (Figure 2). After the words were presented, subjects had to rehearse them for a brief period of 3.4 s. Later, the memory for the word sequences was tested. This allowed for investigating the rehearsal activity associated with successful memory encoding. The key finding was that the alpha activity must be strong in occipital areas in order to support memory encoding (Meeuwissen et al., 2010). This alpha activity is likely to suppress incoming visual information interfering with the memory task. In short, it is now clear that alpha activity plays an important role in neuronal processing. Optimal processing requires alpha activity in task-irrelevant areas. These studies have helped to gain considerable insight into which tasks are effective for modulating the alpha activity. Paradigms in which covert attention is directed to different modalities or spatial locations, seem to strongly modulate the alpha activity. This insight is now being used in the development of novel BCI (Kelly et al., 2005; Van Gerven et al., 2009; Van Gerven and Jensen, 2009; Bahramisharif et al., 2010a, 2011).
Figure 2. An increase in posterior alpha activity predicts successful long-term memory encoding. This suggests that functional inhibition of occipital cortex is a requirement for successful memory formation. (A) Subjects were instructed to rehearse visually presented word triplets during a 3.4-s period. Later memory for the triplets was tested. This allowed for characterizing the brain activity during the rehearsal interval predicting successful memory formation. (B) The alpha activity was stronger for Later Remembered compared to Later Forgotten triplets. (C) The sources of the alpha activity predicting memory encoding were localized in the occipital cortex. Reproduced with permission from Meeuwissen et al. (2010).
Oscillatory activity in the beta band (13–30 Hz) is also modulated in many tasks. The strongest neuronal sources of the beta activity are found in primary motor areas (Salmelin and Hari, 1994; Jensen et al., 2005). Typically the beta activity is depressed during both sensory and motor tasks. After the task, the beta activity “rebounds” (Hari and Salmelin, 1997). This has resulted in the notion that the beta activity reflects active inhibition of the motor system. This view is challenged by other findings implicating the beta activity in more active processing (Brovelli et al., 2004; Tallon-Baudry et al., 2004; Engel and Fries, 2010). For instance it has been demonstrated that beta activity increases during decision-making tasks in the motor and pre-motor regions (Donner et al., 2009; Siegel et al., 2011). Given the relatively strong signal-to-noise ratio of the beta activity, is has also been used as control signal for BCI, in particular in the context of motor imagery (Mcfarland and Wolpaw, 2005; Nam et al., 2011).
Activity in the gamma band (30–150 Hz) is thought to reflect active neuronal processing (Tallon-Baudry and Bertrand, 1999; Fries et al., 2007; Jensen et al., 2007). This is amongst others motivated by the fact that when an ensemble of neurons synchronizes their firing in this frequency range, these neurons will have a strong impact on downstream target regions (Salinas and Sejnowski, 2001). This is because the integration time of the synaptic input is on the order of 10 ms. Thus synaptic inputs aligned within a 10-ms time window or less exercise a strong impact. Numerous studies in both animals and humans have now demonstrated that gamma band activity increases in brain regions involved in cognitive processing. In fact it is now possible to distinguish different features of visual stimuli using multivariate techniques applied to gamma band activity measured by MEG (Duncan et al., 2010). Interestingly, ECoG studies in which brain activity is recorded directly from the brain surface have demonstrated a robust modulation in the higher gamma band (Ramsey et al., 2006; Jerbi et al., 2009; Canolty and Knight, 2010; Edwards et al., 2010; Miller et al., 2010; Vidal et al., 2010; Gaona et al., 2011). Indeed, the high-frequency gamma band activity recorded by ECoG has been tested in BCI setups (Ramsey et al., 2006; Lachaux et al., 2007; Kubanek et al., 2009; Vansteensel et al., 2010). It should be noted that robust accounts of gamma activity in non-invasive studies largely stem from primary visual cortex, whereby the induced signal-changes are distinctively stimulus-driven and become unstable in the absence of stimuli in the visual field (Hoogenboom et al., 2006; Hadjipapas et al., 2007; Muthukumaraswamy and Singh, 2008). Thus such gamma activity is mostly modulated by extrinsic factors rather than being under intrinsic voluntary control and therefore less suited for BCI. However, since gamma activity is highly specific to the stimuli inducing it and persists throughout stimulus presentation (Hadjipapas et al., 2007; Duncan et al., 2010), it has the potential of being an excellent candidate for BSDS, whereby the “gamma state” can be well-controlled by manipulations of stimulus parameters.
In conclusion, different cognitive tasks produce robust modulations in oscillatory activity in various frequency bands and regions. For EEG and MEG data it is particularly the activity in the lower frequency bands (<30 Hz) that is under voluntary control and has a strong signal-to-noise ratio and is thus most suited for BCI applications. Higher frequency ranges might be promising as well but this remains to be further explored.
Alpha Modulated by Attention as a Control Signal for BCI – Toward Two-Dimensional Control
Which tasks might be optimal for modulating the alpha activity in a BCI setting? A now classic EEG study demonstrated that posterior alpha is modulated by covert spatial attention as seen in Figure 3 (Worden et al., 2000). Subjects were required to fixate at the center of a screen while covertly attending to a symbol in either the lower left or right hemifield. This manipulation produces a strong depression in the alpha activity in the hemisphere contralateral the attended hemifield compared to the alpha activity in the ipsilateral hemisphere. Can this hemispheric lateralization in alpha activity serve as a control signal for BCI? This question has been investigated in an MEG experiment in which data were subjected to single trial classification (Van Gerven et al., 2009). In several subjects it was indeed possible to reliably decode the direction of attention in individual trials of about 1 s length. The principle has also been applied in an online BCI setup (Bahramisharif et al., 2010a). The aim of the subjects was to move the “background” of a scene to the left or the right while fixating at the center of the screen. By covertly attending to the left or the right, a few subjects succeeded in moving the background in the desired direction. Previous work has established that BCI gives the feeling of a control if performance is above 70% (Kubler et al., 2004). The performance of classifying the two directions of covert attention is reported to be up to 86% for the offline experiment (Van Gerven and Jensen, 2009), and 83% for the online case (Bahramisharif et al., 2010a). This provides a proof-of-principle that modulation of hemispheric alpha lateralization by covert attention can be used as a control signal for BCI.
Figure 3. Hemispheric alpha lateralization in response to left or right covert attention. This effect has later been tested as a control signal for BCI. When subjects were asked to attend to the right, left hemisphere alpha was depressed compared to when attention was directed to the left. The reverse effect was found over the right hemisphere. Reproduced with permission from Worden et al. (2000).
Not only do changes in covert attention in the horizontal plane modulate posterior alpha activity, so do vertical changes. In an EEG study subjects were asked to covertly direct their attention to eight different directions. When inspecting the topographical maps of the alpha activity, there were strong modulations according to the direction of attention (Rihs et al., 2007). When subjects were attending down, the posterior alpha activity was suppressed more so than when attention was directed up. This implies that the vertical direction of covert attention might be used for BCI control as well. In an MEG study in which subject were asked to covertly attend to a symbol in one of four directions (up, down, left, right) it was indeed possible to decode the attended direction in single trials (Van Gerven et al., 2009). This raises the possibility that an arbitrary direction of covert attention can be decoded by taking the topographic distribution of covert attention into account. This was tested in a setup in which subjects were asked to fixate centrally but covertly track the direction of a moving target. Using a sliding time window of 1700 ms it was possible to decode the direction of attention well above chance with a mean absolute deviation between the actual and predicted directions of about 60 degrees for the best case (Bahramisharif et al., 2010b). This error can be considerably reduced if smooth target movement over time is used as an additional constraint. This constraint can be mathematically expressed using a linear dynamical system where the predicted direction of attention not only depends on the alpha activity but also on the previous predicted direction. By virtue of this approach the mean absolute deviation decreased from 60 to 48 degrees for the best case. See Figure 4 for a visualization of these results.
Figure 4. The topography of the alpha activity is modulated by covert attention in different directions. This opens the possibility of making a BCI system which tracks attention in two dimensions. (A) Subjects were asked to covertly attend to one of eight directions for a 1300-ms period while fixating centrally. During this interval the alpha power was estimated using data from a 128 channel EEG system. The resulting topographies for the eight direction are shown. Both vertical and horizontal directions of covert attention clearly modulated the distribution in the alpha band. Reproduced with permission from Rihs et al. (2007). (B) In an MEG experiment subjects were asked to continuously track the direction of a target moving on a circle while fixating at the central cross. (C) The directions of attention were arbitrarily divided into 16 subparts and the respective alpha power was estimated using at 275 sensor MEG system. The topography of the alpha band activity is clearly modulated by the direction of attention. (D) The direction of covert attention was estimated from 1700 ms data segments. The true angle of the target could be predicted from the alpha activity alone with some error (top panel). After a smoothness constraint was applied (lower panel), the prediction error was much reduced. [(B,C,D), top panel] Reproduced with permission from Bahramisharif et al. (2010b).
This collection of studies demonstrates that modulations in alpha activity with covert attention can be used as a control signal for BCI. How robustly it works is under investigation; nevertheless, the initial findings are very promising. Using visual covert attention for BCI might have several advantages compared to other paradigms used such as motor imagery. First, concerns about EMG artifacts driving the BCI are less. Nevertheless, care has to be taken that eye-movements and blinks do not confound the signals of interest. Second, when constructing a BCI controlled cursor (“brain mouse”) it is quite a natural setup to have the cursor (or the background of a scene) move in the attended direction. Furthermore, the modulation of alpha activity by directing attention vertically allows for constructing a two-dimensional cursor control, something that only has been achieved in a few non-invasive BCI studies (Wolpaw and Mcfarland, 2004; Li et al., 2010). Future research will reveal how robust covert spatial attention is as a control signal for BCI.
It might as well be advantageous to use attention modulations in other modalities as a control signal for BCI. Shifting attention from the visual to the auditory modality is known to produce robust modulations of posterior alpha activity (Fu et al., 2001). Surprisingly, hemispheric alpha lateralization can also be modulated by directing the auditory attention to sounds presented to the left or the right (Kerlin et al., 2010). Finally, the alpha (and beta) activity produced in sensorimotor cortices is strongly modulated by attention. Shifting attention from the left to the right hand provides a very reliable modulation in hemispheric lateralization of the sensorimotor alpha activity (Jones et al., 2010; Van Ede et al., 2010, 2011; Haegens et al., 2011). In sum, various attention manipulations resulting in the modulation of oscillatory activity observed in cognitive studies hold the promise of providing robust control signals for BCI. In particular the shift in attention within and between sensory modalities could be used to increase the dimensionality of BCI control.
Electrophysiological Signals Reflecting the State of the Brain
So far we have addressed how cognitive neuroscience can inform BCI, but what might BCI teach us about the physiological underpinnings of human cognition? Recently there has been a growing interest in understanding the resting state of the brain and how it might impact neuronal processing (Silvanto et al., 2008). This is amongst others motivated by the observation that spontaneous fluctuations in brain activity often are much stronger in magnitude compared to stimulus induced activation. Based on this finding, the fMRI community has been investing the so-called resting state networks (Raichle and Snyder, 2007). The goal has been to uncover regions that are functionally coupled in terms of correlated BOLD fluctuations. Pre-stimulus fluctuations in the BOLD signals are not only informative for understanding functional coupling, they have also been shown to correlate with cognitive processing (Otten et al., 2002; Hesselmann et al., 2008,b; Sadaghiani et al., 2009; Park and Rugg, 2010). The role of spontaneous brain activity in cognitive processing has also been investigated using EEG and MEG recordings in humans. This is mainly done by quantifying how pre-stimulus activity correlates with subsequent perception (Ergenoglu et al., 2004; Linkenkaer-Hansen et al., 2004; Hanslmayr et al., 2005; Van Dijk et al., 2008; Guderian et al., 2009; Musso et al., 2010; Zhang and Ding, 2010). In line with the functional inhibition hypothesis of alpha, it has been demonstrated that strong pre-stimulus alpha activity has a negative impact on perception (Ergenoglu et al., 2004; Van Dijk et al., 2008; Mathewson et al., 2009). Furthermore, somatosensory stimulus detection is modulated by alpha activity produced around the central sulcus (Linkenkaer-Hansen et al., 2004; Zhang and Ding, 2010). These findings do not only hold for hard-to-detect stimuli. Pre-stimulus oscillatory activity was recently shown to affect the performance in Go–noGo task (Mazaheri et al., 2009). Subjects were presented with digits (“1” to “9”). They had to press a button for each digit, but withhold the response when digit “5” was presented. Erroneous button presses were preceded by higher pre-stimulus alpha activity. The higher alpha activity might slow down the processing of the visual stimuli, thus preventing that a stop-signal for the motor system is generated in time.
These studies show that the state of the brain has a strong impact on cognitive processing. Given that the brain-state modulates perception, it implies that how we perceive and remember the world around us is dependent on fluctuations in ongoing brain activity. Thus by presenting stimuli depending on the brain state it should be possible to manipulate perception and memory.
Brain-State Dependent Stimulation
The neuronal substrate of the spontaneous fluctuations of the brain state is reflected in electrophysiological activity that can be measured with EEG, MEG, or intracranial recordings. This raises the possibility of manipulating cognitive processing by taking the online brain activity into account. One way of doing this is by changing stimuli presented to the subject depending on an online characterization of the brain activity (Hartmann et al., 2011). This was done in a recent study in which single neuron firing was recorded from human patients with electrodes implanted in medial temporal lobe regions (Cerf et al., 2010), as shown in Figure 5. The electrodes were implanted in order to identify epileptic foci prior to surgery; however, the subjects consented to participate in a cognitive study as well. What Cerf and colleagues first did was to identify neurons that fired in response to a particular picture of a famous person (e.g., Marilyn Monroe or Josh Brolin). These pictures where then superimposed but a subject was for instance asked to attend to Monroe. If the Monroe cell increased its firing (and the Brolin cell decreased), the contrast of the Monroe picture was increased by the online setup at the expense of the Brolin picture. The key finding was, that by using this manipulation, subjects were able to intentionally manipulate the visual stimuli and thus their own percept. The findings suggest that stimulus specific neurons in the human MTL are under top-down control and that their firing can be modulated by attention.
Figure 5. Online control of the firing of temporal lobe neurons. (A) Electrodes allowing for detecting single unit firing were implanted in the temporal lobe of patients prior to surgery. The firing of multiple neurons was analyzed online. The output of the analysis controlled a visual display. (B) In the visual display photos of famous persons (e.g., Marilyn Monroe and Josh Brolin) were superimposed. Subjects were asked to attend to one of the persons (e.g., Monroe). If the neurons coding for Monroe (C) fired stronger, the Monroe face was made more visible by the control loop. The example in (B) shows eight trials in which the subject was able to make the Monroe picture visible by voluntarily controlling the firing of the temporal lobe neurons. (D) The visibility of the target photo averaged over several trials. Reproduced with permission from Cerf et al. (2010).
Brain-state dependent stimulation has been used to investigate the function of the phase of ongoing oscillations. Gho and Varela (1988) studied the role of alpha phase in the context of the “double flash” illusion using an EEG setup. When two visual stimuli are flashed with a slight delay, they can either be perceived as two flashes or as a moving light. In the EEG study the stimuli were delivered depending on the phase of the ongoing alpha activity. The findings demonstrated that the perception (two flashes versus a single moving light) could be somewhat manipulated depending on the alpha phase. In a more recent study the relationship between alpha phase and evoked responses was addressed (Kruglikov and Schiff, 2003). The phase of the ongoing alpha activity was characterized online and the auditory stimuli were then presented as a function of the phase. This allowed the authors to demonstrate that the magnitude of the P50 was influenced by alpha phase. These two studies suggest that human perception is not continuous, but rather discrete and clocked by ongoing oscillations (Vanrullen and Koch, 2003).
Spontaneous oscillations are not only observed at rest but also during sleep. Oscillations during sleep have been hypothesized to be involved in offline processing of information acquired during the day (Diekelmann and Born, 2010). For instance it has been proposed that slow wave oscillations reflect the reactivation of recently acquired information. This reactivation might support memory consolidation by recoding the neuronal representations in neocortical areas (Rasch and Born, 2007). To test this notion, a recent study investigated the consequences of perturbing slow wave activity. During sleep, EEG was used to record the ongoing brain activity. When slow wave activity increased above a certain threshold, a device started to play sounds. These sounds served to reduce the slow wave activity. After sleep the brain activation following the recall of memories recently acquired was tested. The study demonstrated that MTL activation probed by the memory recall was reduced following the slow wave intervention (Van Der Werf et al., 2009). Using this type of BSDS, it was concluded that slow wave sleep is causally related to memory formation.
This set of studies illustrates a few examples in which BSDS has been applied in order to gain new insight into the functional role of ongoing brain activity. Given the growing interest in the role of brain states for cognition, BSDS is likely to become a more frequently used tool in cognitive neuroscience.
Augmenting Human Behavior Using BCI
There are several approaches for using BCI-type setups to augment human performance (Parra et al., 2008). For instance, it is possible to online assess mental workload as reflected by neurophysiological activity using EEG. Using this measure of workload, the task demands can be adjusted in order to improve performance. This possibility has been investigated in the context of a virtual reality driving environment (Kohlmorgen et al., 2007). Another possibility is to detect errors committed in demanding response tasks using event-related negativity (ERN). The ERN is a frontal negative event-related potential following errors after button presses. In a visual task subjects were asked to discriminate between two visual stimuli by pressing one of two buttons. Since mistakes were followed by an increased ERN, this allowed for correcting erroneous responses and thus increasing performance by more than 20% (Parra et al., 2003).
Posterior alpha activity might also be used in designs to augment human behavior. The finding that visual perception is reduced with strong posterior alpha activity can be used in an online setup where difficult-to-detect stimuli are presented only when the alpha activity is low. This should increase the visual detection rate as compared to when the stimuli are presented without taking the brain state into account. Another possibility is to make use of the finding that memory rehearsal improves encoding when the posterior alpha activity is strong (Khader et al., 2010; Meeuwissen et al., 2010). This allows for predicting to what extent a given memory later can be remembered: If the online alpha activity suggests that an item might be forgotten, it should be presented again. In summary there are several ideas for how BSDS can help augment human behavior; however, many of these ideas need to be tested. The possibility of augmenting memory encoding could have many practical applications. One example would be computer-assisted learning of foreign vocabulary. If the online system could predict how well new word pairs are encoded, the system could be used to optimize learning. When developing these applications a valuable side product might be new insight gained in cognitive neuroscience. For instance, when developing online systems for augmenting memory encoding, we are bound to gain new insight into the brain activity necessary for successful memory formation.
Technical Aspects: Using Multisensor MEG and EEG Systems for BCI
How many sensors are required in a BCI setup and when is it sensible to use MEG rather than EEG? The answers to these questions are of course dependent on the particular application. For portable BCIs to be used in practical settings it is important to have relatively few sensors and a lightweight portable system. However, when BCI is applied as a tool for cognitive neuroscience, it is important to have as much information as possible available. For such applications, a multisensor system will allow for better separation of the various sources being active in different regions. While it has long been appreciated that EEG and MEG signals are a direct consequence of neuronal activity they still come with significant observability constraints (Lopes Da Silva, 1991). EEG/MEG recorded at the scalp will be dominated by a number of components that may either be artifactual or reflect synchronized processing in the underlying neural substrate. This processing may or may not be related to the task at hand. For instance, in the case of covert attention paradigms it is possible that two components are active at the same time: first, a more global alpha component, which is strongly modulated by the vigilance state (akin to the classical “idling” rhythm) and second, a more focal/discrete alpha component, which is modulated according to changes in covert attention. At the scalp, these two components will be mixed together. Hence the main objective when dealing with such macroscopic signals is to first separate the components and then examine their characteristics. MEG will be the most optimal tool given that the spatial smearing at the sensor level is much less compared to EEG (Hamalainen et al., 1993). Due to its biophysical properties, chiefly the fact that magnetic fields do not get distorted by inhomogeneous electrical properties of the volume conductor, MEG is better suited to separate and localize the different components. The isolation of relevant components increases specificity and enhances their signal-to-noise ratio whereas their localization is crucial in creating a link to invasive data and hence to the underlying physiology.
Performing BCI experiments with multisensor EEG and MEG systems is highly demanding given that large amounts of data need to be streamed and processed online. Additionally, one needs to be able to suppress environmental noise in real-time and have access to all the information required to perform online source reconstruction and artifact removal. There are now several initiatives working toward a standardized approached for data streaming and analysis including BCI2000, MatRiver, OpenVIBE (Schalk et al., 2004; Delorme et al., 2010; Renard et al., 2010; Hartmann et al., 2011). As an example for how such a system can work we will here describe an implementation developed using the FieldTrip toolbox (Oostenveld et al., 2011).
The foremost challenge for an online system is to be able to achieve fast streaming of large amounts of data (up to ∼300 sensors at 1 kHz sampling rates). Here we note that not only do MEG and EEG data channels need to be recorded but also EMG and EOG channels. In the case of MEG, reference channels allowing for real-time cancelation of environmental noise are often also collected. Secondly, one needs to access the head position with respect to the MEG sensors. This is necessary for source reconstruction and/or other data projections yielding higher signal-to-noise ratios and specificity. Online monitoring of the participant’s head position is highly desirable in order to check that the decoding in a BCI setup is not influenced by head movement. The process of localizing head positions is dependent on dipole fitting algorithms operating on the fields generated by coils attached to the subject’s head.
The details pertaining to the data writing process and the memory allocated to the online interface are specific to each system and different for systems of different manufacturers. In this light, the best option appears to be to devise a system which does not depend on the specific details of the interface provided by the manufacturers. This is precisely what has been developed at the in the FieldTrip toolbox, namely an access scheme to real-time data, which is platform and hardware independent. This access scheme is part of the open source FieldTrip software package and can be used for real-time streaming of MEG, EEG, and fMRI signals (http://fieldtrip.fcdonders.nl/development/realtime). The implementation of this real-time access scheme is illustrated schematically in Figure 6.
Figure 6. A schematic illustration of how real-time data access can be implemented. This example depicts the approached developed within the FieldTrip package working in conjunction with an MEG system. The MEG system acquisition software (Acq) writes data to a limited capacity memory segment set up on the acquisition computer (shared memory segment), which is the manufacturer-specific online interface. An external, manufacturer-independent buffer (FieldTrip buffer) is set up on the same or a different computer. An agent (acq2ft) continually monitors the shared memory segment for new data and copies them to the external buffer as soon as these become available. Once the data is in the external buffer they can be accessed and analyzed in Matlab for the purposes of the BCI. Crucially, the rapid copying of new data as soon as they become available to an external buffer deems the reverse-engineering of the MEG system data writing process superfluous.
The key motivation behind the real-time access scheme is to make the analysis software as simple and generic as possible, while at the same providing access to crucial information such as sensor locations and other metadata. This is achieved by involving an intermediate in-memory buffer (the “FieldTrip buffer”) to which data is streamed by manufacturer-specific software that runs on the acquisition machine. The buffer data can concurrently be read from by one or multiple analysis agents (e.g., Matlab scripts) on the same or a different computer. This is implemented by a software agent (acq2ft) which constantly monitors the shared memory segment to which the acquisition software (Acq) writes small blocks of data. The very first of these blocks contains a link to the setup file corresponding to the current recording, which is read and decoded by acq2ft and written to the header part of the buffer. Further blocks arrive in rapid succession and wrap around after some time. Since acq2ft copies the data to the buffer as soon as they arrive, the analysis agents do not have to worry about peculiarities of the timing: They can always access the metadata as well as a rather large window of continuously sampled data up to the latest block of samples. On top of that, acq2ft also decodes trigger channels and provides these to the analysis agents in the form of discrete markers. This facilitates the development of synchronous BCIs that are locked to particular trigger events.
In short, there are now useful developments simplifying the design and implementation of the analysis software for BCI and BSDS. These implementations help to abstract from the peculiarities of the hardware and will facilitate the development of standardized platforms.
Conclusion
There are several new exciting developments of BCIs and BSDS. Some of these developments are inspired by new insights from cognitive neuroscience. In particular it is becoming clear that oscillatory activity in the alpha band plays an important role in the allocation of neurocomputational resources. This is for instance evident in tasks where covert attention is modulated within or between modalities. This insight has opened up several new avenues for control signals for BCI, many of which are now actively being explored. BCI also allows for providing new insight into cognitive neuroscience. For instance BCI can be used as a tool to study the functional role of ongoing brain activity which reflects the state of the brain. This can be done using BSDS in which stimuli are introduced as a function of the measured oscillatory brain activity. Beyond providing new insight into the role of the oscillatory brain activity the approach also holds the promise of developing online systems for augmenting human performance.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
The authors gratefully acknowledge the support of the BrainGain Smart Mix Programme of the Netherlands Ministry of Economic Affairs and the Netherlands Ministry of Education, Culture and Science the Netherlands Initiative Brain and Cognition, a part of the Organization for Scientific Research (NWO) under grant number 056-14-011, and The Netherlands Organization for Scientific Research (NWO): Innovational Research Incentive Schemes, VICI grant number: 453-09-002.
References
Allison, B. Z., Wolpaw, E. W., and Wolpaw, J. R. (2007). Brain-computer interface systems: progress and prospects. Expert Rev. Med. Devices 4, 463–474.
Bahramisharif, A., Gerven, M. V., Okazaki, Y., Heskes, T., and Jensen, O. (2010a). Posterior alpha activity modulated by covert attention used as a control signal for BCI. Front Neurosci Conference Abstract: Biomag 2010 - 17th International Conference on Biomagnetism, doi: 10.3389/conf.fnins.2010.06.00258
Bahramisharif, A., Van Gerven, M., Heskes, T., and Jensen, O. (2010b). Covert attention allows for continuous control of brain-computer interfaces. Eur. J. Neurosci. 31, 1501–1508.
Bahramisharif, A., Heskes, T., Jensen, O., and Van Gerven, M. A. (2011). Lateralized responses during covert attention are modulated by target eccentricity. Neurosci. Lett. 491, 35–39.
Berger, H. (1929). Über das Elektroenkephalogramm des Menschen. Arch. Psychiatr. Nervenkr. 87, 527–570.
Blankertz, B., Tangermann, M., Vidaurre, C., Fazli, S., Sannelli, C., Haufe, S., Maeder, C., Ramsey, L., Sturm, I., Curio, G., and Muller, K. R. (2010). The Berlin brain-computer interface: non-medical uses of BCI technology. Front. Neurosci. 4:198. doi: 10.3389/fnins.2010.00198
Boord, P., Craig, A., Tran, Y., and Nguyen, H. (2010). Discrimination of left and right leg motor imagery for brain-computer interfaces. Med. Biol. Eng. Comput. 48, 343–350.
Brovelli, A., Ding, M., Ledberg, A., Chen, Y., Nakamura, R., and Bressler, S. L. (2004). Beta oscillations in a large-scale sensorimotor cortical network: directional influences revealed by Granger causality. Proc. Natl. Acad. Sci. U.S.A. 101, 9849–9854.
Canolty, R. T., and Knight, R. T. (2010). The functional role of cross-frequency coupling. Trends Cogn. Sci. (Regul. Ed.) 14, 506–515.
Cerf, M., Thiruvengadam, N., Mormann, F., Kraskov, A., Quiroga, R. Q., Koch, C., and Fried, I. (2010). On-line, voluntary control of human temporal lobe neurons. Nature 467, 1104–1108.
Cosmelli, D., Lopez, V., Lachaux, J. P., Lopez-Calderon, J., Renault, B., Martinerie, J., and Aboitiz, F. (2011). Shifting visual attention away from fixation is specifically associated with alpha band activity over ipsilateral parietal regions. Psychophysiology 48, 312–322.
Daly, J. J., and Wolpaw, J. R. (2008). Brain-computer interfaces in neurological rehabilitation. Lancet Neurol. 7, 1032–1043.
Delorme, A., Kothe, C., Vankov, A., Bigdely-Shamlo, N., Oostenveldt, R., Zander, T., and Makeig, S. (2010). “MATLAB-based tools for BCI research,” in (B+H)CI: Brain-Computer Interfaces Applying our Minds to Human-Computer Interaction, eds D. Tan and A. Nijholt (London: Springer-Verlag Limited), 241–258.
Donner, T. H., Siegel, M., Fries, P., and Engel, A. K. (2009). Buildup of choice-predictive activity in human motor cortex during perceptual decision making. Curr. Biol. 19, 1581–1585.
Duncan, K. K., Hadjipapas, A., Li, S., Kourtzi, Z., Bagshaw, A., and Barnes, G. (2010). Identifying spatially overlapping local cortical networks with MEG. Hum. Brain Mapp. 31, 1003–1016.
Edwards, E., Nagarajan, S. S., Dalal, S. S., Canolty, R. T., Kirsch, H. E., Barbaro, N. M., and Knight, R. T. (2010). Spatiotemporal imaging of cortical activation during verb generation and picture naming. Neuroimage 50, 291–301.
Engel, A. K., and Fries, P. (2010). Beta-band oscillations-signalling the status quo? Curr. Opin. Neurobiol. 20, 156–165.
Ergenoglu, T., Demiralp, T., Bayraktaroglu, Z., Ergen, M., Beydagi, H., and Uresin, Y. (2004). Alpha rhythm of the EEG modulates visual detection performance in humans. Brain Res. Cogn. Brain Res. 20, 376–383.
Freunberger, R., Werkle-Bergner, M., Griesmayr, B., Lindenberger, U., and Klimesch, W. (2011). Brain oscillatory correlates of working memory constraints. Brain Res. 1375, 93–102.
Fu, K. M., Foxe, J. J., Murray, M. M., Higgins, B. A., Javitt, D. C., and Schroeder, C. E. (2001). Attention-dependent suppression of distracter visual input can be cross-modally cued as indexed by anticipatory parieto-occipital alpha-band oscillations. Brain Res. Cogn. Brain Res. 12, 145–152.
Gaona, C. M., Sharma, M., Freudenburg, Z. V., Breshears, J. D., Bundy, D. T., Roland, J., Barbour, D. L., Schalk, G., and Leuthardt, E. C. (2011). Nonuniform high-gamma (60-500 Hz) power changes dissociate cognitive task and anatomy in human cortex. J. Neurosci. 31, 2091–2100.
Gho, M., and Varela, F. J. (1988). A quantitative assessment of the dependency of the visual temporal frame upon the cortical rhythm. J. Physiol. 83, 95–101.
Gould, I. C., Rushworth, M. F., and Nobre, A. C. (2011). Indexing the graded allocation of visuospatial attention using anticipatory alpha oscillations. J. Neurophysiol. 105, 1318–1326.
Guderian, S., Schott, B. H., Richardson-Klavehn, A., and Duzel, E. (2009). Medial temporal theta state before an event predicts episodic encoding success in humans. Proc. Natl. Acad. Sci. U.S.A. 106, 5365–5370.
Hadjipapas, A., Adjamian, P., Swettenham, J. B., Holliday, I. E., and Barnes, G. R. (2007). Stimuli of varying spatial scale induce gamma activity with distinct temporal characteristics in human visual cortex. Neuroimage 35, 518–530.
Haegens, S., Handel, B. F., and Jensen, O. (2011). Top-down controlled alpha band activity in somatosensory areas determines behavioral performance in a discrimination task. J. Neurosci. 31, 5197–5204.
Haegens, S., Osipova, D., Oostenveld, R., and Jensen, O. (2010). Somatosensory working memory performance in humans depends on both engagement and disengagement of regions in a distributed network. Hum. Brain Mapp. 31, 26–35.
Hamalainen, M., Hari, R., Ilmoniemi, R. J., Knuutila, J., and Lounasmaa, O. V. (1993). Magnetoencephalography – theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev. Mod. Phys. 65, 413–497.
Hanslmayr, S., Klimesch, W., Sauseng, P., Gruber, W., Doppelmayr, M., Freunberger, R., and Pecherstorfer, T. (2005). Visual discrimination performance is related to decreased alpha amplitude but increased phase locking. Neurosci. Lett. 375, 64–68.
Hari, R., and Salmelin, R. (1997). Human cortical oscillations: a neuromagnetic view through the skull. Trends Neurosci. 20, 44–49.
Hartmann, T., Schulz, H., and Weisz, N. (2011). Probing of brain states in real-time: introducing the ConSole environment. Front. Psychol. 2:36. doi: 10.3389/fpsyg.2011.00036
Hesselmann, G., Kell, C. A., Eger, E., and Kleinschmidt, A. (2008a). Spontaneous local variations in ongoing neural activity bias perceptual decisions. Proc. Natl. Acad. Sci. U.S.A. 105, 10984–10989.
Hesselmann, G., Kell, C. A., and Kleinschmidt, A. (2008b). Ongoing activity fluctuations in hMT+ bias the perception of coherent visual motion. J. Neurosci. 28, 14481–14485.
Hoogenboom, N., Schoffelen, J. M., Oostenveld, R., Parkes, L. M., and Fries, P. (2006). Localizing human visual gamma-band activity in frequency, time and space. Neuroimage 29, 764–773.
Jensen, O. (2006). Maintenance of multiple working memory items by temporal segmentation. Neuroscience 139, 237–249.
Jensen, O., Gelfand, J., Kounios, J., and Lisman, J. E. (2002). Oscillations in the alpha band (9-12 Hz) increase with memory load during retention in a short-term memory task. Cereb. Cortex 12, 877–882.
Jensen, O., Goel, P., Kopell, N., Pohja, M., Hari, R., and Ermentrout, B. (2005). On the human sensorimotor-cortex beta rhythm: sources and modeling. Neuroimage 26, 347–355.
Jensen, O., Kaiser, J., and Lachaux, J. P. (2007). Human gamma-frequency oscillations associated with attention and memory. Trends Neurosci. 30, 317–324.
Jensen, O., and Mazaheri, A. (2010). Shaping functional architecture by oscillatory alpha activity: gating by inhibition. Front. Hum. Neurosci. 4:186. doi: 10.3389/fnhum.2010.00186
Jensen, O., and Vanni, S. (2002). A new method to identify multiple sources of oscillatory activity from magnetoencephalographic data. Neuroimage 15, 568–574.
Jerbi, K., Ossandon, T., Hamame, C. M., Senova, S., Dalal, S. S., Jung, J., Minotti, L., Bertrand, O., Berthoz, A., Kahane, P., and Lachaux, J. P. (2009). Task-related gamma-band dynamics from an intracerebral perspective: review and implications for surface EEG and MEG. Hum. Brain Mapp. 30, 1758–1771.
Jokisch, D., and Jensen, O. (2007). Modulation of gamma and alpha activity during a working memory task engaging the dorsal or ventral stream. J. Neurosci. 27, 3244–3251.
Jones, S. R., Kerr, C. E., Wan, Q., Pritchett, D. L., Hamalainen, M., and Moore, C. I. (2010). Cued spatial attention drives functionally relevant modulation of the mu rhythm in primary somatosensory cortex. J. Neurosci. 30, 13760–13765.
Kelly, S. P., Lalor, E. C., Reilly, R. B., and Foxe, J. J. (2005). Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 172–178.
Kerlin, J. R., Shahin, A. J., and Miller, L. M. (2010). Attentional gain control of ongoing cortical speech representations in a “cocktail party”. J. Neurosci. 30, 620–628.
Khader, P. H., Jost, K., Ranganath, C., and Rosler, F. (2010). Theta and alpha oscillations during working-memory maintenance predict successful long-term memory encoding. Neurosci. Lett. 468, 339–343.
Klimesch, W., Sauseng, P., and Hanslmayr, S. (2006). EEG alpha oscillations: the inhibition-timing hypothesis. Brain Res. Rev. 53, 63–88.
Kohlmorgen, J., Dornhege, G., Braun, M., Blankertz, B., MÜller, K. R., Curio, G., Hagemann, K., Bruns, A., Schrauf, M., and Kincses, W. (eds). (2007). Improving Human Performance in a Real Operating Environment Through Real-time Mental Workload Detection. Cambridge: MIT press.
Kruglikov, S. Y., and Schiff, S. J. (2003). Interplay of electroencephalogram phase and auditory-evoked neural activity. J. Neurosci. 23, 10122–10127.
Kubanek, J., Miller, K. J., Ojemann, J. G., Wolpaw, J. R., and Schalk, G. (2009). Decoding flexion of individual fingers using electrocorticographic signals in humans. J. Neural Eng. 6, 066001.
Kubler, A., Neumann, N., Wilhelm, B., Hinterberger, T., and Birbaumer, N. (2004). Brain-computer predictability of brain-computer communication. J. Psychophysiol. 18, 121–129.
Lachaux, J. P., Jerbi, K., Bertrand, O., Minotti, L., Hoffmann, D., Schoendorff, B., and Kahane, P. (2007). A blueprint for real-time functional mapping via human intracranial recordings. PLoS ONE 2, e1094. doi: 10.1371/journal.pone.0001094
Lebedev, M. A., and Nicolelis, M. A. (2006). Brain-machine interfaces: past, present and future. Trends Neurosci. 29, 536–546.
Li, Y., Long, J., Yu, T., Yu, Z., Wang, C., Zhang, H., and Guan, C. (2010). An EEG-based BCI system for 2-D cursor control by combining Mu/Beta rhythm and P300 potential. IEEE Trans. Biomed. Eng. 57, 2495–2505.
Linkenkaer-Hansen, K., Nikulin, V. V., Palva, S., Ilmoniemi, R. J., and Palva, J. M. (2004). Prestimulus oscillations enhance psychophysical performance in humans. J. Neurosci. 24, 10186–10190.
Lopes Da Silva, F. (1991). Neural mechanisms underlying brain waves: from neural membranes to networks. Electroencephalogr. Clin. Neurophysiol. 79, 81–93.
Lou, B., Hong, B., and Gao, S. (2008). Task-irrelevant alpha component analysis in motor imagery based brain computer interface. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2008, 1021–1024.
Mathewson, K. E., Gratton, G., Fabiani, M., Beck, D. M., and Ro, T. (2009). To see or not to see: prestimulus alpha phase predicts visual awareness. J. Neurosci. 29, 2725–2732.
Mazaheri, A., Nieuwenhuis, I. L., Van Dijk, H., and Jensen, O. (2009). Prestimulus alpha and mu activity predicts failure to inhibit motor responses. Hum. Brain Mapp. 30, 1791–1800.
Mcfarland, D. J., and Wolpaw, J. R. (2005). Sensorimotor rhythm-based brain-computer interface (BCI): feature selection by regression improves performance. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 372–379.
Meeuwissen, E. B., Takashima, A., Fernandez, G., and Jensen, O. (2010). Increase in posterior alpha activity during rehearsal predicts successful long-term memory formation of word sequences. Hum. Brain Mapp. doi: 10.1002/hbm.21167. [Epub ahead of print].
Miller, K. J., Schalk, G., Fetz, E. E., Den Nijs, M., Ojemann, J. G., and Rao, R. P. (2010). Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc. Natl. Acad. Sci. U.S.A. 107, 4430–4435.
Musso, F., Brinkmeyer, J., Mobascher, A., Warbrick, T., and Winterer, G. (2010). Spontaneous brain activity and EEG microstates. A novel EEG/fMRI analysis approach to explore resting-state networks. Neuroimage 52, 1149–1161.
Muthukumaraswamy, S. D., and Singh, K. D. (2008). Spatiotemporal frequency tuning of BOLD and gamma band MEG responses compared in primary visual cortex. Neuroimage 40, 1552–1560.
Nam, C. S., Jeon, Y., Kim, Y. J., Lee, I., and Park, K. (2011). Movement imagery-related lateralization of event-related (de)synchronization (ERD/ERS): motor-imagery duration effects. Clin. Neurophysiol. 122, 567–577.
Oostenveld, R., Fries, P., Maris, E., and Schoffelen, J. M. (2011). FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011, 156869.
Otten, L. J., Henson, R. N., and Rugg, M. D. (2002). State-related and item-related neural correlates of successful memory encoding. Nat. Neurosci. 5, 1339–1344.
Palva, S., and Palva, J. M. (2007). New vistas for alpha-frequency band oscillations. Trends Neurosci. 30, 150–158.
Park, H., and Rugg, M. D. (2010). Prestimulus hippocampal activity predicts later recollection. Hippocampus 20, 24–28.
Parra, L. C., Christoforou, C., Gerson, A. D., Dyrholm, M., Luo, A., Wagner, M., Philiastides, M. G., and Sajda, P. (2008). Spatio-temporal linear decoding of brain state: application to performance augmentation in high-throughput tasks. IEEE Signal Process. Mag. 25, 107–115.
Parra, L. C., Spence, C. D., Gerson, A. D., and Sajda, P. (2003). Response error correction – a demonstration of improved human-machine performance using real-time EEG monitoring. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 173–177.
Pfurtscheller, G., and Neuper, C. (2006). Future prospects of ERD/ERS in the context of brain-computer interface (BCI) developments. Prog. Brain Res. 159, 433–437.
Pfurtscheller, G., Stancak, A. Jr., and Neuper, C. (1996). Event-related synchronization (ERS) in the alpha band--an electrophysiological correlate of cortical idling: a review. Int. J. Psychophysiol. 24, 39–46.
Raichle, M. E., and Snyder, A. Z. (2007). A default mode of brain function: a brief history of an evolving idea. Neuroimage 37, 1083–1090; discussion 1097–1089.
Ramsey, N. F., Van De Heuvel, M. P., Kho, K. H., and Leijten, F. S. (2006). Towards human BCI applications based on cognitive brain systems: an investigation of neural signals recorded from the dorsolateral prefrontal cortex. IEEE Trans. Neural Syst. Rehabil. Eng. 14, 214–217.
Rasch, B., and Born, J. (2007). Maintaining memories by reactivation. Curr. Opin. Neurobiol. 17, 698–703.
Renard, Y., Lotte, F., Gibert, G., Congedo, M., Maby, E., Delannoy, V., Bertrand, O., and Lecuyer, A. (2010). OpenViBE: an open-source software platform to design, test, and use brain-computer interfaces in real and virtual environments. Presence (Camb.) 19, 35–53.
Rihs, T. A., Michel, C. M., and Thut, G. (2007). Mechanisms of selective inhibition in visual spatial attention are indexed by alpha-band EEG synchronization. Eur. J. Neurosci. 25, 603–610.
Romei, V., Gross, J., and Thut, G. (2010). On the role of prestimulus alpha rhythms over occipito-parietal areas in visual input regulation: correlation or causation? J. Neurosci. 30, 8692–8697.
Sadaghiani, S., Hesselmann, G., and Kleinschmidt, A. (2009). Distributed and antagonistic contributions of ongoing activity fluctuations to auditory stimulus detection. J. Neurosci. 29, 13410–13417.
Salinas, E., and Sejnowski, T. J. (2001). Correlated neuronal activity and the flow of neural information. Nat. Rev. Neurosci. 2, 539–550.
Salmelin, R., and Hari, R. (1994). Spatiotemporal characteristics of sensorimotor neuromagnetic rhythms related to thumb movement. Neuroscience 60, 537–550.
Sauseng, P., Klimesch, W., Heise, K. F., Gruber, W. R., Holz, E., Karim, A. A., Glennon, M., Gerloff, C., Birbaumer, N., and Hummel, F. C. (2009). Brain oscillatory substrates of visual short-term memory capacity. Curr. Biol. 19, 1846–1852.
Schalk, G., Mcfarland, D. J., Hinterberger, T., Birbaumer, N., and Wolpaw, J. R. (2004). BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51, 1034–1043.
Scheeringa, R., Petersson, K. M., Oostenveld, R., Norris, D. G., Hagoort, P., and Bastiaansen, M. C. (2009). Trial-by-trial coupling between EEG and BOLD identifies networks related to alpha and theta EEG power increases during working memory maintenance. Neuroimage 44, 1224–1238.
Siegel, M., Engel, A. K., and Donner, T. H. (2011). Cortical network dynamics of perceptual decision-making in the human brain. Front. Hum. Neurosci. 5:21. doi: 10.3389/fnhum.2011.00021
Silvanto, J., Muggleton, N., and Walsh, V. (2008). State-dependency in brain stimulation studies of perception and cognition. Trends Cogn. Sci. (Regul. Ed.) 12, 447–454.
Tallon-Baudry, C., and Bertrand, O. (1999). Oscillatory gamma activity in humans and its role in object representation. Trends Cogn. Sci. (Regul. Ed.) 3, 151–162.
Tallon-Baudry, C., Mandon, S., Freiwald, W. A., and Kreiter, A. K. (2004). Oscillatory synchrony in the monkey temporal lobe correlates with performance in a visual short-term memory task. Cereb. Cortex 14, 713–720.
Thut, G., and Miniussi, C. (2009). New insights into rhythmic brain activity from TMS-EEG studies. Trends Cogn. Sci. (Regul. Ed.). 13, 182–189.
Thut, G., Nietzel, A., Brandt, S. A., and Pascual-Leone, A. (2006). Alpha-band electroencephalographic activity over occipital cortex indexes visuospatial attention bias and predicts visual target detection. J. Neurosci. 26, 9494–9502.
Tuladhar, A. M., Ter Huurne, N., Schoffelen, J. M., Maris, E., Oostenveld, R., and Jensen, O. (2007). Parieto-occipital sources account for the increase in alpha activity with working memory load. Hum. Brain Mapp. 28, 785–792.
Van Der Werf, J., Jensen, O., Fries, P., and Medendorp, W. P. (2008). Gamma-band activity in human posterior parietal cortex encodes the motor goal during delayed prosaccades and antisaccades. J. Neurosci. 28, 8397–8405.
Van Der Werf, Y. D., Altena, E., Schoonheim, M. M., Sanz-Arigita, E. J., Vis, J. C., De Rijke, W., and Van Someren, E. J. (2009). Sleep benefits subsequent hippocampal functioning. Nat. Neurosci. 12, 122–123.
Van Dijk, H., Schoffelen, J. M., Oostenveld, R., and Jensen, O. (2008). Prestimulus oscillatory activity in the alpha band predicts visual discrimination ability. J. Neurosci. 28, 1816–1823.
Van Ede, F., De Lange, F., Jensen, O., and Maris, E. (2011). Orienting attention to an upcoming tactile event involves a spatially and temporally specific modulation of sensorimotor alpha- and beta-band oscillations. J. Neurosci. 31, 2016–2024.
Van Ede, F., Jensen, O., and Maris, E. (2010). Tactile expectation modulates pre-stimulus beta-band oscillations in human sensorimotor cortex. Neuroimage 51, 867–876.
Van Gerven, M., Bahramisharif, A., Heskes, T., and Jensen, O. (2009). Selecting features for BCI control based on a covert spatial attention paradigm. Neural Netw. 22, 1271–1277.
Van Gerven, M., and Jensen, O. (2009). Attention modulations of posterior alpha as a control signal for two-dimensional brain-computer interfaces. J. Neurosci. Methods 179, 78–84.
Vanrullen, R., and Koch, C. (2003). Is perception discrete or continuous? Trends Cogn. Sci. (Regul. Ed.) 7, 207–213.
Vansteensel, M. J., Hermes, D., Aarnoutse, E. J., Bleichner, M. G., Schalk, G., Van Rijen, P. C., Leijten, F. S., and Ramsey, N. F. (2010). Brain-computer interfacing based on cognitive control. Ann. Neurol. 67, 809–816.
Vidal, J. R., Ossandon, T., Jerbi, K., Dalal, S. S., Minotti, L., Ryvlin, P., Kahane, P., and Lachaux, J. P. (2010). Category-specific visual responses: an intracranial study comparing gamma, beta, alpha, and ERP response selectivity. Front. Hum. Neurosci. 4:195. doi: 10.3389/fnhum.2010.00195
Wolpaw, J. R., and Mcfarland, D. J. (2004). Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl. Acad. Sci. U.S.A. 101, 17849–17854.
Worden, M. S., Foxe, J. J., Wang, N., and Simpson, G. V. (2000). Anticipatory biasing of visuospatial attention indexed by retinotopically specific alpha-band electroencephalography increases over occipital cortex. J. Neurosci. 20, RC63.
Zander, T. O., and Kothe, C. (2011). Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general. J. Neural Eng. 8, 025005.
Keywords: magnetoencephalography, electroencephalography, alpha, gamma, attention, man machine interface
Citation: Jensen O, Bahramisharif A, Oostenveld R, Klanke S, Hadjipapas A, Okazaki YO and van Gerven MAJ (2011) Using brain–computer interfaces and brain-state dependent stimulation as tools in cognitive neuroscience. Front. Psychology 2:100. doi: 10.3389/fpsyg.2011.00100
Received: 16 March 2011;
Accepted: 06 May 2011;
Published online: 27 May 2011.
Edited by:
Gregor Thut, University of Glasgow, UKReviewed by:
Nathan Weisz, University of Konstanz, GermanyRoman Freunberger, University of Salzburg, Austria
Copyright: © 2011 Jensen, Bahramisharif, Oostenveld, Klanke, Hadjipapas, Okazaki and van Gerven. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.
*Correspondence: Ole Jensen, Donders Institute for Brain, Cognition and Behaviour, Kapittelweg 29, 6525 EN Nijmegen, Netherlands. e-mail: ole.jensen@donders.ru.nl