A single-session preliminary evaluation of an affordable BCI-controlled arm exoskeleton and motor-proprioception platform

Traditional, hospital-based stroke rehabilitation can be labor-intensive and expensive. Furthermore, outcomes from rehabilitation are inconsistent across individuals and recovery is hard to predict. Given these uncertainties, numerous technological approaches have been tested in an effort to improve rehabilitation outcomes and reduce the cost of stroke rehabilitation. These techniques include brain–computer interface (BCI), robotic exoskeletons, functional electrical stimulation (FES), and proprioceptive feedback. However, to the best of our knowledge, no studies have combined all these approaches into a rehabilitation platform that facilitates goal-directed motor movements. Therefore, in this paper, we combined all these technologies to test the feasibility of using a BCI-driven exoskeleton with FES (robotic training device) to facilitate motor task completion among individuals with stroke. The robotic training device operated to assist a pre-defined goal-directed motor task. Because it is hard to predict who can utilize this type of technology, we considered whether the ability to adapt skilled movements with proprioceptive feedback would predict who could learn to control a BCI-driven robotic device. To accomplish this aim, we developed a motor task that requires proprioception for completion to assess motor-proprioception ability. Next, we tested the feasibility of robotic training system in individuals with chronic stroke (n = 9) and found that the training device was well tolerated by all the participants. Ability on the motor-proprioception task did not predict the time to completion of the BCI-driven task. Both participants who could accurately target (n = 6) and those who could not (n = 3), were able to learn to control the BCI device, with each BCI trial lasting on average 2.47 min. Our results showed that the participants’ ability to use proprioception to control motor output did not affect their ability to use the BCI-driven exoskeleton with FES. Based on our preliminary results, we show that our robotic training device has potential for use as therapy for a broad range of individuals with stroke.

Introduction

Nearly 30% of stroke survivors suffer from motor and somatosensory deficits (Connell et al., 2008). Rehabilitation interventions after stroke are important to reduce motor and somatosensory deficits (Cifu and Stewart, 1999). Stroke rehabilitation is unfortunately labor-intensive and expensive. In US alone, the direct cost for stroke rehabilitation is about $36.5 billion per year (Go et al., 2014).

Researchers have been working on various approaches to reduce the cost of rehabilitation while maximizing the rehabilitation outcomes. These approaches include (1) task-specific training therapy in which individuals practice goal-directed motor tasks (Smania et al., 2003; Hubbard et al., 2009; Wong et al., 2012); (2) robotic-assisted therapies that use an exoskeleton or/and a functional electrical stimulation (FES) to conserve time, human labor, and cost (Glanz et al., 1996; Looned et al., 2014); (3) brain–computer interface (BCI) to promote mental practice and user engagement (Dobkin, 2007; Graimann et al., 2007; Daly and Wolpaw, 2008); and (4) proprioceptive feedback that could enhance the motor-related components of electroencephalography (EEG) (Ramos-Murguialday et al., 2012).

In recent years, interest has grown in robotic-assisted rehabilitation (Colombo et al., 2005; Fazekas et al., 2006, 2007; Lo et al., 2010; Loureiro et al., 2011; Poli et al., 2013). Rehabilitation robotic devices aim to assist individuals with stroke who have limited mobility. However, therapies that involve passive robotic-assisted movements are not as efficient, as they do not actively engage the participants during the therapy. The best outcomes are achieved when the users are actively engaged in the task and help to initiate movement of the robotic devices (Lo and Xie, 2012; Formaggio et al., 2013). To encourage user active engagement during the rehabilitation process, BCI-driven robotic devices have been proposed (Adamovich et al., 2004; Ang et al., 2009; Grosse-Wentrup et al., 2011; Ramos-Murguialday et al., 2012; Webb et al., 2012). A BCI system allows the user to translate his/her intention to commands that operate a device such as a computer and/or a robotic arm (Wolpaw et al., 2002). In order to activate a BCI-driven robotic device, the user performs motor imagery or mental practice of motor tasks. Studies have suggested that the combination of exercises and mental practice with BCI helps in neurological recovery perhaps by driving both anatomical and functional reorganization within the central nervous system (Di Pino et al., 2014).

Growing evidence, from both human and animal models, have shown that learning brings about a transformation in the way brain process information (Elbert et al., 1975; Lebedev et al., 2000). These plastic changes are needed to consolidate and store acquired sensorimotor knowledge. Therefore, enhancing the brain plasticity by maximizing the use of remaining undamaged brain, can improve functional recovery following brain injury (Chen et al., 2002; Wang et al., 2010). Further, combining mental practice with robotic motion and feedback can help to augment the sensorimotor feedback to influence functional ability in individuals with stroke (Broetz et al., 2010; Caria et al., 2011; Karolyn, 2011). Interestingly, in a recent study by Ramos-Murguialday et al. (2012), the proprioceptive feedback provided by a BCI-controlled exoskeleton enhanced the motor-related signals of the brain of healthy individuals.

No studies have combined BCI, robotic exoskeletons, FES, and proprioceptive feedback into a rehabilitation platform that facilitates goal-directed motor movements. The primary objective of this paper was to combine all these technologies and test the feasibility the device. We have developed a BCI-driven exoskeleton with FES that can assist individuals with stroke in performing goal-directed motor tasks. This device is lightweight and portable. Besides, it provides proprioceptive feedback similar to Ramos-Murguialday et al. (2012) study. More specifically, the users see and feel their hand with the exoskeleton moving through when working through the goal-directed motor task. We are also interested in understanding whether the motor-proprioception ability of stroke individuals affects their ability to use the BCI-driven device. Thus, a motor task that requires proprioception for completion was developed to assess motor-proprioception ability.

Materials and Methods

This section first provides details about the three main components of the BCI training device: an exoskeleton arm, a FES unit, and the BCI system. Next, it elaborates how these components are integrated and used to assist the users in performing goal-directed motor tasks. Finally, we present the experimental procedures used in this study. All of the methods within this study were in compliance with the declaration of Helsinki (Bosnjak, 2001) and were approved by the Simon Fraser University (SFU) Office of Research Ethics (# 2012s0527).

BCI Training Device

The BCI training device we developed operated to assist stroke individuals in performing a pre-defined goal-directed motor task. This device consists of (1) a lightweight and portable exoskeleton arm, (2) a FES unit, and (3) a BCI. The exoskeleton provides assistance to perform elbow extension and flexion. The FES unit assists in performing finger flexion and extension, which enables grasping or releasing an object. Finally, the BCI allows the users to drive both the exoskeleton arm and the FES unit by motor imagery.

Exoskeleton arm

Figure 1 depicts the prototype of the exoskeleton arm developed in our lab. The exoskeleton arm has two degrees of freedom (DOF), i.e., elbow flexion/extension and forearm pronation/supination. Each DOF can be controlled independently. The elbow flexion/extension and forearm pronation/supination are actuated via “Joint 1” and “Joint 2,” respectively. “Joint 1” is actuated via a brushless DC (BLDC) motor and a customized gearbox. The BLDC motor provides a torque of 69.5 mNn at a nominal speed of 3480-rpm at 79% max efficiency. The customized gearbox consists of a casing and six reduction stages. The casing is rapid prototyped from acrylonitrile butadiene styrene (ABS) plastic. The six reduction stages are made of off-the-shelf plastic spur gears and each stage has a reduction ratio of 1:3. The angular displacement of “Joint 1” is measured via a 256–2048 CPT, 2 channels, with line drive Maxon MILE Encoder. Theoretically, “Joint 1” is able to provide 40 Nm torque. However, for safety purposes, we limited the output torque to 10 Nm. The angular speed is also limited to 2 /s.

FIGURE 1

Figure 1. Exoskeleton prototype. The exoskeleton is composed of three parts: the metallic structure, the elbow joint (Joint 1), and the forearm joint (Joint 2). The metallic structure is an off-the-shelf Breg T-Scope Elbow brace, which has telescoping struts to allow maximum customization of fit. The Breg T-Scope Elbow brace also provides an extension and flexion control from 10° to 120°. Moreover, for safety purposes, it has an adjustable mechanical lockout. Besides, the Breg T-Scope Elbow brace has four straps: two for the forearm and two for the upper arm. These straps ensure full attachment between the brace and the user’s arm.

“Joint 2” is actuated via a brushed DC planetary gear motor and a single-stage customized gearbox. The brushed DC geared motor provides a torque of 1247 mNm at a nominal speed of 38.7 rpm at 24.11% efficiency. The customized single-stage gearbox has a ratio of 1:3.5 and is rapid prototyped from ABS plastic. The angular displacement of the wrist pronation/supination is measured via a low profile and long life EVWAE Panasonic potentiometer. Theoretically, the wrist joint is able to provide 4.36 Nm torque at a nominal speed of 11 rpm. However, for safety purposes, we limited the output torque to 2 Nm. The angular speed was, on the hand, limited to 15°/s. The range of motion (ROM) was limited to 45° for wrist extension and 45° for wrist flexion with the use of a mechanical stop.

There are several advantages of the exoskeleton. First, the exoskeleton is lightweight (total weight approximately 1 kg). Its’ actuator is driven by three 12 V batteries and thus, the exoskeleton is portable. Second, the user can wear the exoskeleton in <30 s when aided by another person. Both of the exoskeleton’s joints can be positioned in such a way that the exoskeleton does not to interfere with the user’s natural arm position when he/she is relaxed or performing tasks. These features are desirable and enhance the usability of the exoskeleton when assisting the users in performing goal-directed movements in daily activities and exercises for rehabilitation purposes.

Functional electrical stimulation

The FES unit used in this study is the RehaStim I (Hasomed GmbH, Magdeburg, Germany), which has eight stimulation channels. RehaStim I has the capacity to generate biphasic rectangle pulses with a frequency range of 1–140 Hz, a pulse width range of 20–500 μs, and a current output range of 0–130 mA.

Brain–computer interface

The BCI used in this study is a non-invasive and EEG-based system. EEG signals are acquired from the users using an inexpensive wireless EEG system, i.e., Emotiv EPOC neuroheadset (Emotiv SDK Research Edition Specifications, 2010). The sampling rate of the EEG signals is 128 Hz. The headset consists of 14 channels: AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, and AF4; with reference electrodes at the P3/P4 locations.

The EEG signals acquired from the headset were first processed by a signal processing unit that extract useful features the signals. Then, the features were translated by a classifier to control signals that were used to activate the exoskeleton and the FES unit. The BCI discriminated two classes of EEG signals, i.e., rest and motor imagery. Thus, the output of the classifier had one of the two discrete states “0” (rest) or “1” (motor imagery). The logical states “1” indicated the user’s intention to activate the device (the exoskeleton arm or the FES in this case). The logical states “0,” on the other hand, implied that the user did not intend to activate the device.

System integration and control

Figure 2 illustrates the functional components of the BCI training device with system software running on a laptop. The BCI system was integrated with the FES unit and the exoskeleton using LabVIEW. First, the EEG data of a user were transmitted from the Emotiv headset to the laptop via Bluetooth. Then, the EEG data were processed using the BCILAB software. The BCI generated an output every 0.5 s. As per established two-class classification, if the BCI output was a “0,” nothing happened. If the BCI output was a “1,” a control signal was sent to: the motor driver through a NI USB-6341 Data Acquisition System (DAQ) to trigger the exoskeleton or the FES unit to assist hand opening. Meanwhile, the angular position of the exoskeleton was continuously sent to the laptop via the DAQ. This data served as a feedback signal to move the exoskeleton to the desired position.

FIGURE 2

Figure 2. Functional components of the training system. EEG data were collected and processed by the BCI software embedded in the laptop. Control signals were then generated by the BCI to trigger the exoskeleton or the FES unit.

The training device operated in a preprogramed movement sequence to assist its users in performing a goal-directed motor task. The task was divided into 11 phases (Figure 3). Each phase involved movements of the shoulder, the elbow, or the hand. Depending on which phase the participant was at, the BCI would trigger (1) the exoskeleton to assist elbow extension/flexion; (2) the FES unit to assist hand opening/closing; or (3) no device for phases that involve shoulder/trunk movements. More details of the different phases of the exercise are provided in Section “Training Protocol.”

FIGURE 3

Figure 3. Flowchart of the goal-directed motor task. The task required one to move a cup from an initial position to a new position. It was divided into 11 phases (movements). All of these phases are triggered by the positive outputs of the BCI, except Phase 3, 6, and 9.

Experimental Protocol

To evaluate the performance of the BCI training device, nine stroke participants were recruited. During the training session of the experiment, the participants went through BCI training (30–45 min) and the FES unit set up (10–20 min). Next, they used the training device to perform goal-directed motor tasks (15–30 min). The participants were also assessed for their motor-proprioception ability before and after the training session using a motor-proprioception assessment protocol we developed.

Participants

Participants with chronic stroke (>6 months post) were recruited through local rehabilitation hospitals and stroke clubs. All potential participants were screened to meet the following inclusion criteria: (a) age range from 35 to 85 years, (b) post-stroke duration ≥6 months, (c) Montreal Cognitive Assessment (MoCA) ≥25 (Aggarwal and Kean, 2010), or pass any other cognitive assessment test, (d) shoulder active ROM in all directions of 10°–15°, (e) elbow passive extension and flexion ROM of 0°–130°, (f) wrist passive extension ROM of 0°–15°, and (g) fingers full passive extension. The exclusion criteria included (a) any other neurological conditions in addition to stroke, (b) unstable cardiovascular disease, (c) contraindications to FES, or (d) other conditions (e.g., poor sitting balance) that precluded them from undergoing the study.

Further, we used the upper-extremity subtest of the Fugl-Meyer (FM) test to examine the impairment severity of all stroke participants (Gladstone et al., 2002). We also used kinesthetic and visual imagery questionnaire short version (KVIQ-10) (Malouin et al., 2007) to quantify the motor imagery ability of the participants.

Nine male stroke participants (mean age 66 ± 11.9 years) agreed to participate in this study. The demographics and pre-assessment results of the participants are presented in Table 1. One participant (P01) had expressive aphasia [15/30 on the Frenchay Aphasia Assessment (Al-Khawaja et al., 1996)] and was unable to complete the MoCA test. Nevertheless, he was still included in this study to explore if individuals with expressive aphasia could operate the training device. The FM scores of the participants ranged from 11 to 63, suggesting mild to severe motor impairments. All participants were also tested on the KVIQ to assess their imagery ability. All participants scored above 29 out of 50.

TABLE 1

Table 1. Stroke participants’ demographics and pre-assessment data.

Training protocol

All the stroke participants learn how to use the training device to perform a pre-defined goal-directed motor task. The training protocol involved three consecutive steps as explained below: the setup of the BCI and the FES unit, and the use of the BCI-driven device to perform the goal-directed motor task.

BCI setup. The Emotiv headset was applied to the participant’s head. Each participant was seated comfortably in front of a laptop. To set up a BCI model, the participant had to first go through a procedure called stimulus presentation. During stimulus presentation, the participant was asked to perform different repetitive imagery tasks according to the stimulus or visual cues displayed on the laptop. Two different visual cues were presented to the participant in this study: rest and imagery grasp movements with the affected arm. Each trial lasted from 9 to 11 s, i.e., the participant was asked to perform each designated task for 5 s, followed by 4–6 s of rest. The stimulus presentation stage consisted of two sessions and each session lasted 7 min. Each session consisted of 20 trials for each task. Throughout the experiment, the participant could take a break whenever needed.

Next, the two classes of EEG data collected from the stimulus presentation procedure (i.e., rest and motor imagery) were processed to generate a customized BCI model for each participant using BCILAB (Delorme et al., 2011). To generate a BCI model, a feature extraction algorithm was first applied to extract relevant features from the EEG data. Then, a classifier was trained to discriminate the two classes of EEG data. In this study, we used the common spatial pattern (CSP) algorithm (Ramoser et al., 2000; Blankertz et al., 2008) to extract features from the EEG signals. For each electrode, three features were generated by the CSP algorithm. Meanwhile, linear discriminant analysis (LDA) was used as a classifier (Christopher, 2006). To evaluate the performance of the BCI, the 10 × 10 cross-validation method was employed (Christopher, 2006). More specifically, the data set was randomized and divided into 10-folds. Nine of the folds were used to set up the classifier and the remaining onefold was used to test the classifier. This procedure was repeated for ten times. Then, the average BCI cross-validation classification accuracy was computed.

For online testing, the BCI classified the EEG features as “0” or “1” and delivered a decision output every 0.5 s. This decision output was used to trigger the exoskeleton arm or the FES unit. An output with a logical state “1” would trigger the next movement of the training device. An output with a logical state “0,” on the other hand, would not trigger anything.

FES setup. The RehaStim FES unit was used in the training device to assist the participants in grasping and releasing an object. For each participant, two self-adhesive rectangular electrodes were attached to the stroke affected extensor digitorum. The stimulus amplitude and the electrode positions were carefully adjusted until hand opening was achieved. Whenever the FES unit was triggered, symmetrical biphasic pulses, with a fixed frequency of 35 Hz and a peak duration of 150 μs (Bigland-Ritchie et al., 1979) were applied to open the hands of the participants. This helped the participants to adjust hand position to get ready to grasp a cup. Then, the FES unit was deactivated to release hand extension and grasp the object.

Goal-directed motor task. After setting up the BCI and the FES unit, the investigators helped the participant to wear the exoskeleton. The exoskeleton was secured to the participant’s arm with four straps that were fastened distally and proximally to the participant forearm and lower arm (Figure 1).

Next, an explanation on how to use the training device to perform a pre-defined goal-directed motor task was given to each participant. This task required the participants to move a cup from an initial position to a new position. This exercise was divided into 11 phases as shown in Figure 3. The participants placed the hand at an initial position where the elbow was totally flexed. At Phase 1, the participants performed motor imagery to activate the BCI and consequently trigger the exoskeleton to extend the elbow. Then, the participant would activate the BCI to trigger the FES unit to open the hand in Phase 2. At Phase 3, the participants moved the shoulder and trunk to bring the hand close to a cup on the table. Once the hand position was adjusted, the participant would use BCI control to deactivate the FES unit in Phase 3. The deactivation of the FES unit would assist the participants to close the hand and grasp the cup. At Phase 5, to lift up the cup, the participants used the BCI-controlled exoskeleton to flex the elbow. At Phase 6 and 7, the participants moved the cup to a new position using shoulder and trunk movements and the assistance from the BCI-controlled exoskeleton (elbow extension), respectively. To release and place the cup on the table, the FES unit was triggered by motor imagery (Phase 8). Finally, to return to the initial position, the participants moved the hand away from the cup using shoulder and trunk movements (Phase 9), performed motor imagery to deactivate the FES unit to close the hand (Phase 10), and performed motor imagery again to flex the elbow (Phase 11). Note that all the phases involved BCI control except shoulder and trunk movements, as the participants had active control to move their shoulder and trunk. Each participant was trained to repeat the task multiple times without break until he/she reported muscle or mental fatigue.

Motor-proprioception assessment protocol

The ability of the stroke participants to use proprioceptive information to update movements was assessed twice during the experiment: once before setting up the FES unit (pre-assessment) and once after the robotic-assisted cup exercises (post-assessment). The duration between the pre- and post-assessment tests was approximately 1 h.

To assess the motor-proprioception ability of the participants, we have developed a protocol using the setup shown in Figure 4. The setup consists of a test bench and a two-DOF passive manipulandum with a handle. Two potentiometers were embedded on the manipulandum handle to measure its position. The circles on the test bench are the initial (Home) and target positions the participants should aim to reach for.

FIGURE 4

Figure 4. (A) Test bench setup for motor-proprioception assessment. The test bench has a table with two identical plates (upper and lower). The size of each plate is 50 cm × 70 cm. Each plate contains six identical circles with a diameter of 2 cm. The circles on the two plates are coaxial. (B) Target positions for the motor-proprioception assessment. Five of the circles on the plate (denoted by the numbers 1–5) are the target positions used for motor-proprioception assessment. H is the initial or home position.

During the assessment protocol, the participant’s vision of the hand, elbow, and shoulder under investigation was occluded with a drape. The participant was requested to move the handle such that the hand was placed on any of the five target positions specified by the investigator. Then, the distance error (DE) between the target position and the position achieved by the participant was calculated and recorded.

Five different sequences of target positions were used in the assessment protocol. These sequences are presented in Table 2. Each sequence contained 10 instructions that specified where the participant should reach. In each sequence, the participant was asked to reach a target position according to vocal instructions from the investigators. In these sequences, the letter “H” means that participant must return to the home position “H” and then advance to the next target position according to the vocal instruction. All sequences start from “H.”

TABLE 2

Table 2. Motor-proprioception task sequences.

Motor-proprioception assessment: validity and reproducibility

To evaluate the validity of the developed method, we compared the obtained results with the test scores obtained from two other standard clinical tests for proprioception, i.e., the “up-or-down” test and the tactile sensation test. The FM test protocol for the “up-or-down” test has been used clinically to evaluate the upper-limb proprioception of a patient (Trinh, 2005). The tactile sensation test, on the other hand, was done to assesses the upper-limb tactile sensation and motor deficits quantitatively (Trinh, 2005; Dros et al., 2009). Both the “up-or-down” test and the tactile sensation test were administered to all the participants by physical therapist.

Ten healthy and eight stroke participants (all in Table 2 except P08) agreed to participate in this part of the study. All participants gave informed written consent to participate in the study. The 10 healthy participants consisted of 8 males and 2 females (aged from 22 to 37, with a mean of 27.9 ± 5.5 years; 9 right-handed, 1 left-handed).

To assess the reproducibility of the results obtained using the developed motor-proprioception assessment method, four healthy participants were asked to perform the assessment protocol twice. Two tests were conducted 7 days apart.

Subjective evaluation (questionnaire feedback)

At the end of the experiment, the participants were asked to complete a questionnaire to evaluate their perceived workload of using the BCI-controlled system to perform the goal-directed motor task. The questionnaire used was the NASA “task load index” (TLX) questionnaire (Hart, 2006). This questionnaire is a multifaceted assessment tool that rates perceived workload with six subscales, i.e., mental demand, physical demand, temporal demand, performance, effort, and frustration. Each participant was asked to evaluate the TLX subscales with a scale of 0–20.

Results

Motor-Proprioception Assessment Test: Validity and Reproducibility

Validity

The validity of the developed motor-proprioception assessment method was examined using the correlation values between the DE obtained from this method and the scores of two other tests (Leibowitz et al., 2008). The results are presented in Table 3. The strong correlation values between DE and the test scores from either of the standard clinical proprioception test suggest that the developed method is valid.

TABLE 3

Table 3. Validation results for the developed motor-proprioception assessment test.

Test–retest reliability

Two tests were conducted among four healthy participants, 7 days apart. For the dominant hand, the deviations (in millimeters) between the two tests’ results for each participant were 0.8, 2.8, 6.7, and 0.8, respectively (mean = 2.87 ± 2.85 mm). For the non-dominant hand, the deviations for each participant are 3.9, 5.9, 2.2, and 0.75, respectively, for non-dominant arm (mean = 3.25 ± 2.25 mm).

To quantify the reliability of the motor-proprioception assessment method, the intra-class correlation coefficient (ICC) between the data from the first test and the second test is calculated. A high intra-class correlation of 0.900 (p < 0.05) was demonstrated. This supports the reliability of the method.

Training Exercise Results

Table 4 presents the BCI cross-validation accuracy for each stroke participant. On average, the cross-validation accuracy achieved is 68.8%, with a standard deviation of 9.0%. The results obtained from this study are similar to those reported in other BCI studies in the literature (Ang et al., 2011).

TABLE 4

Table 4. BCI cross-validation accuracy.

All the stroke participants (n = 9) completed the training exercise at least two times. The average number of training exercise that was completed is 3 ± 0.7 (ranged from 2 to 5). The time taken to complete a trial, T_c was used to evaluate the performance of the participants. On average, the participants completed one exercise in 7.4 ± 2.8 min. All the participants successfully reduced the T_c value of the last trial (with an average of 39.3 ± 16.81 s). For three out of the nine participants (P03, P04, and P06), the reduction was <2 s. For the remaining participants, the reduction ranged from 4.9 to 128.8 s. The time to complete a trial was reduced significantly in some participants (e.g., 128.8 in P05) because after a few trials, they quickly became familiarized with the different phases of the protocol and the BCI system. Next, the difference between the T_c of the first and last trial as well as the difference between the BCI positive output of the first and last trials were also examined. No correlation was found (r = −0.12, p > 0.5) as they had gained a better control of the training device.

For each trial of the training exercise, the following data were recorded in real-time: the BCI output: rest (“0”) or activate (“1”); the FES status: deactivate (“0”) or activate (“1”); the angular position of the exoskeleton; and the time taken to complete a trial (T_c). In Figure 5, an example of the time course of the BCI output [rest (“0”) or activate (“1”)], the FES status, and the angular position of the exoskeleton for participant P05 is presented. This example shows a typical behavior of the system output in real-time when the participant was in good control of the training device. As shown in Figure 5, the participant successfully generated positive BCI outputs (“1”) that triggered the exoskeleton and the FES. However, not all positive BCI outputs in the figure are true positives. Three of the positive BCI outputs labeled as “PO” did not trigger any device. These “PO” occurred at 1.52, 0.55, and 5.65 s, respectively, after a true positive. The first and the last POs were very likely false positives because they took place more than 1 s after a true positive. These undesired positive outputs, however, did not affect the overall system performance. This is because when the exoskeleton was in operation, no other devices could be triggered.

FIGURE 5

Figure 5. Time course of the system response for P05. The following data were recorded in real-time: the BCI output: rest (“0”) or activate (“1”); the FES status: deactivate (“0”) or activate (“1”); and the angular position of the exoskeleton. This figure demonstrates a typical behavior of the system. It consists of 12 regions denoted by letters from “A” to “L.” Section “A” describes the start of the trial till triggering the elbow extension, which initiates elbow extension from the initial position; Section “B” (Phase 1) describes elbow extension (to reach the cup); In Section “C,” the participant generates the trigger to activate the FES (to open the hand, Phase 2) and adjusts the shoulder/trunk (Phase 3); In Section “D,” the participant is supposed to generate a trigger to deactivate the FES (to close the hand and grasp, Phase 4); In Section “E,” the participant is supposed to generate a trigger to initiate the elbow flexion; Section “F” elbow flexion (to elevate the cup, Phase 5); In Section “G,” the participant is supposed to generate a trigger for elbow extension and adjusts the shoulder/trunk (Phase 6); Section “H” elbow extension (to place down the cup, Phase 7); In Section “I” the participant is supposed to generate a trigger to activate the FES (to open the hand and leave the cup, Phase 8); Section “J” describes the section in which the participant moves her/his shoulder/trunk away from the cup (Phase 9), and is supposed to generate a trigger to deactivate the FES (to close the hand, Phase 10); in Section “K” the participant is supposed to generate a trigger to initiate the elbow flexion; and Section “L” elbow flexion (to return to initial position, Phase 11).

Motor-Proprioception Results of Stroke Participants

The motor-proprioception of the stroke participants before (pre-assessment) and after (post-assessment) the BCI training protocol were compared. Motor-proprioception was assessed using the DE values obtained from the motor-proprioception assessment protocol.

For the pre-assessment test, the DE values for the affected and non-affected arms ranged from 24.9 to 146.2 mm (mean = 60.93 ± 39.5) and 24.9 to 88.2 mm (mean = 42.3 ± 11.6), respectively. For the post-assessment test, the DE values the range for the affected arm was 28.4–124.2 mm (mean = 65.4 ± 31.13) and the range for the non-affected arm was 31.8–61.5 mm (mean = 45.47 ± 11.4). Table 5 reports the absolute difference between the DE values for the affected and non-affected arm (DDE). As the data were non-Gaussian, the Wilcoxon signed rank test was used to determine if there was any difference between the DDE values for the pre and post-assessments. The analysis suggested that there was no significant difference (p = 0.36).

TABLE 5

Table 5. Difference between both arms’ DE.

The Wilcoxon signed rank test was also used to determine if there was any difference between the DDE values for the healthy and stroke participants. The analysis showed that the DDE values for the healthy and the pre-assessment DDE values for the stroke participants were statistically significant (p < 0.05). Interestingly, the DDE values for the healthy and the post-assessment DDE values for the stroke participants were not statistically different (p = 0.13). The pre-assessment DDE values for three of the stroke participants (P05, P06, and P07) were within the 95% confidence interval of the DDE values for the healthy participants, i.e., 5.5 ± 4.8 mm.

Questionnaire Feedback

At the end of the experiment, all the stroke participants subjectively evaluated the training device using the NASA TLX questionnaire. The factors taken into consideration in the questionnaire and the average ratings obtained from the participants are presented in Table 6. The results show that the mental demand is the main contributor to the workload as it has the highest average rating (14.4 ± 4.7).

TABLE 6

Table 6. NASA TLX questionnaire’s results.

Correlation between Variables

Based on the data collected from the stroke participants, the bivariate correlations between 15 different variables were examined. This exploratory analysis allowed us to discover the relationships between these variables, which could be useful in generating hypothesis for future work. These variables include:

(1) age of the stroke individuals

(2) duration of stroke (months)

(3) upper-extremity FM scores

(4) KVIQ/KI/VI

(5) BCI cross-validation accuracy

(6) time taken to complete a trial, T_c

(7) motor-proprioception ability (pre-assessment), denoted as DDEpre

(8) motor-proprioception ability (post-assessment), denoted as DDEpost

(9) difference between DDEpost and DDEpre, denoted as DDEdif

(10) mental demand (from TLX)

(11) physical demand (from TLX)

(12) temporal demand (from TLX)

(13) participants’ perceived performance (from TLX)

(14) effort (from TLX)

(15) frustration (from TLX).

Six of the variable pairs have demonstrated significant correlation values (p < 0.05), i.e., motor-proprioception ability before (DDEpre) and after (DDEpost) assessment (r = 0.855); age and frustration (r = 0.756); physical demand and frustration (r = 0.680); temporal and physical demand (r = 0.682); BCI cross-validation accuracy and FM scores (r = −0.678); DDEdif and mental demand (r = −0.805). The scatter plots of these variable pairs are presented in Figure 6.

FIGURE 6

Figure 6. Scatter plots of the variable pairs that demonstrate significant correlation values. (A) The scatter plot of the motor-proprioception ability of the stroke participants before and after the assessment. (B) The scatter plot of the frustration level and age of the stroke participants. (C) The scatter plot of the frustration level and mental demand of the stroke participants. (D) The scatter plot of the temporal and physical demand of the stroke participants. (E) The scatter plot of the BCI accuracy and the Fugl-meyer scores. (F) The scatter plot of the motor-proprioception ability and mental demand of the stroke participants.

The motor-proprioception ability of the stroke participants before and after the assessment was highly correlated (r = 0.855) as demonstrated in Figure 6A. Their ICC value was 0.910 (p < 0.05), which also demonstrated the reproducibility of the results of the developed motor-proprioception assessment method.

The correlation analysis showed that the participant’s age and duration of stroke did not have a significant correlation with either the BCI cross-validation accuracy or the time taken to complete a robotic-assisted exercise, T_c. The age of the participants, however, was correlated with their frustration level when operating the BCI-driven device (see Figure 6B). Another variable that had a positive correlation with frustration was physical demand (r = 0.680) as shown in Figure 6C. On the other hand, mental demand was not correlated with frustration. More specifically, those who considered the BCI-driven exercise mentally demanding (P01 and P08) did not feel more discouraged and frustrated.

As shown in Figure 6E, the BCI cross-validation accuracy had a negative correlation with the upper-extremity FM scores (r = −0.678). This contradicts another study in the literature (Ang et al., 2008), where the authors showed that the upper-extremity FM scores of the hemiparetic stroke patients (n = 35) was not linearly correlated with the BCI performance and thus, motor impairment did not affect the stroke patients’ ability to operate a BCI.

Next, our results showed no correlation between the FM scores and the KVIQ/KI/VI scores. This suggested that motor impairment did not affect the stroke participants’ ability to perform motor imagery. We also found that the BCI cross-validation accuracy was not correlated with KVIQ, KI, or VI (p > 0.2). This was consistent with the results reported in (Zich et al., 2015), in which no correlations were found between the KVIQ/KI/VI and the BCI accuracy. However, the results were not consistent with (Vuckovic and Osuagwu, 2013), where the authors reported a correlation between KI and BCI accuracy (r = 0.458) and VI and BCI accuracy (r = 0.728). In this study, the KVIQ scores were not able to predict the BCI accuracy possibly because the motor imagery task used to operate the BCI was different from the KVIQ motor imagery tasks. Further studies are necessary to investigate the relationship between BCI accuracies and KVIQ scores.

It was also interesting to investigate the relationship between the participants’ proprioceptive function and their feedback/performance during the online exercise protocol. The stroke participants were categorized into three groups according to the difference between their DDE values pre- and post-assessment:

(a) Group A (P01, P07, P08, and P09): decreased DDE values

(b) Group B (P04, P05, and P06): DDE values are in the range of those of the healthy participants

(c) Group C (P02 and P03): increased DDE values.

We found strong correlation between the change in the DDE values and the mental effort in the questionnaire (r = −0.801, p < 0.05). Group A reported that the training exercise was mentally demanding but not physically demanding. Group C reported that the training exercise was neither mentally nor physically demanding. For Group B, the participants (P05 and P06) who reported that the training exercise was mentally demanding experienced a decrease in the DDE values. On the contrary, participant P07 reported that the training exercise was not mentally demanding and showed an increase in the DDE values.

Figure 7 shows the scatter plot of the motor-proprioception ability and the BCI cross-validation accuracy/T_c. The correlation analysis showed that the participants’ motor-proprioception ability did not have a significant relationship with either the BCI cross-validation accuracy or the time taken to complete a robotic-assisted exercise, T_c. Our preliminary results suggested that the motor-proprioception ability of the stroke individuals and their age and duration of stroke did not affect their ability to use the BCI-driven exoskeleton with FES.

FIGURE 7

Figure 7. Scatter plots of the pre- and post-assessment motor-proprioception assessment results of the stroke participants and the BCI cross-validation accuracy and the time to complete a trial. (A) The scatter plot of the BCI accuracy and the motor-proprioception ability (pre-assessment). (B) The scatter plot of the BCI accuracy and the motor-proprioception ability (post-assessment). (C) The scatter plot of the time taken to complete a trial and the motor-proprioception ability (pre-assessment). (D) The scatter plot of the time taken to complete a trial and the motor-proprioception ability (post-assessment).

Discussion and Conclusion

In this study, we developed a comprehensive BCI platform that combines different rehabilitation and technological approaches. The platform consists of a BCI training device and a motor-proprioception assessment protocol. The BCI training device promotes user engagement via the use of motor imagery to trigger the exoskeleton or FES. It also promotes the use of goal-directed motor task as part of the training protocol. The motor-proprioception assessment protocol provided us with a means to assess whether motor-proprioception ability was a critical factor in determining for whom this therapy may be best employed.

Training Device

To evaluate the performance of the training device, two performance metrics were examined: the BCI cross-validation accuracy and the time taken to complete a goal-directed motor task. In addition, the participants also provided subjective feedback about the system using the TLX questionnaire.

The BCI cross-validation accuracy is an important metrics widely used to assess the performance of the BCI model generated offline. Our results show that the BCI cross-validation accuracy achieved is in the range of 55.78–83.20%, which is consistent with those in the literature (Ang et al., 2011). The accuracy for some of the participants (P02, P04, P05, P06, P07, and P08) was not satisfactory (<70%). One reason is that some participants were not able to focus when performing motor imagery possibly due to their age and stroke related problems. Also, the number of electrodes placed around the motor cortex area was sparse.

For the goal-directed motor task of moving a cup, the BCI online accuracy was not available. This is because during this real-time operation, no cues were given to the participants as to when they should perform a motor imagery or rest. In other words, the system was self-paced and the participants had the flexibility of deciding when to activate the BCI and consequently trigger the exoskeleton or the FES unit. As a result, no true labels of the EEG data were available. To measure the training device performance, the time taken by the participants to complete a trial (T_c) was used. On average, the participants were able to complete three trials of the exercise. The number of trials completed was small in this study because the participants experienced fatigue toward the end of the experiment. In real rehabilitation settings, the BCI set up and the use of the BCI device for training could be done on separate days. This will subsequently increase the number of trials the participants can complete. The speed of completing a trial varied across the stroke participants. The T_c values ranged from 2.3 to 6.1 min and the mean was 2.5 min. This mean value is actually quite close to that achieved when the system is ideal, i.e., approximately 2.2 min. Here, an ideal system implies that the BCI system is perfect, i.e., its accuracy is 100%. In addition, we observed that all the participants successfully reduced the T_c value of the last trial. This improvement possibly indicates that they had gained a better control of the training device. The results are encouraging. Also, our results suggest the participant with expressive aphasia was able to use the system. This demonstrates the potential use of the developed system in performing goal-directed motor task multiple times in a 1 h rehabilitation intervention.

One major factor that could affect the online performance of the system is the BCI accuracy. If the participants have a good BCI control, less time is needed to finish an exercise. To improve the performance of the BCI system, denser electrodes could be placed over the scalp especially the motor cortex area. Also, EEG signals have a low signal-to-noise ratio and are frequently contaminated with artifacts. These unwanted noises either originate from the user (e.g., ocular and facial muscle activities) or from other non-physiological sources such as power line interference and created artifacts in the EEG signal. As artifacts could affect the quality of the EEG signals and subsequently degrade the BCI performance, it is important to apply automatic artifact removal algorithms to improve the BCI performance. Another design factor that has not received much attention from BCI researchers is volitional inhibition, which is the ability to suppress a movement due to unexpected changes in the environments (Mirabella, 2012). Neural correlates of volitional inhibition have been investigated in Mirabella et al. (2011). However, more research effort is needed to decode volitional control using EEG signals before this feature is incorporated into the BCI system. The ability of the BCI to predict volitional control may potentially lead to a BCI-controlled exoskeleton/FES that better mimics natural movements (Mirabella, 2012).

Motor-Proprioception Assessment

We employed our new assessment method to identify stroke participants with motor-proprioception deficits. We have demonstrated its validity by comparing its results with two other standard clinical tests for proprioception (i.e., the “up-or-down” test and the tactile sensation test). The claim is supported by the results that the DE values of the developed method has a significant correlation with the test scores obtained from either the “up-or-down” test or the tactile sensation test.

Our results show that participants P05, P06, and P07 have DDE values that were in the range of the DDE values of the healthy participants. This finding is consistent with the results obtained from the two standard clinical tests for proprioception. In addition, no correlation was found between the proprioceptive function of both the hands of the healthy participants. Thus, by comparing the proprioception of the affected and non-affected arms of the stroke participants using DDE, we can quantify the degree of which stroke has affected the proprioceptive function of the affected arm. The difference between the DDE values for the stroke participants for the pre- and post-assessments is not statistically different (p < 0.16). However, the average DDE values were lower after the BCI training session.

Subjective Evaluation

The stroke participants’ subjective feedback on TLX questionnaire has suggested that the developed system and exercise protocol is safe, neither slow nor fast, and not overly physical demanding. Even though the participants perceive the task as mentally demanding, they gave low frustration rating about the experiment. Based on our personal communication with the participants, their ability to control the exoskeleton and the FES unit with motor imagery has boosted their confidence level in regaining the motor functions in the future.

Conclusion

In summary, we have developed a comprehensive platform that consists of (1) a BCI-controlled exoskeleton/FES training device with proprioceptive feedback and (2) a motor-proprioception assessment test. Despite of the small sample size in this study (n = 9) and the limited number of training repetitions, we showed that individuals with stroke can operate the BCI-controlled robotic training device. All the stroke participants successfully completed multiple trials of the goal-directed motor task of moving a cup using the training device. It would be interesting to investigate the use of this comprehensive platform in stroke rehabilitation at various sites of service with a bigger sample size. In our future work, a clinical study will be conducted to evaluate the efficacy of the developed platform in restoring the motor and proprioceptive functions of stroke individuals.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This study was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), the Canadian Institutes of Health Research (CHIR), and the Michael Smith Foundation for Health Research (MSFHR).

References

Adamovich, S. V., Merians, A. S., Boian, R., Tremaine, M., Burdea, G. S., Recce, M., et al. (2004). “A virtual reality based exercise system for hand rehabilitation post-stroke: transfer to function,” in International Conference of the IEEE Engineering in Medicine and Biology Society (San Francisco, CA: IEEE), 4936–4939. doi: 10.1109/IEMBS.2004.1404364

CrossRef Full Text | Google Scholar

Al-Khawaja, I., Wade, D. T., and Collin, C. F. (1996). Bedside screening for aphasia: a comparison of two methods. J. Neurol. 243, 201–204. doi:10.1007/BF02444015

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Ang, K. K., Guan, C., Chua, K. S. G., Ang, B. T., Keong, C. W. K., Wang, C., et al. (2008). “A clinical evaluation of non-invasive motor imagery-based brain-computer interface in stroke,” in International Conference of the IEEE Engineering in Medicine and Biology Society (Vancouver, British of Columbia: IEEE), 4178–4181. doi:10.1109/IEMBS.2008.4650130

CrossRef Full Text | Google Scholar

Ang, K. K., Guan, C., Chua, K. S. G., Ang, B. T., Kuah, C. W. K., Wang, C., et al. (2009). “A clinical study of motor imagery-based brain-computer interface for upper limb robotic rehabilitation,” in International Conference of the IEEE Engineering in Medicine and Biology Society (Minnesota), 5981–5984. doi:10.1109/IEMBS.2009.5335381

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Ang, K. K., Guan, C., Chua, K. S. G., Ang, B. T., Kuah, C. W. K., Wang, C., et al. (2011). A large clinical study on the ability of stroke patients to use an EEG-based motor imagery brain-computer interface. Clin. EEG Neurosci. 42, 253–258. doi:10.1177/155005941104200411

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Aggarwal, A., Kean, E. (2010). Comparison of the Folstein mini mental state examination (MMSE) to the Montreal Cognitive Assessment (MoCA) as a cognitive screening tool in an inpatient rehabilitation setting. Neurosci. Med. 1, 39–42. doi:10.4236/nm.2010.12006

CrossRef Full Text | Google Scholar

Bigland-Ritchie, B., Jones, D. A., and Woods, J. J. (1979). Excitation frequency and muscle fatigue: electrical responses during human voluntary and stimulated contractions. Exp. Neurol. 64, 414–427. doi:10.1016/0014-4886(79)90280-2

CrossRef Full Text | Google Scholar

Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., and Muller, K.-R. (2008). Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Proc. Mag. 25, 41–56. doi:10.1109/MSP.2008.4408441

CrossRef Full Text | Google Scholar

Bosnjak, S. (2001). The declaration of Helsinki – the cornerstone of research ethics. Arch. Oncol. 233, 179–184.

Google Scholar

Broetz, D., Braun, C., Weber, C., Soekadar, S. R., Caria, A., and Birbaumer, N. (2010). Combination of brain-computer interface training and goal-directed physical therapy in chronic stroke: a case report. Neurorehabil. Neural Repair 24, 674–679. doi:10.1177/1545968310368683

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Caria, A., Weber, C., Brötz, D., Ramos, A., Ticini, L. F., Gharabaghi, A., et al. (2011). Chronic stroke recovery after combined bci training and physiotherapy: a case report. Psychophysiology 48, 578–582. doi:10.1111/j.1469-8986.2010.01117.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Chen, R., Cohen, L. G., and Hallett, M. (2002). Nervous system reorganization following injury. Neuroscience 111, 761–773. doi:10.1016/S0306-4522(02)00025-8

CrossRef Full Text | Google Scholar

Christopher, M. B. (2006). Pattern Recognition and Machine Learning. New York, NY: Springer.

Google Scholar

Cifu, D. X., and Stewart, D. G. (1999). Factors affecting functional outcome after stroke: a critical review of rehabilitation interventions. Arch. Phys. Med. Rehabil. 80, S35–S39. doi:10.1016/S0003-9993(99)90101-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Colombo, R., Pisano, F., Micera, S., Mazzone, A., Delconte, C., Carrozza, M. C., et al. (2005). Robotic techniques for upper limb evaluation and rehabilitation of stroke patients. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 311–324. doi:10.1109/TNSRE.2005.848352

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Connell, L. A., Lincoln, N. B., and Radford, K. A. (2008). Somatosensory impairment after stroke: frequency of different deficits and their recovery. Clin. Rehabil. 22, 758–767. doi:10.1177/0269215508090674

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Daly, J. J., and Wolpaw, J. R. (2008). Brain-computer interfaces in neurological rehabilitation. Lancet Neurol. 7, 1032–1043. doi:10.1016/S1474-4422(08)70223-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Delorme, A., Mullen, T., Kothe, C., Akalin Acar, Z., Bigdely-Shamlo, N., Vankov, A., et al. (2011). EEGLAB, SIFT, NFT, BCILAB, and ERICA: new tools for advanced EEG processing. J. Comput. Intell. Neurosci. 2011, 130714. doi:10.1155/2011/130714

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Di Pino, G., Pellegrino, G., Assenza, G., Capone, F., Ferreri, F., Formica, D., et al. (2014). Modulation of brain plasticity in stroke: a novel model for neurorehabilitation. Nat. Rev. Neurol. 10, 597–608. doi:10.1038/nrneurol.2014.162

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Dobkin, B. H. (2007). Brain-computer interface technology as a tool to augment plasticity and outcomes for neurological rehabilitation. J. Physiol. 579, 637–642. doi:10.1113/jphysiol.2006.123067

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Dros, J., Wewerinke, A., Bindels, P. J., and Van Weert, H. C. (2009). Accuracy of monofilament testing to diagnose peripheral neuropathy. Ann. Fam. Med. 7, 555–558. doi:10.1370/afm.1016

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Elbert, T., Pantev, C., Wienbruch, C., Rockstroh, B., and Taub, E. (1975). Increased cortical representation of the fingers of the left hand in string players. Science 270, 305–307. doi:10.1126/science.270.5234.305

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Emotiv SDK Research Edition Specifications. (2010). Available at: http://innovatec.co.jp/content/etc/ResearchEditionSDK.pdf

Google Scholar

Fazekas, G., Horvath, M., and Toth, A. (2006). A novel robot training system designed to supplement upper limb physiotherapy of patients with spastic hemiparesis. Int. J. Rehabil. Res. 29, 251–254. doi:10.1097/01.mrr.0000230050.16604.d9

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Fazekas, G., Horvath, M., Troznai, T., and Toth, A. (2007). Robot-mediated upper limb physiotherapy for patients with spastic hemiparesis: a preliminary study. J. Rehabil. Med. 39, 580–582. doi:10.2340/16501977-0087

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Formaggio, E., Storti, S. F., Galazzo, I. B., Gandolfi, M., Geroin, C., Smania, N., et al. (2013). Modulation of event-related desynchronization in robot-assisted hand performance: brain oscillatory changes in active, passive and imagined movements. J. Neuroeng. Rehabil. 10, 24. doi:10.1186/1743-0003-10-24

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gladstone, D. J., Danells, C. J., and Black, S. E. (2002). The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties. Neurorehabil. Neural Repair 16, 232–240. doi:10.1177/154596802401105171

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Glanz, M., Klawansky, S., Stason, W., Berkey, C., and Chalmers, T. C. (1996). Functional electrostimulation in poststroke rehabilitation: a meta-analysis of the randomized controlled trials. Arch. Phys. Med. Rehabil. 77, 549–553. doi:10.1016/S0003-9993(96)90293-2

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Go, A. S., Mozaffarian, D., Roger, V. L., Benjamin, E. J., Berry, J. D., Blaha, M. J., et al. (2014). Heart disease and stroke statistics – 2014 update: a report from the American heart association. Circulation 129, e28–e292. doi:10.1161/01.cir.0000441139.02102.80

CrossRef Full Text | Google Scholar

Graimann, B., Allison, B., and Gräser, A. (2007). “New applications for non-invasive brain-computer interfaces and the need for engaging training environments,” in Proceedings of the International Conference on Advances in Computer Entertainment (Salzburg), 25–28.

Google Scholar

Grosse-Wentrup, M., Mattia, D., and Oweiss, K. (2011). Using brain-computer interfaces to induce neural plasticity and restore function. J. Neural Eng. 8, 025004. doi:10.1088/1741-2560/8/2/025004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Hart, S. G. (2006). Nasa-task load index (NASA-TLX); 20 years later. Hum. Factors Ergon. Soc. 50, 904–908. doi:10.1037/e577632012-009

CrossRef Full Text | Google Scholar

Hubbard, I. J., Parsons, M. W., Neilson, C., and Carey, L. M. (2009). Task-specific training: evidence for and translation to clinical practice. Occup. Ther. Int. 16, 175–189. doi:10.1002/oti.275

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Karolyn, O. B. (2011). “Brain-Computer Interfaces for Inducing Brain Plasticity and Motor Learning”: Implications for Brain-Injury Rehabilitation. Atlanta, GA: Georgia Institute of Technology.

Google Scholar

Lebedev, M. A., Mirabella, G., Erchova, I., and Diamond, M. E. (2000). Experience-dependent plasticity of rat barrel cortex: redistribution of activity across barrel-columns. Cereb. Cortex 10, 23–31. doi:10.1093/cercor/10.1.23

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Leibowitz, N., Levy, N., Weingarten, S., Grinberg, Y., Karniel, A., Sacher, Y., et al. (2008). Automated measurement of proprioception following stroke. Disabil. Rehabil. 30, 1829–1836. doi:10.1080/09638280701640145

CrossRef Full Text | Google Scholar

Lo, A. C., Guarino, P. D., Richards, L. G., Haselkorn, J. K., Wittenberg, G. F., Federman, D. G., et al. (2010). Robot-assisted therapy for long-term upper-limb impairment after stroke. N. Engl. J. Med. 362, 1772–1783. doi:10.1056/NEJMoa0911341

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Lo, H. S., and Xie, S. Q. (2012). Exoskeleton robots for upper-limb rehabilitation: state of the art and future prospects. Med. Eng. Phys. 34, 261–268. doi:10.1016/j.medengphy.2011.10.004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Looned, R., Webb, J., Xiao, Z. G., and Menon, C. (2014). Assisting drinking with an affordable BCI-controlled wearable robot and electrical stimulation: a preliminary investigation. J. Neuroeng. Rehabil. 11, 51. doi:10.1186/1743-0003-11-51

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Loureiro, R. C. V., Harwin, W. S., Nagai, K., and Johnson, M. (2011). Advances in upper limb stroke rehabilitation: a technology push. Med. Biol. Eng. Comput. 49, 1103–1118. doi:10.1007/s11517-011-0797-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Malouin, F., Richards, C. L., Jackson, P. L., Lafleur, M. F., Durand, A., and Doyon, J. (2007). The kinesthetic and visual imagery questionnaire (KVIQ) for assessing motor imagery in persons with physical disabilities: a reliability and construct validity study. J. Neurol. Phys. Ther. 31, 20–29. doi:10.1097/01.NPT.0000260567.24122.64

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Mirabella, G. (2012). Volitional inhibition and brain-machine interfaces: a mandatory wedding. Front. Neuroeng. 5:20. doi:10.3389/fneng.2012.00020

CrossRef Full Text | Google Scholar

Mirabella, G., Pani, P., and Ferraina, S. (2011). Neural correlates of cognitive control of reaching movements in the dorsal premotor cortex of rhesus monkeys. J. Neurophysiol. 106, 1454–1466. doi:10.1152/jn.00995.2010

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Poli, P., Morone, G., Rosati, G., and Masiero, S. (2013). Robotic technologies and rehabilitation: new tools for stroke patients’ therapy. Biomed Res. Int. 2013, 153872. doi:10.1155/2013/153872

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Ramoser, H., Müller-Gerking, J., and Pfurtscheller, G. (2000). Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans. Rehabil. Eng. 8, 441–446. doi:10.1109/86.895946

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Ramos-Murguialday, A., Schürholz, M., Caggiano, V., Wildgruber, M., Caria, A., Hammer, E. M., et al. (2012). Proprioceptive feedback and brain computer interface (BCI) based neuroprostheses. PLoS ONE 7:e47048. doi:10.1371/journal.pone.0047048

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Smania, N., Montagnana, B., Faccioli, S., Fiaschi, A., and Aglioti, S. M. (2003). Rehabilitation of somatic sensation and related deficit of motor control in patients with pure sensory stroke. Arch. Phys. Med. Rehabil. 84, 1692–1702. doi:10.1053/S0003-9993(03)00277-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Trinh, T. (2005). Physical diagnosis. JAMA 293, 870–875. doi:10.1001/jama.293.7.871

CrossRef Full Text | Google Scholar

Vuckovic, A., and Osuagwu, B. A. (2013). Using a motor imagery questionnaire to estimate the performance of a brain – computer interface based on object oriented motor imagery. Clin. Neurophysiol. 124, 1586–1595. doi:10.1016/j.clinph.2013.02.016

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Wang, L., Yu, C., Chen, H., Qin, W., He, Y., Fan, F., et al. (2010). Dynamic functional reorganization of the motor execution network after stroke. Brain 133(Pt 4), 1224–1238. doi:10.1093/brain/awq043

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Webb, J., Xiao, Z. G., Aschenbrenner, K. P., Herrnstadt, G., and Menon, C. (2012). “Towards a portable assistive arm exoskeleton for stroke patient rehabilitation controlled through a brain computer interface,” in The Fourth IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (Roma), 1299–1304. doi:10.1109/BioRob.2012.6290674.

CrossRef Full Text | Google Scholar

Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., and Vaughan, T. M. (2002). Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791. doi:10.1016/S1388-2457(02)00057-3

CrossRef Full Text | Google Scholar

Wong, J. D., Kistemaker, D. A., Chin, A., and Gribble, P. L. (2012). Can proprioceptive training improve motor learning? J. Neurophysiol. 108, 3313–3321. doi:10.1152/jn.00122.2012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Zich, C., De Vos, M., Kranczioch, C., and Debener, S. (2015). Wireless EEG with individualized channel layout enables efficient motor imagery training. Clin. Neurophysiol. 126, 698–710. doi:10.1016/j.clinph.2014.07.007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar