- 1School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China
- 2Shaanxi Key Laboratory of Intelligent Robots, Xi'an Jiaotong University, Xi'an, China
In the field of lower limb exoskeletons, besides its electromechanical system design and control, attention has been paid to realizing the linkage of exoskeleton robots to humans via electroencephalography (EEG) and electromyography (EMG). However, even the state of the art performance of lower limb voluntary movement intention decoding still faces many obstacles. In the following work, focusing on the perspective of the inner mechanism, a homology characteristic of EEG and EMG for lower limb voluntary movement intention was conducted. A mathematical model of EEG and EMG was built based on its mechanism, which consists of a neural mass model (NMM), neuromuscular junction model, EMG generation model, decoding model, and musculoskeletal biomechanical model. The mechanism analysis and simulation results demonstrated that EEG and EMG signals were both excited by the same movement intention with a response time difference. To assess the efficiency of the proposed model, a synchronous acquisition system for EEG and EMG was constructed to analyze the homology and response time difference from EEG and EMG signals in the limb movement intention. An effective method of wavelet coherence was used to analyze the internal correlation between EEG and EMG signals in the same limb movement intention. To further prove the effectiveness of the hypothesis in this paper, six subjects were involved in the experiments. The experimental results demonstrated that there was a strong EEG-EMG coherence at 1 Hz around movement onset, and the phase of EEG was leading the EMG. Both the simulation and experimental results revealed that EEG and EMG are homologous, and the response time of the EEG signals are earlier than EMG signals during the limb movement intention. This work can provide a theoretical basis for the feasibility of EEG-based pre-perception and fusion perception of EEG and EMG in human movement detection.
Introduction
With the intensifying of the aging problem, and with the military and civilian goals to amplify the human ability, many studies have focused on the development of robotics to break through human motor limitations such as terrain conditions or individual ability (Al-Quraishi et al., 2018). Lower limb exoskeleton robotics is one of the eminent research areas to develop load augmentation, which could further enhance the lower limb's ability, reduce the energy consumption of humans, and increase a human's load capacity (Qiuzhi et al., 2016). Recent notable research on exoskeleton robotics as well as its related technology have extended applications to practical areas. Besides the assistive tool for healthy people, lower limb exoskeleton robotics is also expected to broadly meet social requirements in the fields of medical treatment, walking-assistance for senior and disabled people, industrial and agricultural production, and other fields (Rupal et al., 2017). Despite this field having attracted a considerable level of attention over the last few years, there are still some problems with the prediction of human lower limb voluntary movement intention (Chen et al., 2016).
Nowadays, the mainstream way for human movement intention detection is mainly based on bioelectrical signals or force and position information. For bioelectrical signals, electroencephalogram (EEG) and electromyography (EMG) are especially favored in further analyzing the human movement intention. As for the force and position information methods, since it can only be obtained after limb movement, its shortage in time delay is inevitable. Furthermore, the cost time of information processing and electromechanical system response further reduce real-time performance (Yi et al., 2015). Hence, several efforts have been made to apply EEG and EMG for robotic control, since it can solve this problem very well.
Due to the advantages of EEG and EMG signals, those signals are used to decode the different human movement information. Thirty-two EEG features from 64-channel EEG signals were used to decode multiple classes of different upper limb movements, the average accuracy was above 90%, this is due to the optimal feature set combination which would yield a relatively high decoding accuracy (Samuel et al., 2017). EMG signals contain motor/neural information from which limb movement intent could be identified, and it can recognize more kinds of movement patterns (Samuel et al., 2019). But with the limitation of current technologies, the decoding performance is not very satisfactory under certain conditions. For example, when only using EEG, it is difficult to achieve a satisfactory accuracy, and when only using EMG it is hard to guarantee the stability of recognition. Rui et al. classified the facial action to control a prosthesis, and the results showed that the performance of EMG-based control is better than EEG-based control (Rui et al., 2018b; Xiaodong et al., 2020). Therefore, the fusion method of EEG and EMG signals emerge as the times require, it can improve decoding performance and stability (Tejedor et al., 2019). Leeb et al. discussed the decision fusion of EEG and EMG, and the results showed that different proportion of EMG and EEG fusion had a better and more stable performance (Leeb et al., 2011). Li et al. applied EEG and EMG electrode arrangements using the Sequential Forward Selection (SFS) algorithm, this study demonstrated the feasibility of fusing EMG and EEG signals toward improving motion classification accuracy (Xiangxin et al., 2017). Hazarika presented the real-time implementation of a feature fusion-based learning algorithm using multi-domain discriminant correlation analysis (MDCA), the results demonstrated the effectiveness and reliability of the fusion of EEG and EMG (Hazarika et al., 2019).
However, there are some deficiencies in the current methods of EEG and EEG fusion. Although information of EEG and EMG is fused, its contents are inconsistent. Such inconsistency results from the difference in the response time of EEG and EMG, which leads to the differences in the contents of EEG and EMG signals collected at the same time. Therefore, the performance improvement of EEG and EMG fusion based detection is limited. Only few researchers devoted to explore the mechanism of EEG and EMG for lower limb voluntary movement intention, which can clarify the internal correlation of EEG and EMG and provide a theoretical support for designing a fusion method of EEG and EMG. Hence, this paper presents a numerical simulation and homology analysis of EEG and EMG with lower limb voluntary movement intention. We hypothesized that EEG and EMG signals are homologous and the EEG signal response is earlier than the EMG signal response. In this paper, a mathematical model of EEG and EMG to simulate the EEG and EMG signals and analyze the homology is presented and validated. The organization of this work is as follows. Section Methodologies, including the brain cognitive model with multiple input-multiple output, the mechanism of EEG and EMG and analysis method for homology characteristics of EEG and EMG. Section Numerical Simulation of EEG and EMG for Lower Limb Voluntary Movement Intention. Section Experimental Verification for Homology Characteristic of EEG and EMG, which describes the simulation and experimental results as well as corresponding homology analysis. Sections Discussion and Conclusion, respectively.
Methodologies
Brain Cognitive Model With Multiple-Input Multiple-Output
The human brain is the center where people receive information, process it, and make decisions, so constant efforts are committed to explore and analyze the human brain to better serve human beings. Since Hans Berger recorded EEG signals for the first time in 1924, many scholars have carried out a lot of research work on the establishment of the brain model, EEG signals acquisition, EEG information decoding, and brain computer interfaces (Xiaodong et al., 2014). Therefore, the inevitable problem has arisen in brain control technology of how to establish a universal brain cognitive model, develop a corresponding neural control model, and effectively apply them to robotic interactive controls. In this paper, a brain cognitive model with multiple-input multiple-output was proposed, in which the brain receives outside information and generates a response so that the relative neural pathways are set up as shown in Figure 1 (Xiaodong, 2018).
The brain cognitive model with multiple-input multiple-output consists of an upstream pathway and downstream pathway. The upstream pathway is the revelation of the mechanism of EEG, from sensory information to the cerebral cortex. The upstream pathway shows that the environmental physical information comes from the human visual sense, auditory sense, tactile sense, and other senses, and it will be processed in the brain with the existing intelligent knowledge so that some brain response is excited. The EEG signals will be acquired in the cerebral cortex, where the brain response transmits upward. The downstream pathway is the revelation of the mechanism of EMG, from the brain response to the neural signal, and then to physical expression. The downstream pathway shows that the control signals from brain are transmitted to the limb through the nervous system, after that the external expression of EMG response and limb movement is generated. This model systematically expounds the source of EEG and EMG and reveals the internal relationship between them.
Mechanism of EEG and EMG for Lower Limb Voluntary Movement Intention
On the basis of the proposed brain cognitive model, the mechanism of EEG and EMG of lower limb voluntary movement intention was refined. Human limb movement can be divided into evoked and voluntary movement. Evoked movement is caused by environmental stimulation, which is a response to meet the environmental needs. Voluntary movement is generated by the human's intention to achieve an intended purpose (Waszak et al., 2005; Herwig et al., 2007). The neural pathway of evoked and voluntary movement was shown in Figure 2.
In voluntary movement, the pre-motor cortex and anterior cingulate cortex play important roles in the evocation of movement (Mueller et al., 2007). Regarding evoked movement, the ventral/dorsal pre-motor cortex and posterior parietal cortex are crucial (Obhi and Haggard, 2004; Waszak et al., 2005). The information of evoked and voluntary movement is processed by pre-motor cortex. Processed information will be transmitted to limbs through the peripheral nervous system, causing limb movement. Neuron activity in the motor cortex generates an EEG response including a negative low-frequency potential, which is defined as readiness potential (RP). RP is an ERP (event-related potential), and also is important component of MRCP (movement-related cortical potentials) (Verleger, 2003), it is a remarkable spatiotemporal feature that can be used to analyze the homology and decode movement intention.
Analysis Method for Homology Characteristics of EEG and EMG
The coherence analysis of EEG and EMG is an effective method for homology analysis between two signals representing brain and muscles, respectively. The wavelet coherence gives full play to transform characteristics in the time–frequency domain, which is an important method for processing nonstationary signals (Xugang et al., 2020). In this work, signal sequence x (t) and y (t) denotes EEG and EMG signals respectively. Wx (a, b) denotes the wavelet coefficients, it is obtained by convoluting the scaled wavelet function with x (t).
Where a is the wavelet scale, b is the smoothing parameter, * is the conjugate, ψ[ (t–b)/a] is the wavelet basis function.
|Wxy (a, b)| is the cross-wavelet trans-formation of x (t) and y (t). So, the wavelet cross-spectrum of the EEG signals x (t) and the EMG signals y (t) can be expressed as follows:
The wavelet spectrum needs to be smoothened to calculate the synchronization information of EEG and EMG signals, it is defined as follows:
Where Sa and St are the smoothing operation on the scale axis and time axis, respectively.
Where, c1 and c2 are the normalization coefficients, and ∏ is a matrix function.
The absolute value of the normalized smooth cross-wavelet spectrum is wavelet coherence (Lachaux et al., 2002). Thus, the wavelet coherence coefficient of x (t) and y (t) is as follows:
The wavelet coherence coefficient is in the range of 0–1, the greater the value, the stronger the coherence.
Numerical Simulation of EEG and EMG for Lower Limb Voluntary Movement Intention
Mathematical Model of EEG and EMG
The mechanisms of EEG and EMG signals during lower limb voluntary movement intention provide the theoretical basis for robot control system (Hanzhe et al., 2019b), it can be seen that when humans generate movement intention, it is first processed in the brain to generate EEG signals, and then transmitted to limbs by neural pathway, which cause limb movement to generate surface EMG (sEMG) signals through the peripheral nervous system. So, the mathematical model of EEG and EMG was proposed, the framework of the mathematical model was shown in Figure 3. The mathematical model consists of neural mass model (NMM), neuromuscular junction model, EMG generation model, decoding model, and musculoskeletal biomechanical model. In addition to its ability to systematically simulate EEG and EMG signals, the model can also determine the joint torque and movement intention.
In this paper, P (t) was defined as the input of movement intention. The P (t) will transmit to the motor cortex to generate EEG signals including RP response and the EEG signals simulated by NMM, the movement intention was recognized by the decoding model. Meanwhile, it also transmits to skeletal muscle to generate EMG through the nervous system and neuromuscular junction, the EMG signals are simulated by the neuromuscular junction model and the EMG generation model. Then, the joint torque caused by the joint's multiple muscles is decoded by the musculoskeletal biomechanical model. In this work, the nervous system is equivalent to a delay system, it only has a function of signal transmission and ignores the signal conversion function of the nervous system.
Overall, both EEG and EMG responses are triggered by the same movement intention P (t) during lower limb movement. It provides theoretical guidance for fusion of EEG and EMG.
Mathematical Model of EEG
Due to the complexity of EEG signals, modeling brain activity for the cortex can provide a theory of interpretation for how information generates and is transmitted in the motor cortex for different brain tasks (Rui et al., 2018a). Various mathematical models of brain activities could be subdivided into two major classes: “detailed model” and “neural mass model.” NMM is the dynamic of neural populations at a macroscopic level. To better understand the mechanism of EEG responses, a schematic diagram of NMM was shown in Figure 3. The RP responses were included in the proposed model construction in our previous work (Hanzhe et al., 2019a).
The stimulation of EEG responses can be obtained using coupled NMM based on the Jansen model (Jansen and Rit, 1995). The basic component of this model is the “cortical column” function (Zavaglia et al., 2006). In NMM, there are four neural groups: pyramidal cells, excitatory interneurons, slow inhibitory interneurons, and fast inhibitory interneurons. This model can simulate the EEG signals of the motor cortex in response to limb movement. The impulse responses of excitatory [he (t)] and inhibitory [hi (t)] cells are as follows:
Where A, B and a, b represent the average gain and time constant of excitatory and inhibitory cells, respectively. he = 3.25 mV, hi = 22 mV.
The average postsynaptic membrane potential are the EEG signals that can be collected and observed. This conversion function z (t) is defined as follows:
Where e0 is the maximum firing rate of neural population, r denotes the steepness of z (t), v0 is the potential synapses relative to e0, and v represents the input average postsynaptic membrane potential, s0 = 6 mV, e0 = 2.5 s−1, r = 0.56 mV−1.
The second-order differential equations were employed to combine he (t) and hi (t). Differential equation groups used to calculate are as follows:
Pyramidal neurons:
Excitatory interneurons:
Slow inhibitory interneurons:
Fast inhibitory interneurons:
Where, the vi is the average membrane potentials for the four groups (i = 0, 1, 2, 3). The zi (i = 0, 1, 2, 3) is the input for the sigmoid function which was fired by the neurons. A, B, and G are the average gain for the excitatory, slow inhibitory, and fast inhibitory synapses, respectively; a1, b1, and g1 are the time constant of the excitatory, slow inhibitory, and fast inhibitory synapses, respectively. The yi (i = 0, 1, 2, 3) are outputs of these equations, which represent the postsynaptic membrane potentials of excitatory, slow inhibitory, or fast inhibitory, respectively. Ci represents the connectivity constants of those neurons for interactions among neurons. The P (t) is the input of model from motor or other cortex, which is represented by a Gaussian noise with assigned mean value (m) and variance (σ2) for movement intention.
A single NMM can simulate a unimodal spectrum for EEG response, its position and bandwidth can be adjusted appropriately. However, this model cannot simulate the complexity of EEG in the motor cortical area. Therefore, the model of the cortex is composed by multiple single-models deployed in parallel. Commonly, the coupling of three single-models is sufficient to simulate the motor task of cortical activity. Then, these models will be indicated with the symbols L, M, and H to represent rhythms at low, medium, and high frequencies of EEG, respectively.
Neuromuscular Junction Model
The neuromuscular junction is the contact point between motor nerve and skeletal muscle. The information conversion is achieved through the release of acetylcholine (ACh), and acetylcholine receptors (AChR) will be activated by it (Webster, 2018). Activated AChR will activate related ion channels to generate action potential leading to muscle contraction. In the past, the use of animal models has elucidated its formation and function by the cellular electrophysiology and patch clamp techniques. So, the circuit model of neuromuscular junction was established, the model structure was shown in Figure 3. This model simulates the current change of four kinds of ion flow and the change of membrane potential, it is a two-node circuit with multiple parallel branches (Guan et al., 2013). According to Ohm's law of active circuits, the current of each branch is as follows.
Where Vm is the membrane potential. Ii, Ei, and Ri is the ionic current, equilibrium potentials and resistance of sodium (Na) ions branch, potassium (K) ions branch, chloride (Cl) ions branch and calcium (Ca) ions branch, respectively.
According to the constant field hypothesis proposed by Goldman, Hodgkin, and Katz, the algebraic sum of the branch current of each branch is zero at resting while ignoring the effect of the pump current. The resting potential of the membrane remains constant (Hodgkin and Huxley, 1952).
Membrane potential (Vm) can be obtained by combining Equations (12) and (13).
Where gi is the conductance of each ions branch.
According to the Nernst formula, the calculation of ionic equilibrium potential is as follows.
Where F is the Faraday constant, R is the gas constant, T is the absolute temperature, and z is the changing number of electrons in the electrode reaction. CO and CI are the ion concentration outside and inside the membrane, respectively.
According to the physiological parameters of the human, the equilibrium potential of each ion branch can be obtained.
Mathematical Model of EMG
Human skeletal muscle is composed of a large number of muscle fibers (Wang et al., 2006; Jiangcheng et al., 2016). The end-plate potential is generated in the end-plate area of muscle fibers excited by the impulses of the neuromuscular junction, the action potential of a single fiber (Vf) on muscle fiber membrane caused by end-plate potential. The action potential of the motor unit is generated by the action potential of all the muscle fibers in the motor unit, and the action potential sequence of the motor unit (Vmu) is formed continuously in the time domain. Finally, the action potential sequences of different motor units are superimposed on the skin to form EMG signals (Vs). According to the physiological process of EMG and its physiological basic structure, the EMG generation model was established, as shown in Figure 3. The simplified equation of the model is as follows.
Where ei (z) is the intracellular action potential of a single fiber along the z direction; s is the cross-sectional area of muscle fiber; σi is intracellular conductivity; σm is muscle conductivity; σx is muscle radial conductivity; σz is fiber direction conductivity; v is the conduction velocity of action potential; r is distance between fiber's cross section and observation point. Tst,i is the burst time of action potential.
Equation (17) established the EMG signals model of the observation point. It can be seen that EMG signals are determined by two factors: the recruiting number of muscle fiber in motor units and firing rate of motor unit action potential.
Musculoskeletal Biomechanical Model and Decoding Model
The musculoskeletal model can obtain the joint torque from the original surface EMG signals (Jiangcheng et al., 2017). This model includes the following three sub models, which is muscle bioelectrical activation model, muscle contraction mechanical model, and muscular function model in the lower limb motor system. The first one uses EMG to characterize the degree of muscle activation a (t). The second model converts activation a (t) to output muscle force F (t). The third model converts muscle force F (t) to joint torque output TJ. Those three models are formed according to the mechanical relationship. The output joint torque can be used in the interactive control of the robot.
Where, u (t) is the sEMG signal, A is the non-linear shape conversion factor, FCE and FPE are the muscle-force of contractile element (CE) and parallel elastic element (PE) in Hill model. fl (lm) is force-length equation, fv (vm) is force-velocity equation, fp (lm) is parallel elastic force-length equation, F0 is maximum muscle contraction force, ϕ is pinnation angle, ri is the moment arm of the ith muscle, Fi is the muscle force of the ith muscle.
The decoding model for EEG signals is a classifier which is based on artificial neural network (ANN) and deep learning, it provides a good performance in classifying inconspicuous feature sets. This model is an important way for humans to translate EEG signals into commands, it is also the research focus. Many scholars are committed to the study of excellent methods to improve the performance of this model, many classification methods have been presented in the numerous published EEG-based BCI articles (Rashid et al., 2020).
Simulation Analysis
Simulation Analysis of EEG
A reference value of the parameters in the coupled NMM model have been given to simulate different bands of EEG signals. These values are reported in Table 1.
The theoretical results were obtained from simulation. Under the assumed conditions, the parameters of the activation function P (t) were defined as movement intention. In order to compare the difference between the movement intention state and the rest state, the activation function was defined as changing with the simulation time.
Figure 4A shows the simulation results of EEG signals, and Figure 4B shows an example of an RP stimulus. Clearly, the sharp drop and rapid recovery of low frequency potential is affected by the movement intention [P (t)]. The results also suggested that the coupled NMM could interpret the mechanism of the EEG including RP responses.
Figure 4. Simulation results of EEG. (A) The simulation results of the EEG signals, (B) The RP response from stimulation.
Simulation Analysis of Neuromuscular Junction Model
However, it is difficult to measure the resistance value of each ion branch in the neuromuscular junction model. The membrane potential change of the neuromuscular junction is caused by the difference in ion concentration between the inside and outside of the neuromuscular junction. Therefore, this characteristic is used to derive its voltage or current equation, Goldman, Hodgkin, and Katz deduced the GHK voltage equation based on Nernst-Plank equation. The membrane potential from GHK equation is as follows:
Where, F is the Faraday constant, R is the gas constant, T is the absolute temperature, Pi is the permeability constant of each ions, [i]O, [i]I are the ion concentrations outside and inside the membrane, respectively.
It can be seen from the formula that the membrane potential depends on the transient ion concentration, but it is different to measure the value. At present, it is usually measured by patch clamp recording technique technology, and the I-V curve of the ion branch was obtained (Guan et al., 2013). Although this is a voltage-gated current curve, it can reflect the characteristics of the ion branch. In this work, the resistance value of each ion branch is calculated according to its I-V curve, and the activation function of this model was defined as a trigonometric function similar to the action potential received by the neuromuscular junction, whose shape is shown in Figure 5A.
Figure 5. Simulation of the neuromuscular junction model. (A) The activation function of the model; (B) Simulation result of this model.
The simulation result is shown in Figure 5B, it can be seen that the membrane potential of the neuromuscular junction had a considerable consistency with the real end plate potential. Furthermore, a significant delay in response time was found in the simulation result. Although this change is due to the charge and discharge of membrane capacitance in this model, it still reflects a time required for neuromuscular junction to convert the information. Even though the accurate cost time of conversion cannot be obtained from this model, a qualitative analysis has a certain value. This result suggested that the control signal from the brain and nervous system through neuromuscular junction received by the muscle fiber is further delayed.
Simulation Analysis of EMG
According to the physiological knowledge and experience, the relevant parameters of the model were given. The cross-sectional area of muscle fiber s was 2,300 μm2, the total number of muscle fibers Nf can be calculated as 127,235 by the distribution density of muscle fibers, when muscle diameter was set as 45 mm. The firing frequency of the action potential fr,i follows the Poisson distribution between 8 and 50 Hz, with an average value of 12 Hz. The sampling frequency was set as 2 KHz. The simulation results and its spectrum analysis from this model was obtained, as in Figure 6.
Figure 6. Simulation result of EMG. (A) EMG simulation signals, (B) Spectrum analysis of EMG simulation signals.
The results shown in Figure 6 indicated that the amplitude of EMG signals was between 0 and 10 mv, and its frequency distribution range is mainly between 0 and 400 Hz. This result was consistent with measured sEMG signals, which further proved the correctness of this simulation model.
Each model has well-simulated the EEG signals, conversion process of neural signals, and EMG signals. These simulation results explain the characteristics of EEG and EMG signals very well; they also explained the internal relationship between EEG and EMG signals at the same time. In the simulation of this work, the musculoskeletal biomechanical model was ignored, because this model is a signal decoding function. In general, the simulation of the model also proves the difference in the response time between EEG and EMG. The results indicated that the response of EEG signals is earlier than EMG signals, which proves the proposed theory in this paper.
Simulation Analysis for Homology Characteristic of EEG and EMG
Because the connection characteristics of each functional model in this mathematical model are imperfect, the delay time caused by transfer and conversion is especially unclear. Therefore, different response times of EMG were set to observe the coherence of EEG and EMG. Due to the properties of simulation and real EEG and EMG signals, the simulation EMG signals were normalized, and EMG signals of resting state were added before and after task state. The EMG signal response time (movement onset) was set to three cases, which was set at the early, middle and late stages of the EEG signal response based on an exhaustive method. So, the EMG response onset time was set at 2 5 and 7 s, respectively, and the duration was 3 s. The wavelet coherence of simulation EEG and EMG signals were obtained, as shown in Figure 7.
Figure 7. The Wavelet coherence of the simulation of EEG and EMG. (A) EEG-EMG coherence at early EMG response, (B) EEG-EMG coherence at middle EMG response, (C) EEG-EMG coherence at late EMG response.
The yellow areas in Figure 7 represented the strong simulation of EEG-EMG coherence. It can be seen that the frequency band and time of strong coherence in those figures were different with different EMG response times. Especially for the early and late EMG response (seen in Figures 7A,B, respectively), the time of strong coherence was not consistent with the response time of EMG signals. But for Figure 7B with the EMG response onset at the middle of the EEG signal response, there was a strong coherence at 1 Hz after the EMG response onset (from 5 s to 7 s). This result accords with theory and practice in terms of response time and frequency band, this result indicated that the correct setting of EMG response onset time can obtain a correct simulation result, and response of EEG signals is earlier than that of EMG signals. It also can be seen that the simulation phase of the EEG signals was leading the EMG phase at 1 Hz in Figure 7. The simulation results will be verified with the experimental results, and then used to analyze and verify the time difference of EEG and EMG response. The coherence and phase of simulation EEG and EMG signals proved the homology and different response times of simulation EEG and EMG signals, which was consistent with the theoretical analysis and proposed hypothesis.
Experimental Verification for Homology Characteristic of EEG and EMG
Experimental System Overview
In this work, the EEG and EMG signals of lower limb voluntary movement intention acquisition system comprised an EEG and EMG acquisition device and signals processing equipment. The EEG and EMG data were acquired using NeuroScan NuAmps. A PC with Intel (R) Core (TM) i5-5600 CPU was used to process EEG and EMG signals.
A diagram of the EEG and EMG acquisition system is shown in Figure 8. When the system was working, the subject performed the voluntary movement. At the same time NeuroScan-NuAmps recorded the EEG from the subject's motor cortex and EMG data were recorded with same equipment from subject's tibialis anterior. All the recorded data were transferred to the PC by USB. The computer processed the EEG and EMG data with special algorithms.
Subjects and EEG and EMG Data Recording
Six healthy subjects participated in this experiment (five males and one female, aged 24–28, mean 23.2 years). None of them has limb dysfunction or any known cognitive deficits. After explaining the nature and possible consequences of the experiment, all the subjects signed the informed consent. The Institutional Review Board of Xi'an Jiaotong University approved the proposed experiments and all experiments were conducted in accordance with the declaration of Helsinki.
EEG and EMG data were acquired using NeuroScan-NuAmps with sampling rate at 1,000 Hz. The system had 36 EEG channels. The channel distribution was in accordance with the international 10/20 system. Channels FCz, FC1, FC2, Cz, C1, C2, CP1, and CP2 were selected. Channels AFz and CPz were used as references. The raw EEG signals were first filtered by a 0.5–45 Hz band pass filter. 2 EMG channels from tibialis anterior were selected to offer EMG data detection. The raw EMG data was filtered by a 20–200 Hz band pass filter and a 50 Hz notch filter.
Experimental Procedure
This experiment was conducted to verify the interconnection of EEG and EMG. Each subject completed experiments on several separate days. The subjects were instructed to be in a standing position. In this experiment, the subjects should keep the resting state to avoid unnecessary movement. The subjects were asked to perform the lower limb voluntary movement tasks. There were two parts of voluntary movement tasks, including the left and right leg voluntary movement. Each part consisted of 15 sessions. In each session, all subjects repeated the same lower limb voluntary movement five times. The experiment was conducted in a sequence of intervals between breaks and voluntary movements. Each break time exceeds 10 s, the rest time between each two sessions was decided by the subjects themselves. Figure 9 is the time series for this experiment.
Coherence Analyses for Homology Characteristics of EEG and EMG
The preprocessed EEG signals and the EMG signals were computed to obtain the EEG–EMG coherence curves under the two actions of right leg and left leg voluntary movement. The onset trigger of lower limb voluntary movement was taken as the data center. So, the EEG and EMG data were segmented from continuous data streams. Under this definition, EEG data were segmented to complete the event alignment. The preprocessed EEG and EMG was shown in Figure 10. At the same time, the results were computed after ERP analysis of EEG signals (as shown in Figure 10B), it validated that there had been a significant RP response in the motor cortex (representative channel FC1) during right leg voluntary movement. There are potential changes in accordance with the characteristics of readiness potential at the onset of lower limb voluntary movement. This analysis validated that the experimental results were consistent with the simulation results, which proves that there is a RP response in the EEG of the motor cortex before the onset of lower limb voluntary movement.
Figure 10. Preprocessed EEG and EMG signals from right leg movement. (A) Preprocessed EEG (representative channel FC1) and EMG signals, (B) Single and averaged RP of FC1.
On the basis of the above methods and data, EEG–EMG wavelet coherence can be obtained by selecting the tibialis anterior that dominate the lower limb voluntary movements. The wavelet coherence of EEG and EMG signals from one representative subject (S1) were obtained, as shown in Figure 11.
Figure 11. The wavelet coherence of EEG and EMG from representative subject (S1). (A) EEG (FC1) and EMG signals from right leg movement, (B) EEG (C4) and EMG signals from left leg movement.
The yellow areas in this Figure 11 represented the strong EEG-EMG coherence at 1 Hz, in which the values are substantially increased relative to other frequencies. In Figure 11A, there was a strong coherence in the low frequency after right leg voluntary movement (from 0 to 2 s). At the same time, there also was a strong correlation at 1 Hz at the left leg voluntary movement (as shown in Figure 11B). This result suggested that the EEG and EMG signals are homologous, they were both the expression of lower limb voluntary movement due to the strong correlation at the onset of movement. It also can be seen that the phase of the EEG was leading the EMG phase at 1 Hz, while there was no obvious regularity in the distribution of coherence and phase difference in other frequency bands. Accordingly, this result was consistent with the above theoretical analysis and simulation results, which proved that the response of EEG signals was leading the EMG signals, and had a consistency and universality in the lower limb voluntary movement. Meanwhile, the experimental results also proved the correctness of the simulation results. There is a strong correlation at 1 Hz because there is a low-frequency MRCP in EEG signals response, and the EMG signals respond at this time. Although the low-frequency part of the EMG was filtered in the preprocessing process, the envelope of its amplitude shows a low-frequency characteristic, thus it shows a strong coherence at this frequency and at the movement onset.
The results of EEG and EMG coherence analysis of other subjects were shown in Figure 12. These results showed that there was a strong coherence between EEG and EMG at about 1 Hz during lower limb voluntary movement onset. The arrows pointing indicate the phase difference between the EEG and EMG, the distribution and direction of these arrows are basically consistent with the above results. It suggested that the results and conclusions of the above analysis were universal.
Figure 12. The wavelet coherence of EEG and EMG from other subjects. (A) Wavelet coherence of EEG (C1) and EMG signals from S2, (B) Wavelet coherence of EEG (C1) and EMG signals from S3, (C) Wavelet coherence of EEG (CP1) and EMG signals from S4, (D) Wavelet coherence of EEG (FC2) and EMG signals from S5.
Discussion
In this work, the mathematical model of EEG and EMG was presented and used to explain the homology characteristic between EEG and EMG signals for lower limb voluntary movement.
The Significance of the Mathematical Model
With regards to the EEG model, it verified that the motor cortices are responsible for lower limb voluntary movement. As is known to all, the processing of lower limb voluntary movement in the cortical regions of brain is complicated, requiring different information sources and presentation. However, there is no doubt that the motor cortex provides critical contributions to lower limb voluntary movement processing. The EEG model can simulate the EEG signals in the motor cortex when a human generates a lower limb voluntary movement intention. Compared with our previous modeling (Hanzhe et al., 2019a), the coupling model can better explain the mechanism of the RP responses. Despite its relative simplicity, this mathematic model can explain the response mechanism of RP responses to a lower limb voluntary movement intention.
The neuromuscular junction is an important bridge between nerves and muscles, which realizes the transformation from electrical signals transmission to chemical transmission and then to electrical signals transmission. Under the stimulation of nerve action potential, the release and flow of chemical transmitters and ions in neuromuscular junctions causes changes of its current and membrane potential. Based on its working mechanism, the neuromuscular junction model is proposed, which can simulate the changes of the current and endplate potential.
The EMG generation mode received the control signal from the neuromuscular junction, which activated the skeletal muscle of the lower limb to generate an EMG signal. The simulation result showed the time and frequency characteristics of signals, and it can guide the subsequent signal processing and feature extraction.
Importantly, most of the existing EEG and EMG-based control technologies have some shortcomings, because they are based on empirical evidence or experimental results. Thus, a mathematic modeling can further understand the physiological mechanisms for the responses of EEG and EMG behavior (Rui et al., 2018a). Physiological analyses and model simulations of neural pathways of EEG and EMG show that there are significant differences in the generation and transduction pathway of EEG and EMG, but the EEG and EMG signals were homologous. The EEG signal response times are earlier than EMG signals. The results presented in this work were investigated for the feasibility of EEG-based pre-perception and fusion perception of EEG and EMG for human movement.
The Homology of EEG and EMG Signals
With regards to the homology of EEG and EMG signals, the simulation and experimental results show a high consistency and the strong coherence appears at about 1 Hz during lower limb voluntary movement onset. Compared with the previous research results, the frequency band of the strong coherence of EEG and EMG is significantly different, in which the significant EEG-EMG coherence was observed broadly within 10–30 Hz (Mima et al., 2000a). Since the beta band plays an important role in information processing, it sees the cortico-muscular information flow in this frequency range. The reason for this difference is that this work focused on the readiness potential at low-frequency. Although the there was a divergence, the same conclusion is that the motor control command is from the cortex to the muscle. This conclusion agrees with our previous hypothesis that the EEG and EMG signals are homologous. What is more, the cortical-muscular coherence concerns the time lag between EEG and EMG, the results show that the EEG precedes and drives the synchronization of motor unit activity (Mima et al., 2000b). The phase of EEG-EMG coherence analysis means that the cortical oscillation leads the EMG rhythm, this result proves our previous hypothesis that the EEG response is earlier than EMG response.
Although the work has proved the homology and different response times of EEG and EMG signals, there are still some concerns. The EEG data were acquired for motor cortex, but the EMG data were only collected from the tibialis anterior, if the position of the EMG acquisition is changed, the result may change. What is more, the individual difference is an important factor to the result. It can be seen that the results have good consistency from this work, but it is still clear that the differences of some results are caused by individual differences.
The Limitations of the Study and Further Work
Despite the significance of this work, several limitations should be considered. One of the limitations is that the proposed model was simplified, especially in the part of signal transmission and conversion from brain to muscle. It is known that the transmission and conversion of human nerve signals is very complicated, it involves multiple human organs and neural pathways. The purpose of this paper is to explore the intrinsic relationship between EEG and EMG signals of lower limb voluntary movement, and to provide theoretical support for the decoding and fusion decoding of EEG and EMG. Therefore, the functions of the low-level central nervous system and peripheral nervous system were simplified into a time-delay system with only nerve signal transmission functions. This work was based on this hypothesis to simulate and analyze EMG signals. From the overall perspective of the proposed model, when producing lower limb voluntary movement intention, it will stimulate the relevant regions of the brain to generate responses and EEG signals; at the same time, it will control the muscles through the transmission and conversion of the nervous system and neuromuscular junctions and generate EMG signals. Besides, the closeness between simulated EEG/EMG and real EEG/EMG signals is a lack of quantitative evaluation. In this work, it was the qualitative evaluation based on observation. For the EEG signals, it focused on the RP. While the simulated EEG and real EEG had the same trend of RP, it is considered that they have a good similarity. For the EMG signal, it mainly evaluated the closeness based on the spectrum distribution of EMG signals. In the following work, the non-linear correlation and time-frequency parameters of EEG/EMG signals will be employed for quantitative evaluation.
Another limitation of this study is that only six subjects participated in this experiment. Even though the number of subjects is not enough, it is sufficient to prove the correctness of the proposed viewpoint and highlight some significant experimental results. There is no doubt that the main motivation for this work is to provide a theoretical support for the fusion of EEG and EMG, which improve the movement intention recognition performance. Besides, there was a gender imbalance among subjects. Further study will involve more subjects testing the proposed hypothesis and result. Moreover, significant differences were not found between male and female subjects in this work. Thus, a large sample size and a balance between male and female subjects are desired to further evaluate the correctness of the proposed hypothesis and results. In future works, the specific time difference between EEG and EMG signals will be studied to take advantage of the two signals to improve the performance of human movement intention detection, so as to realize the interactive control applied in robot system.
Conclusion
In this work, a numerical simulation and homology analysis of EEG and EMG from lower limb voluntary movement intention was conducted. The mathematical model of EEG and EMG from lower limb voluntary movement intention was proposed based on the electrophysiological analysis, which revealed the mechanism of EEG and EMG signals and the homology of those two signals. This model systematically reveals the generation and transmission mechanism of EEG and EMG signals. Especially, the internal relationship between EEG and EMG has a clear exposition, from which the difference in response time has been shown from the transmission pathway in this model. The simulation results of the neuromuscular junction model prove that the conversion of the neural signal takes a certain time. Meanwhile, the homology analysis results of the simulation signals show that the correct setting of the EMG response onset time can obtain a correct simulation result. In the experiment, this work synchronously collected the EEG and EMG signals of lower limb voluntary movement, and the experimental results are consistent with the hypothesis and simulation. Overall, both simulation and experimental results demonstrate the homology of EEG and EMG, and that the EEG signals response times are earlier than those of EMG signals. This work can provide a basis for the feasibility of EEG-based pre-perception and fusion perception of EEG and EMG in human movement intention detection.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by Institutional Review Board of Xi'an Jiaotong University. The patients/participants provided their written informed consent to participate in this study.
Author Contributions
XZ proposed the research idea and supervised the work. HL did the research and wrote the manuscript. ZL organized and carried out the experiments. GY revised the manuscript. All authors contributed to the article and approved the submitted version.
Funding
This research work was supported by National Key R&D Program of China (No. 2017YFB1300505) and the Key R&D project of Shaanxi (No. 2019ZDLGY14-09), respectively.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Al-Quraishi, M., Elamvazuthi, I., Daud, S., Parasuraman, S., and Borboni, A. (2018). EEG-based control for upper and lower limb exoskeletons and prostheses: a systematic review. Sensors 18, 1–27. doi: 10.3390/s18103342
Chen, B., Ma, H., Qin, L., Gao, F., Chan, K., Law, S., et al. (2016). Recent developments and challenges of lower extremity exoskeletons. J. Orthop. Transl. 5, 26–37. doi: 10.1016/j.jot.2015.09.007
Guan, B., Zhang, H., and Li, Z. (2013). Basic Principles of Cellular Electrophysiology and Patch Clamp Techniques. Beijing: Science Press.
Hanzhe, L., Xiaodong, Z., Rui, L., Zhufeng, L., Kuncai, Y., and Gui, Y. (2019a). A preperception method for voluntary movement intention of lower limb using readiness potential. J. Xi'an Jiaotong Univer. 53, 16–23. doi: 10.7652/xjtuxb201910003
Hanzhe, L., Xiaodong, Z., Zhufeng, L., and Rui, L. (2019b). “Detection of Readiness Potential for Lower Limb Voluntary Movement Based on EMG Onset”, in The 9th IEEE International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (IEEE-CYBER 2019) (Suzhou).
Hazarika, A., Barman, P., Talukdar, C., Dutta, L., Subasi, A., and Bhuyan, M. (2019). Real-time implementation of a multidomain feature fusion model using inherently available large sensor data. IEEE Trans. Industr. Inform. 15, 6231–6239. doi: 10.1109/tii.2019.2914975
Herwig, A., Prinz, W., and Waszak, F. (2007). Two modes of sensorimotor integration in intention-based and stimulus-based actions. Q. J. Exp. Psych. 60, 1540–1554. doi: 10.1080/17470210601119134
Hodgkin, A., and Huxley, A. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. London 117, 500–544. doi: 10.1113/jphysiol.1952.sp004764
Jansen, B. H., and Rit, V. G. (1995). Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns. Biol. Cyber. 73, 357–366. doi: 10.1007/BF00199471
Jiangcheng, C., Xiaodong, Z., and Gui, Y. (2016). Human gait events fast recognition method via surface electromyography. China Mechan. Eng. 27, 911–916+924. doi: 10.3969/j.issn.1004-132X.2016.07.011
Jiangcheng, C., Xiaodong, Z., Linxia, G., and Carl, N. (2017). Estimating muscle forces and knee joint torque using surface electromyography: a musculoskeletal biomechanical model. J. Mech. Med. Biol. 17, 1–21. doi: 10.1142/s0219519417500695
Lachaux, J., Lutz, A., Rudrauf, D., Cosmelli, D., Le Van Quyen, M., Martinerie, J., et al. (2002). Estimating the time-course of coherence between single-trial brain signals: an introduction to wavelet coherence. Neurophys. Clin. Clin. Neurophys. 32, 157–174. doi: 10.1016/S0987-7053(02)00301-5
Leeb, R., Sagha, H., Chavarriaga, R., and Millan Jdel, R. (2011). A hybrid brain-computer interface based on the fusion of electroencephalographic and electromyographic activities. J. Neural. Eng. 8:025011. doi: 10.1088/1741-2560/8/2/025011
Mima, T., Matsuoka, T., and Hallett, M. (2000a). Information flow from the sensorimotor cortex to muscle in humans. Clin. Neurophys. 112, 122–126. doi: 10.1016/S1388-2457(00)00515-0
Mima, T., Steger, J., Schulman, A. E., Gerloff, C., and Hallett, M. (2000b). Electroencephalographic measurement of motor cortex control of muscle activity in humans. Clin. Neurophys. 111, 326–337. doi: 10.1016/S1388-2457(99)00229-1
Mueller, V. A., Brass, M., Waszak, F., and Prinz, W. (2007). The role of the preSMA and the rostral cingulate zone in internally selected actions. Neuroimage 37, 1354–1361. doi: 10.1016/j.neuroimage.2007.06.018
Obhi, S. S., and Haggard, P. (2004). Internally generated and externally triggered actions are physically distinct and independently controlled. Exp. Brain Res. 156, 518–523. doi: 10.1007/s00221-004-1911-4
Qiuzhi, S., Xiaoguang, W., Xin, W., and Yang, W. (2016). Development of Multi-joint Exoskeleton-assisted Robot and Its Key Technology Analysis: an Overview. Acta Armam. 37, 172–185. doi: 10.3969/j.issn.1000-1093.2016.01.025
Rashid, M., Sulaiman, N., Majeed, A. P. P. A., Musa, R. M., Ab Nasir, A. F., Bar, B. S., et al. (2020). Current status, challenges, and possible solutions of EEG-based brain-computer interface: a comprehensive review. Front. Neurorobot. 14:25. doi: 10.3389/fnbot.2020.00025
Rui, L., Xiaodong, Z., Hanzhe, L., Liming, Z., Zhufeng, L., and Jiangcheng, C. (2018a). An approach for brain-controlled prostheses based on scene graph steady-state visual evoked potentials. Brain Res. 1692, 142–153. doi: 10.1016/j.brainres.2018.05.018
Rui, L., Xiaodong, Z., Zhufeng, L., Chang, L., Hanzhe, L., Weihua, S., et al. (2018b). An approach for brain-controlled prostheses based on a facial expression paradigm. Front. Neurosci. 12, 1–15. doi: 10.3389/fnins.2018.00943
Rupal, B. S., Rafique, S., Singla, A., Singla, E., Isaksson, M., and Virk, G. S. (2017). Lower-limb exoskeletons: research trends and regulatory guidelines in medical and non-medical applications. Int. J. Adv. Robotic Syst. 14, 1–27. doi: 10.1177/1729881417743554
Samuel, O. W., Asogbon, M. G., Geng, Y., Al-Timemy, A. H., Pirbhulal, S., Ji, N., et al. (2019). Intelligent EMG pattern recognition control method for upper-limb multifunctional prostheses: advances, current challenges, and future prospects. IEEE Access 7, 10150–10165. doi: 10.1109/access.2019.2891350
Samuel, O. W., Geng, Y., Li, X., and Li, G. (2017). Towards efficient decoding of multiple classes of motor imagery limb movements based on eeg spectral and time domain descriptors. J. Med. Syst. 41:194. doi: 10.1007/s10916-017-0843-z
Tejedor, J., Garcia, C. A., Marquez, D. G., Raya, R., and Otero, A. (2019). Multiple physiological signals fusion techniques for improving heartbeat detection: a review. Sensors 19:4708. doi: 10.3390/s19214708
Verleger, R. (2003). The bereitschaftspotential. Movement-related cortical potentials. J. Psychophys. 17, 135–136. doi: 10.1027//0269-8803.17.3.135
Wang, W., De Stefano, A., and Allen, R. (2006). A simulation model of the surface EMG signal for analysis of muscle activity during the gait cycle. Comput. Biol. Med. 36, 601–618. doi: 10.1016/j.compbiomed.2005.04.002
Waszak, F., Wascher, E., Keller, P., Koch, I., Aschersleben, G., Rosenbaum, D. A., et al. (2005). Intention-based and stimulus-based mechanisms in action selection. Exp. Brain Res. 162, 346–356. doi: 10.1007/s00221-004-2183-8
Webster, R. G. (2018). Animal models of the neuromuscular junction, vitally informative for understanding function and the molecular mechanisms of congenital myasthenic syndromes. Int. J. Mol. Sci. 19:1326. doi: 10.3390/ijms19051326
Xiangxin, L., Williams, S. O., Xu, Z., Hui, W., Peng, F., and Guanglin, L. (2017). A motion-classification strategy based on sEMG-EEG signal combination for upper-limb amputees. J. Neuroeng. Rehab. 14, 1–13. doi: 10.1186/s12984-016-0212-z
Xiaodong, Z. (2018). “How to realize Brain-control technology under human-machine-environment interaction mechanism?,” in 10000 Selected Problems in Sciences, ed E.B.O.M. Science (Beijing: Science Press), 787–789.
Xiaodong, Z., Rui, L., Hanzhe, L., Zhufeng, L., Yong, H., and Balas, A. A. (2020). Novel approach for electromyography-controlled prostheses based on facial action. Med. Biol. Eng. Comp. 58, 2685–2698. doi: 10.1007/s11517-020-02236-3
Xiaodong, Z., Rui, L., and Yaonan, L. (2014). Research on brain control technology. J. Vibrat. Measur. Diagn. 34, 205–211. doi: 10.16450/j.cnki.issn.1004-6801.2014.02.027
Xugang, X., Cunbin, M., Changmin, Y., Miran, S. M., Xian, H., Yun-Bo, Z., et al. (2020). Enhanced EEG–EMG coherence analysis based on hand movements. Biomed. Signal Proces. Control 56:101727. doi: 10.1016/j.bspc.2019.101727
Yi, L., Zhijiang, D., and Weidong, W. (2015). Control and experiment for exoskeleton robot based on kalman prediction of human motion intent. Robot 37, 304–309. doi: 10.13973/j.cnki.robot.2015.0304
Keywords: numerical simulation, homology analysis, EEG, EMG, coherence
Citation: Zhang X, Li H, Lu Z and Yin G (2021) Homology Characteristics of EEG and EMG for Lower Limb Voluntary Movement Intention. Front. Neurorobot. 15:642607. doi: 10.3389/fnbot.2021.642607
Received: 16 December 2020; Accepted: 31 March 2021;
Published: 18 June 2021.
Edited by:
Minpeng Xu, Tianjin University, ChinaReviewed by:
Oluwarotimi WIlliams Samuel, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences (CAS), ChinaHong Zeng, Southeast University, China
Copyright © 2021 Zhang, Li, Lu and Yin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Hanzhe Li, bGloYW56aGUmI3gwMDA0MDtzdHUueGp0dS5lZHUuY24=