Machine learning corroborates subjective ratings of walking and balance difficulty in multiple sclerosis

Machine learning can discern meaningful information from large datasets. Applying machine learning techniques to raw sensor data from instrumented walkways could automatically detect subtle changes in walking and balance. Multiple sclerosis (MS) is a neurological disorder in which patients report varying degrees of walking and balance disruption. This study aimed to determine whether machine learning applied to walkway sensor data could classify severity of self-reported symptoms in MS patients. Ambulatory people with MS (n = 107) were asked to rate the severity of their walking and balance difficulties, from 1-No problems to 5-Extreme problems, using the MS-Impact Scale-29. Those who scored less than 3 (moderately) were assigned to the “mild” group (n = 35), and those scoring higher were in the “moderate” group (n = 72). Three machine learning algorithms were applied to classify the “mild” group from the “moderate” group. The classification achieved 78% accuracy, a precision of 85%, a recall of 90%, and an F1 score of 87% for distinguishing those people reporting mild from moderate walking and balance difficulty. This study demonstrates that machine learning models can reliably be applied to instrumented walkway data and distinguish severity of self-reported impairment in people with MS.

Introduction

Machine learning is a valuable tool for dealing with enormous amount of data and discovering new knowledge from that data, which has been widely employed in gait detection and analysis from various types of sensors, including videos, inertial measurement unit (IMU), surface electromyography, and insoles (Khera and Kumar, 2020; Saboor et al., 2020; Iosa et al., 2021; Seo et al., 2021). Instrumented walkways, widely used in field of gait biomechanics, sensitively map the spatiotemporal profile of gait. The walkway consists of a dense matrix of embedded sensors to capture walking data from gait. With the sophisticated software package destinated to the walkway system, regular spatiotemporal gait parameters can be estimated and visualized for assisting in various neural-degenerative disease evaluation and diagnosis, such as stroke and multiple sclerosis (Buckley et al., 2018; van de Port et al., 2020; Khera and Kumar, 2022).

MS is a degenerative central nervous system disease (Reich et al., 2018) with a prevalence of 35.9 per 100,000 people worldwide (MSIF, 2020). Patients with MS experience limb weakness, sensory loss, and foot drop which sometimes go undetected until clearly observed by others (Socie et al., 2013; Reich et al., 2018). Patients may sense subtle changes in their walking and balance before these symptoms can be detected by clinicians (Kirkland et al., 2018, 2020). Machine learning methods have been successfully applied to estimate gait parameters from wearables in multiple sclerosis. For example, McGinnis et al. estimated gait speed of MS patients from an array of accelerometers attached to different locations on the body (McGinnis et al., 2017). Our previous published work (Hu et al., 2022) applied machine learning to successfully detect MS patients from healthy volunteers using novel features combined with standard features derived from the raw walkway sensor data. Also based on instrumented walkway raw data, Trentzsch et al. explored different machine learning algorithms (Gaussian Naive Bayes, Decision Tree, k-Nearest Neighbor, and Support Vector Machines) to distinguish mild and moderate MS levels defined by Expanded Disability Status Scale (EDSS) (Trentzsch et al., 2021). However, a recent cross-sectional study showed that spatiotemporal gait measures have great variability within homologous EDSS categories suggesting that EDSS insensitive to subtle differences (Zanotto et al., 2022). In this study, we explored whether machine learning could be applied to raw walkway data to discern the severity of subjective walking difficulties in people with MS (or other neurological disorders). We focused on the patients' perspective because recent evidence suggests that MS has a long prodromal period (up to 5 years) when gait and balance symptoms can go undetected by clinicians (Wijnands et al., 2019).

We grouped patients by their self-reported walking and balance problems using questions in the MS Impact Scale−29 (Phillips et al., 2014) and applied machine learning to gait features derived from the walkway sensors (Hu et al., 2022). It was hypothesized that machine learning models could effectively distinguish the targets.

Materials and methods

Data collection and experimental design

Following approval by the Institutional Health Research Ethics Board (HREB # 2015.103), raw walkway sensor data were collected from the Health Innovation Team in MS (HITMS) project, which studies the health of people with MS in Newfoundland and Labrador, Canada (Chaves et al., 2019; Galloway et al., 2019). Each patient walked across the instrumented walkway (Zeno Walkway, Protokinetics Haverton PA) which measured 90 × 420 cm with or without a walking assistive device (Severini et al., 2017). Spatial data were collected from pressure sensors which represented in x, y coordinates and had an active area of 1.27 × 1.27 cm, 1 cm apart from each other. By calculating Euclidean distance between sensors, the value of the gait features can be obtained¹. Participants performed two different walking tests. Firstly, they walked four times across the walkway at their self-selected comfortable speed. After a rest, the participants performed a dual task walking test at their self-selected speed for four times. During this dual task walking, the participant was asked to walk while subtracting 7 from a preceding number beginning with 100, and speaking aloud the results (Kirkland et al., 2015; Chen et al., 2020; Hu et al., 2022).

Expanded Disability Status Scale (EDSS) rating by the MS neurologist were extracted from health records. Those patients with EDSS scores of higher than 6.5, who could not walk, were excluded. Thus, data from 107 patients were included. Their EDSS scores range from 0 to 6.5, and their average EDSS score was 2.11 ± 1.89.

The replies to a subset of eight MSIS-29 questions on how patients felt about their walking, balance, and movement (Table 1) were used to determine the self-reported rating of walking problems. Patients who scored less than 3 (answering “not at all” or “a little” to the MSIS-29) were considered to have mild walking and balance problems. The remaining patients who scored 3 (“moderate”), 4 (“quite a bit”) and 5 (“extreme”) were considered moderate. There were five different levels of severity, but since there were too few participants in some of the groups, we decided to split the groups into moderate and mild².

TABLE 1

Table 1. Patient demographic data and MSIS-29 scores.

Gait features

A set of gait features was extracted from the raw sensor data based on footprints from each pass. These included 21 features, foot type/length/width/area, unsigned toe angle, step/stride length, step/stride width, base width, step/stride time, step/stride velocity, single/double support time, stance time, toe direction, hull area, base of support (BOS) area, line of progression (LOP) deviation angle. The details regarding each parameter and how they were extracted from the walkway sensor data were described previously (Hu et al., 2022).

Machine learning process

The machine learning process is illustrated in Figure 1. Data were cleaned negative values from time features were excluded, incorrectly clustered data were checked and eliminated from the dataset. The remaining data were scaled to exhibit zero mean and unit variance. Feature selection were completed using ANOVA-SVM based on previous work (Hu et al., 2022). However, in this paper, we used the adaptive synthetic sampling approach (ADASYN) for data balancing and managing different hyperparameters for each model. The importance of each feature to model prediction was also calculated.

FIGURE 1

Figure 1. Machine learning process.

Data balancing

Imbalanced data means that the number of samples belonging to each class in the problem is not evenly distributed. For example, if 90% of the data belong to the same class, models will reach 90% accuracy by classifying all the data into the same class, and such a model is biased. In our case, the data of mild/moderate patients had a ratio of 1:3. Therefore, data balancing was performed (Menardi and Torelli, 2014). The ADASYN (He et al., 2008) was used to synthesize new samples close to minority class samples that are harder to learn, according to the possibility of the correctly classified minority sample's neighbor (K nearest neighbors) (He et al., 2008).

Feature importance

Feature importance explains the model results and which features are vital in distinguishing the targets. We evaluated the feature importance through three different types of models, intended to check which features are consistently important between models. The pipeline ANOVA-SVM (Megantara and Ahmad, 2021) calculates the average F-score for the selected features. Linear models calculate the coefficient of each feature to determine feature importance. Feature importance for tree-based models is calculated based on how the nodes of the tree used in training improve the model results. Then, feature scores are scaled to have their sum equal to one. The higher the score, the higher the importance of the feature. The scores of the features are automatically computed by the provided sciki-learn software (Pedregosa et al., 2011) when training is completed.

Machine learning algorithms

Logistic regression (LR) (Cox, 1958), support vector machine (SVM) (Cortes and Vapnik, 1995), and extreme gradient boosting (XGB) (Chen and Guestrin, 2016) were selected, as these three algorithms represent three accepted classification methods representing linear, non-linear, and decision tree-based classifiers.

LR provides a probability for the target class between 0 and 1 to describe the relationship between the input variables and one or more output targets. The input value x is fitted into the logistic function, whereby the weights or coefficient values are adjusted to predict the output value y.

SVM has the capability to work with high-dimensional data. SVM attempts to define a hyperplane boundary in an N-dimensional space, where N is the number of input features. SVM is supposed to determine the optimal plane that best separates the classes.

XGB is a distributed gradient boosting machine learning library. Applied to an ensemble of weak prediction models, the gradient boosting algorithm trains data with these weak learners forces the poor performance learners to learn to increase their prediction score, and finally combines them into one accurate prediction algorithm.

Training and evaluation

A standard grid search method was used to choose the best hyperparameters. The hyperparameter options and the optimal hyperparameters for each model described in Table 2.

TABLE 2

Table 2. Hyperparameter options for each model.

Grouped 5-fold cross-validation strategy was employed to evaluate the model result (Hu et al., 2022). Accuracy, precision, recall, and F1-score were calculated to score the model's effectiveness.

We compared the means of accuracies between the three models using Student's T-test, and means of areas under curve (AUC) using Delong test (Delong et al., 1988; Sun and Xu, 2014).

Experimental results

Feature selection based on feature correlations

The 19 numerical feature correlations were visualized using a heatmap, as shown in Figure 2. None of the features were significantly correlated with the MSIS-29 questions. However, a strong positive correlation was discovered between the “step” and “stride” parameters and the base/stride width. Therefore, 3 features stride time, stride velocity, and base width were excluded from further analysis.

FIGURE 2

Figure 2. Heatmap for numerical feature correlations. Heatmap regions that are increasingly dark show areas of higher correlations. Q4-Q11 represents the MSIS-29 questions 4 to 11.

ANOVA-SVM then determined that all remaining 16 features contributed to model prediction; respectively they were step time/velocity, single/double support time, stance time, foot length/width/area, hull area, LOP deviation angle, toe angle unsigned, step/stride length, step/stride width, BOS area. Foot type and toe direction were then reintroduced for further analysis.

Feature importance

The average F-score values of features from ANOVA-SVM, the absolute value of feature coefficient of features from LR, feature importance from XGB are presented below.

Table 3 indicates that step width, step length, BOS area, and hull area were the top four most important features for the classification examined by ANOVA-SVM. Step time was the most important feature of LR, followed by step width and area-related features (Figure 3). For the XGB classifier (Figure 4), BOS area was the most important feature, followed by the step length and step time. Step velocity and step width were considered less important. Among all three types of models, step time and step width were considered important.

TABLE 3

Table 3. Average F- score for features.

FIGURE 3

Figure 3. Absolute values of feature coefficient of LR.

FIGURE 4

Figure 4. Feature importance of XGB.

Prediction results

Figure 5 and Table 4 show the prediction results of three models. The student's t-test shows the accuracy of LR (66%) is significantly lower (p < 0.005) than those of XGB (78%) and SVM (77%); there is no significant difference between the accuracies of XGB and SVM. Similarly, LR has significantly lower recall (70%) and f1-score (76%) values than those of XGB (recall: 89%, f1-score: 87%) and SVM (recall: 90%, f1-score: 86%), with no significant differences between XGB and SVM. There was no significant difference between the precision values of the three models. However, the Delong test shows there are significant differences between means of AUC of all three models (p < 0.0001), with XGB being the best (98.1%), SVM (96%), and LR (79.6%). The reason the AUC values of all three models are higher than model accuracies could be that the optimized decision thresholds were chosen for calculating AUCs (Bradley, 1997).

FIGURE 5

Figure 5. Accuracy, precision, recall, and F1 score for each model.

TABLE 4

Table 4. Accuracy, precision, recall, and F1 score for each model.

Discussion

This study demonstrates that different degrees of walking and balance problems reported by MS patients can be distinguished by the gait features calculated from the raw walkway sensor data using machine learning methods. Although only about 78% accuracy was achieved by XGB, it is comparable to the accuracy reported by Trentzsch et al. (2021) in distinguishing moderate and mild MS divided by EDSS score), where an accuracy of about 64% was achieved by using majority decision among six machine learning models. Furthermore, 85% of the ‘moderate’ group was correctly identified (recall) by the XGB classifier and 90% of predicted ‘moderate’ samples identified by SVM were correct (precision), which are also comparable to the corresponding results reported by Trentzsch et al. (2021).

The most important gait features detected in this work were different than those identified in our previous study when distinguishing MS patients from healthy controls (Hu et al., 2022). The previous work identified stride length as the most important feature; a gait characteristic that strongly distinguished MS patients from healthy controls. In this study, step width, step time and base of support area became important, suggesting that problems with stability are distinguishing characteristics when detecting more subtle gait and balance changes. Patients may adapt to feeling unsteady by widening their feet and walking more slowly to prevent falls (Ploughman et al., 2014; Chen et al., 2020). As reported by Brach et al. (2005), people with either narrow or wide step width were more likely to report a fall, and step width could also be an indicator in identifying patients with potential gait disability. Interestingly, foot area-related features including the base of support area, foot area, and hull area were of high importance both for determining the severity of the patient's condition and distinguishing MS patients from healthy controls as reported by our previous study (Hu et al., 2022), which might hint that the patient's balance might be affected by disease progression (Singh and Kelly, 2009).

Machine learning algorithms have been applied to other neurological diseases to gauge severity of walking and balance problems. For instance, machine learning algorithms applied to vertical ground reaction force data predicted severity levels of Parkinson's disease patients according to the Hoehn & Yahr scale (Hoehn and Yahr, 1967; Zhao et al., 2018). Center of pressure coordinates and trajectories were used as prediction features to rate level of gait impairment in Parkinson's disease and cerebral palsy (Mancinelli et al., 2012; Khera and Kumar, 2022). Importantly, these projects used clinician-reported scores as ground truth; impairments that can be observed by another person. The current work uniquely corroborates the patient's perspective regarding their own walking problems, some of which may not be noticeable by others.

It is worth mentioning the limitations of the present work. We employed only two levels (mild vs. moderate) of self-reported (and combined) gait and balance problems using binary class models. Future research should explore expanded ratings, such as mild, moderate, and severe, or divide self-reported mobility problems based on whether the problem relates to gait or balance. Thus, more novel features would be explored, including pressure distribution of the foot, which could potentially be helpful for classification (Kaya et al., 2022) and accuracy. The data used in this project originates from 107 patients, and although this is a fairly large sample in some types of research, a larger sample size may further increase the model performance. Feature importance determination in this study was limited to model specific methods. In the future, more sophisticated and model-agnostic methods such as Lorenz-Zonoid method (Giudici and Raffinetti, 2021) would provide more explainable comparison of the variable importance. Finally, this study included two walking datasets, self-selected speed walking and dual-task walking. Combining the datasets provided a certain level of robustness of the machine learning model, but more walking tests such as fast-paced walking should be included in the future work to exhaustively interrogate the models.

Conclusion

This study demonstrated that machine learning can be used to classify patients' self-reported disability levels using only the raw data collected from an instrumented walkway system. We achieved 78% accuracy, 85% recall, and 90% precision when classifying mild patients from moderate patients using SVM.

Step time, BOS area and step width contributed most to classifying mild patients from moderate patients suggesting changes in balance.

Data availability statement

The datasets used and analyzed during the current study will be made available by the corresponding author upon reasonable request.

Ethics statement

The studies involving human participants were reviewed and approved by Institutional Health Research Ethics Board (HREB No. 2015.103). The patients/participants provided their written informed consent to participate in this study.

Author contributions

WH: experiment design, data analysis and interpretation, and paper drafting and editing. OC: experiment design and data analysis and interpretation. MP and XJ: research concept design, paper revision, and funding support. CN, MW, SB, and AC: subject recruitment, data collection and extraction, and data cleaning. All authors contributed to the article and approved the submitted version.

Funding

This work was supported in part by Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery [XJ, Grant No. RGPIN-2020-05525], the Canada Research Chairs Program (MP Grant No. 950-230457), Canada Foundation for Innovation (MP Project No. 33621), Canadian Institutes for Health Research (MP Grant No. 169649).

Acknowledgments

We would like to acknowledge the participants who donated their time and effort for the study, and the neurologists and clinicians who assessed and helped participants in the project.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^GAITRite Electronic Walkway Technical Reference (47DevM2).

2. ^Multiple Sclerosis Impact Scale (MSIS-29), 2000.

References

Brach, J. S., Berlin, J. E., Vanswearingen, J. M., Newman, A. B., and Studenski, S. A. (2005). Too much or too little step width variability is associated with a fall history in older persons who walk at or near normal gait speed. J. Neuroeng. Rehabilit. 2, 1–8. doi: 10.1186/1743-0003-2-21

PubMed Abstract | CrossRef Full Text | Google Scholar

Bradley, A. E. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recogn. 30, 1145–59. doi: 10.1016/S0031-3203(96)00142-2

CrossRef Full Text | Google Scholar

Buckley, E., Mazzà, C., and McNeill, A. (2018). A systematic review of the gait characteristics associated with Cerebellar Ataxia. Gait Posture 60, 154–163. doi: 10.1016/j.gaitpost.2017.11.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Chaves, A. R., Wallack, E. M., Kelly, L. P., Pretty, R. W., Wiseman, H. D., Chen, A., et al. (2019). Asymmetry of brain excitability: a new biomarker that predicts objective and subjective symptoms in multiple sclerosis. Behav. Brain Res. 359, 281–291. doi: 10.1016/j.bbr.2018.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, A., Kirkland, M. C., Wadden, K. P., Wallack, E. M., and Ploughman, M. (2020). Reliability of gait and dual-task measures in multiple sclerosis. Gait Posture 78, 19–25. doi: 10.1016/j.gaitpost.2020.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, T., and Guestrin, C. (2016). “XGBoost: A Scalable Tree Boosting System,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery Data Mining KDD'16 (ACM). p. 785–794. doi: 10.1145/2939672.2939785

PubMed Abstract | CrossRef Full Text | Google Scholar

Cortes, C., and Vapnik, V. (1995). Support-vector networks. Mach. Learn. 20, 273–297. doi: 10.1007/BF00994018

CrossRef Full Text | Google Scholar

Cox, D. R. (1958). The regression analysis of binary sequences. J. R. Stat. Soc. Series B. 20, 215–232. doi: 10.1111/j.2517-6161.1958.tb00292.x

CrossRef Full Text | Google Scholar

Delong, E. R., Delong, D. M., and Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. doi: 10.2307/2531595

PubMed Abstract | CrossRef Full Text | Google Scholar

Galloway, D. A., Blandford, S. N., Berry, T., Williams, J. B., Stefanelli, M., Ploughman, M., et al. (2019). miR-223 promotes regenerative myeloid cell phenotype and function in the demyelinated central nervous system. Glia 67, 857–869. doi: 10.1002/glia.23576

PubMed Abstract | CrossRef Full Text | Google Scholar

Giudici, P., and Raffinetti, E. (2021). Shapley-Lorenz eXplainable Artificial Intelligence. Expert Syst. Appl. 167, 114104. doi: 10.1016/j.eswa.2020.114104

CrossRef Full Text | Google Scholar

He, H., Bai, Y., Garcia, E. A., and Li, S. (2008). “ADASYN: Adaptive synthetic sampling approach for imbalanced learning,” in Proceedings of the International Joint Conference on Neural Networks p. 1322–1328.

Google Scholar

Hoehn, M. M., and Yahr, M. D. (1967). Parkinsonism : onset, progression, and mortality. Neurology. 50, 318. doi: 10.1212/WNL.50.2.318

PubMed Abstract | CrossRef Full Text | Google Scholar

Hu, W., Combden, O., Jiang, X., Buragadda, S., Newell, C. J., Williams, M. C., et al. (2022). Machine learning classification of multiple sclerosis patients based on raw data from an instrumented walkway. Biomed. Eng. 21, 21. doi: 10.1186/s12938-022-00992-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Iosa, M., Capodaglio, E., Pel,à, S., Persechino, B., Morone, G., Antonucci, G., et al. (2021). Artificial neural network analyzing wearable device gait data for identifying patients with stroke unable to return to work. Front. Neurol. 12, 650542. doi: 10.3389/fneur.2021.650542

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaya, M., Karaku,ş, S., and Tuncer, S. A. (2022). Detection of ataxia with hybrid convolutional neural network using static plantar pressure distribution model in patients with multiple sclerosis. Comput. Methods Programs Biomed. 214, 106525. doi: 10.1016/j.cmpb.2021.106525

PubMed Abstract | CrossRef Full Text | Google Scholar

Khera, P., and Kumar, N. (2020). Role of machine learning in gait analysis: a review. J. Med. Eng. Technol. 44, 441–467. doi: 10.1080/03091902.2020.1822940

PubMed Abstract | CrossRef Full Text | Google Scholar

Khera, P., and Kumar, N. (2022). Age-gender specific prediction model for Parkinson's severity assessment using gait biomarkers. Eng. Sci. Technol. Int. J. 27, 101005. doi: 10.1016/j.jestch.2021.05.009

CrossRef Full Text | Google Scholar

Kirkland, M. C., Chen, A., Downer, M. B., Holloway, B. J., Wallack, E. M., Lockyer, E. J., et al. (2018). Bipedal hopping timed to a metronome to detect impairments in anticipatory motor control in people with mild multiple sclerosis. Clin. Biomech. 55, 45–52. doi: 10.1016/j.clinbiomech.2018.04.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Kirkland, M. C., Wadden, K. P., and Ploughman, M. (2020). Bipedal hopping as a new measure to detect subtle sensorimotor impairment in people with multiple sclerosis. Disabil. Rehabil. 44, 1544–55. doi: 10.1080/09638288.2020.1820585

PubMed Abstract | CrossRef Full Text | Google Scholar

Kirkland, M. C., Wallack, E. M., Rancourt, S. N., and Ploughman, M. (2015). Comparing three dual-task methods and the relationship to physical and cognitive impairment in people with multiple sclerosis and controls. Mult. Scler. Int. 2015, 1–7. doi: 10.1155/2015/650645

PubMed Abstract | CrossRef Full Text | Google Scholar

Mancinelli, C., Patel, S., Deming, L. C., Nimec, D., Chu, J. J., Beckwith, J., et al. (2012). “A novel sensorized shoe system to classify gait severity in children with cerebral palsy,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS p. 5010–5013. doi: 10.1109/EMBC.2012.6347118

PubMed Abstract | CrossRef Full Text | Google Scholar

McGinnis, R. S., Mahadevan, N., Moon, Y., Seagers, K., Sheth, N., Wright, J. A., et al. (2017). A machine learning approach for gait speed estimation using skin-mounted wearable sensors: From healthy controls to individuals with multiple sclerosis. PLoS ONE. 12, e0178366. doi: 10.1371/journal.pone.0178366

PubMed Abstract | CrossRef Full Text | Google Scholar

Megantara, A. A., and Ahmad, T. (2021). ANOVA-SVM for selecting subset features in encrypted internet traffic classification. Int. J. Intell. Eng. Syst. 14, 536–546. doi: 10.22266/ijies2021.0430.48

CrossRef Full Text | Google Scholar

Menardi, G., and Torelli, N. (2014). Training and assessing classification rules with imbalanced data. Data Min. Knowl. Discov. (2014) 28:92–122. doi: 10.1007/s10618-012-0295-5

PubMed Abstract | CrossRef Full Text | Google Scholar

MSIF (2020). Atlas of MS 3 rd edition. third edit, p. 1−36. Available online at: www.atlasofms.org

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830. doi: 10.5555/1953048.2078195

CrossRef Full Text | Google Scholar

Phillips, G. A., Wyrwich, K. W., Guo, S., Medori, R., Altincatal, A., Wagner, L., et al. (2014). Responder definition of the Multiple Sclerosis Impact Scale physical impact subscale for patients with physical worsening. Multiple Scler. J. 20, 1753–1760. doi: 10.1177/1352458514530489

PubMed Abstract | CrossRef Full Text | Google Scholar

Ploughman, M., Deshpande, N., Latimer-Cheung, A. E., and Finlayson, M. (2014). Drawing on related knowledge to advance multiple sclerosis falls-prevention research. Int. J. MS Care. 16, 163–170. doi: 10.7224/1537-2073.2014-052

PubMed Abstract | CrossRef Full Text | Google Scholar

Reich, D. S., Lucchinetti, C. F., and Calabresi, P. A. (2018). Multiple sclerosis. New Engl. J. Med. 378, 169–180. doi: 10.1056/NEJMra1401483

PubMed Abstract | CrossRef Full Text | Google Scholar

Saboor, A., Kask, T., Kuusik, A., Alam, M. M., le Moullec, Y., Niazi, I. K., et al. (2020). Latest research trends in gait analysis using wearable sensors and machine learning: A systematic review. IEEE Access 8, 167830–167864. doi: 10.1109/ACCESS.2020.3022818

CrossRef Full Text | Google Scholar

Seo, K., Chung, B., Panchaseelan, H. P., Kim, T., Park, H., Oh, B., et al. (2021). Forecasting the walking assistance rehabilitation level of stroke patients using artificial intelligence. Diagnostics 11, 1096. doi: 10.3390/diagnostics11061096

PubMed Abstract | CrossRef Full Text | Google Scholar

Severini, G., Manca, M., Ferraresi, G., Caniatti, L. M., Cosma, M., Baldasso, F., et al. (2017). Evaluation of Clinical Gait Analysis parameters in patients affected by Multiple Sclerosis: Analysis of kinematics. Clin. Biomech. 45, 1–8. doi: 10.1016/j.clinbiomech.2017.04.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Singh, A. P., and Kelly, A. J. (2009). A case of charcot's feet in a patient with parkinson's disease: A case report. Cases J. 2, 1–3. doi: 10.1186/1757-1626-2-187

PubMed Abstract | CrossRef Full Text | Google Scholar

Socie, M. J., Motl, R. W., Pula, J. H., Sandroff, B. M., and Sosnoff, J. J. (2013). Gait variability and disability in multiple sclerosis. Gait Posture 38, 51–55. doi: 10.1016/j.gaitpost.2012.10.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, X., and Xu, W. (2014). Fast implementation of DeLong's algorithm for comparing the areas under correlated receiver operating characteristic curves. IEEE Signal Process. Lett. 21, 1389–1393. doi: 10.1109/LSP.2014.2337313

CrossRef Full Text | Google Scholar

Trentzsch, K., Schumann, P., Sliwiński, G., Bartscht, P., Haase, R., Schriefer, D., et al. (2021). Using machine learning algorithms for identifying gait parameters suitable to evaluate subtle changes in gait in people with multiple sclerosis. Brain Sci. 11, 1049. doi: 10.3390/brainsci11081049

PubMed Abstract | CrossRef Full Text | Google Scholar

van de Port, I., Punt, M., and Meijer, J. W. (2020). Walking activity and its determinants in free-living ambulatory people in a chronic phase after stroke: a cross-sectional study. Disabil. Rehabil. 42, 636–641. doi: 10.1080/09638288.2018.1504327

PubMed Abstract | CrossRef Full Text | Google Scholar

Wijnands, J. M. A., Zhu, F., Kingwell, E., Zhao, Y., Ekuma, O., Lu, X., et al. (2019). Five years before multiple sclerosis onset: Phenotyping the prodrome. Multiple Scler. J. 25, 1092–1101. doi: 10.1177/1352458518783662

PubMed Abstract | CrossRef Full Text | Google Scholar

Zanotto, T., Sosnoff, J. J., Ofori, E., Golan, D., Zarif, M., Bumstead, B., et al. (2022). Variability of objective gait measures across the expanded disability status scale in people living with multiple sclerosis: A cross-sectional retrospective analysis. Mult. Scler. Relat. Disord. 59, 103645. doi: 10.1016/j.msard.2022.103645

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhao, A., Qi, L., Li, J., Dong, J., and Yu, H. (2018). A hybrid spatio-temporal model for detection and severity rating of Parkinson's disease from gait data. Neurocomputing 315, 1–8. doi: 10.1016/j.neucom.2018.03.032

CrossRef Full Text | Google Scholar