Oculomotor analysis to assess brain health: preliminary findings from a longitudinal study of multiple sclerosis using novel tablet-based eye-tracking software

A growing body of evidence supports the link between eye movement anomalies and brain health. Indeed, the oculomotor system is composed of a diverse network of cortical and subcortical structures and circuits that are susceptible to a variety of degenerative processes. Here we show preliminary findings from the baseline measurements of an ongoing longitudinal cohort study in MS participants, designed to determine if disease and cognitive status can be estimated and tracked with high accuracy based on eye movement parameters alone. Using a novel gaze-tracking technology that can reliably and accurately track eye movements with good precision without the need for infrared cameras, using only an iPad Pro embedded camera, we show in this cross-sectional study that several eye movement parameters significantly correlated with clinical outcome measures of interest. Eye movement parameters were extracted from fixation, pro-saccade, anti-saccade, and smooth pursuit visual tasks, whereas the clinical outcome measures were the scores of several disease assessment tools and standard cognitive tests such as the Expanded Disability Status Scale (EDSS), Brief International Cognitive Assessment for MS (BICAMS), the Multiple Sclerosis Functional Composite (MSFC) and the Symbol Digit Modalities Test (SDMT). Furthermore, partial least squares regression analyses show that a small set of oculomotor parameters can explain up to 84% of the variance of the clinical outcome measures. Taken together, these findings not only replicate previously known associations between eye movement parameters and clinical scores, this time using a novel mobile-based technology, but also the notion that interrogating the oculomotor system with a novel eye-tracking technology can inform us of disease severity, as well as the cognitive status of MS participants.

Introduction

Multiple sclerosis (MS) is an inflammatory disease of the central nervous system, causing demyelination and neurodegeneration in most individuals, where progression can be gradual and subtle (1, 2). Though historically MS has often been categorized by distinct clinical descriptors (e.g., primary progressive, relapsing–remitting), accumulating evidence suggests that the clinical course of multiple sclerosis is better considered as a continuum (3). Cognitive dysfunction is a common concomitant of MS and recent evidence suggests that the standard clinical interview and neurological examination are not sufficiently sensitive to detect cognitive impairment and more subtle motor impairments in MS (4). It is in part for this reason that special task forces and expert consensus committees developed the Multiple Sclerosis Functional Composite (MSFC) and the Brief International Cognitive Assessment for MS (BICAMS), which when combined allow for more comprehensive assessments of cognitive processing speed, verbal and visuospatial memory, ambulation, and hand visuomotor coordination. Although these assessments have proven invaluable in research and clinical trial settings, their lengthy administration time makes them very difficult and impractical to use as part of the standard of care in routine clinical practice.

Researchers have consequently looked at alternative means to objectively assess the disease and cognitive status of individuals with MS, and one such recent avenue has been through the measurement of oculomotor parameters. Recent work has unequivocally shown that eye movements can reflect certain aspects of brain function and inform on the presence of neurodegeneration and cognitive impairment (5–8). The link between eye movements and brain health should not be too surprising, given that eye movements are controlled by a diverse network of the brainstem and cortical structures and circuits (9, 10) that are susceptible to a variety of degenerative processes (5, 11, 12). Moreover, the analysis of gaze patterns and visual tasks that measure inhibition can provide insights into the integrity of various cognitive processes (6, 13, 14).

The literature for MS is particularly rich with evidence demonstrating oculomotor anomalies, such as fixation instability (15–17), slowed pro-saccades (18–20) with increased onset latencies (21–23), increased anti-saccade error rates (18, 21, 22, 24, 25), and reduced slow pursuit gain (26). Furthermore, several studies linked these anomalies to brain health via correlations between several eye movement parameters and disease or cognitive status as measured by tools such as the Expanded Disability Status Scale (EDSS), the Symbol Digit Modalities Test (SDMT) (17, 18, 23, 25, 27) and the PASAT (13, 24, 25).

These findings and others have led to suggestions that laboratory eye movement recordings can be extremely useful for objective and precise identification of disease status and monitoring of disease progression (5) and assist with differential diagnoses (28–30). Unfortunately, the use of detailed eye movement recordings in clinical settings has been limited, due largely to the cost-prohibitive nature and limited scalability of the required specialized equipment, such as infrared eye-tracking cameras. However, Innodem Neurosciences has recently developed a patented gaze-tracking technology (Eye-Tracking Neurological Assessment: ETNA™) that can reliably and accurately track eye movements with good precision without the need for infrared cameras, using only the embedded camera of an iPad Pro. This technology allows for the precise quantification of several eye movement parameters currently only available with specialized and costly research-grade infrared eye tracking devices, such as the latency, velocity, and accuracy of saccades, and the presence of saccadic intrusions during fixation. The ETNA™ platform was recently used to replicate well-known oculomotor findings in Parkinson’s disease, in addition to showing that several eye movement parameters were significantly correlated with disease severity as assessed via the MDS-UPDRS motor subscale (31). These first set of findings demonstrated that this tablet-based tool has the potential to both accelerate eye movement research via affordable and scalable eye-tracking, and aid with the precise identification of disease status and monitoring of disease progression in clinical settings.

The purpose of this manuscript is to present preliminary findings from an ongoing longitudinal study following a cohort of persons with MS and healthy controls. The results presented herein were produced as the result of an interim device performance analysis using a subset of 20 oculomotor parameters (listed below in the methods section) collected from four different visual tasks (fixation, pro-saccades, antisaccade, and smooth pursuit), which were selected a priori and based on parameters for which there was evidence in the literature of either anomalous values in MS or correlations with our chosen MS-based clinical outcome indicators. For strengthening the evidence, we also wished to identify which of the many eye tracking parameters, alone or in combination, were the most relevant for explaining the clinical outcomes. The overarching goal of the study is to determine if disease and cognitive status can be estimated with high accuracy based on eye movement parameters alone, using Innodem’s patented mobile, scalable, and accessible eye-tracking technology. As a first step, we provide preliminary findings regarding the relationship between the preselected eye movement parameters and validated clinical scale scores linked to disease status and cognitive status. To assess disease and cognitive status, we used three of the most employed MS assessment tools Expanded Disability Status Scale (EDSS) (32), Brief International Cognitive Assessment for MS (BICAMS) (33, 34), and Multiple Sclerosis Functional Composite (MSFC) (35, 36) along with the Symbol Digit Modalities Test (SDMT), which together formed our four clinical outcome measures of interest.

Methods

Study design and subject population

This cross-sectional study included 60 participants and was approved by both the Veritas and the McGill University Health Center (MUHC) research ethics boards (ClinicalTrials.gov Identifier: NCT05061953). All participants were persons with MS (49 with RRMS and 11 with SPMS, 41 female/19 male, see Table 1 for additional demographic and clinical information) and were recruited from one of two study sites: the Montreal Neurological Clinic (MNC) and the Montreal Neurological Institute and Hospital (MNI). All data collection was performed by the clinical research units at the study sites (MNC: Genge Partners, Inc.; MNI: The Clinical Research Unit at the Montreal Neurological Institute). The main inclusion criteria were adults with a confirmed diagnosis of MS with no signs of progressive increase in physical disability within the past 6 months and sufficient corrected visual acuity to allow for the accurate reading of the on-screen visual task instructions. The main exclusion criteria were the presence of comorbid neurological or psychiatric conditions, to avoid eye movement anomaly confounders, the recent start of medications known to influence ocular motor visual function (e.g., benzodiazepines) and participants who experienced an MS relapse at the time of assessment.

TABLE 1

Table 1. Participant demographic data (n = 60) and MS-related clinical test scores.

Clinical and cognitive assessments

To assess disease and cognitive status, we used three of the most commonly employed MS assessment tools: EDSS [Expanded Disability Status Scale; (32)], BICAMS [Brief International Cognitive Assessment for MS; (33, 34)] and two subtests of the MSFC [Multiple Sclerosis Functional Composite; (35, 36)]. The EDSS is a validated disability score based on the neurological examination and takes approximately 20 min to complete and allows for the quantification of physical disability in multiple sclerosis and the monitoring of changes in the level of disability over time and was performed by a certified examiner (neurologist) to assess the level of disability.

The BICAMS consists of a test of information processing speed (SDMT), as well as tests for verbal (CVLT-II) and visual memory (BVMT-R), representing the most frequent cognitive deficits observed in MS (37–39). However, we substituted the CVLT-II with the RAVLT (40) due to it having a validated and normed version for French Canadians. Both tests are nearly identical and have been shown to be highly comparable for detecting learning deficits in MS (41, 42). Moreover, there is precedence for replacing the CLVT-II with the RAVLT in BICAMS validation studies in non-English-speaking countries (43, 44).

The MSFC comprises quantitative functional measures of three key clinical dimensions of MS: leg function/ambulation [timed 25-foot walk test (T25-FW)], arm/hand function 9-hole pegboard test (9-HPT), and cognitive function (the PASAT test) (45). However, here we only performed the two subtests with a motor component (T25-FW and 9-HPT) and replaced the PASAT score with the SDMT score to reduce testing time, and because SDMT was found to be a more valid and reliable measure of cognitive processing speed compared to PASAT (46).

Gaze-tracking experimental setup

All tests were performed using a 12.9-inch iPad Pro tablet with the ETNA™ software installed, which enables simultaneous video recordings of the eyes at 60 frames per second using the embedded front-facing camera and the presentation of visual stimuli on the screen. The ETNA™ software’s gaze-tracking algorithms, as estimated from a normative sample of 196 healthy control individuals, have an accuracy of 0.47 degrees (mean offset between the actual gaze position and the recorded gaze position) and precision of 0.33 degrees as calculated via Root Mean Square (RMS) of the sampled points; an estimate of reliability of the gaze point estimate from one sample to the next, which are comparable values to those of research-grade infrared eye tracking devices.

All participants performed four oculomotor tasks (a fixation task, a pro-saccade task, an anti-saccade task, and a smooth pursuit task, see Figure 1), which were preceded by a calibration step, where participants were instructed to follow a slowly moving target (8 deg./s) across the screen. Calibration and all four tasks were completed in under 10 min.

FIGURE 1

Figure 1. Eye-tracking tasks. (A) Fixation: participants fixated a stationary target for 7 s, at one of 5 locations. (B) Pro-saccades: participants initially fixated a central fixation point, which disappeared after 1.0–3.5 s, after which a different target appeared at one of 8 eccentric locations for 1.5 s. (C) Anti-saccades: participants initially fixated a central fixation point, which disappeared after 1.0–3.5 s, after which a round target appeared at 10° to the left or right from the center. Participants were instructed to move their gaze in the opposite direction to the round target, where after 1,200 ms they were shown a square with an arrow inside that pointed in one of 4 random directions (left, right, up, or down; shown during 400 ms). The users then had to direct their gaze toward the arrow orientation corresponding to the arrow they saw in the preceding step. (D) Smooth pursuit: after initially fixating on a central cross, participants followed a moving target with a constant velocity of 8°/s (in this example, a downward-moving target).

All tasks were performed with the tablet screen placed vertically, camera side up, and secured at eye level using a tablet pole mount. Participants were positioned approximately 45 cm from the tablet screen and were allowed to use their best-corrected visions, with glasses or lenses if necessary. Safeguards within the gaze-tracking software ensured the participant’s head was properly positioned and visible via the embedded camera, at an acceptable angle and distance from the screen.

Fixation task

Participants had to fixate a stationary target for 7 s, at five different locations (one central and 4 eccentric locations). The eccentric positions were located at 10 degrees of visual angle left and right from the center and 14 degrees of visual angle up and down from the center (Figure 1A).

Pro-saccade task

Participants had to initially fixate a central fixation point, which disappeared after a random period of 1.0–3.5 s, after which a different target reappeared at an eccentric location for 1.5 s either to the left or right, above or below the central fixation point. Participants were instructed to move their gaze as quickly as possible to the new target location. Both short (5^o horizontal, 6^o vertical) and large (10^o horizontal,12^o vertical) eccentric target distances were used, and each target location was sampled 3 times, for a total of 24 trials (Figure 1B).

Anti-saccade task

Participants had to initially fixate a central fixation target, which disappeared after a random period of 1.0–3.5 s, after which a different target reappeared at an eccentric location (10^o) to the left or right from the center. Participants were instructed to move their gaze as quickly as possible in the opposite direction to the new target location. After being displayed for only 100 ms, the target disappeared, and the screen was left blank for a predetermined duration of time. Following the blank screen, a symbol appeared in the opposite location of where the initial stimulus appeared (i.e., where the participant should be looking). This symbol consisted of a white square with an arrow inside oriented in one of 4 random directions: either left, right, up, or down. The blank screen period lasted 1,200 ms and the arrow symbol duration of 400 ms. After each trial, a screen was displayed for 5 s prompting the user to answer which symbol they saw by directing their gaze toward the arrow orientation corresponding to what they believe is the correct answer (Figure 1C). This task was inspired by an anti-saccade task used in a previous study (47), whereby participants could only identify the second symbol had they performed the anti-saccade task correctly (i.e., looked in the opposite direction of the initial target).

Smooth pursuit task

Participants here were first required to fixate a central fixation cross of variable duration (1000-2000 ms). Once the fixation cross disappeared a moving target (that could either go up, down, left or right) appeared on screen for which the participants were instructed to follow with their gaze. Step–ramp paradigm of smooth pursuit at constant velocity was used, whereby the initial position of the moving target was positioned offset from the central fixation point, on the opposite side of the motion direction (Figure 1D). For instance, in a trial of rightward smooth pursuit, the motion target would first appear to the left of the central fixation point (i.e., the step) and then moved in the opposite direction (rightward) at a constant velocity (i.e., the ramp). The trial terminated when the target reached the 10° position either left, right, above, or below center. A total of four trials were performed, one in each direction, with a constant velocity of 8.65°/s with a step size of 1.5°.

Parameter extraction and analysis

Offline analysis was performed using ETNA™ ‘s proprietary analysis pipeline to automatically extract the eye movement parameters reported for each task. Before parameter extraction, all gaze signals were processed and non-saccadic artifacts (e.g., blinks) were removed by the software’s analysis pipeline. Saccade detection was performed using an adaptive, velocity-based algorithm based on the work of Schweitzer and Rolfs (48). Saccade parameters were then obtained by fitting a parametric model for saccadic waveforms (49), which reproduces the established relationship between peak saccadic angular velocity and saccadic amplitude (i.e., the saccadic main sequence), to the data acquired during the pro-saccade and anti-saccade tasks. This model fitting provides saccade parameters such as the saccade latency, amplitude and peak velocity. Example saccade traces are provided in Supplementary Figure S1.

The following parameters were extracted from the fixation task gaze recordings (parameters were averaged across the five fixation trials): (1) 95% bivariate contour ellipse area (BCEA 95) of fixation - a measure of fixation stability which encompasses an ellipse that covers the 95% of fixation points that are closest to target, (2) the rate of saccadic intrusions during fixation, (3) the average amplitude of the saccadic intrusions during fixation.

The following parameters were extracted from the pro-saccade task gaze recordings (averaged across all short- and large-eccentricity targets separately): (1) average saccade latency, (2) average total time to reach the peripheral target, (3) average peak saccade velocity, (4) average saccade amplitude gain (i.e., the amplitude of the saccade relative to the eccentricity of the target; a measure of saccade accuracy).

The following parameters were extracted from the anti-saccade task gaze recordings: (1) direction error rate, (2) direction correction rate (proportion of trials where participants directed their gaze in the correct direction following an initial saccade in the wrong direction), (3) correct direction saccade onset latency, (4) incorrect direction saccade onset latency, and (5) time-to-correct latency (time elapsed between a first incorrect saccade and a second corrective saccade in the right direction).

The following parameters were extracted from the smooth pursuit task gaze recordings: (1) pursuit velocity gain, (2) pursuit lag – average distance separating the target from the pursuit gaze point, (3) proportion of time spent in pursuit – the ratio of time spent in pursuit to the total time elapsed between eye movement onset and offset, relative to the time spent performing catch-up saccades, and (4) number of catch-up saccades.

For all correlations between eye movement parameters and the clinical outcome measures of interest (EDSS, SDMT, BICAMS, and MSFC) the Spearman’s ρ correlation coefficient was calculated. Data analyses were performed using SAS statistical software suite. Corrected p-values to adjust for the false discovery rate were computed using the Benjamini-Hochberg procedure evaluated at an alpha level of 0.05 (50). Data visualization was performed using R 4.2.1 in RStudio (build 554), packages dplyr, tidyverse, ggplot2, ggpubr, and rstatix. The scores on SDMT, RAVLT, and BVMT-R were converted to z-scores and their average was calculated to obtain the composite BICAMS score. Similarly, the scores on SDMT, T25FW, and 9HPT were converted to z-scores and their average was calculated to obtain the composite MSFC score (the reciprocal of the 9HPT score was used here and the T25FW score was multiplied by −1 so that higher scores on each test corresponded to better scores).

Comparisons of high vs. low EDSS

For each oculomotor parameter, data were z-scored and then split based on high vs. low EDSS scores (EDSS ≤4 and EDSS ≥4.5) for visualization in Figure 2 and exploratory analyses — Mann–Whitney U tests were used for comparing each ocular parameter value between the high and low EDSS groups.

FIGURE 2

Figure 2. Radar plot illustrating the z-score values for low- and high-EDSS subgroups for each oculomotor parameter. Significant subgroup differences are highlighted with bold labels. S, short amplitude pro-saccade; L, large amplitude pro-saccade. *p < 0.05, **p < 0.01.

Multiple regression analyses

Partial least squares (PLS) analysis was used to identify which of the many eye tracking parameters, alone or in combination, were the most relevant for explaining the clinical outcomes (EDSS, SDMT, MSFC, and BICAMS), while accounting for multicollinearity between oculomotor parameters. For each PLS regression model the number of latent variables maximizing the covariance between the independent and dependent variables was selected using a 5-fold cross validation procedure to minimize the mean square error. An exhaustive feature selection procedure was used to select the parameters that most contributed to the final model. This procedure involved sampling all possible combinations of oculomotor parameters (set sizes from 1 to all parameters) and subsequent linear model fitting. Age was included as an additional predictor for all models. For each set of oculomotor parameters, only patient samples with the full set of parameter values (i.e., not containing any missing values within the specific set of parameters, which for the most part were structurally missing, such as the antisaccade correct direction latency parameter in cases where participants did perform any correct direction anti-saccade trials) were used for the selected linear models. During feature selection, only patient samples with full sets of parameter values for the given set of features were used. This resulted in a sample size of n = 29 for the EDSS and MSFC models, and sample sizes of n = 30 for the SDMT and BICAMS models. For each model, standardized regression coefficients were computed by multiplying regression coefficients by the standard deviation of the predictor variable divided by the standard deviation of the dependent variable. The normalized absolute values of the standardized regression coefficients are used as a measure of oculomotor parameter contribution to the model. The coefficient of determination (R2) was used to assess multiple regression performance (both adjusted and non-adjusted values) and was used as the feature selection metric. Multiple regression analyses and radar plots were conducted using scikit-learn 1.2.2 and matplotlib 3.7.1 in Python 3.9.6.

Results

Spearman correlations between the extracted eye movement parameters and clinical outcome measures (SDMT, BICAMS, MSFC, and EDSS) are shown in Supplementary Table 2 – see Figure 3 for a graphical representation of select representative correlations highlighting the parameter for each task that most strongly correlated with all clinical outcome measures. After correction for multiple comparisons, nine eye-movement parameters were significantly correlated with the SDMT, five with BICAMS, ten with the MSFC, and nine with the EDSS. A greater percentage of pro-saccade (56%) and anti-saccade (44%) parameters were found to significantly correlate with the MS-related clinical scale scores than did the fixation (25%) and smooth pursuit (6%) parameters.

FIGURE 3

Figure 3. Spearman correlations between select eye-tracking parameters and functional scores. (A) Fixation: BCEA95, (B) Pro-Saccades: large amplitude saccade latency, (C) Anti-Saccades: time to target, and (D) Smooth pursuit: average pursuit gain. All Spearman’s rho correlation values were calculated using the raw data. For visualization purposes only, the MSFC x-axes were rescaled [0.1–0.9] and log2-transformed. *p < 0.05 (corrected for multiple comparisons).

We furthermore performed an additional post-hoc exploratory analysis to determine which oculomotor parameters best distinguished high EDSS from low EDSS participants (results are shown in Figure 2 in the form of a radar plot). After z-scoring the parameter values relative to the entire group, high and low EDSS subgroups were compared and significant differences were found for almost all the same oculomotor parameters for which there was a significant correlation with EDSS, except for the two pro-saccade latency parameters and the antisaccade time-to-correct parameter, primarily those for which the monotonic relationship with the EDSS was the weakest amongs those that were significant.

Finally, to further assess the potential of using oculomotor parameters to estimate clinical outcome indicators, we performed multiple regression analyses for each clinical outcome measure using all the oculomotor parameters as predictors. Results are presented in Figures 4A–D and Supplementary Table S1 and show that all models explain upwards of 47% of the variance of the clinical outcome measures, and up to 84% for EDSS specifically. The bottom panel of Figure 4E further illustrates the relative contribution of each oculomotor parameter to each model predictor. Whereas several parameters contribute to all four models (e.g., OSI rate and antisaccade time-to-target), several others are specific to one or two models (e.g., anti-saccade percent correct and pro-saccade peak velocity).

FIGURE 4

Figure 4. (A–D) Scatterplots of the relationship between the study participants’ clinical scores and the corresponding predicted value obtained by partial least squares regression analysis using the oculomotor parameters as predictors. (E) Heatmap visualization of the relative contribution (normalized absolute value of standardized regression coefficients) of each oculomotor parameter to each partial least squares regression predictor. Dark squares indicate lesser contributions to the model whereas lighter/yellow squares indicate greater contributions. Absent squares indicate that the parameter was not used in the final model. S, short amplitude pro-saccade; L, large amplitude pro-saccade.

Discussion

The purpose of this manuscript is to present preliminary findings from an ongoing longitudinal clinical trial following a cohort of persons with MS and to demonstrate whether this novel and scalable tablet-based eye-tracking technology can provide estimates of all these clinical scales used to assess physical and cognitive MS disability. These study-specific aims were developed with the long-term goal of the trial in mind, that is to determine if disease and cognitive status can be estimated with high accuracy using a mobile, scalable, and accessible eye-tracking technology. Although we not only show that individual oculomotor findings are very much in line with those previously reported in the scientific literature on MS, which will be further discussed below, the findings from the partial least squares regression analyses strongly suggest that with a greater sample size and the development of ML-based tools, we may be able to accurately estimate disease severity in MS patients based on eye movement analysis alone across the full EDSS range. Indeed, the linear models explained between 53% (MSFC) and 74% (EDSS) of the variance of the clinical outcome measures using only the most predictive subset of parameters. Taken together, these findings suggest that combining different oculomotor parameters together can be very informative of an MS patient’s current functional and cognitive state, as measured by the various clinical scales in the present study. Moving forward, we believe these could serve as the building blocks required to enable more sophisticated machine learning models to estimate with a high degree of accuracy the clinical state of a patient.

As highlighted above, our oculomotor-clinical scale score correlations are consistent with those previously reported in the literature, where available. To our knowledge, no studies to date have reported correlations between eye movement parameters and the composite BICAMS or MFSC scores (with one exception, see below), but rather report correlations with one of several of their subtests, such as the T25-FW, 9-HPT, PASAT (substituted with the SDMT here), BVTM-R, SDMT, and the CVLT (substituted with the RAVLT) here. Although at first glance the BICAMS may seem less related to oculomotor measures given the fewer number of significant correlations, it should be noted that five correlations had corrected p-values between 0.05 and 0.08 (all coefficients ≥0.27), suggesting that the BICAMS does indeed correlate with similar oculomotor measures, just to a lesser extent than the other examined clinical outcome measures. Finally, several parameters were found to correlate with all outcome measures - this is perhaps not so surprising given that the SDMT score was used to calculate both the BICAMS and MSFC composite scores and the known correlation between SDMT and EDSS (also significant here; Spearman rho = − 0.437, p < 0.001).

To our knowledge only one other study examined the relationship between clinical scale scores and measures of fixational stability, which showed that the average number of microsaccades duration fixation increased as a function of the EDSS (r = 0.35) (17). This finding was further supported by those of both Polet et al. (18) and Nij Bijvank et al. (16), who showed that individuals with high EDSS scores made a greater number of saccadic intrusions than those with lower scores. Sheehy et al. (17) furthermore showed that the average number of microsaccades was negatively correlated with the SDMT (r = −0.35) and positively correlated with the 9HPT non-dominant hand (r = 0.39) (the authors did not find correlations with T25FW, PASAT, or 9HPT dominant hand). Although the technology used here does not capture microsaccades, our measure of fixation stability (BCEA95) showed similar correlation coefficients with SDMT, BICAMS (which includes the SDMT) and MSFC (which includes both the SDMT and 9HPT) (see Supplementary Table 1).

With regards to the relationship between pro-saccade parameters and MS-related clinical outcome measures, Polet et al. (18) showed that individuals with high EDSS had reduced pro-saccade peak velocity compared to those with low EDSS, whereas Nij Bijvank et al. (22) showed that pro-saccade latency was increased in individuals with high EDSS compared to those with low EDSS. Both of these findings are aligned with the negative significant correlation between peak velocities and EDSS and the positive significant correlation between latencies and EDSS observed in the present study. Zangemeister et al. (51) showed that 9-HPT negatively correlated with peak velocity (rho = −0.321), which is consistent with the present MSFC findings (note that here the correlation sign is reversed as the higher MSFC score is better, unlike the 9-HPT). Nygaard et al., (23) showed a negative correlation between saccade latency and SDMT (r = −0.32), which is again consistent with the present findings. Finally, although Finke et al. (19) surprisingly found no correlation between the MFSC composite score and either of the peak velocity, amplitude and latency pro-saccade parameters, they did find significant correlations between the FSS score (a subscore of the EDSS) with both the latency (rho = 0.385) and peak velocity (rho = −0.468) parameters. While the later findings are in line with the present ones, the discrepancy regarding the MSFC correlations could be explained by the fact that we replaced the PASAT with the SDMT in our MSFC composite. Indeed, the PASAT and SDMT are moderately correlated with one another and are hypothesized to depend on different, although partially overlapping, cognitive processes (52).

Regarding anti-saccade parameters, Nij Bijvank et al. (22) showed that anti-saccade latency was increased in individuals with higher EDSS scores compared to those with lower scores, whereas Gajamange et al. (27) showed a moderate correlation (rho = −0.37) between anti-saccade latency and the SDMT (though the correlation did not reach statistical significance due to a small sample size). Here we found that the correct direction anti-saccade latency significantly correlated with not only the SDMT but also the BICAMS and MSFC, whereas the correlation with EDSS just failed to meet the criteria for significance after correction for multiple comparisons. Although the anti-saccade error rate has been shown to significantly correlate with the SDMT (rho = − 0.48) Gajamange et al. (27); (rho = −0.66) Kolbe et al. (25), here the correlation between the error rate (or percent correct rate) and both BICAMS and SDMT failed to meet the criteria for statistical significance following correction for multiple comparisons. Although no studies investigated direction correlations between anti-saccade error rate and the EDSS, Polet et al. (18) showed that individuals with a high EDSS had increased antisaccade error rates than those with low EDSS. Here, however, there was no significant correlation between the percent correct rate and the EDSS. Finally, the two strongest links we found across all task parameters with the MS-related outcome measures have not been, to our knowledge, previously demonstrated. There were significant correlations between all outcome measures and two anti-saccade parameters: time-to-correct and time-to-target. The latter is a measure of the time elapsed between the onset of target appearance and the end of the final saccade, whereas the former is the time elapsed between an initial incorrect saccade and the initiation of a secondary corrective saccade in the opposite direction.

Regarding smooth pursuit, Lizak et al. (26)showed that there was a strong correlation between the onset latency to the slowest velocity and the EDSS (rho =0.31) — a parameter not measured in the present study - they did not find significant correlations between EDSS and pursuit gain, catch-up saccade count or amplitude (correlation coefficients not reported). This is in contrast to the present data where we identified a significant correlation between the EDSS and the pursuit gain. Whereas we found no other significant correlations between any other parameter and either of the clinical scores, Rempe et al. (53) showed both that T25-FW and EDSS significantly correlated with pursuit onset latency (rho = 0.478; rho = 0.564), pursuit gain (rho = −0.519; rho = −0.484), and proportion time in pursuit (rho = −0.527; rho = −0.673) - no correlations were found for the average amplitude of the catch-up saccades. It is currently unclear as to why we did not find as many or as strong correlations in the present study. One possible explanation relates to the velocity of the pursuit stimulus used — while both studies use a step-ramp pursuit paradigm, the velocities reported by Rempe et al. (53) were in the 16–24 deg./s range, whereas here we report findings for a pursuit velocity of 8 deg./s. Fortunately, we should be able to address this issue in the future as we have collected data with higher pursuit target velocities in this study, but parameters measured at these velocities were not selected as part of the a priori parameters to be examined in this preliminary study report.

The present study is not without limitations. Among them are the small sample size and the use of a small number of oculomotor parameters to investigate relationships with, and derive estimates of, clinical scale scores. Upon completion of the study, the sample size will have more than doubled and the number of investigated oculomotor parameters will have significantly increased as well. Another limitation relates to the long-term goal of being able to accurately estimate clinical scale scores in a given individual based on oculomotor parameters. Although here we show very promising results via the partial least squares regression analyses that suggest that it may be possible to do so, such analyses produce inference models, which do not guarantee strong predictive abilities. Indeed, to be able to confidently claim that we can estimate disease status or severity in a single individual, predictive models need to be validated with an independent dataset. Another potential limitation is the inclusion of the SDMT score in the MSFC composite calculation. Although there is a precedent for doing so [e.g., Drake et al. (54)], it will likely limit direct comparability with both past and future studies using the traditional MSFC score composite. We nonetheless felt the PASAT had important limitations in deciding not to include it in the current trial, including a lower test–retest reliability than the SDMT and important ceiling effects (46), and limited use outside of clinical trials due to both being a lengthy assessment time and requiring specialized audio equipment not routinely available in clinics (55).

To conclude, this cross-sectional study shows promising correlations between individual oculomotor parameters and validated clinical assessment scale scores, similar to previously published studies using research-grade eye trackers. When completed, this trial will hopefully demonstrate the reliability of mobile oculomotor assessments for the monitoring of MS progression as a non-invasive, accessible, scalable and sensitive novel digital biomarker of disease progression - both for cognitive and physical disability. Indeed, the results of the multiple regression analyses show that a large portion of the clinical outcome measure variance can be explained using only a subset of oculomotor parameters. The next steps in this project will focus on integrating machine learning models comprising combinations of multiple oculomotor parameters to optimize the reliability and accuracy of the clinical scale estimates in MS patients.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the Veritas IRB and the McGill University Health Center (MUHC) research ethics board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

ÉV-S and PG: conceptualization and supervision. PV, JC-F, AD-P, and NK: formal analysis. JC-F and PG: investigation. ÉV-S, PV, and PG: methodology. JC-F and NK: visualization. PV: writing – original draft preparation. ÉV-S, PV, NB, JC-F, SH, NM, FB, and DG: writing – review and editing. All authors contributed to the article and approved the submitted version.

Funding

This study was funded both by Innodem Neurosciences and Novartis Pharmaceuticals Canada Inc.

Conflict of interest

ÉV-S was a co-founder of Innodem Neurosciences, which developed the Eye-Tracking Neurological Assessment (ETNA™) technology used in this study. PV and AD-P has ownership options in Innodem Neurosciences. JC-F was a part-time employee of Innodem Neurosciences and NK was a research intern at Innodem Neurosciences. DG was an unpaid consultant of Innodem Neurosciences. NB, SH, and FB were employees of Novartis Pharmaceuticals Canada Inc. PG was a paid medical and scientific advisor with Innodem Neurosciences and has stock options in Innodem Neurosciences.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2023.1243594/full#supplementary-material

Supplementary FIGURE 1 | Illustrated the individual trial gaze traces for each participant when performing large rightward saccades, color-coded based on the patient EDSS score (red: EDSS 0-2, blue: 2-4, green: 4-8), along with the average trace for each of the three EDSS groupings (n = 20).

SUPPLEMENTARY TABLE 1 | Multiple regression tables for EDSS, SDMT, MSFC, and BICAMS. Regression coefficients (B), standard error, standardized regression coefficients (ß), t-statistic and associated p-value for each oculomotor parameter. Non-adjusted and adjusted coefficient of determination (R2 and Adj. R2) as well as the F-statistic and associated p-value for each partial least squares regression model.

SUPPLEMENTARY TABLE 2 | Correlations between eye-movement parameters from the four eye-tracking task categories and the MS-related clinical outcome measures of interest.

References

1. Hauser, SL, Oksenberg, JR, Hauser, SL, and Oksenberg, JR. The neurobiology of multiple sclerosis: genes, inflammation, and neurodegeneration. Neuron. (2006) 52:61–76. doi: 10.1016/j.neuron.2006.09.011

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Trapp, BD, and Nave, KA. Multiple sclerosis: an immune or neurodegenerative disorder? Annu Rev Neurosci. (2008) 31:247–69. doi: 10.1146/annurev.neuro.30.051606.094313

CrossRef Full Text | Google Scholar

3. Kuhlmann, T, Moccia, M, Coetzee, T, Cohen, JA, Correale, J, Graves, J, et al. International advisory committee on clinical trials in multiple sclerosis. Multiple sclerosis progression: time for a new mechanism-driven framework. Lancet Neurol. (2023) 22:78–88. doi: 10.1016/S1474-4422(22)00289-7

CrossRef Full Text | Google Scholar

4. Romero, K, Shammi, P, and Feinstein, A. Neurologists′ accuracy in predicting cognitive impairment in multiple sclerosis. Mult Scler Relat Disord. (2015) 4:291–5. doi: 10.1016/j.msard.2015.05.009

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Anderson, TJ, and MacAskill, MR. Eye movements in patients with neurodegenerative disorders. Nat Rev Neurol. (2013) 9:74–85. doi: 10.1038/nrneurol.2012.273

CrossRef Full Text | Google Scholar

6. Bueno, APA, Sato, JR, and Hornberger, M. Eye tracking – the overlooked method to measure cognition in neurodegeneration? Neuropsychologia. (2019) 133:107191. doi: 10.1016/j.neuropsychologia.2019.107191

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Crotty, GF, and Chwalisz, BK. Ocular motor manifestations of movement disorders. Curr Opin Ophthalmol. (2019) 30:443–8. doi: 10.1097/ICU.0000000000000605

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Terao, Y, Fukuda, H, Ugawa, Y, and Hikosaka, O. New perspectives on the pathophysiology of Parkinson's disease as assessed by saccade performance: a clinical review. Clin Neurophysiol. (2013) 124:1491–506. doi: 10.1016/j.clinph.2013.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Goffart, L, Bourrelly, C, and Quinton, JC. Neurophysiology of visually guided eye movements: critical review and alternative viewpoint. J Neurophysiol. (2018) 120:3234–45. doi: 10.1152/jn.00402.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Leigh, RJ, and Zee, DS. The neurology of eye movements. 4th ed. New York, NY, USA: Oxford University Press (2006).

Google Scholar

11. Gorges, M, Pinkhardt, EH, and Kassubek, J. Alterations of eye movement control in neurodegenerative movement disorders. J Ophthalmol. (2014) 2014:658243:1–11. doi: 10.1155/2014/658243

CrossRef Full Text | Google Scholar

12. Serra, A, Chisari, CG, and Matta, M. Eye movement abnormalities in multiple sclerosis: pathogenesis, modeling, and treatment. Front Neurol. (2018) 9:31. doi: 10.3389/fneur.2018.00031

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Fielding, J, Clough, M, Beh, S, Millist, L, Sears, D, Frohman, AN, et al. Ocular motor signatures of cognitive dysfunction in multiple sclerosis. Nat Rev Neurol. (2015) 11:637–45. doi: 10.1038/nrneurol.2015.174

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Liu, Z, Yang, Z, Gu, Y, Liu, H, and Wang, P. The effectiveness of eye tracking in the diagnosis of cognitive disorders: a systematic review and meta-analysis. PLoS One. (2021) 16:e0254059. doi: 10.1371/journal.pone.0254059

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Mallery, RM, Poolman, P, Thurtell, MJ, Full, JM, Ledolter, J, Kimbrough, D, et al. Visual fixation instability in multiple sclerosis measured using SLO-OCT. Invest Ophthalmol Vis Sci. (2018) 59:196–201. doi: 10.1167/iovs.17-22391

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Nij Bijvank, JA, Petzold, A, Coric, D, Tan, HS, Uitdehaag, BMJ, Balk, LJ, et al. Quantification of visual fixation in multiple sclerosis. Invest Ophthalmol Vis Sci. (2019) 60:1372–83. doi: 10.1167/iovs.18-26096

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Sheehy, CK, Bensinger, ES, Romeo, A, Rani, L, Stepien-Bernabe, N, Shi, B, et al. Fixational microsaccades: a quantitative and objective measure of disability in multiple sclerosis. Mult Scler. (2020) 26:343–53. doi: 10.1177/1352458519894712

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Polet, K, Hesse, S, Cohen, M, Morisot, A, Joly, H, Kullmann, B, et al. Video-oculography in multiple sclerosis: links between oculomotor disorders and brain magnetic resonance imaging (MRI). Mult Scler Relat Disord. (2020) 40:101969. doi: 10.1016/j.msard.2020.101969

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Finke, C, Pech, LM, Sömmer, C, Schlichting, J, Stricker, S, Endres, M, et al. Dynamics of saccade parameters in multiple sclerosis patients with fatigue. J Neurol. (2012) 259:2656–63. doi: 10.1007/s00415-012-6565-8

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Jozefowicz-Korczynska, M, Lukomski, M, and Pajor, A. Identification of internuclear ophthalmoplegia signs in multiple sclerosis patients. Saccade test analysis. J Neuroll. (2008) 255:1006–11. doi: 10.1007/s00415-008-0819-5

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Clough, M, Foletta, P, Frohman, AN, Sears, D, Ternes, A, White, OB, et al. Multiple sclerosis: executive dysfunction, task switching and the role of attention. Mult Scler J Exp Transl Clin. (2018) 4:2055217318771781. doi: 10.1177/2055217318771781

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Nij Bijvank, JA, Petzold, A, Coric, D, Tan, HS, Uitdehaag, BMJ, Balk, LJ, et al. Saccadic delay in multiple sclerosis: a quantitative description. Vis Res. (2020) 168:33–41. doi: 10.1016/j.visres.2020.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Nygaard, GO, de Rodez Benavent, SA, Harbo, HF, Laeng, B, Sowa, P, Damangir, S, et al. Eye and hand motor interactions with the symbol digit modalities test in early multiple sclerosis. Mult Scler Relat Disord. (2015) 4:585–9. doi: 10.1016/j.msard.2015.08.003

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Fielding, J, Kilpatrick, T, Millist, L, and White, O. Antisaccade performance in patients with multiple sclerosis. Cortex. (2009) 45:900–3. doi: 10.1016/j.cortex.2009.02.016

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Kolbe, SC, Kilpatrick, TJ, Mitchell, PJ, White, O, Egan, GF, and Fielding, J. Inhibitory saccadic dysfunction is associated with cerebellar injury in multiple sclerosis. Hum Brain Mapp. (2014) 35:2310–9. doi: 10.1002/hbm.22329

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Lizak, N, Clough, M, Millist, L, Kalincik, T, White, OB, and Fielding, J. Impairment of smooth pursuit as a marker of early multiple sclerosis. Front Neurol. (2016) 7:206. doi: 10.3389/fneur.2016.00206

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Gajamange, S, Shelton, A, Clough, M, White, O, Fielding, J, and Kolbe, S. Functional correlates of cognitive dysfunction in clinically isolated syndromes. PLoS One. (2019) 14:e0219590. doi: 10.1371/journal.pone.0219590

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Armstrong, RA . Oculo-visual dysfunction in Parkinson's disease. J Parkinsons Dis. (2015) 5:715–26. doi: 10.3233/JPD-150686

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Kassavetis, P, Kaski, D, Anderson, T, and Hallett, M. Eye Movement Disorders in Movement Disorders. Mov Disord Clin Pract. (2022) 9:284–95. doi: 10.1002/mdc3.13413

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Chalkias, E, Topouzis, F, Tegos, T, and Tsolaki, M. The contribution of ocular biomarkers in the differential diagnosis of Alzheimer's disease versus other types of dementia and future prospects. J Alzheimers Dis. (2021) 80:493–504. doi: 10.3233/JAD-201516

CrossRef Full Text | Google Scholar

31. de Villers-Sidani, VP, Guitton, D, Cisneros-Franco, JM, Koch, NA, and Ducharme, S. A novel tablet-based software for the acquisition and analysis of gaze and eye movement parameters: a preliminary validation study in Parkinson’s disease. Front Neurol. (2023) 14:1204733. doi: 10.3389/fneur.2023.1204733

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Kurtzke, JF . Rating neurological impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. (1983) 33:1444–52. doi: 10.1212/WNL.33.11.1444

CrossRef Full Text | Google Scholar

33. Benedict, RH, Amato, MP, Boringa, J, Brochet, B, Foley, F, Fredrikson, S, et al. Brief international cognitive assessment for MS (BICAMS): international standards for validation. BMC Neurol. (2012) 12:55. doi: 10.1186/1471-2377-12-55

CrossRef Full Text | Google Scholar

34. Langdon, DW, Amato, MP, Boringa, J, Brochet, B, Foley, F, Fredrikson, S, et al. Recommendations for a brief international cognitive assessment for multiple sclerosis (BICAMS). Mult Scler. (2012) 18:891–8. doi: 10.1177/1352458511431076

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Fischer, JS, Rudick, RA, Cutter, GR, and Reingold, SC. The multiple sclerosis functional composite measure (MSFC): an integrated approach to MS clinical outcome assessment. National MS Society clinical outcomes assessment task force. Mult Scler. (1999) 5:244–50. doi: 10.1177/135245859900500409

CrossRef Full Text | Google Scholar

36. Rudick, R, Antel, J, Confavreux, C, Cutter, G, Ellison, G, Fischer, J, et al. Recommendations from the National Multiple Sclerosis Society clinical outcomes assessment task force. Ann Neurol. (1997) 42:379–82. doi: 10.1002/ana.410420318

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Benedict, RH, Cookfair, D, Gavett, R, Gunther, M, Munschauer, F, Garg, N, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc. (2006) 12:549–58. doi: 10.1017/S1355617706060723

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Glanz, BI, Healy, BC, Rintell, DJ, Jaffin, SK, Bakshi, R, and Weiner, HL. The association between cognitive impairment and quality of life in patients with early multiple sclerosis. J Neurol Sci. (2010) 290:75–9. doi: 10.1016/j.jns.2009.11.004

CrossRef Full Text | Google Scholar

39. van Schependom, J, D’hooghe, MB, Cleynhens, K, D’hooge, M, Haelewyck, MC, de Keyser, J, et al. Reduced information processing speed as primum movens for cognitive decline in MS. Mult Scler. (2015) 21:83–91. doi: 10.1177/1352458514537012

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Schmidt, M . Rey auditory verbal learning test: a handbook. Los Angeles, CA: Western Psychological Services (1996).

Google Scholar

41. Beier, M, Alschuler, K, Amtmann, D, Hughes, A, Madathil, R, and Ehde, D. iCAMS: assessing the reliability of a brief international cognitive assessment for multiple sclerosis (BICAMS) tablet application. Int J MS Care. (2020) 22:67–74. doi: 10.7224/1537-2073.2018-108

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Beier, M, Hughes, AJ, Williams, MW, and Gromisch, ES. Brief and cost-effective tool for assessing verbal learning in multiple sclerosis: comparison of the Rey auditory verbal learning test (RAVLT) to the California verbal learning test – II (CVLT-II). J Neurol Sci. (2019) 400:104–9. doi: 10.1016/j.jns.2019.03.016

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Baetge, SJ, Filser, M, Renner, A, Ullrich, S, Lassek, C, and Penner, IK. On the validity of single tests, two-test combinations and the full brief international cognitive assessment for multiple sclerosis (BICAMS) in detecting patients with cognitive impairment. Mult Scler. (2020) 26:1919–28. doi: 10.1177/1352458519887897

CrossRef Full Text | Google Scholar

44. Filser, M, Schreiber, H, Pöttgen, J, Ullrich, S, Lang, M, and Penner, IK. The brief international cognitive assessment in multiple sclerosis (BICAMS): results from the German validation study. J Neurol. (2018) 265:2587–93. doi: 10.1007/s00415-018-9034-1

CrossRef Full Text | Google Scholar

45. Cutter, GR, Baier, ML, Rudick, RA, et al. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain. (1999) 122:871–82. doi: 10.1093/brain/122.5.871

CrossRef Full Text | Google Scholar

46. Sonder, JM, Burggraaff, J, Knol, DL, Polman, CH, and Uitdehaag, BM. Comparing long-term results of PASAT and SDMT scores in relation to neuropsychological testing in multiple sclerosis. Mult Scler. (2014) 20:481–8. doi: 10.1177/1352458513501570

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Guitton, D, Buchtel, HA, and Douglas, RM. Frontal lobe lesions in man cause difficulties in suppressing reflexive glances and in generating goal-directed saccades. Exp Brain Res. (1985) 58:455–72. doi: 10.1007/BF00235863

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Schweitzer, R, and Rolfs, M. An adaptive algorithm for fast and reliable online saccade detection. Behav Res Methods. (2020) 52:1122–39. doi: 10.3758/s13428-019-01304-3

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Dai, W, Selesnick, I, Rizzo, JR, Rucker, J, and Hudson, T. “A parametric model for saccadic eye movement,” in In 2016 IEEE signal processing in medicine and biology symposium (SPMB). IEEE (2016). 1–6.

Google Scholar

50. Benjamini, Y, and Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc. (1995) 57:289–300. doi: 10.1016/s0166-4328(01)00297-2

CrossRef Full Text | Google Scholar

51. Zangemeister, WH, Heesen, C, Röhr, D, and Gold, SM. Oculomotor Fatigue and Neuropsychological Assessments mirror Multiple Sclerosis Fatigue. J Eye Mov Res. (2020) 13:10.16910/jemr.13.4.6. doi: 10.16910/jemr.13.4.6

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Berrigan, LI, LeFevre, JA, Rees, LM, Berard, JA, Francis, A, Freedman, MS, et al. The symbol digit modalities test and the paced auditory serial addition test involve more than processing speed. Mult Scler Relat Disord. (2022) 68:104229. doi: 10.1016/j.msard.2022.104229

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Rempe, T, Dastgheyb, N, Miner, A, Palomino, M, Kinkel, R, Liston, D, et al. Quantification of smooth pursuit dysfunction in multiple sclerosis. Mult Scler Relat Disord. (2021) 54:103073. doi: 10.1016/j.msard.2021.103073

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Drake, AS, Weinstock-Guttman, B, Morrow, SA, Hojnacki, D, Munschauer, FE, and Benedict, RH. Psychometrics and normative data for the multiple sclerosis functional composite: replacing the PASAT with the symbol digit modalities test. Mult Scler. (2010) 16:228–37. doi: 10.1177/1352458509354552

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Cossburn, M, Moore, P, Baker, KE, Pickersgill, TP, and Robertson, NP. 130 a pilot study of a timed short addition test as a tool for the longitudinal study of cognitive dysfunction in multiple sclerosis. J Neurol Neurosurg Psychiatry. (2012) 83:e1.82–e1. doi: 10.1136/jnnp-2011-301993.172

CrossRef Full Text | Google Scholar