Functional MRI of Handwriting Tasks: A Study of Healthy Young Adults Interacting with a Novel Touch-Sensitive Tablet

Karimpoor, Mahta; Churchill, Nathan W.; Tam, Fred; Fischer, Corinne E.; Schweizer, Tom A.; Graham, Simon J.

doi:10.3389/fnhum.2018.00030

ORIGINAL RESEARCH article

Front. Hum. Neurosci., 13 February 2018

Sec. Cognitive Neuroscience

Volume 12 - 2018 | https://doi.org/10.3389/fnhum.2018.00030

Functional MRI of Handwriting Tasks: A Study of Healthy Young Adults Interacting with a Novel Touch-Sensitive Tablet

$\r\nMahta Karimpoor*$ Mahta Karimpoor^1*

Fred Tam¹

¹Department of Medical Biophysics, Sunnybrook Research Institute, University of Toronto, Toronto, ON, Canada
²Department of Neurosurgery, Keenan Research Centre of the Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, ON, Canada
³Geriatric Psychiatry, Department of Psychiatry, St. Michael’s Hospital, Toronto, ON, Canada

Handwriting is a complex human activity that engages a blend of cognitive and visual motor skills. Current understanding of the neural correlates of handwriting has largely come from lesion studies of patients with impaired handwriting. Task-based fMRI studies would be useful to supplement this work. To address concerns over ecological validity, previously we developed a fMRI-compatible, computerized tablet system for writing and drawing including visual feedback of hand position and an augmented reality display. The purpose of the present work is to use the tablet system in proof-of-concept to characterize brain activity associated with clinically relevant handwriting tasks, originally developed to characterize handwriting impairments in Alzheimer’s disease patients. As a prelude to undertaking fMRI studies of patients, imaging was performed of twelve young healthy subjects who copied sentences, phone numbers, and grocery lists using the fMRI-compatible tablet. Activation maps for all handwriting tasks consisted of a distributed network of regions in reasonable agreement with previous studies of handwriting performance. In addition, differences in brain activity were observed between the test subcomponents consistent with different demands of neural processing for successful task performance, as identified by investigating three quantitative behavioral metrics (writing speed, stylus contact force and stylus in air time). This study provides baseline behavioral and brain activity results for fMRI studies that adopt this handwriting test to characterize patients with brain impairments.

Introduction

Handwriting is a complex everyday skill that requires a combination of cognition, language processing, kinematics, motor planning, eye-hand coordination and visual-motor integration (Reisman, 1993; Rosenblum et al., 2003). Given this set of complex dependencies, it is not surprising that handwriting performance can be affected by many different brain impairments including Alzheimer’s disease (AD) (Platel et al., 1993; Croisile, 1999; Slavin et al., 1999; Schröter et al., 2003; Werner et al., 2006), developmental learning difficulties (Rosenblum et al., 2003), schizophrenia (Tigges et al., 2000), cerebrovascular disease (Auerbach and Alexander, 1981; Otsuki et al., 1999) and traumatic brain injury (Yorkston et al., 1997).

The functional neuroanatomy that supports handwriting performance has been investigated previously in multiple studies that have administered appropriate behavioral tests to patients with brain lesions. Whereas lesions of the angular gyrus produce impaired handwriting and impaired reading (Benson et al., 1982), other lesion studies have suggested the existence of handwriting-specific sites in parietal and frontal regions of the brain where lesions have been shown to affect handwriting but not other related tasks (Basso et al., 1978). Impaired handwriting but intact reading has been associated with lesions of the SPL (Yokota et al., 1990). Patients with lesions of the GMFA, a region of the MiFG close to the SFG, have shown impaired handwriting with preserved reading skills as well as intact simple motor functions (e.g., finger tapping) (Roux et al., 2009).

The exact contributions of regions such as the GMFA and the SPL in the control of handwriting remain incompletely understood (Planton et al., 2013). This uncertainty extends to the interpretation of neuropsychological (NP) tests developed to evaluate handwriting performance and associated brain impairments. It is generally known that the relationship between NP test scores and impaired brain function is complicated. The brain regions that support performance of specific NP tests may not have been fully and unambiguously identified, and a poor NP test score may arise from damage to one or more of the supporting interconnected brain regions (Lezak, 2004; Strauss et al., 2006). Therefore, it is difficult to localize the site of brain injury in patients with handwriting impairments using the appropriate NP tests. The limitations of using lesion studies to characterize the functional neuroanatomical correlates of a specific NP test are also well known (Hebben and Milberg, 2009).

An important approach to address these issues involves the use of task-based functional neuroimaging technology to probe the brain activity associated with handwriting. In particular, the fMRI method has been developed over approximately the last 25 years and is widely recognized as a safe, non-invasive and indirect probe of neural activity by neurovascular coupling mechanisms (Ogawa et al., 1990, 1992), at approximately millimeter spatial resolution throughout the entire brain.

Performing complex pen-and-paper tests during fMRI is challenging, however, from a logistical perspective. The fMRI methodology must be developed carefully to ensure that handwriting tasks are as ecologically valid as possible during imaging, recruiting similar aspects of brain function and behavior as handwriting administered under conventional testing conditions in the “real world.” The interplay between visuospatial, eye-hand coordination, and fine motor skills is particularly important in this context, and has led to methodological limitations in several previous studies. In two examples (Beeson et al., 2003; Segal and Petrides, 2012), subjects were instructed to write one word on top of the other using pen and paper without receiving visual feedback of their handwriting. Another study mapped brain activity while handwriting was performed with the index finger (Katanoda et al., 2001). Although the fMRI findings of such studies are partly consistent with the previous lesion work, they suffer from the limitation that the handwriting tasks were somewhat limited in their ecological validity.

For enhanced ecological validity of handwriting and drawing tasks, our lab previously designed and validated a computerized fMRI-compatible tablet system, consisting of a touch-sensitive surface that could be operated using a stylus (Tam et al., 2011). In an additional refinement, a video camera and augmented reality display were added to provide enhanced VFHP, which improved writing performance in young healthy adults (Karimpoor et al., 2015). Other studies have also shown that VFHP is an important human factor that should be included particularly for patients with brain impairments, as they may rely on this form of visuomotor integration to maintain performance on tasks that healthy individuals do not find challenging (Halligan et al., 1996). For example, AD patients show impaired performance when moving a cursor to a target, or when handwriting, when VFHP was not provided (Ghilardi et al., 1999, 2000; Slavin et al., 1999). Thus, providing VFHP may be advantageous to ensure that when patients with brain impairments are assessed using a specific NP test, the resultant test scores reflect impairment on the specific domains of interest attributed to the NP test, rather than confounds associated with difficulties in tablet interaction.

The purpose of the present work is to use fMRI methods together with the above-mentioned tablet technology to characterize the brain activity associated with an NP test developed specifically to assess handwriting performance and kinematics. This test was developed for use with a digitizing tablet to assess handwriting impairments (agraphia) in AD patients (Werner et al., 2006). Agraphia was observed in the original case study of AD (Maurer et al., 1997), and is recognized as a manifestation of the disease that includes both language processing difficulties as well as motor impairments such as reduced pen pressure and writing velocity (LaBarge et al., 1992; Croisile, 1999; Slavin et al., 1999). The NP test in question consists of three simple tasks: copying a grocery list; a phone number list; and a paragraph (sentence), that may require different levels of demand on handwriting behavior, language processing and brain activity. In the present study, the rationale for choosing this NP test was that (1) the initial work by Werner et al. (2006) indicated strong ability of this test to differentiate patients with mild cognitive impairment (MCI) and patients with early probable AD from healthy elderly controls; (2) this NP test has direct relevance to activities of daily living and especially day-to-day handwriting activities; (3) the functional neuroanatomical correlates of this NP test are not well determined; and (4) the test potentially provides a highly complementary adjunct to other NP tests typically applied to assess cognitive impairment in AD patients, such as the Mini-Mental State Examination (MMSE) (Folstein et al., 1975) and those that focus on memory (Blackwell et al., 2003). As a prelude to undertaking fMRI studies to investigate the performance of AD patients using this NP test, the present study consists of preliminary proof-of-concept validation work involving young healthy adults. It is hypothesized that the brain activity supporting this NP test (a) consists of a distributed network of regions in reasonable agreement with previous neuropsychological and neuroimaging studies of handwriting performance; and (b) shows differences between the test subcomponents that are consistent with different demands of neural processing for successful task performance.

Materials and Methods

Subjects

The study was conducted with the approval (# 080-2011) of the Research Ethics Board at Sunnybrook Health Sciences Centre in Toronto, and with the free and informed consent of the volunteer subjects. Twelve young, healthy adults (five male and seven female; mean age 25 years) were recruited from the population of graduate students at the University of Toronto. All subjects were right-handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971); free from standard MRI exclusion criteria (e.g., claustrophobia, ferromagnetic implants); and free from any past or present neurological or psychiatric impairments.

fMRI Tasks

Three simple writing tasks were developed following an approach similar to that of Werner et al. (2006). The three tasks (Figure 1) consisted of (a) copying a grocery list; (b) copying phone numbers, and (c) copying a sentence. For the first two tasks, subjects were required to copy four grocery items and two phone numbers on one display screen, respectively. Visual stimuli were listed on the left side of the display screen, with a boxed area located on the right side for making handwriting responses. The procedure was similar for the third task, except that the response box was located below the sentence. Subjects were instructed to write in their usual writing style, in lowercase cursive script, as quickly as possible while maintaining accuracy. The maximum duration for each task was 35 s. It was anticipated that subjects would have variable completion times for each task block and that in some cases, the completion time would be less than the maximum duration. Subjects were therefore instructed to lift the stylus off the tablet surface once they had finished performing all elements of a given task, such that the stylus contact force data could be used to record task completion (see fMRI-compatible tablet technology, below). Each task was separated from the next by a baseline condition of visual fixation, consisting of a white screen with a central black fixation cross, of 10 s duration. Each task was repeated four times with different words and numbers for copying, and with the tasks presented in pseudo-random order. The fMRI time series data were subsequently collected in one “run” of 9 min and 10 s duration, containing all three handwriting tasks and baseline conditions. The time series also included an initial 10 s of dead time to ensure that the fMRI signal was at equilibrium prior to presenting the first handwriting task block.

FIGURE 1

FIGURE 1. Handwriting tasks for fMRI and representative behavioral responses: (A) copying a grocery list; (B) copying phone numbers; (C) copying a sentence.

Outside the magnet prior to fMRI, subjects practiced one trial of each writing task to become familiar with using the tablet. The practice trials involved novel stimuli that were not repeated during imaging.

fMRI-Compatible Tablet Technology

Subjects operated the tablet while lying supine in the magnet bore, with the tablet mounted on an adjustable support stand that lifted the touch-sensitive surface away from the torso. The support ensured that respiratory motion did not move the tablet and thereby affect task performance. The position and angle of the support were also adjusted for each subject to enable tablet interactions comfortably with forearm or wrist support. Foam padding was used to eliminate possible discomfort at pressure points, such as the elbow in contact with the magnet bore. The tablet system included the same resistive touch-sensitive surface used in previous fMRI studies for converting localized contact force to position coordinate values, and for locating these values on a computer display (Tam et al., 2011). A force sensor (FSR 400, 30-49649, Interlink Electronics, Camarillo, CA, United States) was also located at the stylus tip to record relative force values as a function of time. Both (x, y) coordinates and force parameters were sampled and logged to a computer file at a rate of approximately 40 Hz using E-prime software (Psychology Software Tools, Inc., Sharpsburg, PA, United States).

The tablet was also equipped with an MRI-compatible video camera with a color CMOS sensor (12M-i, MRC Instruments Gmbh, Germany) and custom illuminator (Karimpoor et al., 2015). Using custom programs written in MATLAB (The Mathworks Inc., Natick, MA, United States) a real-time segmentation procedure was used to isolate the hand and stylus from each camera video frame, using a simple skin color detection algorithm performed in Red–Green–Blue (RGB) space (Kovac et al., 2003). In addition, the color properties of the stylus were adjusted to fall within the RGB distribution of skin color, using a red plastic cover. Touching the stylus to the tablet and pressing downward beyond a preconfigured force threshold resulted in “ink” marks at the appropriate locations on the display. The camera and stimulus/response video signals were then superimposed in an augmented reality display, projected on a rear-projection screen and viewed by the subject via a mirror on the MRI system’s head coil.

Behavioral Data Analysis

Digitized tablet recordings were processed for each task and for each subject to extract three metrics similar to those reported by Werner et al. (2006): mean handwriting speed (pixels/s); the mean time that the stylus was held in the air during handwriting; and mean stylus contact force. Writing speed, s, was quantified by dividing the number of pixels “inked” during a given block, by the performance time (the completion time or the block duration, as appropriate). The time that the stylus was held in the air (in air time, IAT) during writing performance was quantified as the total time that zero force was recorded during each task. The stylus contact force, F, was quantified as the temporal average of the force recording data for each task. Mean values were determined for each metric by averaging over all trials of each task. The data obtained for each metric were also submitted to one-sample Kolmogorov–Smirnov tests to assess for normal distributions. Statistically significant differences in each metric were subsequently assessed between tasks using the Friedman test in MATLAB (The Mathworks Inc., Natick, MA, United States), with a Bonferroni post hoc correction for multiple comparisons (three comparisons; corrected alpha value = 0.017).

Imaging Parameters

All imaging was conducted at 3.0 T using a research-dedicated whole-body MRI system (MR750, GE Healthcare, Waukesha, WI, United States), with a standard 8-channel head coil receiver. An angled mirror was attached to the head coil so that the subject could view visual stimuli on a rear-projection screen mounted at the rear of the magnet bore. Anatomical MRI was undertaken using inversion recovery-prepped three dimensional (3D) fast spoiled gradient echo imaging [3DFSPGR, inversion time (TI) = 300 ms, repetition time (TR) = 7.0 ms, echo time (TE) = 3.1 ms, flip angle = 15°, FOV = 22 cm × 22 cm, matrix = 256 × 192, number of slices = 190, slice thickness = 1 mm]. These images subsequently served as an anatomical underlay to the color maps of brain activity generated from the fMRI data. Functional MRI was undertaken using a T2^∗-weighted spiral in/out pulse sequence (Glover and Law, 2001) to record brain activity via the BOLD effect (Ogawa et al., 1990, 1992) (TR = 2 s, TE = 30 ms, flip angle = 70°, FOV = 20 cm × 20 cm, matrix = 64 × 64, number of slices = 30, slice thickness = 4.5 mm). Cardiac and respiratory signals were measured during fMRI using a photoplethysmograph attached to the finger of the left hand and a respiratory belt strapped around the torso, respectively.

fMRI Analysis

Previously, it has been demonstrated that the choices made in fMRI pre-processing pipelines (the procedures conducted to remove noise sources from fMRI data prior to estimating brain activity) significantly affect the reliability of the end results at both the individual and group level (Churchill et al., 2012a,b, 2015). This is especially the case when dealing with weaker contrasts, such as the differences in brain activity that are anticipated between the handwriting tasks in the present study. Furthermore, the present study provides preliminary, proof-of-concept results for a small sample size of volunteers. The Optimization of Preprocessing Pipelines for NeuroImaging (OPPNI) methodology was chosen for fMRI analysis, therefore, specifically for its ability to provide robust results while suppressing bias from subject-dependent noise sources – achieving improved signal detection and reliability of brain activations compared to pre-processing use of fixed pipelines for groups of subjects (Churchill et al., 2012a,b, 2015). Preprocessing pipelines were optimized for each single subject (prior to group analysis) based on metrics of prediction and reproducibility, generating the most reliable and task-predictive activation maps using the Non-parametric Prediction, Activation, Influence, and Reproducibility reSampling (NPAIRS) method (Strother et al., 2002) as briefly described below.

For each subject, the fMRI time series data were divided into two “split-halves,” corresponding to the first and second halves of the data in temporal order. The first two time points from each instance of each handwriting task and each baseline condition were discarded to avoid fMRI signal transients and to model stabilized BOLD hemodynamic responses. The fastest time for a subject to complete one writing task was 14 s, and thus only the first 14 s of each handwriting task was subsequently analyzed for all subjects. This choice simplified the fMRI analysis because it ensured that all subjects were engaged in task performance during the time designated in the analysis model for the task condition, removing the need to model effects of subject-specific completion time across the allotted 35 s time window for tablet interactions. The optimal preprocessing pipelines were selected by computing metrics of prediction (P) and reproducibility (R) for all preprocessing pipelines, and then selecting the pipeline minimizing the Euclidean distance D (P, R) relative to perfect model performance (P = 1, R = 1). The P-values were computed using a classifier model based on a single split-half, and measuring the ability to predict experimental conditions in the other split-half using Bayesian posterior probability from a LDA. The R-values were computed from the Pearson correlation between split-half activation maps. Pipeline optimization was conducted for each subject, by measuring D (P, R) for all possible combinations of the following preprocessing algorithms, implemented using calls to Analysis of Functional NeuroImages freeware (AFNI) (Cox, 1996) or directly in the OPPNI software. These algorithms were applied in the following fixed order: rigid-body motion correction using 3dvolreg; cardiac and respiratory physiological noise correction (Glover et al., 2000) using 3dretroicor; slice timing correction using 3dTshift; spatial smoothing using a 6 mm FWHM Gaussian filter using 3dmerge; temporal detrending with zero to third order Legendre polynomials using 3dDetrend; motion covariate regression using PCA of 3dvolreg motion parameter estimates and regressing principal components (PCs) that accounted for >85% of motion variance; task paradigm covariate regression to protect against over-regression of task-related BOLD signals, after convolving the paradigm with the AFNI ‘SPMG1’ function (Rasmussen et al., 2012); and global signal regression by removing the first component of PCA (Carbonell et al., 2011). Pipeline optimization was conducted by testing the impact of including/excluding all of the above pipeline steps except spatial smoothing, and selecting the pipeline minimizing D (P, R) for each subject. Brain masks were generated using the Oxford Centre for Functional MRI of the Brain (FMRIB) Software Library (FSL) Brain Extraction Tool (Smith et al., 2004).

The individual subject datasets were analyzed to obtain (P, R) values and SPMs using LDA, a predictive multivariate model, regularized by projecting onto a PCA basis prior to analysis [where the optimal number of PCs was chosen to minimize D (P, R)]. The LDA–PCA algorithm was chosen as a simple, robust multivariate classifier model, enabling computation of (P, R) metrics for each pipeline. Recent work has shown that compared to other multivariate linear classifiers, LDA–PCA exhibits relatively high prediction accuracy and highly reproducible activation maps (Churchill et al., 2014; Yourganov et al., 2014), of particular importance in the low sample size scenario. For each subject, the resulting brain maps (generated by LDA–PCA) were then converted into reproducible Z-scored statistical parametric maps (rSPMs) using the procedure described in (Strother et al., 2002) prior to group-level analysis. Subjects with low reproducibility have low rSPM Z-scores, such that they contribute less to the most reproducible activation pattern for the overall group-level analysis (Strother et al., 2002). The rSPMs were subsequently transformed into a standard brain atlas space (MNI 152, Mazziotta et al., 2001) as follows: FSL flirt was used to compute the rigid-body (6-parameter) transform from fMRI images to T1-weighted images, and the affine (12-parameter) transform from T1-weighted images to the MNI template. The net transformation matrix from fMRI images to the MNI template was then computed, and applied to the rSPMs.

Group-level analysis was then performed by applying PCA to the set of individual subject rSPMs, and extracting the spatial PCs that explained the most data covariance. A split-half cross-validation framework was used at this point in the analysis as in Churchill et al. (2015) with 100 resampling iterations, to obtain a Z-scored map of brain regions associated with each PC. This procedure is very stable, such that only a small number of resampling iterations is required for robust parameter estimation (Churchill et al., 2013). Previous studies have used as few as 50 iterations (Strother et al., 2004). The contrasts that were subsequently investigated included (a) all handwriting tasks vs. baseline, identifying areas of common brain activity across the tasks; (b) copying paragraphs vs. phone numbers; and (c) copying paragraphs vs. grocery lists (results for the contrast of copying phone numbers vs. the grocery list were not statistically significant). Each contrast was optimized separately using OPPNI to address the potential concern that there might be different optimized pipeline steps required for each subject between two different contrasts to achieve maximum signal detection and reproducibility of the output activation maps. However, the resampling framework used in OPPNI is not presently able to account for repeated measures multivariate analysis of variance. The maps of brain activity for each contrast were thresholded using the two-tailed False-Discovery Rate (FDR) to correct for multiple comparisons (Genovese et al., 2002), rather than using a family-wise error (FWE) approach with a cluster-size threshold (Eklund et al., 2016). The FDR method is well established and although it provides somewhat weaker control over type I error than FWE methods (Nichols and Hayasaka, 2003), it is simple to implement and does not require an estimation of the likelihood of a given cluster size occurring by chance. Given the preliminary nature of the study, activation maps were reported at a conservative FDR of q = 0.01.

Results

After a brief training session outside the MRI system, all subjects reported that they were able to interact with the tablet easily during fMRI. Overall, subjects had completion times (group mean with standard deviation shown in brackets) of 23.1 (4.6) s for the grocery list task, 23.0 (5.4) s for the phone number task, and 26.2 (4.1) s for the sentence task. No statistically significant differences were observed between these completion times based on a one-way Friedman test with a Bonferroni correction for multiple comparisons. The behavioral results that were obtained for the three writing tasks during fMRI are shown as box and whisker plots in Figure 2. These plots show the median and interquartile range (IQR) for each metric (s, IAT, F) with the box bounding the first and third quartiles, and the whiskers extending to the most extreme data points not considered outliers (2.7 times the sample standard deviation). Outlier data points are shown as crosses. The sentence task showed significantly elevated s-values (Figure 2A) and reduced IAT values (Figure 2B) compared to the phone number task (Bonferroni corrected alpha value = 0.017; p < 0.01). In addition, the sentence task had a non-significant trend of increased s-values (p < 0.06), and significantly reduced IAT values (p < 0.002) compared to the grocery list task (Figures 2A,B). No statistically significant differences in s and IAT values were observed between the phone number and grocery list tasks. Furthermore, no statistically significant results were observed in the F-values across the three handwriting tasks (Figure 2C).

FIGURE 2

FIGURE 2. Box and whisker plots of behavioral performance metrics (A) handwriting speed s, (B) stylus in air time, IAT, and (C) stylus contact force, F for three handwriting tasks during fMRI, involving copying a grocery list, phone numbers, and a sentence. For each box, the interior horizontal line shows the median value, and edges of the box are estimates of the first and third quartile. The whiskers extend to the most extreme data points not considered outliers (2.7 times the sample standard deviation assuming a normal distribution). Outlier points are shown as crosses. Statistically significant differences between tasks (p < 0.05) are indicated by an asterisk.

Figure 3a shows group maps of selected slice locations for the first principal component (PC1), reflecting the most common pattern of brain activity for all writing tasks vs. baseline. The associated complete set of regions activated is listed in Table 1. Results for all other PCs were negligible. Activations (not all shown in Figure 3) included the left-lateralized primary somatosensory and motor cortex, as well as the bilateral SMA and pre-motor, primary visual and visual association areas. In addition, activity involved the bilateral SPL, IPL, MiFG/SFG, STG, Wernicke’s area, postCB, angular gyrus, SuMG, bilateral IFG/Broca’s area, as well as ventral pre-motor cortex, and the left lateralized VWFA located in ITS/MiOG.

FIGURE 3

FIGURE 3. Activation maps of the first principal component (PC1) for (a) all handwriting tasks vs. baseline; (b) copying a sentence vs. copying phone numbers; and (c) copying a sentence vs. copying a grocery list. All activations are thresholded at the false discovery rate of q = 0.01. Activation maps were not statistically significant for any other principal components. IFG, inferior frontal gyrus; MiFG, middle frontal gyrus; STG, superior temporal gyrus; ITG, inferior temporal gyrus; MiOG, middle occipital gyrus; IPL, inferior parietal lobule; SPL, superior parietal lobule; SMA, supplementary motor area; PostCB, posterior cerebellum; VWFA, visual word form area; aPCU, anterior precuneus; GMFA, graphomotor frontal area; SuMG, supramarginal gyrus.

TABLE 1

TABLE 1. Activated brain regions identified for all handwriting tasks vs. baseline in MNI coordinate space.

Figure 3b shows PC1 for sentence vs. phone number tasks, with the complete list of differentially activated regions given in Table 2. Activity involved the bilateral SPL/IPL, pre-cuneus, IFG, SMA, premotor, MiFG/SFG (GMFA), precentral gyrus, and IPL/SuMG, as well as left lateralized somatosensory and primary motor cortex, ITG/MiOG (VWFA), and angular gyrus. Right-lateralized activity was also observed in the postCB.

TABLE 2

TABLE 2. Activated brain regions identified for copying a sentence vs. copying phone numbers in MNI coordinate space.

Figure 3c shows PC1 for sentence vs. grocery list tasks, with all activated regions listed in Table 3. Activity involved the bilateral visual areas and lingual gyrus as well as ITG/MioG (VWFA), left lateralized SPL, IFG, insula, SuMG, post-central gyrus and the medial aPCu. Activity in the right thalamus and SMA was also observed.

TABLE 3

TABLE 3. Activated brain regions identified for copying a sentence vs. copying a grocery list in MNI coordinate space.

Discussion

The present work illustrates the brain activity and behavior of young adults who performed an NP test that was originally designed for assessment of handwriting impairments in patients with AD (Werner et al., 2006). The original test required patients to interact with a digitizing tablet, enabling various kinematic variables to be quantified. These capabilities were mirrored in the present study by incorporating a fMRI-compatible tablet system for handwriting with VFHP, with good ecological validity (Karimpoor et al., 2015). The results supported both hypotheses posited in the Introduction, namely that the observed brain activity (a) consists of a distributed network of regions in reasonable agreement with previous neuropsychological and neuroimaging studies of handwriting performance; and (b) shows differences between the NP test subcomponents consistent with different demands of neural processing for successful task performance. The following discussion evaluates these findings in more detail, focusing first on the behavioral performance and kinematic analysis of the handwriting tasks, and then on the brain activity.

Behavior

The behavioral performance results of the present study are encouraging, as they replicate some of the observations found by Werner et al. (2006). Werner et al. (2006) observed that across handwriting tasks, the IAT metric was a good discriminator of the differences in handwriting performance across AD, MCI, and healthy control subjects. The IAT metric was also sensitive in the present study, exhibiting a statistically significant decrease when young healthy adults performed the sentence task in comparison to the phone number task and the grocery list task. Similarly, the two studies consistently found that the sentence task was associated with increased writing speed in comparison to the other two tasks, and that all three tasks showed little differences in terms of force production. Beyond this qualitative agreement, however, it is not useful to perform a direct comparison of the absolute metric values reported in both studies for at least three reasons: (1) age differences in the control groups; (2) language differences [English in the present study, Hebrew in Werner et al. (2006)]; and (3) differences in completion time, as the present study involved some task modification to accommodate use of the custom tablet [which has reduced surface area compared to that used by Werner et al. (2006), and to enable a simple block design paradigm followed by group fMRI analysis].

In addition to the qualitative agreement between the two studies, there is a plausible explanation for the dependencies of the handwriting metrics across the three tasks – an issue not addressed by Werner et al. (2006). Compared to the grocery list and phone number tasks, the sentence task was performed with reduced IAT and increased speed. Furthermore, the decrease in IAT occurred even though the sentence task included 13–14 more characters on average in comparison to the other two tasks. The likely reason for these results is that writing a sentence is a more “fluid” task than the other two tasks. Subjects were easily able to read text and then copy it while handwriting from left to right, as in normal everyday activities, moving smoothly from word to word. Both the other tasks are less fluid in the sense that they require copying of random words or numbers that are not assembled in a strong semantic framework, with successive words and phone numbers proceeding from top to bottom, necessitating comparatively large diagonal movements to start each response on a new line. However, it must be emphasized that this interpretation is conjecture at present without support from scientific evidence and literature.

Brain Activity: All Handwriting Tasks

To our knowledge, no other fMRI study characterized brain activity associated with handwriting using a device similar to that used in the present study, that closely approximates the complex tactile, visual-spatial, and eye-hand coordination elements of real-world performance. Brain activity has previously been reported for writing in the air with the index finger (Katanoda et al., 2001), and writing letters on top of each other on a piece of paper without VFHP (Beeson et al., 2003). Several other studies focusing on language, reading, and spelling aspects of handwriting have involved responses that also have ecological validity limitations (Beeson et al., 2003; Longcamp et al., 2003; Dehaene et al., 2015). Most recently, Planton et al. (2017) also used a tablet for writing during a fMRI study, although without VFHP and without visual feedback of writing production. With “ground truth” lacking, the brain activations observed in the present study are compared with the existing pertinent fMRI literature available on tasks that attempt to engage similar domains of brain function. This provides an initial assessment of consistency, recognizing that additional research will be required in the future to provide stronger evidence and characterization of the neural correlates of this NP test of handwriting performance.

Considering all three writing tasks grouped together vs. baseline, the observed brain activity was consistent with performing a sensorimotor task using the dominant hand for writing and language responses. Predominantly positive z-scores were observed in the left lateralized primary somatosensory and motor cortex are consistent with the tactile and proprioceptive sensory inputs used to direct complex hand and arm movements during writing. Activation of pre-central, premotor, SMA, and the VPMc regions is consistent with planning stylus movements required to perform handwriting while visually guiding hand actions. Activation of the SPL is consistent with visual-spatial processing involved in handwriting. In particular, the involvement of the SPL (especially close to the IPL) has been observed in several neuroimaging and lesion studies of writing, and has been referred to as a writing-specific region (Katanoda et al., 2001; Menon and Desmond, 2001; Sugihara et al., 2006; Roux et al., 2009; Segal and Petrides, 2012). In addition, Segal and Petrides (2012) reported the role of the SPL in higher-level motor control involving interactions with language-processing regions during writing performance. Activation of the left angular gyrus and SuMG in the parietal lobe is consistent with the reading and semantic processing aspects of handwriting (Basso et al., 1978; Roeltgen and Heilman, 1984; Segal and Petrides, 2012).

Next, activation of the GMFA located in the MiFG/SFG, is consistent with orthographic processing, processing of graphomotor shapes of words and higher-level motor planning of a complex motor task. The observed activation of the GMFA is consistent with the findings of Planton et al. (2013) who conducted meta-analysis of cerebral network of areas commonly activated during handwriting in 18 neuroimaging studies. The GMFA has been suggested as a writing-specific region in previous fMRI studies (Roux et al., 2009; Segal and Petrides, 2012; Planton et al., 2017). In particular, Roux et al. (2009) conducted direct cortical electrical stimulation of this region and observed impaired handwriting in six patients, without disturbing hand movements or oral language tasks. The GMFA is believed to be involved in bridging between orthography and motor programs specifically during handwriting (Roux et al., 2009). Rapp and Dufor (2011) also suggested that the GMFA is engaged in orthographic-specific working memory during writing.

The observed activation of the IFG and posterior STG is consistent with language processing during handwriting performance (Turkeltaub et al., 2002). Although the NP test under investigation is described above as involving three different copying tasks, a strong language processing component is expected as subjects inevitably engaged in silent reading and comprehension of the stimuli. The left Broca’s area located in IFG is known to be involved in language production and semantic aspects of language processing (Friederici et al., 2003; Winhuisen et al., 2005). Activation of the posterior STG, known as Wernicke’s area, is consistent with language processing, phonetic sequencing and comprehension (Hickok and Poeppel, 2007).

The bilateral involvement of the insula is also consistent with sensory imagery experience and emotional context of writing tasks, as known to be processed by these regions (Craig, 2009). Furthermore, activation of the left MiOG, close to the posterior ITG is referred to as the VWFA and is known to be involved in the storage or recovery of the visual graphic images of words (Nakamura et al., 2000, 2002; Beeson et al., 2003; Rapcsak and Beeson, 2004; Planton et al., 2013).

Lastly, bilateral activity was observed in the cerebellum with involvement of the right posterior region (postCB), as indicated by positive Z-scores. Whereas the left cerebellar activation is consistent with representation of finger movements, right cerebellar activation has been similarly observed in previous fMRI studies of writing as approximated by complex, coordinated movement of the index finger, in contrast with simple movements (Katanoda et al., 2001; Segal and Petrides, 2012).

Brain Activity: Differences between Tasks

Considering first the contrast between copying sentences and copying phone numbers, the observed brain activity was consistent with the differing levels of engagement of language processing areas during complex sentence processing vs. more simple number processing. In particular, the observed activation of left IFG is consistent with more processing requirements for motor and language representation of letters compared to digits. The left IFG is part of the left VPMc and is known to be involved in the storage of motor representation of letters (Longcamp et al., 2003, 2005), or phonological processing (Omura et al., 2004). Involvement of the left SuMG and angular gyrus was also observed, consistent with complex semantic sentence processing vs. comparatively less processing of digits. In addition, activation of the VWFA is consistent with the storage and recovery of the visual graphic images of the words required for processing a sentence, rather than numbers (Nakamura et al., 2000, 2002; Beeson et al., 2003; Rapcsak and Beeson, 2004). The activation of SFG/MiFG (part of the GMFA) suggests preferential involvement of the GMFA during writing letters compared to writing digits, in agreement with the previous findings of Longcamp et al. (2014).

Activation of the left anterior SPL/IPL was also observed, suggesting more engagement of this region associated with the act of handwriting as well as high-level motor control functions required during writing a sentence (Segal and Petrides, 2012; Planton et al., 2017), compared to writing phone numbers, as measured by predominantly positive Z-scores in this region. Activation of anterior SPL/IPL and GMFA regions have been consistently associated with writing (Roux et al., 2009; Segal and Petrides, 2012; Planton et al., 2017), highly coordinated fine motor movements and, more recently, drawing (Planton et al., 2013, 2017). Planton et al. (2017) also showed that all regions known as “writing specific” regions were activated during drawing tasks whereas a subset were activated during spelling tasks, thus showing that the specialization of these regions was not absolute but preferential.

Rapp and Dufor (2011) also suggested that activity in these two writing-specific brain regions is modulated by word length (letters) during writing. The contrasted brain activity of the present study is consistent with such an effect, when taken together with the behavioral data observed for the three different handwriting tasks. Specifically, handwriting speed was elevated when copying sentences, in relation to the other two tasks, and thus a greater number of characters were written during the fixed task analysis length of 14 s. However, no firm conclusions can be drawn concerning whether relative involvement of GMFA and SPL/IPL in the current study depends much more on character length, much more on the language and motor requirements for writing words compared to digits, or relatively equal proportions of both mechanisms. This can be considered a limitation of the present study design. Future research to study these mechanisms in detail can be undertaken by designing tasks with appropriate stimulus and control conditions. Lastly for this contrast, the observed involvement of left sensorimotor regions as well as medial aPCu is consistent with use of more sensorimotor and proprioceptive resources (Nachev et al., 2008; Filimon et al., 2009) during copying of a sentence in comparison to copying phone numbers; and with the observed behavioral differences in handwriting metrics (higher writing speed and lower IAT when copying sentences).

Turning to the activation contrasts between copying a sentence and copying a grocery list, differences were again expected due to the different levels of language processing involved for the two tasks (more for copying complex sentences, less for copying a word list). Notably, less regional activity was observed for this contrast in comparison to that for copying sentences vs. copying phone numbers, as the latter contrast involves a larger difference in the level of language processing between the two tasks. In particular, copying sentences vs. copying a grocery list was associated with activation of the left SuMG, and the left MiFG/IFG. This is consistent with the findings of Segal and Petrides (2012), who observed preferential engagement of the left SuMG during writing English words in a strong semantic framework.

Furthermore, copying sentences vs. copying a grocery list showed increased activity in the lingual gyrus and secondary visual areas, SMA, precuneus, and left medial aPCU. This pattern of brain activity is consistent with the significant decrease in IAT values (i.e., more tablet contact time) associated with elevated sensorimotor planning during the former task. The activity of secondary visual areas is known to be associated with processing of complex shapes (Kravitz et al., 2013). The observed bilateral activation of the VWFA is consistent with requirements for storage and recovery of the visual graphic images of words, and as for the contrast discussed immediately above, the word-count was larger for copying sentences. One proviso concerning this interpretation is that the VWFA is known to be associated with reading and visually recognizing shapes of words, but some studies also suggest that the high-order function of this visual processing area is not only limited to reading (Cohen et al., 2000; Dehaene et al., 2005). Planton et al. (2017), observed bilateral activity of the VWFA during drawing that was stronger than the activity during writing, concluding that the activity in this region is vision-related and due to complex object recognition and visual recognition of the words. Additional research is required to provide converging evidence and resolve these differences in interpretation.

Conclusion

A fMRI study was performed in young healthy adults to identify the neural correlates of an NP test originally designed to assess handwriting impairments in patients with AD. The behavioral findings were consistent with those of Werner et al. (2006) reaffirming that the IAT kinematic metric is sensitive for discriminating performance differences across the three subcomponents of the NP test. The observed brain activity across all tasks showed good consistency with the previous fMRI literature, which to date has involved handwriting tasks implemented with less ecological validity. In particular, activity in the SPL, GMFA, and VWFA are consistent with handwriting specific regions in literature, as well as the involvement of regions in VPMc that are involved in guiding hand actions. Regarding brain activity contrasting the differences between handwriting tasks, the results were consistent with lower, intermediate and higher levels of mental processing for copying phone numbers, items of a grocery list, and sentences, which were further supported by the differences quantified in the behavioral performance data.

These findings are relevant for future fMRI studies that aim to clarify in more detail the functional roles of brain regions that process handwriting behavior. They are also important for providing baseline behavioral and brain activity results in relation to fMRI studies that adopt this handwriting NP test to characterize patients with brain impairments, such as those with AD or prodromal dysfunction, as part of larger testing protocols. The present results are preliminary, given the small sample size, and need to be replicated. Nevertheless, robust analysis procedures have been used to obtain the results and the findings are sufficiently strong to consider using the tablet with VFHP for fMRI of handwriting tasks in AD patients in the near future.

Ethics Statement

This study was carried out in accordance with the recommendations of ‘Human Research Protections Program, Sunnybrook’s Research Ethics Board (REB)’ with written informed consent from all subjects. All subjects gave written informed consent in accordance with the REB guidelines. The protocol was approved by the ‘Sunnybrook’s research ethics board committee’ (approval number 080-2011).

Author Contributions

MK: Designing the entire work, data acquisition and experimental setup, data analysis and interpretation; writing and drafting the work; final approval of the work; ensuring questions related to the work accuracy are investigated and resolved. NC: Contributions to the work conception; data analysis, and interpretation of data; revising the work critically; final approval; ensuring questions related to the work accuracy are investigated and resolved. FT: Contributions to the work conception, experimental setup; revising the work critically; final approval of the work; ensuring questions related to the work accuracy are investigated and resolved. CF: Contributions to the work conception; critically revising the work; final approval of the work; ensuring questions related to the work accuracy are investigated and resolved. TS: Contributions to the work conception and data interpretation; revising the work critically; final approval of the work; ensuring questions related to the work accuracy are investigated and resolved. SG: Designing the work; analysis and interpretation of data; revising the work critically; final approval of the work; ensuring questions related to the work accuracy are investigated and resolved.

Funding

This work was supported by the Heart and Stroke Foundation of Canada grant number G-13-00002824, and the Canadian Institutes of Health Research (CIHR) grant number 114813. MK received student stipend funds from the Department of Medical Biophysics and the University of Toronto.

Conflict of Interest Statement

This work has used a technology which was provisionally patented in 2014. Patent title: Systems and Methods for Providing Visual Feedback of Touch Panel Input during Magnetic Resonance Imaging. Publication Number: US 2016/0120437 A1.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We are very grateful to all those who participated in this study; they gave freely and generously of their time, and were a joy to work with. We would also like to thank Ruby Endre for helping us with data collection.

Abbreviations

AC, anterior cingulate; AFNI, analysis of functional neuroimaging; aPCu, anterior precuneus; BOLD, blood oxygen level dependent; FMRI, functional magnetic resonance imaging; FOV, field of view; FWHM, full width half maximum; GMFA, graphomotor frontal area; IAT, in air time; IFG, inferior frontal gyrus; IOG, inferior occipital gyrus; IPL, inferior parietal lobule; ITG, inferior temporal gyrus; LDA, linear discriminant analysis; MeFG, medial frontal gyrus; MiFG, middle frontal gyrus; MiOG, middle occipital gyrus; MiTG, middle temporal gyrus; MNI, Montreal Neurological Institute; PCA, principal component analysis; PostCB, posterior cerebellum; SFG, superior frontal gyrus; SMA, supplementary motor area; SOG, superior occipital gyrus; SPL, superior parietal lobule; SPMs, statistical parametric map; STG, superior temporal gyrus; SuMG, Supramarginal gyrus; VFHP, visual feedback of hand position; VPMc, ventral premotor cortex; VWFA, visual word form area.

References

Auerbach, S. H., and Alexander, M. P. (1981). Pure agraphia and unilateral optic ataxia associated with a left superior parietal lobule lesion. J. Neurol. Neurosurg. Psychiatry 44, 430–432. doi: 10.1136/jnnp.44.5.430