Jerry Slotkin ^1* Aaron J Kaat ² Stephanie Young ² Elizabeth M. Dworak ² Miriam Novack ² Yusuke Shono ³ Hubert Adam ² Cindy Nowinski ² Sarah Pila ² Zahra Hosseinian ² Maria Varela Diaz ² Anyelo Almonte-Correa ² Keith Alperin ⁴ Monica R Camacho ^5,6 Bernard Landavazo ^5,6 Rachel Nosheny ^5,6 Michael W Weiner ^5,6 Richard Gershon ²

• ¹ Center for Health Assessment Research and Translation, University of Delaware, Newark, United States
• ² Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
• ³ School of Global Health, Claremont Graduate University, Claremont, California, United States
• ⁴ Helium Foot Software, Inc., Chicago, United States
• ⁵ University of California, San Francisco, San Francisco, California, United States
• ⁶ San Francisco Veteran's Administration Medical Center, Northern California Institute for Research and Education (NCIRE), San Francisco, California, United States

The ability to assess cognitive skills remotely is increasing with the widespread use and availability of smartphones. The Mobile Toolbox (MTB) is a measurement system that includes Sequences, a new measure of working memory designed specifically for smartphones. This paper describes the development of Sequences, and the studies conducted to assess its psychometric properties.We developed a new measure of working memory for remote self-administration on an iOS or Android smartphone. In Sequences, a series of numbers and letters is shown on screen one at a time, and the participant must first tap the letters they saw in alphabetical order, and then the numbers from smallest to largest. Sequences was evaluated for usability and feasibility across two pilot studies, then assessed in this validation study (total N=1,246). Psychometric properties of the new measure were evaluated in three studies with participants ages 18-90. In Study 1 (N=92), participants completed MTB measures in the lab and were administered both an equivalent NIH Toolbox (NIHTB) measure and external measures of similar constructs. In Study 2 (N=1,007), participants were administered NIHTB measures in the lab and then completed MTB measures remotely on their own devices. In Study 3 (N=147), participants completed MTB measures twice remotely on their own devices, two weeks apart.Results: Sequences exhibited moderately high correlations with a comparable NIHTB test and with external measures of a similar construct, and a lower correlation with an unrelated test, as hypothesized. Internal consistency was high, but test-retest reliability was moderate. When controlling for age, phone operating system (iOS vs. Android) and sex assigned at birth did not significantly impact performance, but there was a significant difference between those who completed college compared to those with a high school or lower education.Results support the validity of Sequences as a measure of working memory for remote, self-administered use. Internal consistency was strong, with moderate test-retest reliability that is likely a function of the test's unproctored self-administration method. Findings suggest that Sequences is appropriate for use across the adult lifespan (ages 18-90) in remote, self-administered designs that focus on group results.