Hostname: page-component-5c6d5d7d68-lvtdw Total loading time: 0 Render date: 2024-08-19T09:18:38.976Z Has data issue: false hasContentIssue false
  • English
  • Français

Mobile toolbox (MTB) remote measures of executive function and processing speed: development and validation

Published online by Cambridge University Press:  11 July 2024

Miriam A. Novack Open the ORCID record for Miriam A. Novack [Opens in a new window] ,
Stephanie Ruth Young Open the ORCID record for Stephanie Ruth Young [Opens in a new window] ,
Elizabeth M. Dworak ,
Aaron J. Kaat Open the ORCID record for Aaron J. Kaat [Opens in a new window] ,
Jerry Slotkin ,
Cindy Nowinski ,
Lihua Yao ,
Hubert Adam ,
Jordan Stoeger  and
Zahra Hosseinian
...Show all authors
Miriam A. Novack
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Stephanie Ruth Young*
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Elizabeth M. Dworak
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Aaron J. Kaat
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Jerry Slotkin
Affiliation:
Center for Health Assessment Research and Translation, University of Delaware, Newark, DE, USA
Cindy Nowinski
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Lihua Yao
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Hubert Adam
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Jordan Stoeger
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Zahra Hosseinian
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Saki Amagai
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Sarah Pila
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Maria Varela Diaz
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Anyelo Almonte Correa
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
Keith Alperin
Affiliation:
Helium Foot Software, Inc, Chicago, IL, USA
Sonia Carlson
Affiliation:
Sage Bionetworks, Seattle, WA, USA
Michael Kellen
Affiliation:
Sage Bionetworks, Seattle, WA, USA
Larsson Omberg
Affiliation:
Sage Bionetworks, Seattle, WA, USA
Monica R. Camacho
Affiliation:
University of California, San Francisco, NCIRE, San Francisco, CA, USA Northern California Institute for Research and Education, San Francisco Veteran’s Administration Medical Center, San Francisco, CA, USA
Bernard Landavazo
Affiliation:
University of California, San Francisco, NCIRE, San Francisco, CA, USA Northern California Institute for Research and Education, San Francisco Veteran’s Administration Medical Center, San Francisco, CA, USA
Rachel L. Nosheny
Affiliation:
University of California, San Francisco, NCIRE, San Francisco, CA, USA Northern California Institute for Research and Education, San Francisco Veteran’s Administration Medical Center, San Francisco, CA, USA
Michael W. Weiner
Affiliation:
University of California, San Francisco, NCIRE, San Francisco, CA, USA Northern California Institute for Research and Education, San Francisco Veteran’s Administration Medical Center, San Francisco, CA, USA
Richard C. Gershon
Affiliation:
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
*
Corresponding author: Stephanie Ruth Young; Email: stephanieruth.young@northwestern.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

The ability to remotely monitor cognitive skills is increasing with the ubiquity of smartphones. The Mobile Toolbox (MTB) is a new measurement system that includes measures assessing Executive Functioning (EF) and Processing Speed (PS): Arrow Matching, Shape-Color Sorting, and Number-Symbol Match. The purpose of this study was to assess their psychometric properties.

Method:

MTB measures were developed for smartphone administration based on constructs measured in the NIH Toolbox® (NIHTB). Psychometric properties of the resulting measures were evaluated in three studies with participants ages 18 to 90. In Study 1 (N = 92), participants completed MTB measures in the lab and were administered both equivalent NIH TB measures and other external measures of similar cognitive constructs. In Study 2 (N = 1,021), participants completed the equivalent NIHTB measures in the lab and then took the MTB measures on their own, remotely. In Study 3 (N = 168), participants completed MTB measures twice remotely, two weeks apart.

Results:

All three measures exhibited very high internal consistency and strong test-retest reliability, as well as moderately high correlations with comparable NIHTB tests and moderate correlations with external measures of similar constructs. Phone operating system (iOS vs. Android) had a significant impact on performance for Arrow Matching and Shape-Color Sorting, but no impact on either validity or reliability.

Conclusions:

Results support the reliability and convergent validity of MTB EF and PS measures for use across the adult lifespan in remote, self-administered designs.

Keywords

Cognitionexecutive functionprocessing speedmobile assessmentNIH Toolboxvalidation
Type
Research Article
Information
Journal of the International Neuropsychological Society , First View , pp. 1 - 9
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of International Neuropsychological Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ben-Zeev, D., & Atkins, D. C. (2017). Bringing digital mental health to where it is needed most. Nature Human Behaviour, 1(12), 849–851.CrossRefGoogle Scholar
Carlozzi, N. E., Beaumont, J. L., Tulsky, D. S., & Gershon, R. C. (2015). The NIH toolbox pattern comparison processing speed test: Normative data. Archives of Clinical Neuropsychology, 30(5), 359–368.CrossRefGoogle ScholarPubMed
Carlozzi, N. E., Tulsky, D. S., Chiaravalloti, N. D., Beaumont, J. L., Weintraub, S., Conway, K., & Gershon, R. C. (2014). NIH toolbox cognitive battery (NIHTB-CB): The NIHTB pattern comparison processing speed test. Journal of the International Neuropsychological Society, 20(6), 630–641.CrossRefGoogle ScholarPubMed
Delis, D. C., Kaplan, E., & Kramer, J. H. Delis-Kaplan executive function system. Assessment (2001).CrossRefGoogle Scholar
Dunn, D. Peabody picture vocabulary test fifth edition (PPVT-5) (2018).Google Scholar
Eriksen, B. A., & Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophysics, 16(1), 143–149.CrossRefGoogle Scholar
Germine, L., Reinecke, K., & Chaytor, N. S. (2019). Digital neuropsychology: Challenges and opportunities at the intersection of science and software. Clinical Neuropsychologist, 33(2), 271–286. https://doi.org/10.1080/13854046.2018.1535662 Google ScholarPubMed
Gershon, R. C., Cook, K. F., Mungas, D., Manly, J. J., Slotkin, J., Beaumont, J. L., & Weintraub, S. (2014). Language measures of the NIH toolbox cognition battery. Journal of the International Neuropsychological Society, 20(6), 642–651.CrossRefGoogle ScholarPubMed
Gershon, R. C., Sliwinski, M. J., Mangravite, L., King, J. W., Kaat, A. J., Weiner, M. W., & Rentz, D. M. (2022). The mobile toolbox for monitoring cognitive function. The Lancet Neurology, 21(7), 589–590.CrossRefGoogle ScholarPubMed
Gershon, R. C., Wagster, M. V., Hendrie, H. C., Fox, N. A., Cook, K. F., & Nowinski, C. J. (2013). NIH toolbox for assessment of neurological and behavioral function. Neurology, 80(11_supplement_3), S2–S6.CrossRefGoogle ScholarPubMed
Harris, P. A., Swafford, J., Serdoz, E. S., Eidenmuller, J., Delacqua, G., Jagtap, V., Taylor, R. J., Gelbard, A., Cheng, A. C., & Duda, S. N. (2022). MyCap: A flexible and configurable platform for mobilizing the participant voice. JAMIA Open, 5(2), ooac047.CrossRefGoogle ScholarPubMed
Harris, P. A., Taylor, R., Minor, B. L., Elliott, V., Fernandez, M., O'Neal, L., McLeod, L., Delacqua, G., Delacqua, F., Kirby, J., & Duda, S. N. (2019). The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics, 95, 103208.CrossRefGoogle ScholarPubMed
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381.CrossRefGoogle ScholarPubMed
Healy, W., & Fernald, G. M. (1911). Tests for practical mental classification. The Psychological Monographs, 13(2), i–54.CrossRefGoogle Scholar
Heaton, R. K. Wisconsin card sorting test manual (1981), Psychological assessment resources.Google Scholar
Koo, B. M., & Vizer, L. M. (2019). Mobile technology for cognitive assessment of older adults: A scoping review. Innovation in Aging, 3(1), igy038.CrossRefGoogle ScholarPubMed
Naito, A., Wills, A.-M., Tropea, T. F., Ramirez-Zamora, A., Hauser, R. A., Martino, D., Turner, T. H., Rafferty, M. R., Afshari, M., Williams, K. L., Vaou, O., McKeown, M. J., Ginsburg, L., Ezra, A., Iansek, R., Wallock, K., Evers, C., Schroeder, K., DeLeon, R., Yarab, N., Alcalay, R. N., & Beck, J. C. (2021). Expediting telehealth use in clinical research studies: Recommendations for overcoming barriers in North America. npj Parkinson’s Disease, 7(1), 34.CrossRefGoogle ScholarPubMed
Passell, E., Strong, R. W., Rutter, L. A., Kim, H., Scheuer, L., Martini, P., Grinspoon, L., & Germine, L. (2021). Cognitive test scores vary with choice of personal digital device. Behavior Research Methods, 53(6), 2544–2557.CrossRefGoogle ScholarPubMed
R Core Team (2023). R: A language and environment for statistical computing. In R Foundation for Statistical Computing. https://www.R-project.org/ Google Scholar
Reynolds, J. R., West, R., & Braver, T. (2009). Distinct neural circuits support transient and sustained processes in prospective memory and working memory. Cerebral Cortex, 19(5), 1208–1221.CrossRefGoogle ScholarPubMed
Salthouse, T. A. (1996). The processing-speed theory of adult age differences in cognition. Psychological Review, 103(3), 403–428.CrossRefGoogle ScholarPubMed
Salthouse, T. A. (2000). Aging and measures of processing speed. Biological Psychology, 54(1-3), 35–54.CrossRefGoogle ScholarPubMed
Tinker, M. A. (1963). Influence of simultaneous variation in size of type, width of line, and leading for newspaper type. Journal of Applied Psychology, 47(6), 380–382.CrossRefGoogle Scholar
Wechsler, D. (2008). Wechsler Adult Intelligence Scale—Fourth Edition Administration and Scoring Manual. Pearson.Google Scholar
Weiner, M. W., Aaronson, A., Eichenbaum, J., Kwang, W., Ashford, M. T., Gummadi, S., Santhakumar, J., Camacho, M. R., Flenniken, D., Fockler, J., Truran-Sacrey, D., Ulbricht, A., Scott Mackin, R., & Nosheny, R. L. (2023). Brain health registry updates: An online longitudinal neuroscience platform. Alzheimer’s & Dementia, 19(11), 4935–4951.CrossRefGoogle Scholar
Weiner, M. W., Aaronson, A., Eichenbaum, J., Kwang, W., Ashford, M. T., Gummadi, S., Santhakumar, J., Camacho, M. R., Flenniken, D., Fockler, J., Truran-Sacrey, D., Ulbricht, A., Mackin, R. S., & Nosheny, R. L. (2023).Brain health registry updates: An online longitudinal neuroscience platform. Alzheimer’s Dementia, https://doi.org/10.1002/alz.13077 CrossRefGoogle Scholar
Weiner, M. W., Nosheny, R., Camacho, M., Truran‐Sacrey, D., Mackin, R. S., Flenniken, D., Ulbricht, A., Insel, P., Finley, S., Fockler, J., Veitch, D. (2018). The brain health registry: An internet-based platform for recruitment, assessment, and longitudinal monitoring of participants for neuroscience studies. Alzheimer’s & Dementia, 14(8), 1063–1076. https://doi.org/10.1016/j.jalz.2018.02.021 CrossRefGoogle Scholar
Weintraub, S., Dikmen, S. S., Heaton, R. K., Tulsky, D. S., Zelazo, P. D., Bauer, P. J., Carlozzi, N. E., Slotkin, J., Blitz, D., Wallner-Allen, K., Fox, N. A., Beaumont, J. L., Mungas, D., Nowinski, C. J., Richler, J., Deocampo, J. A., Anderson, J. E., Manly, J. J., Borosh, B., Havlik, R., Conway, K., Edwards, E., Freund, L., King, J. W., Moy, C., Witt, E., & Gershon, R. C. (2013). Cognition assessment using the NIH toolbox. Neurology, 80(11_supplement_3), S54–S64.CrossRefGoogle ScholarPubMed
Woltz, D. J., & Was, C. A. (2006). Availability of related long-term memory during and after attention focus in working memory. Memory & Cognition, 34(3), 668–684.CrossRefGoogle ScholarPubMed
Zelazo, P. D., Anderson, J. E., Richler, J., Wallner-Allen, K., Beaumont, J. L., Conway, K. P., Gershon, R., & Weintraub, S. (2014). NIH toolbox cognition battery (CB): Validation of executive function measures in adults. Journal of the International Neuropsychological Society, 20(6), 620–629.CrossRefGoogle ScholarPubMed