Currently, it is estimated that 6.5 million persons over the age of 65 years are living with Alzheimer’s disease (AD) and related dementias in the USA, and the number is projected to be 12.7 million by the year 2050 (Gaugler et al., Reference Gaugler, James, Johnson, Reimer, Solis, Weuve and Hohman2022). Studies have shown that proactive management of AD and related dementias can improve the quality of life of affected individuals and their caregivers (Grossberg et al., Reference Grossberg, Christensen, Griffith, Kerwin, Hunt and Hall2010; Vickrey et al., Reference Vickrey, Mittman, Connor, Pearson, Della Penna, Ganiats, DeMonte, Chodosh, Cui, Vassar, Duan and Lee2006; Voisin & Vellas, Reference Voisin and Vellas2009). As more treatments are under study or become available for AD, it is increasingly important to identify people at risk for AD and related dementias as early as possible, in part through accurately identifying individuals with mild cognitive impairment (MCI), as well as those with normal cognition.
A diagnosis of MCI refers to cognitive decline that is not normal for a person’s age but generally does not affect that person’s ability to carry out most activities of daily living (Gauthier et al., Reference Gauthier, Reisberg, Zaudig, Petersen, Ritchie, Broich, Belleville, Brodaty, Bennett, Chertkow, Cummings, de Leon, Feldman, Ganguli, Hampel, Scheltens, Tierney, Whitehouse and Winblad2006). MCI is classified as one of two types based on a person’s symptoms: amnestic (memory issues predominate) or non-amnestic (other cognitive issues predominate; Alzheimer’s Association, 2022; Petersen et al., Reference Petersen, Lopez, Armstrong, Getchius, Ganguli, Gloss, Gronseth, Marson, Pringsheim, Day, Sager, Stevens and Rae-Grant2018; Ward et al., Reference Ward, Tardiff, Dye and Arrighi2013). Though a portion of those diagnosed with MCI may remain stable or revert to preclinical cognition (Petersen et al., Reference Petersen, Lopez, Armstrong, Getchius, Ganguli, Gloss, Gronseth, Marson, Pringsheim, Day, Sager, Stevens and Rae-Grant2018), the risk for AD is significantly higher in amnestic MCI versus non-amnestic MCI (Alzheimer’s Association, 2022; Kaduszkiewicz et al., Reference Kaduszkiewicz, Eisele, Wiese, Prokein, Luppa, Luck, Jessen, Bickel, Mosch, Pentzek, Fuchs, Eifflaender-Gorfer, Weyerer, Konig, Brettschneider, van den Bussche, Maier, Scherer and Riedel-Heller2014; Ward et al., Reference Ward, Tardiff, Dye and Arrighi2013). It is estimated 10%–15% of individuals with MCI go on to develop a form of dementia each year (Alzheimer’s Association, 2022) and about 1/3 of people with MCI develop dementia due to AD within five years (Alzheimer’s Association, 2022; Ward et al., Reference Ward, Tardiff, Dye and Arrighi2013). Accurate identification and diagnosis of those with MCI is critical for helping individuals, their families, and physicians prepare for future treatment and care.
As the older adult population with dementia grows, disparities have emerged in the prevalence of all-cause dementia among different races. Older non-Hispanic Black/African American (B/AA) and Hispanic Americans are disproportionately more likely than older Whites to have AD or other dementias (Dilworth-Anderson et al., Reference Dilworth-Anderson, Hendrie, Manly, Khachaturian and Fazio2008; Power et al., Reference Power, Bennett, Turner, Dowling, Ciarleglio, Glymour and Gianattasio2021; Rosselli et al., Reference Rosselli, Uribe, Ahne and Shihadeh2022; Steenland et al., Reference Steenland, Goldstein, Levey and Wharton2016; Yaffe et al., Reference Yaffe, Falvey, Harris, Newman, Satterfield, Koster, Ayonayon and Simonsick2013). There is also evidence that a missed or delayed diagnosis of AD and other dementia types is more common among B/AA and Hispanic older adults than among White older adults (Clark et al., Reference Clark, Kutner, Goldstein, Peterson-Hazen, Garner, Zhang and Bowles2005; Fitten et al., Reference Fitten, Ortiz and Pontón2001; Gianattasio et al., Reference Gianattasio, Prather, Glymour, Ciarleglio and Power2019; Lin et al., Reference Lin, Daly, Olchanski, Cohen, Neumann, Faul, Fillit and Freund2021), which then contributes to a delay of care that may impact disease trajectory and outcomes. Further, despite the increased risk posed to B/AA older adults for developing all-cause dementia, B/AA adults are largely underrepresented in research seeking to understand these diseases (Rosselli et al., Reference Rosselli, Uribe, Ahne and Shihadeh2022).
The NIH Toolbox Cognition Battery (NIHTB-CB) is one module within the larger computerized NIH Toolbox for the Assessment of Neurological and Behavioral Function that was developed as an assessment tool to provide clinical researchers a common metric for cross-study comparisons (Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer, Carlozzi, Slotkin, Blitz, Wallner-Allen, Fox, Beaumont, Mungas, Nowinski, Richler, Deocampo, Anderson, Manly, Borosh, Havlik, Conway, Edwards, Freund, King, Moy, Witt and Gershon2013). The NIHTB-CB was designed to be a brief (30-min), computerized, widely accessible, and easily administered cognitive screener for ages 3–85 that is available in both English and Spanish (Gershon et al., Reference Gershon, Wagster, Hendrie, Fox, Cook and Nowinski2013). The battery consists of seven tests measuring five cognitive domains (i.e., executive functioning, episodic memory, processing speed, working memory, language; Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer, Carlozzi, Slotkin, Blitz, Wallner-Allen, Fox, Beaumont, Mungas, Nowinski, Richler, Deocampo, Anderson, Manly, Borosh, Havlik, Conway, Edwards, Freund, King, Moy, Witt and Gershon2013), which are separated broadly into “fluid” or dynamic thinking skills (executive function, episodic memory, processing speed, working memory) and “crystallized” or skills that remain relatively stable in adulthood (language; Heaton et al., Reference Heaton, Akshoomoff, Tulsky, Mungas, Weintraub, Dikmen, Beaumont, Casaletto, Conway, Slotkin and Gershon2014). The original NIHTB-CB was unadjusted for demographic factors (Heaton et al., Reference Heaton, Akshoomoff, Tulsky, Mungas, Weintraub, Dikmen, Beaumont, Casaletto, Conway, Slotkin and Gershon2014; Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer, Carlozzi, Slotkin, Blitz, Wallner-Allen, Fox, Beaumont, Mungas, Nowinski, Richler, Deocampo, Anderson, Manly, Borosh, Havlik, Conway, Edwards, Freund, King, Moy, Witt and Gershon2013), with subsequent normative samples providing corrections for age, sex, race/ethnicity, and education (Casaletto et al., Reference Casaletto, Umlauf, Beaumont, Gershon, Slotkin, Akshoomoff and Heaton2015).
Although the NIHTB-CB may have potential use as a clinical cognitive screener to help identify individuals appropriate for referral for more comprehensive neuropsychological assessment, the utility of the newer computerized NIHTB-CB for tablet has not been well established for clinical characterization. The measures within the NIHTB-CB have demonstrated acceptable reliability and construct validity as compared to traditional paper–pencil methods (Casaletto et al., Reference Casaletto, Umlauf, Beaumont, Gershon, Slotkin, Akshoomoff and Heaton2015; Hackett et al., Reference Hackett, Krikorian, Giovannetti, Melendez‐Cabrero, Rahman, Caesar, Chen, Hristov, Seifan, Mosconi and Isaacson2018; Heaton et al., Reference Heaton, Akshoomoff, Tulsky, Mungas, Weintraub, Dikmen, Beaumont, Casaletto, Conway, Slotkin and Gershon2014; Scott et al., Reference Scott, Sorrell and Benitez2019; Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer, Carlozzi, Slotkin, Blitz, Wallner-Allen, Fox, Beaumont, Mungas, Nowinski, Richler, Deocampo, Anderson, Manly, Borosh, Havlik, Conway, Edwards, Freund, King, Moy, Witt and Gershon2013), but the test battery lacks clear support as a standalone replacement for traditional neuropsychological assessment methods (Garcia et al., Reference Garcia, Askew, Kavcic, Shair, Bhaumik, Rose, Campbell, May, Hampstead, Dodge, Heidebrink, Paulson and Giordani2023; Hackett et al., Reference Hackett, Krikorian, Giovannetti, Melendez‐Cabrero, Rahman, Caesar, Chen, Hristov, Seifan, Mosconi and Isaacson2018; Scott et al., Reference Scott, Sorrell and Benitez2019). Further, despite substantial efforts devoted to the development of representative normative samples for the NIHTB-CB, there remains a scarcity of published studies delineating the performance of underrepresented populations on this cognitive assessment tool. This knowledge gap may inadvertently disregard the potential for performance disparities among distinct racial and ethnic groups.
Aims
In our study, we aimed to determine how well NIHTB-CB tablet subtest scores differentiate those characterized by consensus diagnosis as either healthy controls (HCs) or those with amnestic mild cognitive impairment (aMCI), using National Alzheimer’s Coordinating Center (NACC) criteria. We further sought to compare the discriminant ability between a model using “Norm Adjusted” T-scores provided by NIHTB-CB that have been a priori adjusted for age, sex, race/ethnicity, and education (Casaletto et al., Reference Casaletto, Umlauf, Beaumont, Gershon, Slotkin, Akshoomoff and Heaton2015) to a second model using NIHTB-CB “Unadjusted” standard scores with the same demographic variables controlled for within the model. No prior predictions were made, as this aim was largely exploratory. We also aimed to examine possible differences between the models when stratified by race (B/AA and White). We hypothesized that there would be no significant differences between the two subsamples, as scores would be either norm adjusted or adjusted in the model for age, sex, race/ethnicity, and education prior to entering the model. We also sought to identify which subtest(s) within the NIHTB-CB for tablet accounted for the largest proportion of difference between HCs and aMCIs. On this point the literature is mixed. Traditionally, Fluid measures have been shown to be particularly sensitive to changes in cognitive status (Heaton et al., Reference Heaton, Akshoomoff, Tulsky, Mungas, Weintraub, Dikmen, Beaumont, Casaletto, Conway, Slotkin and Gershon2014; Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer, Carlozzi, Slotkin, Blitz, Wallner-Allen, Fox, Beaumont, Mungas, Nowinski, Richler, Deocampo, Anderson, Manly, Borosh, Havlik, Conway, Edwards, Freund, King, Moy, Witt and Gershon2013). However, studies of older adults in an all B/AA sample (Kairys et al., Reference Kairys, Daugherty, Kavcic, Shair, Persad, Heidebrink, Bhaumik and Giordani2022), a majority B/AA sample (Garcia et al., Reference Garcia, Askew, Kavcic, Shair, Bhaumik, Rose, Campbell, May, Hampstead, Dodge, Heidebrink, Paulson and Giordani2023), and a majority White sample (Hackett et al., Reference Hackett, Krikorian, Giovannetti, Melendez‐Cabrero, Rahman, Caesar, Chen, Hristov, Seifan, Mosconi and Isaacson2018) found that crystalized measures on the NIHTB-CB were also important in differentiating those with MCI from HCs.
Methods
Participants
Participants were recruited through the Michigan Alzheimer’s Disease Research Center and allied projects. This research was completed in accordance with the Helsinki Declaration. This study was reviewed and approved by the human subjects Institutional Review Board at the University of Michigan Medical School in Ann Arbor, MI, USA. All participants signed consent as per the human subjects University of Michigan Medical School Institutional Review Board in Ann Arbor, MI, USA prior to participation in the study. If the competency of a participant was questionable, a trained study team member administered a decision-making assessment tool to gauge their understanding of the research study and their rights as a participant. For those participants deemed not able to give informed consent, the participant’s assent as well as the written informed consent of their legal representative (durable power of attorney, guardian, or next-of-kin as applicable by local laws and regulations) was obtained. All participants then completed the NACC – Unified Data Set (UDS) Version 3 evaluation which included a multidomain medical, neurological, social, and neuropsychological evaluation; participants were then diagnosed at the Michigan Alzheimer’s Disease Research Center using NACC consensus conference criteria (Weintraub et al., Reference Weintraub, Salmon, Mercaldo, Ferris, Graff-Radford, Chui, Cummings, DeCarli, Foster, Galasko, Peskind, Dietrich, Beekly, Kukull and Morris2009; Weintraub et al., Reference Weintraub, Besser, Dodge, Teylan, Ferris, Goldstein, Giordani, Kramer, Loewenstein, Marson, Mungas, Salmon, Welsh-Bohmer, Zhou, Shirk, Atri, Kukull, Phelps and Morris2018).
Participants were community-dwelling older adults and were included in analyses if they were between 65 and 85 years of age, classified by consensus diagnosis as either having no clinically significant cognitive impairment/healthy control (HC; n = 96) or probable MCI with amnestic features (aMCI; n = 62), identified as either B/AA (n = 80) or White (n = 78), and were not missing any scores. Our sample originally included those classified as non-amnestic MCI (n = 23); however, our subsample of non-amnestic MCI was so small that it could not be reasonably included in the analyses and was, therefore, excluded from all analyses. NIHTB-CB assessments using the tablet version for iPad were conducted in English up to 10 days before UDS visits and up to 18 days after UDS assessments with 96.8% of assessments taking place on the same day. NIHTB-CB results were not available to the consensus panel.
Assessment measures
National Institutes of Health Toolbox Cognition Battery (NIHTB-CB): The NIHTB-CB is a computerized cognitive screener that takes approximately 30 minutes to administer and is validated to use from ages 3 to 85 (Gershon et al., Reference Gershon, Wagster, Hendrie, Fox, Cook and Nowinski2013). Individual subtest performances (Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer, Carlozzi, Slotkin, Blitz, Wallner-Allen, Fox, Beaumont, Mungas, Nowinski, Richler, Deocampo, Anderson, Manly, Borosh, Havlik, Conway, Edwards, Freund, King, Moy, Witt and Gershon2013) as well as composite summary scores for Crystallized, Fluid, and Total Cognition are provided (Heaton et al., Reference Heaton, Akshoomoff, Tulsky, Mungas, Weintraub, Dikmen, Beaumont, Casaletto, Conway, Slotkin and Gershon2014). The Crystallized Cognition Composite includes the subtests: Oral Reading Recognition (reading and pronunciation) and Picture Vocabulary (receptive vocabulary). Measures comprising the Fluid Composite include the subtests: Dimensional Change Card Sort (executive function/set-shifting), Flanker Inhibitory Control and Attention (executive function/attention), List Sorting Working Memory (working memory), Pattern Comparison Processing Speed (complex processing speed), and Picture Sequence Memory (episodic memory). Specific test details, procedures, and extensive psychometric evaluation are available elsewhere (Heaton et al., Reference Heaton, Akshoomoff, Tulsky, Mungas, Weintraub, Dikmen, Beaumont, Casaletto, Conway, Slotkin and Gershon2014; Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer, Carlozzi, Slotkin, Blitz, Wallner-Allen, Fox, Beaumont, Mungas, Nowinski, Richler, Deocampo, Anderson, Manly, Borosh, Havlik, Conway, Edwards, Freund, King, Moy, Witt and Gershon2013).
Statistical analyses
This study is unique because it provides analyses of two types of NIHTB-CB reported scores. The first are “Norm Adjusted” T-scores (M = 50, SD = 10) that have been a priori adjusted for age, sex, race/ethnicity, and education (Casaletto et al., Reference Casaletto, Umlauf, Beaumont, Gershon, Slotkin, Akshoomoff and Heaton2015). The second are “Unadjusted” standard scores (M = 100, SD = 15) that were standardized using the overall NIHTB-CB norming sample (without regard to demographic variables) and then, for research purposes in this paper, have been adjusted for age, sex, race/ethnicity, and education of our participants through statistical modeling.
Prior to analysis, all measures were screened visually for univariate and multivariate outliers. Univariate outliers were assessed with boxplots, and several were identified. However, only one multivariate outlier was identified among the Norm Adjusted scores based on a chi-square statistic of the Mahalanobis distance with a p-value < .001; this outlier was removed from the analytic sample with 158 participants remaining in the final sample. The assumption of multivariate normality was assessed with a multivariate qq plot and was reasonably met. Demographic data and NIHTB-CB subtest performance were examined for group differences using independent measures t-test on continuous variables and chi-square on categorical variables (see Table 1). Deeper exploration of the frequency, range, means, and standard deviations of the subtest Picture Sequence Memory was done post hoc to investigate possible difference by diagnosis when stratified by race (see Appendix A).
Table 1. Sample characteristics
Note: To examine for group differences, independent measures t-test was used on continuous variables and chi-square statistic on categorical. Scores used in the NIH Toolbox portion of this table were unadjusted standard scores (M = 100, SD = 15) available through NIH Toolbox Cognition for tablet. B/AA = Black/African American; HC = Healthy Controls; aMCI = Mild Cognitive Impairment with amnestic features; M/SD = Mean/Standard Deviation.
* Significance p < .05.
To determine how well NIHTB-CB tablet subtest scores differentiate those characterized by consensus diagnosis (the “Gold Standard”) as either HCs or those with aMCI, a series of discriminant function analyses (DFAs) with leave-one-out cross-validation were performed. All individual NIHTB-CB subtests scores were used in the DFA analyses (summary scores were not included due to overlapping correlations). DFA allows for evaluation of the unique contribution of each variable in rank order. A positive or negative coefficient loading, respectively, increases or decreases the final total score used to discriminate groups. It is, therefore, recommended that the absolute value of the coefficients (see Tables 2 and 3) be used to interpret which tests had more influence over the model outcome (see Table 4). In each model, leave-one-out cross-validation was used to assess model accuracy; meaning that each observation was left out of the model in turn and group membership can be predicted from the loadings of each variable across functions (see Cross-Validation percentage in each model; Table 4).
Table 2. Discriminant function analyses differentiating those with amnestic mild cognitive impairment from healthy controls in total sample and by race using the norm adjusted NIH Toolbox Cognition for tablet T-scores
Note: Coefficients are standardized structure matrix scores that identified the discriminant function. The absolute value of coefficient loadings represents the unique contribution of each measure in rank order. All variables contributed to the model. Scores used were the norm a priori adjusted (age, sex, race/ethnicity, and education) T-scores (M = 50, SD = 10) available through NIH Toolbox Cognition for tablet. B/AA = Black/African American.
* Coefficients with an absolute value of at least .30 were interpreted.
Table 3. Discriminant function analyses differentiating those with amnestic mild cognitive impairment from healthy controls in the total sample and by race using the unadjusted NIH toolbox cognition for tablet standard scores
Note: Coefficients are standardized structure matrix scores that identified the discriminant function. The absolute value of coefficient loadings represents the unique contribution of each measure in rank order. All variables contributed to the model. Scores used were unadjusted standard scores (M = 100, SD = 15) available through NIH Toolbox Cognition for tablet, and then correcting for differences within the model using age, sex, race/ethnicity, and education as covariates. B/AA = Black/African American.
* Coefficients with an absolute value of at least .30 were interpreted.
Table 4. Discriminant function analysis results by model type
Note: Norm Adjusted = model using a priori norm adjusted (age, sex, race/ethnicity, and education) T-scores (M = 50, SD = 10); Unadjusted = model using unadjusted standard scores (M = 100, SD = 15) and, then, correcting for differences within the model using age, sex, race/ethnicity, and education as covariates; p-value = a two-sample proportional test was used to determine whether there was a statistically significant difference between the proportion of diagnoses correctly predicted by each model (based on a p < .05) between the Norm Adjusted and Unadjusted models; B/AA = Black/African American; HC = Healthy Controls; aMCI = Mild Cognitive Impairment with amnestic features.
Separate DFA analyses were run for the Total Sample (N = 158) and then separately for B/AA participants (n = 80) and White participants (n = 78). The same analytic samples were used in the Norm Adjusted score models and the Unadjusted score models to allow for direct comparison of the models by sample (Total Sample (N = 158), B/AA (n = 80), and White (n = 78)). In the DFA models using the Norm Adjusted scores (a priori adjusted for age, sex, race/ethnicity, education), all variables were entered into the models in a single step with no additional control variables included in the models (see Tables 2–4). In the DFA models using Unadjusted scores, the control variables (age, sex, race/ethnicity, and education) were added in a single step concurrent with the subtest variables (see Tables 3 and 4). A two-sample proportional test was used to determine whether there was a statistically significant (p < .05) difference between the proportion of diagnoses correctly differentiated across model type (Norm Adjusted versus Unadjusted) in the Total Sample and when stratified by race (Table 4). A two-sample proportional test was also used to determine whether there was a statistically significant difference (p < .05) between the proportion of diagnoses correctly differentiated by race (B/AA versus White) in the Norm Adjusted model and Unadjusted model, respectively (see results section).
Results
Sample characteristics can be found in Table 1. HCs did not significantly differ from those with aMCI in education. Age differed significantly between HCs and those with aMCI in the Total Sample, with HCs being approximately 3 years younger than those with aMCI on average. There were significantly more female HCs in the total sample than there were females with aMCI. HCs consistently scored higher on NIHTB-CB subtests than those with aMCI when using non-demographically corrected scores. Apart from the crystalized measure Oral Reading Recognition, all differences between HCs and those with aMCI were significant. In a deeper post hoc exploration of the subtest Picture Sequence Memory by diagnosis when stratified by race we found that the range of scores was more restricted in the White sample than in the B/AA when using both Norm Adjusted and Unadjusted scores (see Appendix A). There was also greater score range overlap between the performance of HCs and those with aMCI in the White sample than in the B/AA sample when using both Norm Adjusted and Unadjusted scores (see Appendix A).
The B/AA sample did not significantly differ from the White sample in education or age, and there was no significant difference between the samples on these variables when further broken down into HCs and those with aMCI. However, there were significantly more female participants in the B/AA sample than the White sample. White participants scored significantly higher on all NIHTB-CB subtests than B/AA participants when using non-demographically corrected scores; though, it should be noted that the percentage of those with aMCI was greater in the B/AA sample (59.7%) than in the White sample (40.3%).
Table 2 shows the individual contribution of tests within the NIHTB-CB using the Norm Adjusted scores (a priori adjusted for age, sex, race/ethnicity, and education) in the Total Sample and stratified by race. Picture Sequence Memory was the subtest that accounted for the largest proportion of the between-group difference in the Total Sample and B/AA sample, followed by List Sorting Working Memory. In the White sample, Picture Sequence Memory was again the coefficient with the highest loading followed by Oral Reading Recognition and Picture Vocabulary.
DFAs that controlled for age, sex, race/ethnicity, and education within the models using the Unadjusted NIHTB-CB standard scores can be seen in Table 3. This table shows the individual contribution of NIHTB-CB subtests in the Total Sample and stratified by race. In the Total Sample analysis, Picture Sequence Memory was the subtest that accounted for the largest proportion of the between-group difference followed by sex. In the B/AA sample, Picture Sequence Memory was again the coefficient with the highest loading followed by sex, education, and Oral Reading Recognition, respectively. In the White sample, Picture Sequence Memory was the coefficient with the highest loading followed by Picture Vocabulary.
Table 4 shows a side-by-side comparison of DFA results by model type (Norm Adjusted versus Unadjusted) in the Total Sample and stratified by race (B/AA; White). The Norm Adjusted and Unadjusted DFAs did not significantly differ in their ability to discriminate consensus-confirmed HCs from those with a consensus diagnosis of aMCI across samples (Total Sample, B/AA, White). Further, neither the Norm Adjusted (p = .68) nor Unadjusted models (p = .80) significantly differed in their overall discriminant ability when comparing model type by race. Though the overall accuracy of the models did not significantly differ by race, the White sample models were notably worse at identifying those with aMCI than the B/AA sample models. This finding indicates that in the White sample the ability to discriminate between the two diagnoses (HC versus aMCI) is relatively weak in both the Fully Adjusted and Unadjusted models.
Discussion
We found that the DFA models using Norm Adjusted scores produced a similar pattern of results to the models using Unadjusted scores that were corrected within the analyses for age, sex, race/ethnicity, and education. Both White sample models were notably worse at identifying those with aMCI than the B/AA sample models. To explore this difference, we reexamined frequency plots, ranges, means, and the standard deviations of the subtest that accounted for the largest proportion of the between-group difference by sample type in each model—Picture Sequence Memory. We found that the White sample had a restricted range of scores when compared to the B/AA sample and there was greater score range overlap between HCs and those with aMCI in the White sample than in the B/AA sample. Thus, the coefficient loadings in the White models are describing a small between-group difference that is being reflected in the relatively low discriminant accuracy of the models. Similar to our findings in the White sample, a study found that the NIHTB-CB yielded a moderate level of specificity but demonstrated chance level sensitivity to subtle cognitive impairment in a racially diverse sample (Buckley et al., Reference Buckley, Sparks, Papp, Dekhtyar, Martin, Burnham, Sperling and Rentz2017).
Picture Sequence Memory (a test of episodic memory) was the subtest that accounted for the largest proportion of between-group difference in each model. This finding is unsurprising as episodic memory is often one of the earliest cognitive domains to decline in population-based studies of preclinical Alzheimer’s dementia, and measures of episodic memory are one of the best predictors of progression from MCI to Alzheimer’s dementia in most longitudinal studies (Bastin & Salmon, Reference Bastin and Salmon2014). Other studies of the NIHTB-CB have found Picture Sequence Memory to be an important contributor to differentiating HCs from those with MCI (Garcia et al., Reference Garcia, Askew, Kavcic, Shair, Bhaumik, Rose, Campbell, May, Hampstead, Dodge, Heidebrink, Paulson and Giordani2023; Kairys et al., Reference Kairys, Daugherty, Kavcic, Shair, Persad, Heidebrink, Bhaumik and Giordani2022) and differentiating those with aMCI from those with non-amnestic MCI (Garcia et al., Reference Garcia, Askew, Kavcic, Shair, Bhaumik, Rose, Campbell, May, Hampstead, Dodge, Heidebrink, Paulson and Giordani2023). However, a study conducted in a memory clinic setting found that many cognitively impaired participants had difficulty completing the task Picture Sequence Memory and it was ultimately dropped from the analyses and replaced by another memory task (Rey Auditory Verbal Learning Test; Hackett et al., Reference Hackett, Krikorian, Giovannetti, Melendez‐Cabrero, Rahman, Caesar, Chen, Hristov, Seifan, Mosconi and Isaacson2018). Thus, while memory tasks are important in differentiating those with a memory deficit from those without, Picture Sequence Memory may be too difficult to use as a standalone screener in a clinical setting.
When stratifying by race using NIHTB-CB standard scores, Oral Reading Recognition (a single word reading task) was among the largest unique contributors accounting for between-group difference in the B/AA sample, while Picture Vocabulary (a receptive language ability task) was in the White sample. Similarly, a recent study with a majority B/AA older adult sample found Oral Reading Recognition to be one of the strongest unique contributors in identifying HCs from those with MCI (both amnestic and non-amnestic MCI) when using the NIHTB-CB standard scores for tablet (Garcia et al., Reference Garcia, Askew, Kavcic, Shair, Bhaumik, Rose, Campbell, May, Hampstead, Dodge, Heidebrink, Paulson and Giordani2023). Another study using NIHTB-CB with an all B/AA older adult sample found that Oral Reading Recognition and Picture Vocabulary best differentiated HCs from those with aMCI when using NIHTB-CB Norm Adjusted scores for laptop (Kairys et al., Reference Kairys, Daugherty, Kavcic, Shair, Persad, Heidebrink, Bhaumik and Giordani2022). In our sample, we found, when using the NIHTB-CB Norm Adjusted scores for tablet, that crystalized abilities were among the strongest contributors to the model identifying HCs from those with aMCI in the White sample but not in the B/AA sample. These findings are interesting, as both receptive language ability tasks and single word reading tasks are thought to generally remain stable across the lifespan or become “crystallized” and are, thus, thought to provide a reasonably accurate estimate of premorbid cognitive functioning in older adults with and without neurodegenerative disease (Grober et al., Reference Grober, Sliwinsk and Korey1991; Snitz et al., Reference Snitz, Bieliauskas, Crossland, Basso and Roper2000). However, individuals with MCI have been shown to perform worse than HCs on a receptive language task (Peabody Picture Vocabulary Test; Jokel et al., Reference Jokel, Seixas Lima, Fernandez and Murphy2019) and those with AD have been shown to perform worse than those with MCI on that same task (Snitz et al., Reference Snitz, Bieliauskas, Crossland, Basso and Roper2000). Thus, decline in language ability may be more important than is generally acknowledged for identifying those declining cognitively. However, neither Oral Reading Recognition nor Picture Vocabulary were strong discriminating variables in the Unadjusted or Norm Adjusted models when the total (race combined) sample was used; this was due to the opposing coefficient loadings (i.e., negative versus positive) between the B/AA and White samples. Because this effect washed away once the race stratified samples were unified, it is important to consider how combining racial samples can sometimes obscure important differences—lending further support to the call to conduct more studies that focus on understudied minority populations and cross-study comparisons of factor invariance.
When looking at the models using unadjusted NIHTB-CB scores, sex was a strong discriminating variable in the Total Sample and in the B/AA models, but not in the White models. Sex effects have been consistently shown across studies with differences typically favoring males in visuospatial tasks specifically and favoring females more broadly but particularly in episodic memory tasks (Heaton, Reference Heaton2004; Lippa et al., Reference Lippa, Collaer and Peters2010). However, there were significantly more females in the HC group than in the aMCI group and there were more female HCs in the B/AA sample than in the White. Thus, it is difficult to parse apart whether the relatively high coefficient loading is related to sex effects, or the uneven distribution of sex among variables (i.e., HC versus aMCI). Similarly, the B/AA group differences are likely driving the sex effects seen in the Total Sample model using Unadjusted scores.
Despite there being no significant difference in average level of education between participants when stratified by race, we found that White participants scored significantly higher on average than B/AA participants on all NIHTB-CB subtests. This finding is consistent with a previous finding using NIHTB-CB (Casaletto et al., Reference Casaletto, Umlauf, Beaumont, Gershon, Slotkin, Akshoomoff and Heaton2015) and a finding using traditional neuropsychological measures (Manly et al., Reference Manly, Miller, Heaton, Byrd, Reilly, Velasquez, Saccuzzo and Grant1998). Further, education was among the largest unique contributors in identifying HCs from those with MCI in the B/AA sample in the model using Unadjusted NIHTB-CB scores. Conversely, education was not a strong discriminating variable in the Total Sample model or the White sample model using Unadjusted NIHTB-CB scores. Together, these results reiterate the meaningful and complicated impact of demographic factors on assessment outcomes. Historically, neuropsychologists have sought to address cultural disparities by making demographically adjusted norms. However, such norms are not able to account for all sociocultural or individual factors (Byrd & Rivera-Mindt, Reference Byrd and Rivera-Mindt2022; Manly, Reference Manly2005; Rosselli et al., Reference Rosselli, Uribe, Ahne and Shihadeh2022). Nevertheless, when such norms are used appropriately, they offer greater diagnostic accuracy (Manly, Reference Manly2005; Werry et al., Reference Werry, Daniel and Bergström2019) and have been recently shown to reduce the association between education and cognitive performance in a racially diverse sample of older adults (Mungas et al., Reference Mungas, Shaw, Hayes-Larson, DeCarli, Farias, Olichney, Saucedo, Gilsanz, Glymour, Whitmer and Mayeda2021). The NIHTB-CB norms specifically have been shown to successfully reduce the impact of demographic variables on performance (Casaletto et al., Reference Casaletto, Umlauf, Beaumont, Gershon, Slotkin, Akshoomoff and Heaton2015); thus, use of the norm corrected scores provided by NIHTB-CB or within model demographic correction is suggested.
Limitations and future directions
One limitation of this study is that we were not able to include older adults over the age of 85 due to a lack of norms in the field for those above the age of 85. Other studies such as the Advancing Reliable Measurement in Alzheimer’s Disease and Cognitive Aging Study are seeking to address this problem by extending the NIHTB norms to those over the age of 85 (Weintraub et al., Reference Weintraub, Karpouzian‐Rogers, Peipert, Nowinski, Slotkin, Wortman, Ho, Rogalski, Carlsson, Giordani, Goldstein, Lucas, Manly, Rentz, Salmon, Snitz, Dodge, Riley, Eldes, Ustsinovich and Gershon2022). HCs in our sample were younger by 3 years on average than those diagnosed with aMCI. This finding is unsurprising as increasing age in adults is consistently and strongly associated with poorer performances (Heaton, Reference Heaton2004) as well as a higher risk of cognitive impairment associated with MCI and all-cause dementia (Alzheimer’s Association, 2022; Gaugler et al., Reference Gaugler, James, Johnson, Reimer, Solis, Weuve and Hohman2022). Those that participated in this study with a consensus diagnosis of MCI all had amnestic features. Future studies may attempt to recruit larger numbers of those with a diagnosis of non-amnestic MCI to examine potential differences. There was an uneven distribution of sex among HCs and those with aMCI in our study sample, as well as differences in distribution of sex by race. Future studies may aim to recruit an equal number of males and females, though this continues to be an issue in many ongoing longitudinal studies of cognitive change. Differences were found between models when stratified by race. For example, although both models using the B/AA sample were more accurate at identifying those with aMCI than the models using the White sample, it should be noted that the B/AA participants were better balanced in terms of the number of aMCI versus HCs, and the White sample demonstrated a restricted range with small between-group difference on the strongest discriminating variable (Picture Sequence Memory). It is likely that these differences also impacted the results of the combined sample model. Though, the NIHTB-CB norms have been shown to successfully reduce the impact of demographic variables on performance (Casaletto et al., Reference Casaletto, Umlauf, Beaumont, Gershon, Slotkin, Akshoomoff and Heaton2015), it is unclear to what degree the differences between racial groups in our study are due to differences in sample distribution and demographic factors. Future DFA studies should aim to recruit equal numbers of racial groups by diagnosis. Further, because the between-group difference of HCs versus those with aMCI appears small, future studies should aim to recruit larger sample sizes.
Conclusions
In our study we compared two types of methods using scores available through NIHTB-CB: a priori adjusted (age, sex, race/ethnicity, and education) T-scores and unadjusted standard scores that were corrected for age, sex, race/ethnicity, and education within the analyses. Our findings indicate that either method can be used to produce similar results when identifying HCs and those with aMCI. However, despite the use of normalized scores or demographic covariates, differences between models when stratified by race were noted—emphasizing the need to continue efforts to include underrepresented groups in research seeking to understand AD and other dementia types. We found that the White sample models were less successful at identifying those with aMCI across model type than the B/AA sample. The use of the norm corrected scores is strongly recommended for use in a clinical setting, and norm corrected scores or within model demographic correction are suggested when using NIHTB-CB in research settings.
Our findings do not provide clear support for use of the NIHTB-CB as a standalone screener in a clinical setting; however, NIHTB-CB has relative ease and efficiency of administration when compared to traditional neuropsychological methods and has been shown to perform as well as traditional neuropsychiatric test batteries at classifying aMCI in a clinical setting (Hackett et al., Reference Hackett, Krikorian, Giovannetti, Melendez‐Cabrero, Rahman, Caesar, Chen, Hristov, Seifan, Mosconi and Isaacson2018). Though, further work is needed to optimize computerized batteries for use in clinical settings. For example, there were subtests in the NIHTB-CB that provided comparatively little information in differentiating HCs from those with aMCI, and some tasks may be too difficult for cognitively impaired individuals to complete reliably (Hackett et al., Reference Hackett, Krikorian, Giovannetti, Melendez‐Cabrero, Rahman, Caesar, Chen, Hristov, Seifan, Mosconi and Isaacson2018). Further, pairing the complete or partial NIHTB-CB (a measure with specificity) with a more sensitive measure like the Montreal Cognitive Assessment may be helpful (Larner, Reference Larner2019; Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin, Cummings and Chertkow2005), but more research is needed. In research settings, it will be important to consider the aims and constraints of the study when determining whether to use NIHTB-CB (e.g., ease and efficiency of administration, minimizing false negatives/sensitivity versus minimizing false positives/specificity).
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1355617724000213.
Acknowledgments
We would like to thank Dr Ana Daugherty, Assistant Professor at the Wayne State Institute of Gerontology and Department of Psychology, and Jian Kang, Professor of Biostatistics at the University of Michigan School of Public Health, for conferring with us regarding the statistical modeling included in this paper. We would also like to thank Dr Subhamoy Pal, Statistician at the Michigan Alzheimer’s Disease Center, for his helpful input.
Funding statement
This work was supported by the US National Institute on Aging (P30 AG053760, P30 AG072931) and the US National Institute of Health (V.K., R01 AG054484, R21 AG046637; B.H., RO1 AG058724; U2C AG057441).
Competing interests
None.
