Importance Early detection of cognitive impairment in older adults is crucial for identifying reversible causes, managing symptoms, and enabling future planning; yet primary care settings face significant barriers to implementing effective cognitive screening.

Objective To clinically validate an innovated self-administered, tablet-based cognitive screening tool (“MyCog”, adapted from the NIH Toolbox Cognitive Battery) and to develop an optimized scoring algorithm for detecting cognitive impairment in older adults during primary care visits.

Design Cross-sectional clinical validation study conducted between summer 2021 and fall 2024, comparing cognitive performance between impaired and non-impaired participants using multiple machine learning approaches to identify optimal predictors of impairment.

Setting Academic medical centers and community-based general population recruitment.

Participants 248 adults aged 65 and older, including 70 participants with clinically diagnosed cognitive impairment (MCI or dementia), 81 participants confirmed to be cognitively intact from clinical settings, and 97 participants from the general population. Participants were predominantly White (88%) with varied educational backgrounds and a mean age of 76.3 years (SD = 6.9).

Main Outcomes and Measures Sensitivity, specificity, and area under the receiver operating characteristic curve (ROC AUC) were the primary metrics used for evaluating model fit in classifying cognitive impairment of known groups from the MyCog Picture Sequence Memory and Dimensional Change Card Sort tasks.

Results Across nine machine learning model classes, Picture Sequence Memory exact match score and Dimensional Change Card Sort rate-correct score were identified as the most robust predictors of cognitive impairment. The final optimized model achieved a sensitivity of 82.5% (bootstrapped 95% CI [0.733, 0.913]) and a specificity of 76.8% (bootstrapped 95% CI [0.709, 0.828]) with an ROC AUC of 0.892, comparable to traditional, long form cognitive screening measures while requiring less than 7 minutes to self-administer.

Conclusions and Relevance MyCog demonstrates strong diagnostic accuracy for detecting cognitive impairment in older adults with a parsimonious model focusing on episodic memory and executive function. As a brief, self-administered screening tool that can be integrated into electronic health records, MyCog addresses key barriers to cognitive screening in primary care settings, which may help increase screening and early detection rates.

The authors have declared no competing interest.

This study was funded by a grant from the NIH (R01AG074245-01).

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

All methods were approved by the Northwestern University Institutional Review Board (STU00214921).

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

All data produced in the present study are available upon reasonable request to the authors