Evidence map›Paper›PMID 37266996›Full record

ArticleJournal of medical Internet research2023

Ecological Momentary Assessment of Cognition in Clinical and Community Samples: Reliability and Validity Study.

Shifali Singh, Roger Strong, Irene Xu, Luciana M Fonseca, Zoe Hawks, Elizabeth Grinspoon, Lanee Jung, Frances Li, Ruth S Weinstock, Martin J Sliwinski and 2 more

Open access · goldAbstract read
In one paragraph

Article in Journal of medical Internet research, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 32 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
32citing papers in PubMed, 2 pooled it
9.5field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

32 citing papers in PubMed, 2 syntheses or guidelines pooled it, 37 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Personality trait and state associations with cognitive functioning and variability in middle-aged and older adults.The journals of gerontology. Series B, Psychological sciences and social sciences · 2026
    Article
  4. Technology-natives and technology-adopters: Age differences in remote cognitive assessment in Alzheimer disease.Alzheimer's & dementia : the journal of the Alzheimer's Association · 2026
    Article
  5. Depressive symptom instability predicts incident mild cognitive impairment and dementia in older adults.Alzheimer's & dementia : the journal of the Alzheimer's Association · 2026
    Article
  6. Review
  7. Article
  8. Article
  9. Article
  10. Article
  11. Review
  12. Observational
  13. Review
  14. Idiographic momentary profiles of personality facets.Journal of personality and social psychology · 2026
    Article
  15. Article
  16. Article
  17. Computational Phenotyping of Cognitive Decline With Retest Learning.The journals of gerontology. Series B, Psychological sciences and social sciences · 2025
    Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

12 authors at 6 institutions in 2 countries.

Shifali SinghMcLean Hospital, Belmont, MA, United States.ORCID 0000-0003-1312-665X
Roger StrongMcLean Hospital, Belmont, MA, United States.ORCID 0000-0002-6645-8116
Irene XuMcLean Hospital, Belmont, MA, United States.ORCID 0000-0001-5018-0837
Luciana M FonsecaElson S Floyd College of Medicine, Washington State University, Pullman, WA, United States.ORCID 0000-0002-2849-0545
Zoe HawksMcLean Hospital, Belmont, MA, United States.ORCID 0000-0002-3796-0289
Elizabeth GrinspoonMcLean Hospital, Belmont, MA, United States.ORCID 0000-0002-5608-8254
Lanee JungMcLean Hospital, Belmont, MA, United States.ORCID 0000-0003-3555-5287
Frances LiMcLean Hospital, Belmont, MA, United States.ORCID 0000-0002-2612-6336
Ruth S WeinstockDepartment of Medicine, State University of New York (SUNY) Upstate Medical University, Syracuse, NY, United States.ORCID 0000-0001-5859-5666
Martin J SliwinskiDepartment of Human Development and Family Studies, The Pennsylvania State University, State College, PA, United States.ORCID 0000-0002-9611-7558
Naomi S ChaytorElson S Floyd College of Medicine, Washington State University, Pullman, WA, United States.ORCID 0000-0001-8821-289X
Laura T GermineMcLean Hospital, Belmont, MA, United States.ORCID 0000-0001-8690-8412
Harvard University · USMcLean Hospital · USPennsylvania State University · USSUNY Upstate Medical University · USUniversidade de São Paulo · BRWashington State University · US

Funding

Ambulatory Methods for Measuring Cognitive ChangeU2CAG060408 · NIA · PENNSYLVANIA STATE UNIVERSITY, THE · PI SLIWINSKI, MARTIN J · 2018 to 2022
$14.9M
Mobile Measures of Threat Sensitivity for Cross-sectional and Longitudinal AssessmentR01MH121617 · NIMH · MCLEAN HOSPITAL · PI GERMINE, LAURA THI · 2020 to 2025
$3.9M
Glycemic Variability and Fluctuations in Cognitive Status in Adults with Type 1 DiabetesR01DK121240 · NIDDK · MCLEAN HOSPITAL · PI CHAYTOR, NAOMI SAGE, GERMINE, LAURA THI · 2019 to 2022
$3.1M
Precisely quantifying the dynamic impact of sleep quality on cognitive fluctuationsF32MH127812 · NIMH · MCLEAN HOSPITAL · PI HAWKS, ZOE · 2022 to 2023
$101k
NIA NIH HHS U2C AG060408NIDDK NIH HHS R01 DK121240NIMH NIH HHS F32 MH127812NIMH NIH HHS R01 MH121617
6 · The paper itself

Abstract

backgroundThe current methods of evaluating cognitive functioning typically rely on a single time point to assess and characterize an individual's performance. However, cognitive functioning fluctuates within individuals over time in relation to environmental, psychological, and physiological contexts. This limits the generalizability and diagnostic utility of single time point assessments, particularly among individuals who may exhibit large variations in cognition depending on physiological or psychological context (eg, those with type 1 diabetes [T1D], who may have fluctuating glucose concentrations throughout the day).

objectiveWe aimed to report the reliability and validity of cognitive ecological momentary assessment (EMA) as a method for understanding between-person differences and capturing within-person variation in cognition over time in a community sample and sample of adults with T1D.

methodsCognitive performance was measured 3 times a day for 15 days in the sample of adults with T1D (n=198, recruited through endocrinology clinics) and for 10 days in the community sample (n=128, recruited from TestMyBrain, a web-based citizen science platform) using ultrabrief cognitive tests developed for cognitive EMA. Our cognitive EMA platform allowed for remote, automated assessment in participants' natural environments, enabling the measurement of within-person cognitive variation without the burden of repeated laboratory or clinic visits. This allowed us to evaluate reliability and validity in samples that differed in their expected degree of cognitive variability as well as the method of recruitment.

resultsThe results demonstrate excellent between-person reliability (ranging from 0.95 to 0.99) and construct validity of cognitive EMA in both the sample of adults with T1D and community sample. Within-person reliability in both samples (ranging from 0.20 to 0.80) was comparable with that observed in previous studies in healthy older adults. As expected, the full-length baseline and EMA versions of TestMyBrain tests correlated highly with one another and loaded together on the expected cognitive domains when using exploratory factor analysis. Interruptions had higher negative impacts on accuracy-based outcomes (β=-.34 to -.26; all P values <.001) than on reaction time-based outcomes (β=-.07 to -.02; P<.001 to P=.40).

conclusionsWe demonstrated that ultrabrief mobile assessments are both reliable and valid across 2 very different clinic versus community samples, despite the conditions in which cognitive EMAs are administered, which are often associated with more noise and variability. The psychometric characteristics described here should be leveraged appropriately depending on the goals of the cognitive assessment (eg, diagnostic vs everyday functioning) and the population being studied.

Indexed as

Diabetes Mellitus, Type 1Ecological Momentary AssessmentAgedCognitionData CollectionHumansReproducibility of Resultscognitioncognitive functioningcommunitydigital neuropsychologydigital technologyecological momentary assessmentglucosephysiologicalpsychologicalreliabilityremote assessmenttype 1 diabetes, teleneuropsychologyvalidity

Identifiers

PMID37266996
PMCPMC10276323
OpenAlexW4361800978

What Socratic holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the Socratic graph.