The Reliability and Usability of Ecological Momentary Assessment Platform ReCoUPS to Monitor Concussion Symptoms and Psychological HRQoL

Abstract

Background: The psychometric properties and usability of concussion symptom checklists and psychological health-related quality of life (PHRQoL) inventories as measured by a novel, mobile ecological momentary assessment (EMA) platform, Recovering Concussion Update on Progression of Symptoms (ReCoUPS) remain unknown. Objectives: To evaluate evidence of the psychometric properties of these measures and the usability of ReCoUPS in healthy individuals. Methods: We enrolled 79 healthy individuals (female=54; _ age=20.85 years, SD=2.49) into our test-retest reliability study using the ReCoUPS platform on participants' smartphones. Thirty survey questions from the Sport Concussion Assessment Tool6 (SCAT6) symptom checklist and the PROMIS Emotional Distress Shorts Forms Anxiety and Depression (PROMISAnx and PROMISDepress), were administered at random times, daily, for 7 days via ReCoUPS text messages. Eight days after enrollment, participants completed recalled PHRQoL (rPHRQoL) inventories (i.e., "within the past 7 days") and a ReCoUPS User Feedback survey. For agreement coding, total momentary PHRQoL (mPHRQoL) assessments ranged from 0-24 (i.e., Likert Scale of 0-6 for 4 questions; >0 indicates impairment present) to match SCAT6 Likert Scale (0-6), while total rPHRQoL ranged from 4-20 (i.e., Likert Scale of 1-5 for 4 questions; >4 indicates impairment present). Cronbach's alpha (a) determined the internal consistency of mPHRQoL. Intraclass correlation coefficients (ICC), 2-way mixed effects model (95% confidence intervals,[CI]) and Cohen's Kappa measured agreement between rHRQoL and mPHRQoL. Spearman's rho (rs) correlation coefficients determined linear relationships between mPHRQoL and SCAT6 symptom clusters. Descriptive statistics (frequencies) demonstrated participants' perceived usability of the ReCoUPS platform. Results: There was strong internal consistency among all mPHRQoL items (PROMISAnx _=0.82; PROMISDepress _=0.94). rPHRQoL demonstrated excellent agreement with mPHRQoL (PROMISAnx ICC=0.99, 95% CI: 0.98-0.99; PROMISDepress ICC=0.99, 95% CI: -0.99-0.99). Following agreement coding, mPROMIS Anxiety demonstrated moderate agreement with rPROMIS Anxiety (Kappa=0.51; Expected: 50.76%, Actual: 75.95%, p<0.001). Additionally, mPROMIS Depression demonstrated substantial agreement with recalled PROMIS Depression (Kappa=0.66; Expected: 51.85%, Actual: 83.54%, p<0.001). Both mPROMISAnx (affective rs= 0.79, p<0.001; cognitive-fatigue rs= 0.66, p<0.001; migraine rs= 0.45; p<0.001) and mPROMISDepress (affective rs= 0.67, p<0.001; cognitive-fatigue rs= 0.58, p<0.001; migraine rs= 0.25; p<0.001) were significantly correlated with all SCAT6 symptom clusters. Twenty-six (33%) of participants completed the ReCoUPS User Feedback Survey, with >77% indicating the platform was easy to use, quick and effective, and accurately captured how they were feeling in the present moment; >73.1% indicating they felt that the platform was beneficial to their health care and that they would recommend others to use the platform following a concussion; 73.1% indicating that they responded to all of the survey text messages; and <=11.5% indicating that the platform was difficult to use and burdensome. Conclusions: These findings ensure the robustness, generalizability, and usability of administering the SCAT-6 symptom checklist, PROMISAnx, and PROMISDepress via ReCoUPS for monitoring concussion symptoms and PHRQoL in healthy participants.