User Testing an mHealth Behavioral Health App for Hopi/Tewa Youth During the COVID-19 Pandemic: Usability Study.

Background: American Indian/Alaska Native (AI/AN) people represent a culturally diverse people group within the United States. AI/AN people experience some of the most severe health disparities in the United States, including behavioral health. A quarter of AI/AN people in the United States live on tribal lands, experiencing significant barriers to mental health resources and broadband infrastructure for telehealth. We developed Amplifying Resilience Over Restricted Internet Access (ARORA)-a mobile health (mHealth) smartphone app, promoting mindfulness practices and community building through AI/AN culture and values. Originally co-designed with both Hopi/Tewa and Navajo youth and adults, this study evaluated app resonance among Hopi/Tewa youth, supporting its iterative design. While we initially planned in-person user testing, this was moved online due to the COVID-19 pandemic. Objective: This study assessed the potential and acceptability of an mHealth app supporting Hopi/Tewa youth practicing mindfulness inspired by their culture, values, and beliefs. This research served as preliminary work for an ongoing, iterative participatory action research study, identifying points of improvement to align with our partner community's goals. Methods: After meeting with 6 community advisory board members and focus groups prior to this study, we developed a prototype for ARORA. This study evaluated intuitiveness and usability through testing and interviews with Hopi/Tewa youth. All meetings with stakeholders were moved online due to the COVID-19 pandemic. Using screen-sharing via Zoom (Zoom Communications, Inc) and Android emulators, we received feedback for the iterative design process. Results: This study involved 9 participants aged 16-24 years. Of these participants, 1 was male and 8 were female; all identified as Hopi/Tewa and/or Tewa. This study included a quantitative assessment using a modified version of the User Version of the Mobile Application Rating Scale. The mean score across all questions was 3.71 (SD 0.427), suggesting generally positive reception. Qualitative results from thematically analyzing open-ended focus group data produced 5 open codes and 12 axial themes, reaching thematic saturation after engaging with 9 participants. Qualitative feedback revealed that while its use was generally enjoyable, the ARORA app could be more specific to Hopi/Tewa culture. Finally, we reflect on adaptations made to our initial protocol in response to the COVID-19 pandemic, offering guidelines for future mHealth work involving rural or hard-to-reach communities. Conclusions: In this evaluation and usability testing of the ARORA prototype, participants expressed interest and engagement in the mindfulness activities. Participants also identified spaces in which the app could improve, both in usability and in cultural groundedness, especially with the visual dimensions of the app. Reflecting on our experience in facilitating remote user testing, we encourage future work in rural mHealth to consider practices for conducting research when in-person meetings are not feasible.