Views of People With Psychosis About Algorithm-Based Relapse Prediction and Data Sharing: Qualitative Study.

Background: Preventing relapses of psychosis is difficult and important. Digital remote monitoring (DRM) systems are being developed and tested to support this. Increasingly, these systems use algorithm-based relapse prediction. Hence, understanding stakeholder views about algorithmic prediction is crucial. Existing qualitative work has explored health professionals' views, but very few studies have examined the perspectives of people with psychosis on this topic. Objective: This paper aimed to provide an in-depth examination of the views of people with psychosis regarding algorithmic relapse prediction within a DRM system that incorporates active symptom monitoring and passive sensing data. Methods: People with psychosis (n=58) were recruited from 6 geographically distinct areas of the United Kingdom. They participated in semistructured qualitative interviews exploring their views about using a DRM system that predicts psychosis relapse based on a machine learning algorithm. Transcripts were analyzed using reflexive thematic analysis. People with lived experience of psychosis were involved extensively in study design, analysis, and reporting. Results: Findings were described across 4 themes. First, accuracy was a prominent theme. Participants emphasized that transparency about algorithm sensitivity and specificity is crucial and discussed the risks of the relapse prediction algorithm producing false positives (flagging that someone was relapsing when they were not) and false negatives (missing actual relapses). In both cases, participants said that errors may be partially mitigated through a human-in-the-loop approach (theme 2), with DRM blended with human oversight, from clinicians or a dedicated digital monitoring team, and calibrated based on service user, carer, and clinician feedback. The third theme, trust, fears, and choice, noted the interplay between users' trust in the DRM system and their relationship with the clinical team. This theme described participants' fears about potential overreactions (hospitalization or excessive medication) or underreactions (no additional support) from the clinical team in response to algorithm-generated relapse predictions. It emphasized the importance of retaining choice around the use of relapse detection algorithms and the sharing of personal data. The final theme described participants' views about the benefits of using a relapse prediction algorithm, including facilitating early intervention, triaging care according to need, minimizing human bias in assessment, and efficiency in saving staff time. Conclusions: People with psychosis acknowledged potential benefits of algorithm-assisted relapse prediction for receiving timely or efficient care, but with several caveats. Algorithm-generated relapse alerts need to be sufficiently accurate and must be interpreted, with understanding of their limitations, by a trustworthy human who is aware of the relevant context. Algorithm-based relapse predictions should only be used with valid consent, in a way that promotes and respects the autonomy and voice of service users and avoids increasing the use of excessive restriction.