Using Large Language Models to Assess Burnout Among Health Care Workers in the Context of COVID-19 Vaccine Decisions and Health Beliefs: Retrospective Cohort Study.

Background: Burnout among health care workers affects their well-being and decision-making, influencing patient and public health outcomes. Health care workers' health beliefs and COVID-19 vaccine decisions may affect the risks of burnout. Therefore, understanding the interplay between these crucial factors is essential for identifying at-risk staff, providing targeted support, and addressing workplace challenges to prevent further escalation of burnout-related issues. Objective: This study examines how burnout is impacted by health beliefs and COVID-19 vaccine decisions among health care workers. Building on our previously developed Health Belief Model (HBM) classifier based on the HBM framework, which explains how individual perceptions of health risks and benefits influence behavior, we focused on key HBM constructs, including the perceived severity of COVID-19, perceived barriers to vaccination, and their relationship to burnout. We aim to leverage natural language processing techniques to automatically identify theoretically grounded burnout symptoms from comments authored by nurses in a large-scale, national survey and assess their associations with vaccine hesitancy and health beliefs. Methods: We analyzed 1944 open-ended comments written by 1501 vaccine-hesitant nurses, using data from the Nurses' Health Study surveys. We fine-tuned LLaMA 3, an open-source large language model with few-shot prompts and enhanced performance with structured annotation guidance and reasoning-aware inference. Comments were classified into burnout dimensions-Emotional Exhaustion, Depersonalization, and Inefficacy-based on the Maslach Burnout Inventory framework. Results: The model achieved a high weighted accuracy of 92% and an F1-score of 91% for Depersonalization. Emotional Exhaustion was identified in 52% (1003/1944) of comments, correlating strongly with perceived severity (189/323, 59%) and barriers to vaccination (281/650, 43%). Demographic analyses revealed significant variations in burnout prevalence, with older age groups reporting greater burnout. Conclusions: This study highlights the relationship between burnout and vaccine decision-making among health care workers, uncovering areas for further exploration. By exploring the complex interplay between psychological strain and vaccine hesitancy, this study sets the stage for developing transformative interventions and policies that could redefine workforce resilience and public health strategies.