Potential of ChatGPT in youth mental health emergency triage: Comparative analysis with clinicians.

Aim: Large language models, such as GPT-4, are increasingly integrated into healthcare to support clinicians in making informed decisions. Given ChatGPT's potential, it is necessary to explore such applications as a support tool, particularly within mental health telephone triage services. This study evaluates whether GPT Models can accurately triage psychiatric emergency vignettes and compares its performance to that of clinicians. Methods: A cross-sectional study was performed to assess the performance of three different GPT-4 models (GPT-4o, GPT-4o Mini, and GPT-4 Legacy) in psychiatric emergency triage. Twenty-two psychiatric emergency vignettes, intended to represent realistic prehospital triage scenarios, were initially drafted using ChatGPT and subsequently reviewed and refined by the research team to ensure clinical accuracy and relevance. The GPT-4 models independently generated clinical responses to the vignettes over three iterations to ensure consistency. Thereafter, two advanced practice nurse practitioners independently assessed these responses utilizing a 3-point Likert-type scale for the main triage criteria: risk level ( Results: GPT Models had an average admission score of 1.73 (standard deviation [SD] = 0.45; scale: Conclusion: This study indicates that GPT Models may serve as supportive decision-support tools in mental health telephone triage, particularly for psychiatric emergencies. Although response variability across iterations was minimal, most discrepancies in admission decisions were identified as false positives, reflecting that GPT Models may have a tendency to over-triage relative to clinician judgment. Further investigation is needed to establish robust structure to increase alignment with clinical decisions and response relevance in clinical practice.