Development and evaluation of a large language model-based, retrieval-augmented generation application for query response in early oncology clinical trials.

Background: Early-phase oncology trials involve complex protocols and extensive documents, making timely resolution of study queries challenging. We developed the Study Document Assistant (SDA), a retrieval-augmented generation (RAG) system that integrates semantic search with large language models to generate context-specific answers from clinical trial documentation. Materials and methods: SDA processes study documents and delivers responses through a secure, conversational interface. A two-arm experiment was conducted in which sponsor clinical study team members answered protocol-related queries either with or without SDA. Questions were stratified by difficulty and assessed by blinded subject matter experts (SMEs) and artificial intelligence (AI) models. The primary endpoint was mean response time. Main secondary endpoints included response time as a time-to-event outcome and overall accuracy. SDA user satisfaction was also assessed. Data were analyzed using Welch Results: In the experiment ( Conclusions: Our findings demonstrate that SDA accelerates query resolution without compromising answer quality, highlighting the potential of RAG-based digital assistants to streamline clinical trial operations and enhance research efficiency.