Integrating curation into scientific publishing to train AI models.

motivationHigh-throughput extraction and structured labeling of data from academic articles are crucial for enabling downstream machine learning applications and secondary analyses. Current approaches lack integration with the publishing process and comprehensive annotation of experimental roles and methodologies alongside bioentity recognition.

resultsWe embedded multimodal data curation into the academic publishing process to annotate segmented figure panels and captions, combining natural language processing with authors' feedback to increase annotation accuracy. The resulting dataset, SourceData-NLP, comprises over 620 000 annotated biomedical entities, curated from 18 689 figures in 3223 articles in molecular and cell biology. Annotations include eight classes of bioentities (small molecules, gene products, subcellular components, cell lines, cell types, tissues, organisms, and diseases), plus additional classes that delineate the entities' roles in experimental designs and methodologies. We evaluate the utility of the dataset for training AI models using named-entity recognition, segmentation of figure captions into their constituent panels, and a novel context-dependent semantic task that assesses whether an entity is a controlled intervention target or a measurement object. We also demonstrate multi-modal applications for segmenting figures into panel images and their corresponding captions. AVAILABILITY AND IMPLEMENTATION: Trained models are available at https://huggingface.co/EMBO. The SourceData-NLP dataset and code are available at https://github.com/source-data/soda-data, https://github.com/source-data/soda-model, and https://github.com/source-data/soda_image_segmentation.