Applying LLMs and Semantic Technologies for Data Extraction in Literature Reviews: A Pilot Study in LIS

Abstract

This pilot study evaluates the capabilities of two LLMs, Mistral Small 3.1 and GPT-4o mini, in performing ontology-based data extraction to support literature reviews in library and information science (LIS). A sample of four published systematic reviews was selected as ground truth data. The open-access publications included in these reviews (n = 47) were collected as inputs for the models to perform semantic information extraction, using classes from the Document Components Ontology (DoCO). These preliminary findings highlight the opportunities and challenges of using AI and semantic technologies to streamline literature reviews in the social sciences.

Camille Demers

École de bibliothéconomie et sciences de l’information, Université de Montréal

Camille Demers is a PhD student at the School of Library and Information Science (EBSI) at the University of Montreal. She holds a bachelor’s degree in cognitive neuroscience and a master’s degree in information science. Her research explores the application of natural language processing (NLP) to support scientific knowledge synthesis, with a focus on ontology-based information extraction and analysis. She is also involved in research projects at the intersection of information science and the digital humanities.