Chiumento, Francesco and Liu, Mingming ORCID: 0000-0002-8988-2104
(2025)
Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer’s Disease.
In: IEEE International Conference on Big Data (BigData), 15-18 December 2024, Washington, DC, USA.
Abstract
The rapid advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) have shown great potential in medical diagnostics, particularly in radiology, where datasets such as X-rays are paired with human-generated diagnostic reports. However, a significant research gap exists in the neuroimaging field, especially for conditions such as Alzheimer’s disease, due to the lack of comprehensive diagnostic reports that can be utilized for model fine-tuning. This paper addresses this gap by generating synthetic diagnostic reports using GPT-4o-mini
on structured data from the OASIS-4 dataset, which comprises
663 patients. Using the synthetic reports as ground truth for
training and validation, we then generated neurological reports
directly from the images in the dataset leveraging the pre-trained
BiomedCLIP and T5 models. Our proposed method achieved
a BLEU-4 score of 0.1827, ROUGE-L score of 0.3719, and
METEOR score of 0.4163, revealing its potential in generating
clinically relevant and accurate diagnostic reports.
Metadata
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Event Type: | Conference |
Refereed: | Yes |
Subjects: | Computer Science > Artificial intelligence Computer Science > Machine learning |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering Research Institutes and Centres Research Institutes and Centres > INSIGHT Centre for Data Analytics |
Published in: | Conference Proceedings, 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA. . IEEE. |
Publisher: | IEEE |
Official URL: | https://www.computer.org/csdl/proceedings-article/... |
Funders: | Research Ireland Insight Centre for Data Analytics, SFI Centre for Research Training in Machine Learning (ML-Labs) at DCU |
ID Code: | 30714 |
Deposited On: | 29 Jan 2025 09:56 by Mingming Liu . Last Modified 29 Jan 2025 09:56 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Creative Commons: Attribution-Noncommercial-No Derivative Works 4.0 1MB |
Metrics
Altmetric Badge
Dimensions Badge
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record