Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer’s Disease

Chiumento, Francesco and Liu, Mingming orcid logoORCID: 0000-0002-8988-2104 (2025) Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer’s Disease. In: IEEE International Conference on Big Data (BigData), 15-18 December 2024, Washington, DC, USA.

Abstract
The rapid advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) have shown great potential in medical diagnostics, particularly in radiology, where datasets such as X-rays are paired with human-generated diagnostic reports. However, a significant research gap exists in the neuroimaging field, especially for conditions such as Alzheimer’s disease, due to the lack of comprehensive diagnostic reports that can be utilized for model fine-tuning. This paper addresses this gap by generating synthetic diagnostic reports using GPT-4o-mini on structured data from the OASIS-4 dataset, which comprises 663 patients. Using the synthetic reports as ground truth for training and validation, we then generated neurological reports directly from the images in the dataset leveraging the pre-trained BiomedCLIP and T5 models. Our proposed method achieved a BLEU-4 score of 0.1827, ROUGE-L score of 0.3719, and METEOR score of 0.4163, revealing its potential in generating clinically relevant and accurate diagnostic reports.
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Subjects:Computer Science > Artificial intelligence
Computer Science > Machine learning
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering
Research Institutes and Centres
Research Institutes and Centres > INSIGHT Centre for Data Analytics
Published in: Conference Proceedings, 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA. . IEEE.
Publisher:IEEE
Official URL:https://www.computer.org/csdl/proceedings-article/...
Funders:Research Ireland Insight Centre for Data Analytics, SFI Centre for Research Training in Machine Learning (ML-Labs) at DCU
ID Code:30714
Deposited On:29 Jan 2025 09:56 by Mingming Liu . Last Modified 29 Jan 2025 09:56
Documents

Full text available as:

[thumbnail of Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimers Disease.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Creative Commons: Attribution-Noncommercial-No Derivative Works 4.0
1MB
Metrics

Altmetric Badge

Dimensions Badge

Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record