Skip to main content
DORAS
DCU Online Research Access Service
Login (DCU Staff Only)
Multimodal image news article alignment

Jeyaram, Haree, Hasanuzzaman, Mohammed ORCID: 0000-0003-1838-0091, Calixto, Iacer ORCID: 0000-0001-6244-7906 and Way, Andy ORCID: 0000-0001-5736-5930 (2019) Multimodal image news article alignment. In: 20th International Conference on Computational Linguistics and Intelligent Text Processing (CICLing 2019), 07-Apr 2019, La Rochelle, France.

Full text available as:

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
585kB

Abstract

Multimodal learning has received a lot of attention in the recent years. Associating a description to an image in any language is a challenging task as it involves identifying the objects within the image and determining the relationships between them. Often, the documents are multimodal, and hence they may contain text as well as images. Various methodologies have been put forward to match an image to its corresponding description at sentence level. In this work, we are the first to propose a novel joint image-paragraph (i.e. news article) ranking model trained with images and its corresponding paragraphs (i.e. news articles). The image-paragraph ranking model works in such a way that, given an image, the model ranks the best matching news articles and vice-versa. We achieve this correspondence by using a pairwise ranking function and evaluate the model performance on benchmark datasets using Image-Sentence Ranking task evaluation metric. The experimental results show that our model achieves comparable performance to the cutting edge technique.

Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Subjects:UNSPECIFIED
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Initiatives and Centres > ADAPT
Copyright Information:© 2019 The Authors
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
ID Code:24609
Deposited On:16 Jun 2020 10:54 by Vidatum Academic . Last Modified 06 Jan 2022 17:47

Downloads

Downloads per month over past year

Archive Staff Only: edit this record

  • Student Email
  • Staff Email
  • Student Apps
  • Staff Apps
  • Loop
  • Disclaimer
  • Privacy
  • Contact Us