Assens, Marc, Giró-i-Nieto, Xavier ORCID: 0000-0002-9935-5332, McGuinness, Kevin ORCID: 0000-0003-1336-6477 and O'Connor, Noel E. ORCID: 0000-0002-4033-9135 (2018) Scanpath and saliency prediction on 360 degree images. Signal Processing: Image Communication, 69 . pp. 8-14. ISSN 0923-5965
Abstract
We introduce deep neural networks for scanpath and saliency prediction trained on 360-degree images. The scanpath prediction model called SaltiNet is based on a temporal-aware novel representation of saliency information named the saliency volume. The first part of the network consists of a model trained to generate saliency volumes, whose parameters are fit by back-propagation using a binary cross entropy (BCE) loss over downsampled versions of the saliency volumes. Sampling strategies over these volumes are used to generate scanpaths over the 360-degree images. Our experiments show the advantages of using saliency volumes, and how they can be used for related tasks. We also show how a similar architecture achieves state-of-the-art performance for the related task of saliency map prediction. Our source code and trained models available at https://github.com/massens/saliency-360salient-2017.
Metadata
Item Type: | Article (Published) |
---|---|
Refereed: | Yes |
Uncontrolled Keywords: | Deep learning; saliency; scanpath; visual attention |
Subjects: | Computer Science > Artificial intelligence Computer Science > Image processing Computer Science > Machine learning |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering Research Institutes and Centres > INSIGHT Centre for Data Analytics |
Publisher: | Elsevier |
Official URL: | https://doi.org/10.1016/j.image.2018.06.006 |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License |
Funders: | TEC2013- 290 43935-R and TEC2016-75976-R, funded by the Spanish Ministerio de Economia y Competitividad and the European Regional Development Fund (ERDF), SGR14 Consolidated Research Group recognized and sponsored by AGAUR, the Catalan Government (Generalitat de Catalunya), Science Foundation Ireland under Grant No 15/SIRG/3283. |
ID Code: | 22817 |
Deposited On: | 03 Dec 2018 14:29 by Kevin Mcguinness . Last Modified 31 May 2019 08:22 |
Documents
Full text available as:
Preview |
PDF (Accepted Manuscript)
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
2MB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record