We introduce deep neural networks for scanpath and saliency prediction trained on 360-degree images. The scanpath prediction model called SaltiNet is based on a temporal-aware novel representation of saliency information named the saliency volume. The first part of the network consists of a model trained to generate saliency volumes, whose parameters are fit by back-propagation using a binary cross entropy (BCE) loss over downsampled versions of the saliency volumes. Sampling strategies over these volumes are used to generate scanpaths over the 360-degree images. Our experiments show the advantages of using saliency volumes, and how they can be used for related tasks. We also show how a similar architecture achieves state-of-the-art performance for the related task of saliency map prediction. Our source code and trained models available at https://github.com/massens/saliency-360salient-2017.
Item Type:
Article (Published)
Refereed:
Yes
Uncontrolled Keywords:
Deep learning; saliency; scanpath; visual attention
This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:
TEC2013- 290 43935-R and TEC2016-75976-R, funded by the Spanish Ministerio de Economia y Competitividad and the European Regional Development Fund (ERDF), SGR14 Consolidated Research Group recognized and sponsored by AGAUR, the Catalan Government (Generalitat de Catalunya), Science Foundation Ireland under Grant No 15/SIRG/3283.
ID Code:
22817
Deposited On:
03 Dec 2018 14:29 by
Kevin Mcguinness
. Last Modified 31 May 2019 08:22