This thesis explores the field of visual attention modeling, focusing on saliency prediction for both 2D and 360° visual data. The research is grounded in integrating deep learning techniques with human fixation data, aiming to refine current methods and address existing approaches’ limitations. Key contributions include the analysis of self-attention maps in Vision Transformers for improved saliency prediction and developing a transformer-based model for image saliency. Then, we target the field of 360° data and introduce ATSal, which integrates attention mechanisms with expert instances for omnidirectional scene analysis. Additionally, the thesis explores the application of contrastive learning to 360° scenes, presenting an unsupervised framework that effectively learns general representations. The final chapter introduces a new probabilistic metric suited for 360° saliency. This comprehensive study advances the understanding and application of deep learning in visual attention modeling.