Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Enhancing XR Theatre with Remote Scent Delivery Using Audio Speech Recognition and Convolutional Neural Network-Based Scene Detection

Simiscuka, Anderson Augusto orcid logoORCID: 0000-0002-0851-2452, Picoreau, Matis, Muntean, Gabriel-Miro orcid logoORCID: 0000-0002-9332-4770, Fadda, Gianluca, Murroni, Maurizio, Montagud, Mario, Mancini, Massimo, Carli, Marco, Battisti, Federica and Popescu, Vlad (2025) Enhancing XR Theatre with Remote Scent Delivery Using Audio Speech Recognition and Convolutional Neural Network-Based Scene Detection. In: IEEE International Symposium on the Internet of Sounds, 29–31 October 2025, L'Aquila, Italy. ISBN 979-8-4007-2037-6

Abstract
This paper presents a multi-sensory solution de- signed to enhance the immersive experience of remote audiences for theatrical plays. Using the Macbettu play as a case study, the approach synchronises olfactory and visual stimuli for remote viewers. The solution considers two approaches for generating scents: audio detection that identify the scenes within the play that require specific scents, based on automatic speech recogni- tion (ASR), and convolutional neural networks (CNN) for scene recognition triggering the scent dispensers. When the key scenes are detected during the live play, via network communications, olfaction dispensers release the corresponding scent for remote users viewing the play on VR headsets. Real-time communication is managed through an MQTT-based approach. The paper also compares user feedback for CNN-based recognition versus the audio-based approach, focussing on scent delay and user perceived Quality of Experience (QoE) of the generated remote scents in multiple distances from a scent source. This work illustrates how scent-enhanced XR technologies can be integrated into virtual theatre experiences to engage remote audiences.
Metadata
Item Type:Conference or Workshop Item (Speech)
Event Type:Conference
Refereed:Yes
Uncontrolled Keywords:Extended Reality, Olfaction, CNNs, Audio, Speech Recognition, Multi-Sensory Experiences, Theatre Technology
Subjects:Computer Science > Computer engineering
Computer Science > Computer networks
DCU Faculties and Centres:Research Institutes and Centres > INSIGHT Centre for Data Analytics
Published in: UIST '25: Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology. (17). Association for Computing Machinery. ISBN 979-8-4007-2037-6
Publisher:Association for Computing Machinery
Official URL:https://dl.acm.org/doi/proceedings/10.1145/3746059
Copyright Information:Authors
ID Code:31858
Deposited On:18 Nov 2025 11:03 by Anderson Augusto Simiscuka . Last Modified 18 Nov 2025 11:03
Documents

Full text available as:

[thumbnail of IS2_25___Macbettu (6).pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Creative Commons: Attribution 4.0
10MB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record