Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Use of neural signals to evaluate the quality of generative adversarial network performance in facial image generation

Wang, Zhengwei orcid logoORCID: 0000-0001-7706-553X, Healy, Graham orcid logoORCID: 0000-0001-6429-6339, Smeaton, Alan F. orcid logoORCID: 0000-0003-1028-8389 and Ward, Tomás E. orcid logoORCID: 0000-0002-6173-6607 (2019) Use of neural signals to evaluate the quality of generative adversarial network performance in facial image generation. Cognitive Computation, 12 (1). pp. 13-24. ISSN 1866-9964

There is a growing interest in using generative adversarial networks (GANs) to produce image content that is indistinguishable from real images as judged by a typical person. A number of GAN variants for this purpose have been proposed; however, evaluating GAN performance is inherently difficult because current methods for measuring the quality of their output are not always consistent with what a human perceives. We propose a novel approach that combines a brain-computer interface (BCI) with GANs to generate a measure we call Neuroscore, which closely mirrors the behavioral ground truth measured from participants tasked with discerning real from synthetic images. This technique we call a neuro-AI interface, as it provides an interface between a human’s neural systems and an AI process. In this paper, we first compare the three most widely used metrics in the literature for evaluating GANs in terms of visual quality and compare their outputs with human judgments. Secondly, we propose and demonstrate a novel approach using neural signals and rapid serial visual presentation (RSVP) that directly measures a human perceptual response to facial production quality, independent of a behavioral response measurement. The correlation between our proposed Neuroscore and human perceptual judgments has Pearson correlation statistics: r(48) = − 0.767, p = 2.089e − 10. We also present the bootstrap result for the correlation i.e., p ≤ 0.0001. Results show that our Neuroscore is more consistent with human judgment compared with the conventional metrics we evaluated. We conclude that neural signals have potential applications for high-quality, rapid evaluation of GANs in the context of visual image synthesis.
Item Type:Article (Published)
Uncontrolled Keywords:Generative adversarial networks; Rapid serial visual presentation; Neuroscore; Brain-computer interface; Neuro-AI interface
Subjects:Biological Sciences > Neuroscience
Computer Science > Artificial intelligence
Computer Science > Machine learning
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Institutes and Centres > INSIGHT Centre for Data Analytics
Publisher:Springer Nature
Official URL:http://dx.doi.org/10.1007/s12559-019-09670-y
Copyright Information:© 2019 Springer Nature
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:Insight Centre for Data Analytics which is supported by Science Foundation Ireland under Grant Number SFI/12/RC/2289
ID Code:24648
Deposited On:18 Jun 2020 15:42 by Graham Healy . Last Modified 18 Jun 2020 15:42

Full text available as:

[thumbnail of Cognitive_Computation_revision_1.pdf]
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader


Downloads per month over past year

Archive Staff Only: edit this record