Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Synthetic-Neuroscore: using a neuro-AI interface for evaluating generative adversarial networks

Wang, Zhengwei orcid logoORCID: 0000-0001-7706-553X, She, Qi, Smeaton, Alan F. orcid logoORCID: 0000-0003-1028-8389, Ward, Tomás E. orcid logoORCID: 0000-0002-6173-6607 and Healy, Graham orcid logoORCID: 0000-0001-6429-6339 (2020) Synthetic-Neuroscore: using a neuro-AI interface for evaluating generative adversarial networks. Neurocomputing, 405 . pp. 26-36. ISSN 0925-2312

Generative adversarial networks (GANs) are increasingly attracting attention in the computer vision, natural language processing, speech synthesis and similar domains. Arguably the most striking results have been in the area of image synthesis. However, evaluating the performance of GANs is still an open and challenging problem. Existing evaluation metrics primarily measure the dissimilarity between real and generated images using automated statistical methods. They often require large sample sizes for evaluation and do not directly reflect the human perception of the image quality. In this work, we introduce an evaluation metric we call Neuroscore, for evaluating the performance of GANs, that more directly reflects psychoperceptual image quality through the utilization of brain signals. Our results show that Neuroscore has superior performances to the current evaluation metrics in that:(1) It is more consistent with human judgment;(2) The evaluation process needs much smaller numbers of samples; and (3) It is able to rank the quality of images on a per GAN basis. A convolutional neural network based brain-inspired framework is also proposed to predict Neuroscore from GAN-generated images. Importantly, we show that including neural responses during the training phase of the network can significantly improve the prediction capability of the proposed model.
Item Type:Article (Published)
Uncontrolled Keywords:Neuroscore; Generative adversarial networks; Neuro-AI interface; Brain-computer interface
Subjects:Biological Sciences > Neuroscience
Computer Science > Artificial intelligence
Computer Science > Machine learning
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Institutes and Centres > INSIGHT Centre for Data Analytics
Official URL:http://dx.doi.org/10.1016/j.neucom.2020.04.069
Copyright Information:© 2020 Elsevier
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:This work is funded as part of the Insight Centre for Data Analytics which is supported by Science Foundation Ireland under Grant Number SFI/12/RC/2289.
ID Code:24650
Deposited On:18 Jun 2020 15:31 by Graham Healy . Last Modified 13 Apr 2022 03:30

Full text available as:

[thumbnail of Neurocomputing_latest.pdf]
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader


Downloads per month over past year

Archive Staff Only: edit this record