Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Do stochastic parrots have feelings too? Improving neural detection of synthetic text via emotion recognition

Cowap, Alan orcid logoORCID: 0000-0002-6300-6034, Graham, Yvette orcid logoORCID: 0000-0001-6741-4855 and Foster, Jennifer orcid logoORCID: 0000-0002-7789-4853 (2023) Do stochastic parrots have feelings too? Improving neural detection of synthetic text via emotion recognition. In: 2023 Conference on Empirical Methods in Natural Language Processing: EMNLP Findings, 6-10 Dec 2023, Singapore.

Abstract
Recent developments in generative AI have shone a spotlight on high-performance synthetic text generation technologies. The now wide availability and ease of use of such models highlights the urgent need to provide equally powerful technologies capable of identifying synthetic text. With this in mind, we draw inspiration from psychological studies which suggest that people can be driven by emotion and encode emotion in the text they compose. We hypothesize that pretrained language models (PLMs) have an affective deficit because they lack such an emotional driver when generating text and consequently may generate synthetic text which has affective incoherence i.e. lacking the kind of emotional coherence present in human-authored text. We subsequently develop an emotionally aware detector by fine-tuning a PLM on emotion. Experiment results indicate that our emotionally-aware detector achieves improvements across a range of synthetic text generators, various sized models, datasets, and domains. Finally, we compare our emotionally-aware synthetic text detector to ChatGPT in the task of identification of its own output and show substantial gains, reinforcing the potential of emotion as a signal to identify synthetic text. Code, models, and datasets are available at https://github.com/alanagiasi/emoPLMsynth
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Uncontrolled Keywords:Synthetic Text; Neural Text; Emotion Recognition; Pretrained Language Models; Large Language Models
Subjects:Computer Science > Artificial intelligence
Computer Science > Computational linguistics
Computer Science > Machine learning
Humanities > Linguistics
Social Sciences > Journalism
Social Sciences > Mass media
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Published in: Findings of the Association for Computational Linguistics: EMNLP 2023. . Association for Computational Linguistics (ACL).
Publisher:Association for Computational Linguistics (ACL)
Official URL:https://doi.org/10.18653/v1/2023.findings-emnlp.66...
Copyright Information:© Association for Computational Linguistics (ACL)
Funders:Science Foundation Ireland Grant number 18/CRT/6183
ID Code:29158
Deposited On:04 Dec 2023 14:01 by Alan Cowap . Last Modified 23 Feb 2024 16:31
Documents

Full text available as:

[thumbnail of Do Stochastic Parrots have Feelings Too? Improving Neural Detection of Synthetic Text via Emotion Recognition - EMNLP Findings 2023]
Preview
PDF (Do Stochastic Parrots have Feelings Too? Improving Neural Detection of Synthetic Text via Emotion Recognition - EMNLP Findings 2023) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Creative Commons: Attribution-Share Alike 4.0
461kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record