Skip to main content
DORAS
DCU Online Research Access Service
Login (DCU Staff Only)
Contrastive representation learning: a framework and review

Le-Khac, Phúc H. ORCID: 0000-0002-0504-5844, Healy, Graham ORCID: 0000-0001-6429-6339 and Smeaton, Alan F. ORCID: 0000-0003-1028-8389 (2020) Contrastive representation learning: a framework and review. IEEE Access, 8 . pp. 1-28. ISSN 2169-3536

Full text available as:

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
3MB

Abstract

Contrastive Learning has recently received interest due to its success in self-supervised representation learning in the computer vision domain. However, the origins of Contrastive Learning date as far back as the 1990s and its development has spanned across many fields and domains including Metric Learning and natural language processing. In this paper, we provide a comprehensive literature review and we propose a general Contrastive Representation Learning framework that simplifies and unifies many different contrastive learning methods. We also provide a taxonomy for each of the components of contrastive learning in order to summarise it and distinguish it from other forms of machine learning. We then discuss the inductive biases which are present in any contrastive learning system and we analyse our framework under different views from various sub-fields of Machine Learning. Examples of how contrastive learning has been applied in computer vision, natural language processing, audio processing, and others, as well as in Reinforcement Learning are also presented. Finally, we discuss the challenges and some of the most promising future research directions ahead.

Item Type:Article (Published)
Refereed:Yes
Uncontrolled Keywords:Contrastive learning; representation learning; self-supervised learning; unsupervised learning; deep learning; Contrastive Learning; Representation Learning; Self-supervised Learning;
Subjects:Computer Science > Machine learning
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Initiatives and Centres > INSIGHT Centre for Data Analytics
Publisher:IEEE
Official URL:http://dx.doi.org/10.1109/ACCESS.2020.3031549
Copyright Information:2020 The Authors. Open Access CC-BY
Funders:Science Foundation Ireland (SFI) Centre for Research Training in Machine Learning (18/CRT/6183) and in part by the Insight Centre for Data Analytics (SFI/12/RC/2289_P2).
ID Code:25121
Deposited On:28 Oct 2020 12:49 by Alan Smeaton . Last Modified 18 Dec 2020 17:12

Downloads

Downloads per month over past year

Archive Staff Only: edit this record

Altmetric
- Altmetric
+ Altmetric
  • Student Email
  • Staff Email
  • Student Apps
  • Staff Apps
  • Loop
  • Disclaimer
  • Privacy
  • Contact Us