Skip to main content
DORAS
DCU Online Research Access Service
Login (DCU Staff Only)
How important is importance sampling for deep budgeted training?

Eric, Arazo, Diego, Ortego ORCID: 0000-0002-1011-3610, Paul, Albert, Noel E., O'Connor ORCID: 0000-0002-4033-9135 and Kevin, McGuinness ORCID: 0000-0003-1336-6477 (2021) How important is importance sampling for deep budgeted training? In: 32nd British Machine Vision Conference (BMVC) 2021, 22 - 25 Nov 2021, Virtual conference.

Full text available as:

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
339kB

Abstract

Long iterative training processes for Deep Neural Networks (DNNs) are commonly required to achieve state-of-the-art performance in many computer vision tasks. Importance sampling approaches might play a key role in budgeted training regimes, i.e. when limiting the number of training iterations. These approaches aim at dynamically estimating the importance of each sample to focus on the most relevant and speed up convergence. This work explores this paradigm and how a budget constraint interacts with importance sampling approaches and data augmentation techniques. We show that under budget restrictions, importance sampling approaches do not provide a consistent improvement over uniform sampling. We suggest that, given a specific budget, the best course of action is to disregard the importance and introduce adequate data augmentation; e.g. when reducing the budget to a 30% in CIFAR-10/100, RICAP data augmentation maintains accuracy, while importance sampling does not. We conclude from our work that DNNs under budget restrictions benefit greatly from variety in the training set and that finding the right samples to train on is not the most effective strategy when balancing high performance with low computational requirements. Source code available at: https://git.io/JKHa3

Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Subjects:Computer Science > Artificial intelligence
Computer Science > Image processing
Computer Science > Machine learning
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering
Research Initiatives and Centres > INSIGHT Centre for Data Analytics
Published in: 32nd British Machine Vision Conference (BMVC) 2021, Proceedings. . BMVC 2021.
Publisher:BMVC 2021
Official URL:https://www.bmvc2021-virtualconference.com/conference/papers/paper_0591.html
Copyright Information:© 2021 The Authors
Funders:Science Foundation Ireland (SFI) under grant number SFI/15/SIRG/3283 and SFI/12/RC/2289 P2.
ID Code:26408
Deposited On:22 Nov 2021 15:02 by Eric Arazo Sánchez . Last Modified 25 Apr 2022 13:08

Downloads

Downloads per month over past year

Archive Staff Only: edit this record

  • Student Email
  • Staff Email
  • Student Apps
  • Staff Apps
  • Loop
  • Disclaimer
  • Privacy
  • Contact Us