Skip to main content
DORAS
DCU Online Research Access Service
Login (DCU Staff Only)
Performance and energy optimization on terasort algorithm by task self-resizing

Song, Jie, Xu, Shu, Zhang, Li, Pahl, Claus and Yu, Ge (2014) Performance and energy optimization on terasort algorithm by task self-resizing. Information Technology And Control, 44 (1). pp. 30-40. ISSN 1392-124X

Full text available as:

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
939kB

Abstract

In applications of MapReduce, Terasort is one of the most successful ones, which has helped Hadoop to win the Sort Benchmark three times. While Terasort is known for its sorting speed on big data, its performance and energy consumption still can be optimized. We have analyzed the characteristics of Terasort and have identified the existence of idle notes, which does not only waste energy but also loses performance. Therefore, we optimize Terasort through a single-task distributed algorithm and a task self-resizing algorithm to save time and reduce the energy that is consumed by map nodes, which is caused by waiting for tasks and reduce nodes waiting for input. The algorithm proposed in this paper has proved to be effective in optimizing performance and energy consumption through a series of experiments. It can also be adapted to other applications in the MapReduce environment.

Item Type:Article (Published)
Refereed:Yes
Uncontrolled Keywords:Big data; Energy consumption; MapReduce; Terasort; Task resizing
Subjects:Computer Science > Algorithms
Computer Science > Software engineering
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Publisher:Kaunas University of Technology
Official URL:http://www.itc.ktu.lt/index.php/ITC/article/view/5772
Copyright Information:© 2014 KUT
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
ID Code:20346
Deposited On:09 Apr 2015 10:36 by Claus Pahl . Last Modified 19 Jul 2018 15:05

Downloads

Downloads per month over past year

Archive Staff Only: edit this record

Altmetric
- Altmetric
+ Altmetric
  • Student Email
  • Staff Email
  • Student Apps
  • Staff Apps
  • Loop
  • Disclaimer
  • Privacy
  • Contact Us