Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Performance and energy optimization on terasort algorithm by task self-resizing

Song, Jie, Xu, Shu, Zhang, Li, Pahl, Claus orcid logoORCID: 0000-0002-9049-212X and Yu, Ge (2014) Performance and energy optimization on terasort algorithm by task self-resizing. Information Technology And Control, 44 (1). pp. 30-40. ISSN 1392-124X

Abstract
In applications of MapReduce, Terasort is one of the most successful ones, which has helped Hadoop to win the Sort Benchmark three times. While Terasort is known for its sorting speed on big data, its performance and energy consumption still can be optimized. We have analyzed the characteristics of Terasort and have identified the existence of idle notes, which does not only waste energy but also loses performance. Therefore, we optimize Terasort through a single-task distributed algorithm and a task self-resizing algorithm to save time and reduce the energy that is consumed by map nodes, which is caused by waiting for tasks and reduce nodes waiting for input. The algorithm proposed in this paper has proved to be effective in optimizing performance and energy consumption through a series of experiments. It can also be adapted to other applications in the MapReduce environment.
Metadata
Item Type:Article (Published)
Refereed:Yes
Uncontrolled Keywords:Big data; Energy consumption; MapReduce; Terasort; Task resizing
Subjects:Computer Science > Algorithms
Computer Science > Software engineering
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Publisher:Kaunas University of Technology
Official URL:http://www.itc.ktu.lt/index.php/ITC/article/view/5...
Copyright Information:© 2014 KUT
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
ID Code:20346
Deposited On:09 Apr 2015 10:36 by Claus Pahl . Last Modified 21 Jan 2021 16:51
Documents

Full text available as:

[thumbnail of MapReduce_Terasort_ITC_final.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
939kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record