Song, Jie, Xu, Shu, Zhang, Li, Pahl, Claus ORCID: 0000-0002-9049-212X and Yu, Ge (2014) Performance and energy optimization on terasort algorithm by task self-resizing. Information Technology And Control, 44 (1). pp. 30-40. ISSN 1392-124X
Abstract
In applications of MapReduce, Terasort is one of the most successful ones, which has helped Hadoop to win the Sort Benchmark three times. While Terasort is known for its sorting speed on big data, its performance and energy consumption still can be optimized. We have analyzed the characteristics of Terasort and have identified the existence of idle notes, which does not only waste energy but also loses performance. Therefore, we optimize Terasort through a single-task distributed algorithm and a task self-resizing algorithm to save time and reduce the energy that is consumed by map nodes, which is caused by waiting for tasks and reduce nodes waiting for input. The algorithm proposed in this paper has proved to be effective in optimizing performance and energy consumption through a series of experiments. It can also be adapted to other applications in the MapReduce environment.
Metadata
Item Type: | Article (Published) |
---|---|
Refereed: | Yes |
Uncontrolled Keywords: | Big data; Energy consumption; MapReduce; Terasort; Task resizing |
Subjects: | Computer Science > Algorithms Computer Science > Software engineering |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing |
Publisher: | Kaunas University of Technology |
Official URL: | http://www.itc.ktu.lt/index.php/ITC/article/view/5... |
Copyright Information: | © 2014 KUT |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License |
ID Code: | 20346 |
Deposited On: | 09 Apr 2015 10:36 by Claus Pahl . Last Modified 21 Jan 2021 16:51 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
939kB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record