Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Efficient large scale clustering based on data partitioning

Bendechache, Malika orcid logoORCID: 0000-0003-0069-1860, Le-Khac, Nhien-An and Kechadi, M-Tahar orcid logoORCID: 0000-0002-0176-6281 (2016) Efficient large scale clustering based on data partitioning. In: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), 17-19 Oct 2016, Montreal, Canada.

Abstract
Clustering techniques are very attractive for extracting and identifying patterns in datasets. However, their application to very large spatial datasets presents numerous challenges such as high-dimensionality data, heterogeneity, and high complexity of some algorithms. For instance, some algorithms may have linear complexity but they require the domain knowledge in order to determine their input parameters. Distributed clustering techniques constitute a very good alternative to big data challenges (e.g., Volume, Variety, Veracity, and Velocity). Usually, these techniques consist of two phases. The first phase generates local models or patterns and the second one tends to aggregate the local results to obtain global models. While the first phase can be executed in parallel on each site and, therefore, efficient, the aggregation phase is complex, time-consuming and may produce incorrect and ambiguous global clusters and therefore incorrect models. In this paper we propose a new distributed clustering approach to deal efficiently with both phases; generation of local results and generation of global models by aggregation. For the first phase, our approach is capable of analysing the datasets located in each site using different clustering techniques. The aggregation phase is designed in such a way that the final clusters are compact and accurate while the overall process is efficient in time and memory allocation. For the evaluation, we use two well-known clustering algorithms; K-Means and DBSCAN. One of the key outputs of this distributed clustering technique is that the number of global clusters is dynamic; no need to be fixed in advance. Experimental results show that the approach is scalable and produces high-quality results.
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Uncontrolled Keywords:Big Data; spatial data; clustering; distributed mining; data analysis; k-means; DBSCAN
Subjects:Computer Science > Algorithms
Computer Science > Computational complexity
Computer Science > Machine learning
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Institutes and Centres > INSIGHT Centre for Data Analytics
Published in: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Proceedings. . IEEE.
Publisher:IEEE
Official URL:http://dx.doi.org/10.1109/DSAA.2016.70
Copyright Information:© 2016 The Authors
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:Science Foundation Ireland under Grant Number SFI/12/RC/2289.
ID Code:24624
Deposited On:16 Jun 2020 15:39 by Malika Bendechache . Last Modified 16 Jun 2020 15:39
Documents

Full text available as:

[thumbnail of Efficient_Large_Scale_Clustering_based_on_Data_Partitioning_DSAA_.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
3MB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record