期刊名称：International Journal of Innovative Research in Computer and Communication Engineering
印刷版ISSN：2320-9798
电子版ISSN：2320-9801
出版年度：2017
卷号：5
期号：4
页码：8989
DOI：10.15680/IJIRCCE.2017.05040352
出版社：S&S Publications
摘要：Existing parallel mining algorithms for frequent itemsets lack a mechanism that enables automaticparallelization,load balancing, data distribution, and fault tolerance on large clusters. As a solution to this problem, wedesign a parallel frequent itemsets mining algorithm called FiDoop using the MapReduce programming model. Toachieve compressed storage and avoid building conditional pattern bases, FiDoop incorporates the frequent itemsultrametric tree, rather than conventional FP trees. In FiDoop, three MapReduce jobs are implemented to complete themining task. In the crucial third MapReduce job, the mappers independently decompose itemsets, the reducers performcombination operations by constructing small ultrametric trees, and the actual mining of these trees separately. FiDoopon the cluster is sensitive to data distribution and dimensions, because itemsets with different lengths have differentdecomposition and construction costs. To improve FiDoop’s performance, we develop a workload balance metric tomeasure load balance across the cluster’s computing nodes.
