首页    期刊浏览 2024年07月03日 星期三
登录注册

文章基本信息

  • 标题:An Improved Data Placement Strategy in a Heterogeneous Hadoop Cluster
  • 本地全文:下载
  • 作者:Wentao Zhao ; Lingjun Meng ; Jiangfeng Sun
  • 期刊名称:The Open Cybernetics & Systemics Journal
  • 电子版ISSN:1874-110X
  • 出版年度:2014
  • 卷号:8
  • 期号:1
  • 页码:957-963
  • DOI:10.2174/1874110X01408010957
  • 出版社:Bentham Science Publishers Ltd
  • 摘要:

    Hadoop Distributed File System (HDFS) is designed to store big data reliably, and to stream these data at high bandwidth to user applications. However, the default HDFS block placement policy assumes that all nodes in the cluster are homogeneous, and randomly place blocks without considering any nodes’ resource characteristics, which decreases self-adaptability of the system. In this paper, we take account nodes heterogeneities, such as utilization of nodes’ disk space, and put forward an improved blocks placement strategy for solving some drawbacks in the default HDFS. The simulation experiments indicate that our improved strategy performs much better not only in the data distribution but also significantly saves more time than the default blocks placement.

国家哲学社会科学文献中心版权所有