首页    期刊浏览 2024年11月23日 星期六
登录注册

文章基本信息

  • 标题:Optimizing Workflow Data Footprint
  • 本地全文:下载
  • 作者:Gurmeet Singh ; Karan Vahi ; Arun Ramakrishnan
  • 期刊名称:Scientific Programming
  • 印刷版ISSN:1058-9244
  • 出版年度:2007
  • 卷号:15
  • 期号:4
  • 页码:249-268
  • DOI:10.1155/2007/701609
  • 出版社:Hindawi Publishing Corporation
  • 摘要:

    In this paper we examine the issue of optimizing disk usage and scheduling large-scale scientific workflows onto distributed resources where the workflows are data-intensive, requiring large amounts of data storage, and the resources have limited storage resources. Our approach is two-fold: we minimize the amount of space a workflow requires during execution by removing data files at runtime when they are no longer needed and we demonstrate that workflows may have to be restructured to reduce the overall data footprint of the workflow. We show the results of our data management and workflow restructuring solutions using a Laser Interferometer Gravitational-Wave Observatory (LIGO) application and an astronomy application, Montage, running on a large-scale production grid-the Open Science Grid. We show that although reducing the data footprint of Montage by 48% can be achieved with dynamic data cleanup techniques, LIGO Scientific Collaboration workflows require additional restructuring to achieve a 56% reduction in data space usage. We also examine the cost of the workflow restructuring in terms of the application's runtime.

国家哲学社会科学文献中心版权所有