摘要:This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies ( i.e. , automated code analysis and instrumentation), in-situ infrastructure ( i.e. , ADIOS) and big data analysis engines ( i.e. , SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. The in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.
关键词:In-SituData Analysis;Source Code Analysis;Data Staging;ADIOS;Earth System Model;Machine Learning;SciKit-Learn;E3SM