期刊名称：Bulletin of the Technical Committee on Data Engineering
出版年度：2013
卷号：36
期号：4
出版社：IEEE Computer Society
摘要：Creating and testing reproducible computational experiments is hard. Researchers must derive a com- pendium that encapsulates all the components needed to reproduce a result. Reviewers must unpack the encapsulated components, run them in an environment that could be different from the source en- vironment, and verify the results. Although many tools support some aspect of reproducibility, there is no common benchmark against which single or multiple tools can be tested. This paper describes a benchmark that can be used to categorize and better understand existing systems. The benchmark will also serve as the basis for a competition whereby tool builders will demonstrate if and how their systems support end-to-end reproducibility
