期刊名称：International Journal of Software Engineering and Its Applications
印刷版ISSN：1738-9984
出版年度：2014
卷号：8
期号：11
页码：115-128
DOI：10.14257/ijseia.2014.8.11.10
出版社：SERSC
摘要：Managing and storing big data makes use of powerful systems mainly based on the concept of data warehousing. Data changes usually occur in operational data sources that are mostly heterogeneous and remote from each other. To gather them in a single location called the data warehouse, they may undergo many treatments and processes according to the organization's policies and rules. Once in the data warehouse, they become ready for decision-making and analytical tools. For fast and good decisions for the future, based on fresh data, it is necessary that the data warehouse reflects the real operational data changes and provides the freshest data to the analytical systems. This paper addresses the problem of integrating big data into the data warehouse in short time and proposes a new model called DJ-DI. Based on Division of data changes by adapting table joins, this model increases the data integration's rate and thus data reach the data warehouse in shorter time. We have conducted different simulations of the DJ-DI model under our experimental platform. The obtained results show that the DJ-DI model offers a remarkable improvement of data integration's rate.
关键词：Big data; data changes; division; data warehouse; real-time; data integration; ; join
