出版社：Academy & Industry Research Collaboration Center (AIRCC)
摘要：In this paper we investigate the problem of providing scalability to near-real-time ETL+Q(Extract, transform, load and querying) process of data warehouses. In general, data loading,transformation and integration are heavy tasks that are performed only periodically duringsmall fixed time windows.We propose an approach to enable the automatic scalability and freshness of any datawarehouse and ETL+Q process for near-real-time BigData scenarios. A general framework fortesting the proposed system was implementing, supporting parallelization solutions for eachpart of the ETL+Q pipeline. The results show that the proposed system is capable of handlingscalability to provide the desired processing speed.
