Data WareHouse :: Technology, Database

Being a market leader today requires competitive advantage over rival organizations. By investing in data warehouses, organizations can better predict the trends in market and offer services best suited to the needs of their customers. A Data Warehouse (DW) can be defined as a subject-oriented, non-volatile database having records over years [1,2]. DWs support the strategic decision-making process and help to answer questions such as "Who was our best customer for this item last year?"[3]. Different DW systems consists of different components, however, some core components are shared by most DW systems. The first component is the data sources. DW receives input from different data sources (such as Point-Of-Sales (POS) systems, Automated Teller Machines (ATM) in banks, checkout terminals etc). The second component is the data staging area. The data comes from data sources and it is placed in the staging area, where the data is treated with different transformations and cleansed of any anomalies. After this transformation, the data is placed in the third component which is known as storage area, which is usually a Relational Database Management System (RDBMS). This process of data extraction from data sources, transformation and finally loading in storage area is regarded as Extract, Transform and Load (ETL). The saved data from the storage can be viewed by reporting units. Different On-line Analytical Processing (OLAP) tools assist in generating reports based on the da ta saved in the storage area [4,5,6,7,8]. We believe that testing should be ingrained in DW development. Thus, each of the DW components should be tested. One of the main challenges in testing the DW systems is the fact that DW systems are different among organizations, each organization has its own DW system that conforms with its own requirements and needs, which leads to having differences between DW systems in several aspects (such as database technology, tools used, size, number of users, number of data sources, how the components are connected, etc.)[9]. Another big challenge that is faced by the DW testers is regarding the test data preparation. Making use of real data for testing purpose is a violation of citizen’s privacy laws in some countries (for example, using real data of bank accounts and other information is illegal in many countries). For a proper testing of a DW, presence of a huge amount of test data is necessary. In real-time environment, the system may behave differently in the presence of terabyt es of data [10].