ETL TESTING

This guide provides the following sections:1. Data warehouse concepts 2. Etl development life cycle 3. Etl test plan 4. Etl testing life cycle (or) Etl test process 5. Types of etl testing 6. Types of etl bugs 7. Bug reporting 8. Testing templates(test case, bug reporting & etc..) 9. Etl performance testing 10. Etl interview questions 11. Project with example 12. SQL 13. Unix

Data warehouse concepts

Data ware house is relational database which is subject oriented, integrated, time-variant and non volatile collection of data used to support strategic decision making process Data warehouse Architecture:

Etl development life cycle

Hi, To learn etl testing, sql is mandatory and should have knowledge in unix. Any way I will guide you in last section.

ETL Testing:ETL testing is similar to manual testing which we have to do manually with human interaction. Once after inserting or updating the data into data mart by etl developer then we will test that data mart before loading into the centralized data ware house. This test is called ETL Testing. Etl development life cycle:

REQUIREMENT ANLAYSIS HIGH LEVEL DESIGN LOW LEVEL DESIGN DEVELOPMENT SIT(system integration testing) REVIEW TESTING--------->>Etl Testing life Cycle UAT(user acceptance testing) PRODUCTION

Etl test plan

<<< >>>

Test Plan for banking project Introduction Back Ground Test Items Features to be tested Approach Testing levels Banking Informatica, oracle 10g Fixed Deposit, Withdrawls. Like password.non secure field to tested Types of Etl testing Sanity, smoke

Features Pass & How many tc pass, tc fail Fail criteria Suspension criteria Company will make some rules Test Environment Test deliverables Scheduled tasks Staff & training Staging server, client server(Alpha), production server(Beta), live server Test cases, bug logging, test procedure Its a time table of the project or module. Required persons

Risk and mitigation General Holidays, seek leaves Sign off Features not to be tested Higher authority Secure fiels, tables

Etl testing life cycle (or) Etl test process

ETL TESTING LIFE CYCLE:-

Types of Etl testing

Types of ETL Testing:1) Constraint Testing: In the phase of constraint testing, the test engineers identifies whether the data is mapped from source to target or not. The Test Engineer follows the below scenarios in ETL Testing process. a) NOT NULL b) UNIQUE c) Primary Key d) Foreign key e) Check f) Default g) NULL 2) Source to Target Count Testing: In the Source to Target data is matched or not. A Tester can check in this view whether it is ascending order or descending order it doesnt matter .Only count is required for Tester. Due to lack of time a tester can follow this type of Testing. 3) Source to Target Data Validation Testing: In this Testing, a tester can validate the each and every point of the source to target data. Most of the financial projects, a tester can identify the decimal factors. 4) Threshold/Data Integrated Testing: In this Testing, the Ranges of the data, A test Engineer can usually identifies the population calculation and share marketing and business finance analysis (quarterly, halferly, Yearly) MIN 4 MAX 10 RANGE 6

5) Field to Field Testing: In the field to field testing, a test engineer can identify that how much space is occupied in the database. The data is integrated in the table cum datatypes. NOTE: To check the order of the columns and source column to target column. 6) Duplicate Check Testing: In this phase of ETL Testing, a Tester can face duplicate value very frequently so, at that time the tester follows database queries why because huge amount of data is present in source and Target tables. Select ENO, ENAME, SAL, COUNT (*) FROM EMP GROUP BY ENO, ENAME, SAL HAVING COUNT (*) >1; Note: 1) There are no mistakes in Primary Key or no Primary Key is allotted then the duplicates may arise.

2) Sometimes, a developer can do mistakes while transferring the data from source to target at that time duplicates may arise. 3) Due to Environment Mistakes also duplicates arise (Due to improper plugins in the tool). 7) Error/Exception Logical Testing: 1) Delimiter is available in Valid Tables 2) Delimiter is not available in invalid tables(Exception Tables) 8) Incremental and Historical Process Testing: In the Incremental data, the historical data is not corrupted. When the historical data is corrupted then this is the condition where bugs raise. 9) Control Columns and Defect Values Testing: This is introduced by IBM 10) Navigation Testing: Navigation Testing is the End user point of view testing. An end user cannot follow the friendly of the application that navigation is called as bad or poor Navigation. At the time of Testing, A tester can identify this type of navigation scenarios to avoid unnecessary navigation. 11) Initialization testing: A combination of hardware and software installed in platform is called the Initialization Testing 12) Transformation Testing: At the time of mapping from source table to target table, Transformation is not in mapping condition, then the Test Engineer raises bugs. 13) Regression Testing: Code modification to fix a bug or to implement a new functionality which makes us to to find errors. These introduced errors are called regression . Identifying for regression effect is called regression testing. 14) Retesting: Re executing the failed test cases after fixing the bug. 15) System Integration Testing: Integration testing: After the completion of programming process . Developer can integrate the modules there are 3 models a) Top Down b) Bottom Up c) Hybrid Types of Etl Bugs 1. User interface bugs/cosmetic bugs:-

Related to GUI of application Navigation, spelling mistakes, font style, font size, colors, alignment. 2. BVA Related bug:Minimum and maximum values 3. ECP Related bug:Valid and invalid type 4. Input/output bugs:Valid values not accepted Invalid values accepted 5. Calculation bugs:Mathematical errors Final output is wrong 6. Load condition bugs:Does not allows multiple users Does not allows customer expected load 7. Race condition bugs:System crash & hang System cannot run client plat forms 8. Version control bugs:No logo matching No version information available This occurs usually in regression testing 9. H/W bugs:Device is not responding to the application 10. Source bugs:Mistakes in help documents Bug Life Cycle (or) Defect Tracking Process

DETECT DEFECT REPRODUCED DEFECT

REPORT DEFECT BUG FIXING BUG RESOLVING BUG CLOSING

Testing templates
ETL Testing Templates: 1. 2. 3. 4. Issue log/Clarification template Test case template Bug reporting template Metrics template

Etl performance testing

<<< >>>

ETL Performance Tuning: In the Phase of ETL Performing testing , A tester can involve in database Level or Core Database Level. As well as database tester and the same time ETL tester can involve in Performance tuning also. Performance tuning means server side based work. What is a Performance Testing : To test the Server response with different user loads. The Purpose of performance testing is to find bottle neck in the application. What is a Bottle Neck ? Bottle Neck is a break point where the server will be in peak (or) the bottle neck is a pin point (or) break point when the server responds where the server will be busy with the user request. ETL Performing Life cycle : Work flow requirements

Performing Objective Performing testing Performing Measurements Performance Tuning ETL Workflow requirements:In the Phase of work flow req, ETL Tester can identify the performing scenarios how to connect the database to server which environment supports the performance testing and to check the front end and back end environment and batch jobs, data merging, file system components finally reporting events. Performing Objective : The performing objective is to start end to end performance testing most Of the time performing objective will be decided by the client. Performing Testing : To calculate the speed of the project , ETL Tester can test the DataBase Level . The data base is loading the target properly or not. When ETL Developer doesnt loads the data in proper conditions then some damage is caused in the performance of the system. Performing Measurements : At the time of Performing execution, we need to measure the below metrics. Client side metrics 2.hits/sec 3. Through put 4. Memory allocation,5. Process resources 6. Database statistics database user conditions. Performance Tuning : It is a mechanism to get a fixed performance related issues as a Performance tester , we are going to give some suggest recommendations to tuning department. Code Level ---------- Developer Data Base Level-----------DBA Network Level------------Administrator System Level-------------S/A Server Level------------Server side People.

Project with example

<<< >>> Project Here I am taking emp table as example. For this I will write test scenarios and test cases, that means we are testing emp table.

Check List or Test Scenarios:1. To validate the data in table (emp) 2. To validate the table structure. 3. To validate the null values of the table. 4. To validate the null values of very attribute. 5. To check the duplicate values of the table. 6. To check the duplicate values of each attribute of the table 7. To check the field value or space (length of the field size) 8. To check the constraints (foreign ,primary key) 9. To check the name of the employer who has not earned any commission 10. To check the all employers who are work in dept no (Account dept,sales dept) 11. To check the row count of each attribute. 12. To check the row count of the table. 13. To check the max salary from emp table. 14. To check the min salary from emp table.