FULLY AUTOMATED DATA WAREHOUSE FRAMEWORK USING ETL PROCESS FOR DECISION SUPPORT SYSTEM

Abstract

Consider data comparison tools, ETL testing frameworks, and data visualisation tools when choosing tools for your data warehouse architecture, ETL framework, and testing requirements. These tools are all important to consider. An assortment of data comparison tools, including Informatica Data Validation Option, Talend Data Quality, and SQL Server Data Tools, can assist in the identification of inconsistencies or mistakes that exist between the source data sets and the target data sets. To automate ETL test cases, scenarios, and workflows, ETL testing frameworks such as Pytest-ETL, ETL Validator, and ETL Robot offer a method that is both structured and portable. The data and the ETL process can also be visualised and analysed with the assistance of data visualisation tools like as Tableau, Power BI, and Qlik Sense. To supply this company with information that is helpful, this study makes use of the nine-step process that was created to implement an Online Analysis Processing (OLAP) database and a data warehouse. Small and medium-sized businesses will find it simpler to analyse data because of the data being performed into dashboards. Additionally, a great deal of helpful information can be shared in a short amount of time and in an effective manner.