About
Senior Data Engineering and Enterprise Data Architecture professional with 14+ years of experience designing, modernizing, and optimizing large-scale data integration platforms across financial services, insurance, healthcare, and retail industries. Specialized in cloud data modernization, AI-enabled data engineering, enterprise ETL architecture, and data product development using AWS, Snowflake, DBT, IBM DataStage, Informatica, Teradata, Netezza, and other enterprise technologies.
Currently focused on enterprise data migration and modernization initiatives, enabling organizations to transform legacy on-premises ecosystems into scalable cloud-native architectures. Proven expertise in data modeling, source-to-target mapping, ETL framework design, data governance, automation, performance optimization, and enterprise analytics.
As a Technology leader, I actively contribute to advancing the fields of Data Engineering, Artificial Intelligence, Cloud Data Platforms, and Enterprise Analytics through research, technical publications, and industry thought leadership. Passionate about building intelligent data ecosystems that drive innovation, operational excellence, and business transformation at scale.
Skills & Expertise
IBM DataStage
DBT
Snowflake
AWS Databricks
Python
Shell Scripting
SQL
Oracle
IBM DB2
Netezza
Teradata
AirFlow
Control-M
GIT
Data Warehousing
ETL
Data Migration
Data Analysis
Agile
Tableau
Power BI
Spark
Kafka
PL/SQL
Pyspark
Pandas
NumPy
Waterfall
Research Interests
No research interests added yet.
Connect With Me
Experience
Senior Data Engineer
Refactored resource-heavy legacy stored procedures and SSIS packages into optimized, scalable Data bricks Spark jobs, increasing daily pipeline execution efficiency by 35%. Migrated highly fragmented on-premises transactional databases to an centralized AWS/Azure cloud data lake platform with zero data loss or downtime during peak fulfillment schedules. Decommissioned multi-generational data silos across decentralized distribution centers, modernizing the ingestion workflows into unified cloud architecture. Designed and deployed a modern Cloud Data Lakehouse architecture using Delta Lake/Parquet formats, partitioning multi-terabyte inventory sets by regional distribution center. Transitioned daily batch inventory reporting to event-driven architectures utilizing Kafka and cloud event streaming, achieving real-time visibility into supply chain metrics. Built automated pipelines to extract, transform, and load historical supply chain datasets directly into AI-driven inventory planning platforms like RELEX Solutions. Developed robust feed synchronization loops that fed localized demand signals back into financial planning tools, drastically reducing fresh food spoilage. Implemented automated data validation and reconciliation scripts to verify data parity (row counts, keys, data types) between legacy sources and cloud targets. Enforced strict data governance using Role-Based Access Controls (RBAC) to securely mask sensitive vendor contract margins and protect proprietary supply chain metrics. Improved ETL reliability and maintainability through automation and performance optimization. Supported cloud migration and modernization efforts for enterprise data platforms. Increased ETL scalability and maintainability through AWS Daatbricks and Pyspark adoption.
Senior Data Engineer
Participated in the requirements session to gather requirements from Business users, Business owners to convert Business requirement documents into technical specification documents. Worked on the production Tickets to resolve the existing Datastage Issues based on the priority. Worked on ETL code Migration from Datastage 11.7 to DBT (Cloud based ELT tool) & also Netezza to Snowflake Database. Created various Models to extract data from DL2 and load in to DL3 and DL4 layer (EDW and Data Marts). Created and modified the SQL Source Extraction Queries for Oracle Database as part of Performance Tuning. Worked on bringing account Payable data from oracle 11g database to load it into EDW on a weekly frequency for various Affinity groups for processing Payments. Performed in-depth research on existing process and technologies to migrate legacy UDA's(from SAS) to IT Supported applications (to cloud based DBT and Snowflake), and created high level process/ architecture work flow diagram for converting the UDA's to IT Supported system. Built and developed the ETL process using the Datastage tool to transform the data between Source & Target Database. Modified/Enhanced the existing ETL Processes to load in to both Netezza EDW (Enterprise Data Warehouse) & snowflake database as well. Worked on various ETL Development tasks on a regular on-going basis to fulfill the business priorities. Worked Extensively on Talon and Control-M to Schedule the ETL Control Jobs to automate the ETL Process. Experienced with CI/CD Pipeline and GIT hub Repository for Code deployment and code version control. Experienced in basic pre and post sanity check programs using Python language. Good knowledge on using Python Data manipulation libraries like Pandas & NumPy. Supported cloud migration and modernization efforts for enterprise data platforms. Increased ETL scalability and maintainability through DBT and Snowflake adoption. Improved operational efficiency through ETL automation and scheduling enhancements.
DataStage ETL Developer
Worked on Converting server Jobs into Parallel Jobs for as part of 11.7 Datastage Migration. Replaced Hash Files with Sequential/Datasets for server Jobs to Parallel Jobs Conversion. Built and developed the ETL process using the DataStage tool to transform the data between Source & Target Database. Created ETL Process to push data from EDW (Enterprise Data Warehouse) to SQL Server Data Marts for Claims, Provider and Appeal related data to feed data in to Power BI Dashboard. Worked on Parallel, Sequencer and Server Jobs for Converting the Server Jobs in to Parallel for DataStage 11.5 Version. Worked on ETL Enhancements, Fixes and Development based on the business Demand. Used unstructured stage to read the data from Microsoft Excel Files Provided by the external Vendors. Created reusable UNIX scripts to perform data Manipulation for various source files. Created Complex SQL Queries using Common Table Expressions, Row over Partition and Stored Procedure Concepts. Worked Extensively on ZENA Work Station to Schedule the ETL Control Jobs to automate the ETL Process. Implemented Shared Containers, Multiple Instance and Routines Concepts in the DataStage Designer For code reusability. Gained Knowledge on GIT Process for Code Deployment Using Azure Cloud Services. Worked on Converting and rebuilding the PL/SQL Code/process in to Datastage for Easier Maintenance. Improved ETL efficiency through modernization and reusable framework implementation. Enhanced reporting capabilities through optimized data integration into dashboards. Reduced maintenance efforts with reusable ETL components and automation.
Senior Software Engineer
Implemented the Data warehouse concepts like creating Dimension and Fact tables using Star and Snowflake Schema approach based on the business requirements for storing the credit/debit card information in the Enterprise Data Warehouse (EDW). Involved in gathering the requirements, defining the technical feasibility, software development and coordinating with the offshore team. Created Sanity Check ETL Scripts for various Mainframes and vendor files to perform data validations and data manipulations. Created ETL Parallel jobs to consume fixed width Mainframes files with the help of copybook provided by the upstream for data mapping. Involved in all phases of the projects starting from requirement gathering to Production deployment along the warranty support for the ETL Jobs. Gained deep knowledge in understanding the Agility that includes Analyzing the Epic, Creating Features, Writing User Stories, Analyzing the effort to determine the Story Points. Attended the Backlog Grooming Sessions, Iteration Planning & Iteration Demos. Involved in High level design, Low level design, Code Review Documentation. Performed Peer to Peer Testing and Manual Testing Such as Unit Testing and Defect Fixing during SIT and UAT phase. Readily adapted to new environments. Worked on Data Model creation and UML Creation using ERWIN Data Modeling Tool. Extensively used Parallel Stages like Join, Merge, Lookup, Filter, Aggregator, Copy, Sort, Funnel, Change Data Capture, Remove Duplicates, Surrogate key Generator, Row Generator, and Column Generator for development and de-bugging purposes. Created several Job Sequences for the maintenance of Datastage jobs using Wait for file, Job Activity and Executive command stage. Wrote Unix Scripts to parse XML Files and load the data in target Databases. Created Shared containers for the common logic to use across various Projects. Created Universal Parameter Set for ETLS. Improved ETL maintainability through reusable architecture and automation. Strengthened enterprise banking data integration capabilities. Increased project delivery efficiency through agile practices and collaboration.
Software Engineer
Involved in the requirements session to gather requirements from Business users, Business owners to convert business requirement documents in to technical specification documents. Performed data analysis and prepared Source to Target Mappings based on the source and target systems. Built and developed the ETL process using the DataStage Designer tool to transform the data between Source & Target Database. Gained experience in using Netezza as a source and target databases for storing the historical data in to Netezza database for performing analytics. Worked on data Extraction from various source systems like Oracle, Flat files and XML files using the DataStage Designer 11.3 version. Implemented the slowly changing Dimensions techniques like (SCD) Type1, Type 2 to capture the data changes over the period of time using the Data Warehouse and ETL Concepts. Analyzed and created the impact analysis documents for the enhancements in the existing ETL Process. Created Wiki pages and technical documents for the developed end to end ETL Process for a future reference. Created reusable UNIX scripts to perform the data transformation for various source files to perform the sanity check and data validations. Worked on Error handling techniques, optimizing techniques and tuning the ETL flow for better and faster performance. Worked Extensively on Control-M Work Station tool to Schedule the ETL Jobs based on the business requirements to automate the ETL Process for running them on a Daily, Weekly, Monthly and Ad-hoc basis in the Production Environment. Worked with Oracle SQL Developer, Business Data Analyst and ETL Tester to perform smoke test, System testing and regression testing for the ETL DataStage Jobs. Worked on Production support to support Daily, Weekly, Monthly and Yearly Cycles for various ETL Process. Created the unit testing and functional testing documents for the various business cases. Improved reliability and performance of ETL systems through optimization. Supported enterprise analytics through scalable data processing pipelines. Increased process efficiency with automated scheduling and monitoring.
Education
Kent State University
GITAM - Gandhi Institute of Technology and Management (Deemed to be University), Visakhapatnam
Certificates & Licenses (1)
IBM DATASTAGE
Awards & Achievements (2)
🏆 TITAN Business Awards Individual Achievement Winners
https://thetitanawards.com/winner-individual-achievement.php?page=28&competition_id=&category=&rank=
🏆 Noble Business Awards
Professional Memberships (3)
Association for Computing Machinery (ACM)
Country: United States
The institution of Analysts & Programmers
Country: United Kingdom
IEEE - Institute of Electrical and Electronics Engineers
Country: United States
Role in Research Journals (1)
Editor-in-Chief
Publications (3)
AI-Enhanced ETL Processes: Leveraging Artificial Intelligence for Optimized Data Integration Systems
The integration of Artificial Intelligence into traditional Extract, Transform, Load processes represents a transformative
evolution in data integration systems, addressing fundamental constraints of...
This article presents a comprehensive examination of centralized data lake architecture as a strategic solution for
enterprise-wide data integration challenges. By consolidating disparate data types...
Metadata-focused data integration frameworks are revolutionising how
companies handle data. The basic ideas of data integration, their application procedures, and
their usage patterns in practical a...
dd