Paper Title

Architecting Scalable Data Engineering Pipelines for Real-Time Processing in High-Dimensional Data Ecosystems

Authors

Anete Fossen

Keywords

scalable data pipelines
real-time processing
high-dimensional data
data ecosystems
fault tolerance
data engineering

Article Type

Research Article

Journal

INTERNATIONAL JOURNAL OF DATA ENGINEERING RESEARCH AND DEVELOPMENT

Issue

Volume : 5 | Issue : 1 | Page No : 1-7

Published On

March, 2024

Downloads

FULL PDF

CITATION

COPY LINK

Abstract

The exponential growth of high-dimensional data has necessitated scalable and efficient real-time data engineering pipelines. This paper explores the architecture and design of such pipelines, focusing on scalability, fault tolerance, and real-time processing capabilities. The study synthesizes prior literature, identifies key trends, and presents insights into optimal practices for managing complex data ecosystems.