Explainable AI Framework for Policy-Compliant Anomaly Detection in Data Pipelines

Abstract

AI design for detecting anomalies in data pipelines with the Isolation Forest algorithm has been done in this research. The purpose of the study is to increase the interpretability and performance of anomaly detection models based on the issues of precision and recall. The model has a high accuracy that is 90.4% that shows its ability to correctly detect anomalies with high levels of false positives and false negatives. The metrics used to evaluate the worth of the assessments, such as the precision-recall curve and F1-score, indicated threshold optimization as a requirement. The results propose that a future study will aim at combining better models, real-world data, and real-time anomaly detection methods and methods to enhance their practical applications