End-to-End Data Architecture for Regulatory Compliance-Driven Environment in Banking

Authors: Ravikumar Mani Naidu Gunasekaran

DOI: https://doi.org/10.37082/IJIRMPS.v14.i1.232909

Short DOI: https://doi.org/hbnbz4

Country: United States

Full-text Research PDF File:   View   |   Download

Abstract: In today’s highly regulated financial sector, data engineering plays a central role in ensuring compliance with global data protection and financial regulations such as Basel, 2052a, SDR, CCPA, AML, KYC and SOX. Traditional data architecture often falls short in supporting the agility, transparency and auditability required by regulators. This article presents a comprehensive, end-to-end data architecture framework designed specifically for compliance-driven banking environments, drawing on real-world implementation at a Fortune 500 financial Institution.

The proposed architecture integrates streaming and batch ingestion, immutable data storage, fine-grained access control and metadata-driven governance to create a secure, scalable and transparent data platform. By leveraging modern tools such as Apache Kafka, Apache Airflow, dbt and data cataloging systems, the solution ensures complete data lineage, traceability, and real-time regulatory reporting. It addresses core challenges including policy-based data access, audit readiness, and cross-border data retention requirements.

Through practical examples and system-level design patterns, this article demonstrates how compliance can be embedded into the data engineering lifecycle-not treated as an afterthought. The architecture has enabled measurable outcomes such as reduced reporting times, improved audit pass rates, and real-time anomaly detection, reinforcing the strategic role of data engineers in modern financial compliance. This contribution serves as a blueprint for technologist building resilient, regulation-aligned data platforms in any highly regulated industry.

Keywords: Data, Governance, Compliance, ETL, Privacy, Data Quality, Data Model, Financial Services industry.

Paper Id: 232909

Published On: 2026-01-28

Published In: Volume 14, Issue 1, January-February 2026

Share this