Scaling Machine Learning Model Training with CI/CD Pipelines in Cloud Environments

Authors: Swamy Prasadarao Velaga

DOI: https://doi.org/https://doi.org/10.5281/zenodo.12805504

Short DOI: https://doi.org/gt442g

Country: India

Full-text Research PDF File:   View   |   Download

Abstract: As machine learning (ML) continues to advance, the need for scalable, efficient, and reliable model training has become critical. Traditional approaches to ML model training often struggle to meet these demands, prompting the integration of Continuous Integration and Continuous Deployment (CI/CD) practices with cloud environments. This survey paper explores the intersection of CI/CD pipelines and cloud-based solutions in scaling ML model training. We provide a comprehensive review of the current state of CI/CD practices tailored for ML workflows, examine the benefits and offerings of cloud environments, and identify best practices, tools, and frameworks that facilitate this integration. Additionally, we address the challenges associated with resource management, data handling, distributed training, model versioning, and security. By leveraging cloud-native tools and adhering to best practices, organizations can optimize their ML workflows, ensuring efficient and consistent model updates. Furthermore, we highlight future research directions, including advanced resource management techniques, federated learning, AI-driven automation, standardization, enhanced security frameworks, explainability, fairness, and sustainable AI practices. This paper aims to serve as a valuable resource for researchers, practitioners, and organizations seeking to optimize their ML workflows through the effective implementation of CI/CD pipelines in cloud environments, ultimately leading to more robust, reliable, and ethical AI systems.

Keywords: Continuous Deployment, AI Systems, Machine Learning Models, Cloud Environments

Paper Id: 230794

Published On: 2020-01-03

Published In: Volume 8, Issue 1, January-February 2020

Cite This: Scaling Machine Learning Model Training with CI/CD Pipelines in Cloud Environments - Swamy Prasadarao Velaga - IJIRMPS Volume 8, Issue 1, January-February 2020. DOI https://doi.org/10.5281/zenodo.12805504

Share this