Sampling for Reliability of High Scale Systems

Authors: Arjun Reddy Lingala

DOI: https://doi.org/10.5281/zenodo.14382845

Short DOI: https://doi.org/g8vbhq

Country: USA

Full-text Research PDF File:   View   |   Download

Abstract: As companies grow they often deal with vast amounts of data and with modern data analytics use cases companies would like to capture as much data as possible and use it for multiple reasons like improving user experience and understanding which new products to build. Building new systems alone is not going to improve the user experience and bring more users to applications. High volume systems these days demands having reliability at each step of the product to not harm the user experience of the product and bring in more users. Few of the reliability use cases that a product has to built at minimum include — Are the users able to land on the webpage of the product and get response? How is p95 latency of a web page request or internal service request? Is latency affected the app for all regions across the world or is it for specific regions only? Is there any change is type of data uploaded or searched in your app? Applications with reliability needs to have monitoring for all these scenarios and more. With growing amount of data across the world, the cost of capturing the data to get answers to reliability questions increases. In this paper, I discuss various sampling strategies that can be used for reliability of high volume systems and compare various approaches in terms of implementation complexity and associated cost

Keywords: Reliability, High Scale Systems, Sampling Strategies, Static Sampling, Random Sampling, Stratified Sampling, Dynamic Sampling, Adaptive Sampling, Monitoring, Time Series Data

Paper Id: 231827

Published On: 2021-01-08

Published In: Volume 9, Issue 1, January-February 2021

Cite This: Sampling for Reliability of High Scale Systems - Arjun Reddy Lingala - IJIRMPS Volume 9, Issue 1, January-February 2021. DOI 10.5281/zenodo.14382845

Share this