Reducing Runtime Overhead in Distributed Congestion Monitoring Systems

Authors: Vijaya Krishna Namala

DOI: https://doi.org/10.37082/IJIRMPS.v9.i4.232961

Short DOI: https://doi.org/hbqqnq

Country: United States

Full-text Research PDF File:   View   |   Download

Abstract: Distributed congestion monitoring plays a critical role in maintaining reliability and performance in modern cloud and large-scale network environments where numerous nodes continuously exchange traffic. To ensure visibility into network behavior, monitoring frameworks collect runtime telemetry such as queue statistics, packet counts, flow information, and system logs. Conventional monitoring designs process and store these metrics independently at each node while simultaneously forwarding large volumes of data to centralized collectors. Although this approach provides detailed diagnostics, it introduces substantial memory overhead due to continuous buffering, local storage of metrics, duplicated state maintenance, and repeated aggregation of similar information across nodes. Each node maintains separate monitoring buffers and intermediate analysis results. Central collectors further require additional memory to aggregate, cache, and process incoming telemetry streams. Consequently, overall memory usage increases significantly even under moderate workloads. In large deployments, monitoring components consume a considerable share of available memory resources, reducing the capacity available for application services and affecting system stability. Excessive memory pressure may also trigger frequent garbage collection and paging activity, which further degrades performance. These limitations reveal that existing congestion monitoring frameworks are not memory efficient and do not scale effectively with growing infrastructure size. Persistent duplication of telemetry data and independent processing create avoidable storage overhead that impacts overall resource utilization. This paper addresses the problem of excessive memory consumption in distributed congestion monitoring systems and focuses on improving memory efficiency to enable scalable and resource conscious monitoring across large scale environments.

Keywords: Congestion, Monitoring, Memory, Utilization, Telemetry, Runtime, Overhead, Distributed, Scalability, Buffers, Aggregation, Efficiency, Diagnostics, Resources.

Paper Id: 232961

Published On: 2021-08-11

Published In: Volume 9, Issue 4, July-August 2021

Share this