Real-time data center's telemetry reduction and reconstruction using Markov chain models
Visualitza/Obre
Cita com:
hdl:2117/186371
Tipus de documentArticle
Data publicació2019-12
Condicions d'accésAccés obert
Tots els drets reservats. Aquesta obra està protegida pels drets de propietat intel·lectual i
industrial corresponents. Sense perjudici de les exempcions legals existents, queda prohibida la seva
reproducció, distribució, comunicació pública o transformació sense l'autorització del titular dels drets
ProjecteHi-EST - Holistic Integration of Emerging Supercomputing Technologies (EC-H2020-639595)
COMPUTACION DE ALTAS PRESTACIONES VII (MINECO-TIN2015-65316-P)
COMPUTACION DE ALTAS PRESTACIONES VII (MINECO-TIN2015-65316-P)
Abstract
Large-scale data centers are composed of thousands of servers organized in interconnected racks to offer services to users. These data centers continuously generate large amounts of telemetry data streams (e.g., hardware utilization metrics) used for multiple purposes, including resource management, workload characterization, resource utilization prediction, capacity planning, and real-time analytics. These telemetry streams require costly bandwidth utilization and storage space, particularly at medium-long term for large data centers. This paper addresses this problem by proposing and evaluating a system to efficiently reduce bandwidth and storage for telemetry data through real-time modeling using Markov chain based methods. Our proposed solution was evaluated using real telemetry datasets and compared with polynomial regression methods for reducing and reconstructing data. Experimental results show that data can be lossy compressed up to 75% for bandwidth utilization and 95.33% for storage space, with reconstruction accuracy close to 92%.
CitacióBaig, S. [et al.]. Real-time data center's telemetry reduction and reconstruction using Markov chain models. "IEEE systems journal", Desembre 2019, vol. 13, núm. 4, p. 4039-4050.
ISSN1932-8184
Versió de l'editorhttps://ieeexplore.ieee.org/document/8734756
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
Telemetry_Reduction_and_Reconstruction.PDF | Versió publicada pel l'editor. En accés obert a IEEE | 647,5Kb | Visualitza/Obre |