Interconnect Energy Savings and Lower Latency Networks in Hadoop Clusters: The Missing Link
Visualitza/Obre
Cita com:
hdl:2117/115590
Tipus de documentComunicació de congrés
Data publicació2017
EditorIEEE
Condicions d'accésAccés obert
Tots els drets reservats. Aquesta obra està protegida pels drets de propietat intel·lectual i
industrial corresponents. Sense perjudici de les exempcions legals existents, queda prohibida la seva
reproducció, distribució, comunicació pública o transformació sense l'autorització del titular dels drets
Abstract
An important challenge of modern data centres running Hadoop workloads is to minimise energy consumption, a significant proportion of which is due to the network. Significant network savings are already possible using Energy Efficient Ethernet, supported by a large number of NICs and switches, but recent work has demonstrated that the packet coalescing settings must be carefully configured to avoid a substantial loss in performance. Meanwhile, Hadoop is evolving from its original batch concept to become a more iterative type of framework. Other recent work attempts to reduce Hadoop's network latency using Explicit Congestion Notifications. Linking these studies reveals that, surprisingly, even when packet coalescing does not hurt performance, it can degrade network latency much more than previously thought. This paper is the first to analyze the impact of packet coalescing in the context of network latency. We investigate how to design and configure interconnects to provide the maximum energy savings without degrading cluster throughput performance or network latency.
CitacióFischer e Silva, R.; Carpenter, P. M. Interconnect Energy Savings and Lower Latency Networks in Hadoop Clusters: The Missing Link. A: "2017 IEEE 42nd Conference on Local Computer Networks (LCN)". IEEE, 2017, p. 514-517.
ISBN978-1-5090-6523-3
ISSN0742-1303
Col·leccions
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
Interconnect Energy Savings and Lower Latency.pdf | 452,0Kb | Visualitza/Obre |