Low-latency multi-threaded ensemble learning for dynamic big data streams
Tipus de documentText en actes de congrés
EditorInstitute of Electrical and Electronics Engineers (IEEE)
Condicions d'accésAccés obert
Real–time mining of evolving data streams involves new challenges when targeting today’s application domains such as the Internet of the Things: increasing volume, velocity and volatility requires data to be processed on–the–fly with fast reaction and adaptation to changes. This paper presents a high performance scalable design for decision trees and ensemble combinations that makes use of the vector SIMD and multicore capabilities available in modern processors to provide the required throughput and accuracy. The proposed design offers very low latency and good scalability with the number of cores on commodity hardware when compared to other state–of–the art implementations. On an Intel i7-based system, processing a single decision tree is 6x faster than MOA (Java), and 7x faster than StreamDM (C++), two well- known reference implementations. On the same system, the use of the 6 cores (and 12 hardware threads) available allow to process an ensemble of 100 learners 85x faster that MOA while providing the same accuracy. Furthermore, our solution is highly scalable: on an Intel Xeon socket with large core counts, the proposed ensemble design achieves up to 16x speed-up when employing 24 cores with respect to a single threaded execution.
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
CitacióMarron, D., Ayguade, E., Herrero, J., Read, J., Bifet, A. Low-latency multi-threaded ensemble learning for dynamic big data streams. A: IEEE International Conference on Big Data. "2017 IEEE International Conference on Big Data (Big Data 2017): Boston, Massachusetts, USA: 11-14 December 2017". Institute of Electrical and Electronics Engineers (IEEE), 2018, p. 223-232.
Versió de l'editorhttps://ieeexplore.ieee.org/document/8257930/