DNN pruning with principal component analysis and connection importance estimation
Visualitza/Obre
10.1016/j.sysarc.2021.102336
Inclou dades d'ús des de 2022
Cita com:
hdl:2117/358029
Tipus de documentArticle
Data publicació2022-01
Condicions d'accésAccés obert
Llevat que s'hi indiqui el contrari, els
continguts d'aquesta obra estan subjectes a la llicència de Creative Commons
:
Reconeixement-NoComercial-SenseObraDerivada 4.0 Internacional
ProjecteCoCoUnit - CoCoUnit: An Energy-Efficient Processing Unit for Cognitive Computing (EC-H2020-833057)
ARQUITECTURAS DE DOMINIO ESPECIFICO PARA SISTEMAS DE COMPUTACION ENERGETICAMENTE EFICIENTES (AEI-PID2020-113172RB-I00)
ARQUITECTURAS DE DOMINIO ESPECIFICO PARA SISTEMAS DE COMPUTACION ENERGETICAMENTE EFICIENTES (AEI-PID2020-113172RB-I00)
Abstract
DNN pruning reduces memory footprint and computational work of DNN-based solutions to improve performance and energy-efficiency. An effective pruning scheme should be able to systematically remove connections and/or neurons that are unnecessary or redundant, reducing the DNN size without any loss in accuracy. In this paper we show that some of the most popular pruning schemes, such as the Near Zero Weights, require an extremely time-consuming iterative process that requires retraining the DNN many times to tune the pruning hyperparameters. Then, we propose a DNN pruning scheme based on Principal Component Analysis and relative importance of each neuron’s connection (PCA+DIRIE) that automatically finds the optimized DNN in one shot without requiring hand-tuning of multiple parameters. The experimental results show the effectiveness of our method on several benchmarks. Notably, on ImageNet, PCA+DIRIE can prune up to 60% of ResNet-50 with negligible impact on accuracy.
CitacióRiera, M.; Arnau, J.; González, A. DNN pruning with principal component analysis and connection importance estimation. "Journal of systems architecture", 2022, vol. 122, article 102336, p. 1-11.
ISSN1383-7621
Versió de l'editorhttps://www.sciencedirect.com/science/article/pii/S1383762121002307
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
1-s2.0-S1383762121002307-main.pdf | 1,024Mb | Visualitza/Obre |