Show simple item record

dc.contributor.authorYazdani Aminabadi, Reza
dc.contributor.authorArnau Montañés, José María
dc.contributor.authorGonzález Colás, Antonio María
dc.contributor.authorRiera Villanueva, Marc
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Enginyeria Minera, Industrial i TIC
dc.identifier.citationYazdani, R., Riera, M., Arnau, J., Gonzalez Colas, A. The dark side of DNN pruning. A: International Symposium on Computer Architecture. "2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA 2018): Los Angeles, California, USA: 1-6 June 2018". Institute of Electrical and Electronics Engineers (IEEE), 2018, p. 790-801.
dc.description.abstractDNN pruning has been recently proposed as an effective technique to improve the energy-efficiency of DNN-based solutions. It is claimed that by removing unimportant or redundant connections, the pruned DNN delivers higher performance and energy-efficiency with negligible impact on accuracy. However, DNN pruning has an important side effect: it May reduce the confidence of DNN predictions. We show that, although top-1 accuracy May be maintained with DNN pruning, the likelihood of the class in the top-1 is significantly reduced when using the pruned models. For applications such as Automatic Speech Recognition (ASR), where the DNN scores are consumed by a successive stage, the workload of this stage can be dramatically increased due to the loss of confidence in the DNN. An ASR system consists of a DNN for computing acoustic scores, followed by a Viterbi beam search to find the most likely sequence of words. We show that, when pruning the DNN model used for acoustic scoring, the Word Error Rate (WER) is maintained but the execution time of the ASR system is increased by 33%. Although pruning improves the efficiency of the DNN, it results in a huge increase of activity in the Viterbi search since the output scores of the pruned model are less reliable. Based on this observation, we propose a novel hardware-based ASR system that effectively integrates a DNN accelerator for pruned models with a Viterbi accelerator. In order to avoid the aforementioned increase in Viterbi search workload, our system loosely selects the N-best hypotheses at every time step, exploring only the N most likely paths. To avoid an expensive sort of the hypotheses based on their likelihoods, our accelerator employs a set-associative hash table to keep track of the best paths mapped to each set. In practice, this solution approaches the selection of N-best, but it requires much simpler hardware. Our approach manages to efficiently combine both DNN pruning and Viterbi search, and achieves 9x energy savings and 4.2x speedup with respect to the state-of-the-art ASR solutions.
dc.format.extent12 p.
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)
dc.subjectÀrees temàtiques de la UPC::Informàtica::Arquitectura de computadors::Arquitectures paral·leles
dc.subject.lcshComputer architecture
dc.subject.otherAutomatic speech recognition (ASR)
dc.subject.otherDeep learning
dc.subject.otherDNN pruning
dc.subject.otherHardware accelerator
dc.subject.otherViterbi search
dc.subject.otherComputer architecture
dc.subject.otherComputer hardware
dc.subject.otherDeep learning
dc.subject.otherDeep neural networks
dc.subject.otherEnergy conservation
dc.subject.otherEnergy efficiency
dc.subject.otherViterbi algorithm
dc.subject.otherAutomatic speech recognition
dc.subject.otherDNN pruning
dc.subject.otherHardware accelerators
dc.subject.otherN-best hypothesis
dc.subject.otherRedundant connections
dc.subject.otherSolution approach
dc.subject.otherState of the art
dc.subject.otherViterbi search
dc.subject.otherSpeech recognition
dc.titleThe dark side of DNN pruning
dc.typeConference report
dc.subject.lemacArquitectura d'ordinadors
dc.contributor.groupUniversitat Politècnica de Catalunya. ARCO - Microarquitectura i Compiladors
dc.contributor.groupUniversitat Politècnica de Catalunya. CERCLE - Cercle d'Arquitectura
dc.description.peerreviewedPeer Reviewed
dc.rights.accessRestricted access - publisher's policy
dc.description.versionPostprint (published version)
local.citation.authorYazdani, R.; Riera, M.; Arnau, J.; Gonzalez Colas, A.
local.citation.contributorInternational Symposium on Computer Architecture
local.citation.publicationName2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA 2018): Los Angeles, California, USA: 1-6 June 2018

Files in this item


This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder