Ir al contenido (pulsa Retorno)

Universitat Politècnica de Catalunya

    • Català
    • Castellano
    • English
    • LoginRegisterLog in (no UPC users)
  • mailContact Us
  • world English 
    • Català
    • Castellano
    • English
  • userLogin   
      LoginRegisterLog in (no UPC users)

UPCommons. Global access to UPC knowledge

58.918 UPC E-Prints
You are here:
View Item 
  •   DSpace Home
  • E-prints
  • Programes de doctorat
  • Doctorat en Arquitectura de Computadors
  • Ponències/Comunicacions de congressos
  • View Item
  •   DSpace Home
  • E-prints
  • Programes de doctorat
  • Doctorat en Arquitectura de Computadors
  • Ponències/Comunicacions de congressos
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Neuron-level fuzzy memoization in RNNs

Thumbnail
View/Open
MICRO2019.pdf (954,9Kb)
Share:
 
 
10.1145/3352460.3358309
 
  View Usage Statistics
Cita com:
hdl:2117/176878

Show full item record
Silfa Feliz, Franyell AntonioMés informacióMés informació
Dot Artigas, Gem
Arnau Montañés, José MaríaMés informacióMés informació
González Colás, Antonio MaríaMés informacióMés informacióMés informació
Document typeConference report
Defense date2019
PublisherAssociation for Computing Machinery (ACM)
Rights accessOpen Access
All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder
ProjectCoCoUnit - CoCoUnit: An Energy-Efficient Processing Unit for Cognitive Computing (EC-H2020-833057)
Abstract
Recurrent Neural Networks (RNNs) are a key technology for applications such as automatic speech recognition or machine translation. Unlike conventional feed-forward DNNs, RNNs remember past information to improve the accuracy of future predictions and, therefore, they are very effective for sequence processing problems. For each application run, each recurrent layer is executed many times for processing a potentially large sequence of inputs (words, images, audio frames, etc.). In this paper, we make the observation that the output of a neuron exhibits small changes in consecutive invocations. We exploit this property to build a neuron-level fuzzy memoization scheme, which dynamically caches the output of each neuron and reuses it whenever it is predicted that the current output will be similar to a previously computed result, avoiding in this way the output computations. The main challenge in this scheme is determining whether the new neuron's output for the current input in the sequence will be similar to a recently computed result. To this end, we extend the recurrent layer with a much simpler Bitwise Neural Network (BNN), and show that the BNN and RNN outputs are highly correlated: if two BNN outputs are very similar, the corresponding outputs in the original RNN layer are likely to exhibit negligible changes. The BNN provides a low-cost and effective mechanism for deciding when fuzzy memoization can be applied with a small impact on accuracy. We evaluate our memoization scheme on top of a state-of-the-art accelerator for RNNs, for a variety of different neural networks from multiple application domains. We show that our technique avoids more than 24.2% of computations, resulting in 18.5% energy savings and 1.35x speedup on average.
Description
The final publication is available at ACM via http://dx.doi.org/10.1145/3352460.3358309
CitationSilfa, F.A. [et al.]. Neuron-level fuzzy memoization in RNNs. A: Annual IEEE/ACM International Symposium on Microarchitecture. "MICRO-52: the 52nd Annual IEEE/ACM International Symposium on Microarchitecture: proceedings: October 12-16, 2019: Columbus, Ohio, USA". New York: Association for Computing Machinery (ACM), 2019, p. 782-793. 
URIhttp://hdl.handle.net/2117/176878
DOI10.1145/3352460.3358309
ISBN978-1-4503-6938-1
Publisher versionhttps://dl.acm.org/doi/abs/10.1145/3352460.3358309
Collections
  • Doctorat en Arquitectura de Computadors - Ponències/Comunicacions de congressos [222]
  • ARCO - Microarquitectura i Compiladors - Ponències/Comunicacions de congressos [178]
  • Departament d'Arquitectura de Computadors - Ponències/Comunicacions de congressos [1.827]
Share:
 
  View Usage Statistics

Show full item record

FilesDescriptionSizeFormatView
MICRO2019.pdf954,9KbPDFView/Open

Browse

This CollectionBy Issue DateAuthorsOther contributionsTitlesSubjectsThis repositoryCommunities & CollectionsBy Issue DateAuthorsOther contributionsTitlesSubjects

© UPC Obrir en finestra nova . Servei de Biblioteques, Publicacions i Arxius

info.biblioteques@upc.edu

  • About This Repository
  • Contact Us
  • Send Feedback
  • Inici de la pàgina