Talk 2: Recurrent neural nets and differentiable memory mechanism

Cita com:
hdl:2117/96655
Document typeConference report
Defense date2016-09-10
PublisherBarcelona Supercomputing Center
Rights accessOpen Access
This work is protected by the corresponding intellectual and industrial property rights.
Except where otherwise noted, its contents are licensed under a Creative Commons license
:
Attribution-NonCommercial-NoDerivs 3.0 Spain
Abstract
This past year, RNNs have seen a lot of attention as powerful models that are able to decode sequences from signals. The key component of such methods are the use of a recurrent neural network architecture that is trained end-to-end to optimize the probability of the output sequence given those signals. In this talk, I’ll define the architecture and review some recent successes in my group on machine translation, image understanding, and beyond. On the second part of the talk, I will introduce a new paradigm — differentiable memory — that has enabled learning programs (e.g., planar Traveling Salesman Problem) using training instances via a powerful extension of RNNs with memory. This effectively turns a machine learning model into a “differentiable computer”. I will conclude the talk giving a few examples (e.g., AlphaGo) on how these recent Machine Learning advances have been the main catalyst in Artificial Intelligence in the past years.
CitationVinyals, Oriol. Talk 2: Recurrent neural nets and differentiable memory mechanism. A: 2nd Severo Ochoa Research Seminar Lectures at BSC, Barcelona, 2015-2016 : Deep Learning Seminar. "Book of abstracts". Barcelona: Barcelona Supercomputing Center, 2016, p. 42-43.
Files | Description | Size | Format | View |
---|---|---|---|---|
43-44 2015-16 S ... Recurrent Neural Nets.pdf | 129,6Kb | View/Open |