GANimation: anatomically-aware facial animation from a single image

Cita com:
hdl:2117/125337
Document typeConference report
Defense date2018
PublisherSpringer
Rights accessOpen Access
European Commission's projectAEROARMS - AErial RObotic system integrating multiple ARMS and advanced manipulation capabilities for inspection and maintenance (EC-H2020-644271)
Abstract
Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis. The most successful architecture is StarGAN, that conditions GANs' generation process with images of a specific domain, namely a set of images of persons sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combine several of them. Additionally, we propose a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit attention mechanisms that make our network robust to changing backgrounds and lighting conditions. Extensive evaluation show that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild.
Description
The final publication is available at link.springer.com
CitationPumarola, A., Agudo, A., Martinez, A., Sanfeliu, A., Moreno-Noguer, F. GANimation: anatomically-aware facial animation from a single image. A: European Conference on Computer Vision. "Computer Vision – ECCV 2018. 15th European Conference, Munich, Germany, September 8-14, 2018, proceedings, part I". Berlín: Springer, 2018, p. 835-851.
Award-winningAward-winning
Publisher versionhttps://link.springer.com/chapter/10.1007%2F978-3-030-01249-6_50
Collections
- IRI - Institut de Robòtica i Informàtica Industrial, CSIC-UPC - Ponències/Comunicacions de congressos [463]
- Departament d'Enginyeria de Sistemes, Automàtica i Informàtica Industrial - Ponències/Comunicacions de congressos [1.275]
- VIS - Visió Artificial i Sistemes Intel·ligents - Ponències/Comunicacions de congressos [266]
- ROBiri - Grup de Robòtica de l'IRI - Ponències/Comunicacions de congressos [174]
Files | Description | Size | Format | View |
---|---|---|---|---|
2052-GANimation ... on-from-a-Single-Image.pdf | 1,291Mb | View/Open |
All rights reserved. This work is protected by the corresponding intellectual and industrial
property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public
communication or transformation of this work are prohibited without permission of the copyright holder