GANimation: one-shot anatomically consistent facial animation
Cita com:
hdl:2117/331813
Tipus de documentArticle
Data publicació2019-01-01
Condicions d'accésAccés obert
Tots els drets reservats. Aquesta obra està protegida pels drets de propietat intel·lectual i
industrial corresponents. Sense perjudici de les exempcions legals existents, queda prohibida la seva
reproducció, distribució, comunicació pública o transformació sense l'autorització del titular dels drets
Abstract
Recent advances in generative adversarial networks (GANs) have shown impressive results for the task of facial expression synthesis. The most successful architecture is StarGAN (Choi et al. in CVPR, 2018), that conditions GANs’ generation process with images of a specific domain, namely a set of images of people sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content and granularity of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on action units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combining several of them. Additionally, we propose a weakly supervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit a novel self-learned attention mechanism that makes our network robust to changing backgrounds, lighting conditions and occlusions. Extensive evaluation shows that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild. The code of this work is publicly available at https://github.com/albertpumarola/GANimation.
Descripció
The final publication is available at link.springer.com
CitacióPumarola, A. [et al.]. GANimation: one-shot anatomically consistent facial animation. "International journal of computer vision", 1 Gener 2019, vol. 128, p. 698-713.
ISSN0920-5691
Versió de l'editorhttps://link.springer.com/article/10.1007%2Fs11263-019-01210-3
Col·leccions
- ROBiri - Grup de Percepció i Manipulació Robotitzada de l'IRI - Articles de revista [163]
- Departament d'Enginyeria de Sistemes, Automàtica i Informàtica Industrial - Articles de revista [1.390]
- Doctorat en Automàtica, Robòtica i Visió - Articles de revista [163]
- IRI - Institut de Robòtica i Informàtica Industrial, CSIC-UPC - Articles de revista [376]
- VIS - Visió Artificial i Sistemes Intel·ligents - Articles de revista [139]
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
2253-GANimation ... stent-facial-animation.pdf | 16,68Mb | Visualitza/Obre |