Single-view 3d body and cloth reconstruction under complex poses
Cita com:
hdl:2117/385206
Tipus de documentText en actes de congrés
Data publicació2022
EditorScitepress
Condicions d'accésAccés obert
Llevat que s'hi indiqui el contrari, els
continguts d'aquesta obra estan subjectes a la llicència de Creative Commons
:
Reconeixement-NoComercial-SenseObraDerivada 4.0 Internacional
Abstract
Recent advances in 3D human shape reconstruction from single images have shown impressive results, leveraging on deep networks that model the so-called implicit function to learn the occupancy status of arbitrarily dense 3D points in space. However, while current algorithms based on this paradigm, like PiFuHD (Saito et al., 2020), are able to estimate accurate geometry of the human shape and clothes, they require high-resolution input images and are not able to capture complex body poses. Most training and evaluation is performed on 1k-resolution images of humans standing in front of the camera under neutral body poses. In this paper, we leverage publicly available data to extend existing implicit function-based models to deal with images of humans that can have arbitrary poses and self-occluded limbs. We argue that the representation power of the implicit function is not sufficient to simultaneously model details of the geometry and of the body pose. We, therefore, propose a coarse- to-fine approach in which we first learn an implicit function that maps the input image to a 3D body shape with a low level of detail, but which correctly fits the underlying human pose, despite its complexity. We then learn a displacement map, conditioned on the smoothed surface and on the input image, which encodes the high-frequency details of the clothes and body. In the experimental section, we show that this coarse-to-fine strategy represents a very good trade-off between shape detail and pose correctness, comparing favorably to the most recent state-of-the-art approaches. Our code will be made publicly available.
CitacióUgrinovic, N. [et al.]. Single-view 3d body and cloth reconstruction under complex poses. A: International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. "Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. Volume 4: VISAPP". Setúbal: Scitepress, 2022, p. 192-203. ISBN 978-989-758-555-5. DOI 10.5220/0010896100003124.
ISBN978-989-758-555-5
Versió de l'editorhttps://www.scitepress.org/Link.aspx?doi=10.5220/0010896100003124
Altres identificadorshttps://arxiv.org/abs/2205.04087
Col·leccions
- IRI - Institut de Robòtica i Informàtica Industrial, CSIC-UPC - Ponències/Comunicacions de congressos [575]
- Departament d'Enginyeria de Sistemes, Automàtica i Informàtica Industrial - Ponències/Comunicacions de congressos [1.497]
- Doctorat en Intel·ligència Artificial - Ponències/Comunicacions de congressos [46]
- VIS - Visió Artificial i Sistemes Intel·ligents - Ponències/Comunicacions de congressos [292]
- ROBiri - Grup de Percepció i Manipulació Robotitzada de l'IRI - Ponències/Comunicacions de congressos [251]
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
2643-Single-vie ... n-under--Complex-Poses.pdf | 1,435Mb | Visualitza/Obre |