3D Human Modelling from Image and Text Guidance

Speaker

Yukang Cao is a final-year Ph.D. student at The University of Hong Kong with HKU-PS scholarship. He is under the supervision of Dr. Kenneth Wong and works closely with Dr. Kai Han. Throughout his academic journey, he has gained valuable experiences at UT-Austin, PKU, THU, and Tencent. His research interests lie in 3D modelling of the virtual world, with a particular focus on 3D human reconstruction and the generation of 3D avatars. More details can be found in https://yukangcao.github.io/.

Abstract

Over the years, there have been significant developments in 3D human models, particularly in the areas of reconstruction and generation. Technically, reconstruction methods rely on large and hard-to-obtain 3D human dataset, while generation approaches still need a higher-fidelity and more efficient pipeline. In this talk, I will introduce our recent works in these two areas: (1) 3D human reconstruction: JIFF for face-enhance quality from a single image and SeSDF for uncalibrated multi-view scenarios; and (2) 3D human avatar generation: DreamAvatar for controllable 3D human avatar generation, HeadSculpt for editing 3D head avatars while preserving the original identity, and Guide3D for creating 3D human by combining the metrics from reconstruction and generation.

Video