Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos
Method presented for constructing animatable dog avatars from monocular videos, utilizing a 4D solution for pose and appearance modeling, showing superior results on challenging datasets compared to existing approaches.
The paper introduces a novel method for reconstructing animatable 3D animals from casual videos, specifically focusing on dogs. It addresses the limitations of existing approaches by replacing sparse keypoint supervision with Continuous Surface Embeddings (CSE) for denser supervision. Transforming CSE descriptors to the SMAL mesh provides stronger keypoint loss for reprojection constraints in rear and side views. Additionally, the method enhances fits by incorporating the smoothness of animal movements over time, optimizing the SMAL deformation using Structure-from-Motion (SfM) camera and animal motion. This allows for proper temporal regularization and more accurate shape fitting through rendering-based loss terms.
Furthermore, the method enables texturing of the SMAL mesh by leveraging it as a scaffold for an implicit duplex-mesh neural radiance field. This involves defining implicit shape and color functions on a subset of the 3D domain bounded by enlarged and downsized versions of the mesh template, articulating the implicit surface by posing the boundary meshes similarly to the original mesh. It also factors in the rigid motion of the camera and the shape separately, leading to improved performance across all metrics. By utilizing Continuous Surface Embeddings, optimizing SMAL deformation, and implementing implicit duplex-mesh rendering, the method achieves superior results in reconstructing textured animatable 3D models of dogs from monocular videos.


Comments
None