The proposed LayGA introduces a novel approach to creating photorealistic human avatars by utilizing a layered Gaussian representation for animatable clothing transfer. The method involves a two-stage training process, starting with single-layer reconstruction and segmentation, followed by multi-layer fitting. In the single-layer reconstruction stage, geometric constraints are introduced to ensure that the 3D Gaussians lie on a smooth surface, facilitating collision handling between the body and clothing. Additionally, a segmentation label channel is learned to separate body and clothing. In the multi-layer fitting stage, two separate models are trained to represent the body and clothing, with the reconstructed clothing geometry serving as 3D supervision for accurate garment tracking. A rendering layer is also introduced to maintain high-quality geometric reconstruction and rendering.

During the single-layer modeling with geometric constraints, various loss functions are utilized to enforce smooth surface reconstruction and clear clothing boundaries. Geometric constraints such as offset regularization, total variational loss, and edge regularization are employed to penalize large offsets, encourage neighboring pixels to remain close, and maintain edge lengths between the base model and deformed model. Additionally, a segmentation loss function is introduced to obtain segmentation between body and clothing, with cross-entropy loss used to compare rendered segmentation with ground truth segmentation. The rendering loss function includes L1, SSIM, and perceptual losses on rendered RGB images, ensuring high-fidelity rendering quality.

LayGA method the challenges of existing approaches by providing a layered Gaussian representation for animatable clothing transfer. By incorporating geometric constraints, segmentation, and rendering layers, the model can generate realistic animations under novel poses and transfer clothing across identities. The two-stage training process enables the creation of high-quality reconstructions with explicit geometries for collision handling and accurate tracking of clothing boundaries. The method outperforms baseline approaches in terms of photorealistic avatar creation and virtual try-on capabilities.