Hybrid mesh-Gaussian head avatars (MeGA) combines neural mesh modeling for the face with 3D Gaussian splatting (3DGS) for hair modeling. To enhance facial modeling, the FLAME mesh is improved, and a UV displacement map is decoded for geometric details. Facial colors are decoded from a neural texture map consisting of disentangled diffuse, view-dependent, and dynamic textures. For hair modeling, a static 3DGS hair model is used with an MLP-based deformation field for animation. The final renderings are achieved through an occlusion-aware blending module that blends the hair and head parts seamlessly.

This approach involves several key components. Disentangled texture maps play a crucial role in capturing detailed skin appearance and handling view-dependent effects. A UV displacement map is utilized to improve geometric details in the renderings. Different blending strategies are explored, with an occlusion-aware blending approach proving to be effective in achieving realistic results. Regularizations are also employed to ensure the generation of a reasonable facial mesh without issues like face crossing or reversing. The method supports various editing functionalities, including hairstyle alteration and texture editing, allowing for easy modifications to the head avatars.

The method is trained using multi-view video inputs and leverages photometric and geometric losses to refine the facial mesh and ensure high-quality renderings. The training process involves joint optimization of the hybrid mesh-Gaussian avatar to improve the quality of the face-hair overlapping region. Ablation studies are conducted to verify the effectiveness of the proposed components, demonstrating the importance of disentangled texture maps, UV displacement maps, occlusion-aware blending, and regularizations in achieving high-quality results. Overall, the method offers a comprehensive framework for creating realistic and editable head avatars with detailed facial and hair modeling.