AniArtAvatar introduces a one-shot framework for generating animatable avatars from a single art portrait image. This new method allows for precise control over facial expressions, head poses, and torso poses, enabling various applications including exaggerated cartoon generation. The pipeline involves utilizing conditioned diffusion models to obtain initial multi-view images from the input image. Subsequently, an SDF-based implicit surface is trained to synthesize the artistic avatar using these multi-view images. To animate the avatar, the front image is rendered, 2D landmarks are extracted, and then projected onto the implicit surface to obtain 3D landmarks. Expression animation is achieved through the transfer of human face landmarks' relative motion to drive the 3D landmarks and deform the implicit surface. For avatar pose animation, head and torso cages are set up, and transformations are applied to them.

The method leverages recent advances in zero-shot 3D generation with diffusion models, particularly focusing on conditioned diffusion models for multi-view image synthesis. By extracting 3D landmarks from 2D landmarks and deforming the implicit surface based on human face landmark motion, AniArtAvatar enables precise control over avatar expressions and poses. The approach involves synthesizing an SDF-based implicit surface for the art avatar, allowing for detailed and controllable animation. The use of head and torso cages, along with neck deformation, facilitates pose animation, while additional landmarks are introduced to enhance the visual quality of expression animation.