Text-to-3D Shape Generation
Explores advancements in representation learning and differentiable rendering for text-to-3D shape generation
The paper provides a comprehensive survey of recent advancements in text-to-3D shape generation methods, focusing on the underlying technology and methods enabling this capability. It highlights the rapid progress in this research direction, driven by advances in generative models for text and images, as well as learned 3D representations and generative models. The survey categorizes recent work on text-to-3D shape generation based on the type of supervision data required, systematically summarizing methods based on training data, 3D representation type, generative model, and training setup.
The survey classifies recent text-to-3D research into four primary families: methods that utilize paired text with 3D data (3DPT), methods reliant on 3D data but not requiring paired 3D and text data (3DUT), methods that require no 3D data training data at all, and recent work that leverages a combination of text-to-image and image-to-3D. The focus is primarily on the third family of works, which have not been addressed in detail by prior surveys. The survey further divides the methods that rely on no 3D data into sections that discuss methods that leverage pre-trained text-image embeddings, formulate or improve upon ways to use diffusion models as a prior, and use different 3D representations, rendering techniques, or improvements to the training setup to enhance the quality of the results.
Furthermore, the survey addresses the computational aspects of the methods, providing a summary of reported speed and memory consumption for the surveyed methods. It also outlines the types of evaluation protocols in prior work, focusing on user studies and evaluation against ground-truth shapes. Overall, the survey provides a comprehensive overview of the state-of-the-art in text-to-3D shape generation methods, categorizing and summarizing recent advancements in this rapidly evolving research area.
Comments
None