ThemeStation is a two-stage approach for theme-aware 3D-to-3D generation. The goal is to generate novel 3D assets with unity and diversity based on just one or a few 3D exemplars. In the first stage, a pre-trained text-to-image (T2I) diffusion model is fine-tuned to create concept images that align with the theme of the input exemplars. This mimics the concept art design process in practical 3D modeling. The second stage involves reference-informed 3D asset modeling, where a rough initial 3D model is generated based on the concept image and then refined into the final 3D model using an optimization-based method. The key innovation is the introduction of dual score distillation (DSD) to leverage two diffusion priors simultaneously - a concept prior from the concept image and a reference prior from the input exemplars.

In the first stage of ThemeStation, the customized T2I diffusion model produces concept images that share a consistent theme with the input exemplars. This step is crucial for setting the theme and style of the generated 3D models. The second stage focuses on refining the initial 3D model generated from the concept image using an optimization-based approach. The dual score distillation (DSD) loss is employed to combine the guidance from both the concept image and the input exemplars, ensuring that the final 3D models maintain thematic consistency while incorporating details and features from the exemplars.

Paper: ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplars