Gaussian Splatting in Style (GSS) performs real-time neural scene stylization based on 3D Gaussian splatting. The motivation for this work stems from the need for a specialized method that considers spatial information when stylizing a scene. The authors argue that simply extending 2D style transfer to 3D scenes can result in visual artifacts such as blurriness and inconsistency across different views. To address this, the proposed GSS method leverages pre-trained Gaussians conditioned on a style image to obtain stylized views of complex 3D scenes. By utilizing these Gaussians as the backbone of the approach, spatial consistency is ensured. The method does not add any overhead to the existing real-time rendering speed of 3D Gaussian Splatting, making it suitable for applications in augmented and virtual reality ecosystems.

In terms of implementation, the authors build the GSS method on top of the 3D Gaussian Splatting (3DGS) framework and use it to pretrain the Gaussians for each scene. They also incorporate a 2D stylization module using a pretrained VGG and AdaIN decoder, as well as a pretrained image encoder for obtaining the latent code of input style images. The hyperparameters are set, and the training dataset consists of 120 diverse style images sampled from the WikiArt dataset, including paintings and popular artworks of various artists.