Compositional Neural Textures
Fully unsupervised approach for representing textures using a compositional neural model based on textons represented as Gaussian functions
The paper introduces a novel framework for compositional neural texture representation, focusing on texture diversification, transfer, modification, interpolation, direct texton manipulation, and animated textures. The method involves reshuffling appearance features in latent Gaussians to generate different versions of textures. The discriminator D is used for downsampling and cropping images, while the encoder E generates spatial Gaussian parameters. The dataset consists of synthetic and real stock images, augmented with transformations and color matching.
Applications of the framework include texture diversification, transfer, modification, interpolation, direct texton manipulation, and animated textures. The method allows for manipulating compositional latent Gaussians to efficiently generate output textures. The feed-forward processing of images using the autoencoder is fast, taking 0.16 seconds on a single NVIDIA A10G. The framework showcases various dynamic visualizations and applications for texture editing.
The methodology autonomously extracts compositional textons from textures using unsupervised techniques, focusing on segmentation and conversion into Gaussians. A comparison with the "Segment Anything" model highlights the versatility of the proposed method in extracting textons. The approach excels in disentangled structure-appearance texture modeling, capturing both structure and appearance effectively.
Comments
None