The paper introduces a novel approach that involves converting between implicit representations, such as Neural Radiance Fields (NeRFs), and explicit representations, like Gaussian Splatting (GS), to leverage the strengths of both methods. The process involves iteratively training NeRFs for mapping and feature distillation, converting them to GS for real-time localization, and then back to NeRF for updating the map and features. This method allows for efficient handling of sparse views commonly encountered in robotics scenarios.

To implement this approach, the authors modify Nerfacto to predict spherical harmonics for each RGB channel, creating a "NeRF-SH" model. They then generate a point cloud of the scene using rays rendered from training views and initialize isotropic Gaussians at these points based on the NeRF-SH predictions. This results in a representation called "NeRFGS," which captures geometric and photometric properties of the scene effectively. Fine-tuning can be done using training views to further enhance the representation.

Rendering images using NeRFGS from training views and updating a NeRF-SH model leads to the creation of "GSNeRF." Training NeRFs using GS-rendered views improves the quality metrics compared to using original images, possibly due to the absence of high-frequency structures in the GS-rendered views. This conversion between explicit and implicit representations facilitates easy updates to NeRFs or feature distillation, as demonstrated by editing out a lamp-post in an example scene.