Gradient-Weighted Feature Back-Projection: A Fast Alternative to Feature Distillation in 3D Gaussian Splatting

Joji Joseph, Bharadwaj Amrutur, Shalabh Bhatnagar

SIGGRAPH Asia 2025




(a) (b) (c)

Instantaneous 3D Segmentation (Prompt: Table, Negative Prompt: Vase, Other)

(a) Rendering without any segmentation. (b) The result after extracting the segmented regions. (c) The result after deleting the segmented regions.

Abstract

We introduce a training-free method for feature field rendering in Gaussian splatting. Our approach back-projects 2D features into pre-trained 3D Gaussians, using a weighted sum based on each Gaussian's influence in the final rendering. While most training-based feature field rendering methods excel at 2D segmentation but perform poorly at 3D segmentation without post-processing, our method achieves high-quality results in both 2D and 3D segmentation. Experimental results demonstrate that our approach is fast, scalable, and offers performance comparable to training-based methods.

Highlights

Feature Back-Projection

$$ \mathbf{f}_k = \frac{\sum_{(x,y,n)} \mathbf{F}_{2D}(x,y,n)\alpha_k(x,y,n)T_k(x,y,n)}{\sum_{(x,y,n)} \alpha_k(x,y,n)T_k(x,y,n)} $$ \begin{aligned} &\textbf{Terms Explanation:} \\ &\mathbf{f}_k: \text{Feature vector for the \( k \)-th Gaussian. Represents the aggregated feature assigned to the Gaussian in 3D.} \\ &\mathbf{F}_{2D}(x, y, n): \text{Feature vector at the 2D pixel coordinate \( (x, y) \) in the \( n \)-th view. Comes from input feature maps.} \\ &\alpha_k(x, y, n): \text{Opacity of the \( k \)-th Gaussian at pixel \( (x, y) \) in view \( n \).} \\ &T_k(x, y, n): \text{Transmittance of the \( k \)-th Gaussian at pixel \( (x, y) \) in view \( n \), accounting for occlusion effects.} \\ \end{aligned}

Gallery

2D Segmentation

Feature 3DGS Ours
Prompt: Table, Negative Prompt: Vase, Other
Prompt: Plant, Negative Prompt: Other
Prompt: Plant, Negative Prompt: Vase, Other
Prompt: Truck, Negative Prompt: Other

3D Segmentation

Extraction Deletion
Feature 3DGS Ours Feature 3DGS Ours
Prompt: Table, Negative Prompt: Vase, Other
Prompt: Plant, Negative Prompt: Other
Prompt: Plant, Negative Prompt: Vase, Other
Prompt: Truck, Negative Prompt: Other

Citation

If you find this paper or the code helpful for your work, please consider citing the following preprint: 


@inproceedings{joseph2024gradientweightedfeaturebackprojection,
    author = {Joseph, Joji and Amrutur, Bharadwaj and Bhatnagar, Shalabh},
    title = {Gradient-Weighted Feature Back-Projection: A Fast Alternative to Feature Distillation in 3D Gaussian Splatting},
    year = {2025},
    isbn = {9798400721373},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3757377.3763926},
    doi = {10.1145/3757377.3763926},
    abstract = {We propose a training-free method for feature field rendering in 3D Gaussian Splatting, enabling fast and scalable embedding of high-dimensional features into 3D scenes. Unlike training-based feature distillation methods, which are computationally expensive and often yield feature embeddings that poorly reflect the rendered semantics, our approach back-projects 2D features onto pre-trained 3D Gaussians using influence weights derived from the rendering equation. This projection produces a queryable 3D feature field, validated on tasks including 2D and 3D segmentation, affordance transfer, and identity encoding, spanning queries using language, pixel, and synthetic embeddings. These capabilities, in turn, enable downstream applications in augmented and virtual reality, interactive scene editing, and robotics. Across different tasks, our method achieves performance comparable to or better than training-based approaches, while significantly reducing computational cost. The project page is at https://jojijoseph.github.io/3dgs-backprojection.},
    booktitle = {Proceedings of the SIGGRAPH Asia 2025 Conference Papers},
    articleno = {178},
    numpages = {12},
    keywords = {3DGS, Feature Field Distillation, Feature Lifting},
    location = {
    },
    series = {SA Conference Papers '25}
}