Figure 1: OV-Foam pipeline and qualitative text-query example. Cropped from the author-uploaded arXiv paper; credit: Yousef Rajeh, Alejandra Esparza Pelcastre, Ivan Viola, and Dominik Engel.At a glance
- Uses non-overlapping PowerFoam cells to reduce semantic mixing when lifting 2D features into a 3D scene.
- Separates reconstruction-induced ambiguity from upstream semantic errors using matched lifting procedures and ground-truth image labels.
- Improves the tested open-vocabulary segmentation methods on ScanNet and LERF while keeping their SAM/CLIP observations fixed.
Motivation
Open-vocabulary 3D segmentation lets users identify objects and regions with free-form text queries. A common approach lifts vision-language features from 2D images onto reconstructed 3D primitives. With Gaussian splatting, many overlapping primitives contribute to each pixel, entangling semantic evidence across objects and views. OV-Foam asks how much of this ambiguity comes from the underlying representation rather than the feature-lifting procedure itself.
Method
OV-Foam replaces overlapping Gaussians with PowerFoam, whose bounded power-diagram cells partition space without overlapping interiors. Starting from a fixed reconstruction, it uses the renderer’s contribution weights to backproject SAM-region CLIP features into one feature vector per primitive. These vectors can then be compared with text embeddings for open-vocabulary queries.
The method instantiates the Splat Feature Solver one-shot weighted estimator on PowerFoam and studies its relationship to a least-squares reference. Non-overlapping cells do not imply that every ray meets only one primitive: a ray can traverse several cells. Instead, PowerFoam concentrates contributions onto fewer primitives, reducing feature mixing and bringing the one-shot estimate closer to the reference. The analysis distinguishes ray-level dispersion, the fitting gap to least squares, and the class purity of evidence accumulated by each primitive.
Evaluation
On ten ScanNet scenes, the controlled comparison starts both representations from the same ground-truth vertices and fixes primitive centers and counts. PowerFoam averages 2.32 contributors per ray, compared with 33.63 for 3DGS, and reduces the per-ray fitting gap to the least-squares reference by 82.1%. With correct image labels encoded as class text embeddings, it reaches 90.59% mIoU, versus 85.40% for 3DGS; this isolates reconstruction and lifting errors from upstream semantic mistakes.
Using real SAM/CLIP features, replacing 3DGS with PowerFoam improves mIoU and mean class accuracy for all four tested lifting methods—VALA, Occam’s LGS, SFS, and LUDVIG—across the three ScanNet class sets. The main ScanNet comparison retains all valid labeled points rather than filtering out low-opacity regions. A separate rendered text-query evaluation on the four LERF scenes Figurines, Ramen, Teatime, and Waldo kitchen also improves both reported metrics for every tested method.
Limitations
The ScanNet study covers ten scenes initialized from ground-truth geometry, not image-derived structure-from-motion reconstructions. Its controlled ground-truth-label experiment is not an upper bound over all possible lifting methods. Reduced fitting loss does not guarantee better segmentation, and high class purity can still reflect consistently incorrect image features. Cells may cross semantic boundaries, and multiple cells or partially transparent surfaces can still contribute along a ray. The reported advantages therefore depend on reconstruction quality, input features, and the evaluation setting.
