SceneBind: Semantic-Spatial Scene Embeddings

SceneBind combines a global semantic embedding with object-centric semantic-spatial slots for scene retrieval, object grounding, and audio-visual localization.

TL;DR — SceneBind represents realistic scenes across vision, audio, and language by combining a global semantic embedding with object-centric semantic-spatial slots. The abstract says these slots capture object semantics, spatial attributes, and uncertainty. SceneBind Matching combines global scene similarity with object alignment for cross-modal scene retrieval and object grounding. The abstract claims state-of-the-art scene and spatial retrieval and strong zero-shot transfer to audio-visual localization, but gives no numerical metrics.

Problem and core idea

SceneBind addresses a gap stated directly in the abstract: existing omni-modal encoders can be strong at instance-level semantics, or “what is present,” while often lacking explicit spatial structure, or “where it is.” The proposed goal is joint semantic and 3D spatial understanding across vision, audio, and language.

The central idea is to represent each scene as a semantic-spatial entity rather than only as a global semantic vector. That entity combines scene-level semantic information with object-level spatial structure.

Full analysis, extracted claims, numerical results, entity graph, FAQ, related work, applications, and BibTeX are available via x402 micropayment.