arXiv 2610.22602
Stuck on Shuffle: On Attribute-Object Binding in Vision-Language Models
AI14 upvotes
Authors: Netanel Y. Tamir, Michael Green, Issar Tzachor, Or Shimshi, Dvir Samuel, Or Patashnik, Rami Ben-Ari
Overview
The authors introduce SABER, a controlled benchmark that tests attribute-object binding across all four image and text modality pairings, with scenes of up to ten objects. They then finetune an MLLM-based embedder using only synthetic attribute-binding text, which is cheap to generate and allows precisely controlled hard negatives.
Introduction
Multimodal retrieval needs embeddings that capture not only which concepts appear but also how they are composed, and attribute-object binding (which attribute belongs to which object) is a key part of that. Existing benchmarks cover too few modality pairings and too simple scenes to show how robust models are at this.
Results
Finetuning an MLLM-based embedder solely on synthetic binding text achieves SOTA on SABER, and the gains transfer to cross-modal and image-image retrieval without losing general multimodal performance. Both CLIP-style dual encoders and recent MLLM-based embedders show substantial binding failures before this training, and negation and instance-retrieval benchmarks also improve.
Conclusion
Cheap, precisely controlled text supervision offers a practical way to fix attribute binding in multimodal embedders without costly image collection.