Thinking-with-Visual-Primitives

Thinking with Visual Primitives introduces a novel approach to Multimodal Large Language Models by interleaving spatial markers directly into the reasoning process. This method addresses the reference gap in complex structural tasks by anchoring abstract language to concrete physical coordinates. The framework achieves frontier-competitive performance while maintaining high visual token efficiency through a compressed architecture.

The project introduces a novel paradigm for Multimodal Large Language Models by addressing the 'Reference Gap,' where natural language fails to precisely describe dense spatial layouts. By interleaving spatial markers like points and bounding boxes directly into the reasoning trajectory, the model anchors abstract concepts to physical coordinates, effectively mimicking human cognitive behavior. This approach prioritizes structural reasoning and visual grounding, utilizing a highly efficient architecture to maintain performance while significantly reducing the computational overhead of image tokens.