Abstract:Vision language models (VLMs) have demonstrated impressive performance across a wide range of downstream tasks. However, their proficiency in spatial reasoning remains limited, despite its crucial role in tasks involving navigation and interaction with physical environments. Specifically, much of the spatial reasoning in these tasks occurs in two-dimensional (2D) environments, and our evaluation reveals that state-of-the-art VLMs frequently generate implausible and incorrect responses to composite spatial reasoning problems, including simple pathfinding tasks that humans can solve effortlessly at a glance. To address this, we explore an effective approach to enhance 2D spatial reasoning within VLMs by training the model on basic spatial capabilities. We begin by disentangling the key components of 2D spatial reasoning: direction comprehension, distance estimation, and localization. Our central hypothesis is that mastering these basic spatial capabilities can significantly enhance a model's performance on composite spatial tasks requiring advanced spatial understanding and combinatorial problem-solving. To investigate this hypothesis, we introduce Sparkle, a framework that fine-tunes VLMs on these three basic spatial capabilities by synthetic data generation and targeted supervision to form an instruction dataset for each capability. Our experiments demonstrate that VLMs fine-tuned with Sparkle achieve significant performance gains, not only in the basic tasks themselves but also in generalizing to composite and out-of-distribution spatial reasoning tasks (e.g., improving from 13.5% to 40.0% on the shortest path problem). These findings underscore the effectiveness of mastering basic spatial capabilities in enhancing composite spatial problem-solving, offering insights for improving VLMs' spatial reasoning capabilities.

Sparcle: Boosting the Accuracy of Data Cleaning Systems through Spatial Awareness

A New Design of High-Performance Large-Scale GIS Computing at a Finer Spatial Granularity: A Case Study of Spatial Join with Spark for Sustainability

A general framework for spatial data inspection and assessment

Distance-based Data Cleaning: A Survey (Technical Report)

Data Cleaning Based on Multi-sensor Spatiotemporal Correlation

GPSClean: A Framework for Cleaning and Repairing GPS Data

A data cleaning framework based on user feedback

ESCORT: Fine-Grained Urban Crime Risk Inference Leveraging Heterogeneous Open Data

An Effective High-Performance Multiway Spatial Join Algorithm with Spark

Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Composite Spatial Reasoning

Cleaning Relations Using Knowledge Bases.

Interactive and Deterministic Data Cleaning

LocationSpark: In-memory Distributed Spatial Query Processing and Optimization

BoostClean: Automated Error Detection and Repair for Machine Learning

Temporal rules discovery for web data cleaning

Data Cleaning Using Belief Propagation

A Novel Data Cleaning Framework Based on Knowledge Graph

A Sample-Based Approach to Data Quality Assessment in Spatial Databases with Application to Mobile Trajectory Nearest-Neighbor Search

A Common-Factor Approach for Multivariate Data Cleaning with an Application to Mars Phoenix Mission Data

Recover Fine-Grained Spatial Data from Coarse Aggregation

STREAK: An Efficient Engine for Processing Top-k SPARQL Queries with Spatial Filters