Measure Anything: Real-time, Multi-stage Vision-based Dimensional Measurement using Segment Anything

Yongkyu Lee,Shivam Kumar Panda,Wei Wang,Mohammad Khalid Jawed

2024-12-05

Abstract:We present Measure Anything, a comprehensive vision-based framework for dimensional measurement of objects with circular cross-sections, leveraging the Segment Anything Model (SAM). Our approach estimates key geometric features -- including diameter, length, and volume -- for rod-like geometries with varying curvature and general objects with constant skeleton slope. The framework integrates segmentation, mask processing, skeleton construction, and 2D-3D transformation, packaged in a user-friendly interface. We validate our framework by estimating the diameters of Canola stems -- collected from agricultural fields in North Dakota -- which are thin and non-uniform, posing challenges for existing methods. Measuring its diameters is critical, as it is a phenotypic traits that correlates with the health and yield of Canola crops. This application also exemplifies the potential of Measure Anything, where integrating intelligent models -- such as keypoint detection -- extends its scalability to fully automate the measurement process for high-throughput applications. Furthermore, we showcase its versatility in robotic grasping, leveraging extracted geometric features to identify optimal grasp points.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the challenges faced in vision - based dimensional measurement, especially the problem of accurately identifying target objects and extracting their geometric features in complex and cluttered environments. Specifically, the paper proposes a comprehensive framework named **Measure Anything**, which utilizes the Segment Anything Model (SAM) to achieve real - time, multi - stage measurement of key geometric features (such as diameter, length, and volume) of objects with circular cross - sections.

#### Main problems and challenges include:

1. **Identifying target objects in cluttered environments**:
- Complex geometric shapes and occluders make it difficult for existing methods to accurately identify and segment target objects.

2. **Extracting accurate measurement values from object contours**:
- Existing methods face difficulties when dealing with non - uniform and slender objects (such as rape stems), and the size measurement of these objects is crucial in fields such as agriculture and manufacturing.

3. **The need for automation and high - throughput applications**:
- A scalable method is required to automatically obtain point prompts to support efficient operation in large - scale application scenarios.

4. **Adaptability to multi - domain applications**:
- The framework needs to be applicable to tasks in different domains, such as precision agriculture, robotic grasping, etc., and be able to maintain performance in highly cluttered environments.

### Specific application scenarios:

- **Precision agriculture**:
- Measuring the diameter of rape stems, which is a phenotypic feature closely related to crop health and yield.

- **Robotic grasping**:
- Using the extracted geometric features to determine the optimal grasping points, thereby achieving stable operation.

By combining advanced image segmentation techniques and geometric processing methods, **Measure Anything** provides a comprehensive solution to the above challenges and demonstrates its broad application potential in multiple fields.