Precision fruit tree pruning using a learned hybrid vision/interaction controller

Alexander You,Hannah Kolano,Nidhi Parayil,Cindy Grimm,Joseph R. Davidson

DOI: https://doi.org/10.48550/arXiv.2109.13162

2021-09-28

Abstract:Robotic tree pruning requires highly precise manipulator control in order to accurately align a cutting implement with the desired pruning point at the correct angle. Simultaneously, the robot must avoid applying excessive force to rigid parts of the environment such as trees, support posts, and wires. In this paper, we propose a hybrid control system that uses a learned vision-based controller to initially align the cutter with the desired pruning point, taking in images of the environment and outputting control actions. This controller is trained entirely in simulation, but transfers easily to real trees via a neural network which transforms raw images into a simplified, segmented representation. Once contact is established, the system hands over control to an interaction controller that guides the cutter pivot point to the branch while minimizing interaction forces. With this simple, yet novel, approach we demonstrate an improvement of over 30 percentage points in accuracy over a baseline controller that uses camera depth data.

Robotics

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The paper aims to address the issue of precise control in pruning fruit trees in orchards. Specifically, robotic pruning requires high-precision control of the robotic arm to ensure that the cutting tool can accurately align with the target pruning point and cut at the correct angle. At the same time, the robot needs to avoid exerting excessive force on rigid parts of the environment (such as trees, support poles, and wires).

Traditional position-based controllers face numerous challenges in handling these tasks, such as:
1. **Precise Alignment**: How to accurately align the cutting tool with the target pruning point in a complex and irregular environment.
2. **Force Control**: How to avoid damaging the environment while ensuring cutting precision when contacting branches.
3. **Robustness**: How to cope with environmental uncertainties and sensor noise to ensure system stability and reliability.

To address these issues, the paper proposes a hybrid control system that combines a vision-based controller and a force-based interaction controller. Specifically:
- **Vision-Based Controller**: Guides the cutting tool near the target pruning point by learning from image data. This controller is entirely trained in a simulated environment but can perform well in real environments by using neural networks to convert real images into simplified segmented representations.
- **Force-Based Interaction Controller**: Takes over control upon detecting contact with a branch, guiding the pivot point of the cutting tool to align with the branch through force feedback to achieve precise cutting.

Through this simple yet novel approach, the paper demonstrates that compared to traditional position-based controllers, this system improves cutting accuracy by over 30 percentage points and can cut branches more consistently at the preset length.