Abstract:Trajectory planning in robotics aims to generate collision-free pose sequences that can be reliably executed. Recently, vision-to-planning systems have garnered increasing attention for their efficiency and ability to interpret and adapt to surrounding environments. However, traditional modular systems suffer from increased latency and error propagation, while purely data-driven approaches often overlook the robot's kinematic constraints. This oversight leads to discrepancies between planned trajectories and those that are executable. To address these challenges, we propose iKap, a novel vision-to-planning system that integrates the robot's kinematic model directly into the learning pipeline. iKap employs a self-supervised learning approach and incorporates the state transition model within a differentiable bi-level optimization framework. This integration ensures the network learns collision-free waypoints while satisfying kinematic constraints, enabling gradient back-propagation for end-to-end training. Our experimental results demonstrate that iKap achieves higher success rates and reduced latency compared to the state-of-the-art methods. Besides the complete system, iKap offers a visual-to-planning network that seamlessly integrates kinematics into various controllers, providing a robust solution for robots navigating complex and dynamic environments.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the limitations of traditional modular systems and pure data - driven methods in robot trajectory planning. Specifically:
1. **Limitations of traditional modular systems**:
- **Delay and error propagation**: In traditional modular systems, the perception module and the planning module work independently, which leads to potential delays in the transfer of intermediate results, increased communication overhead, and increased storage requirements. In addition, an error in one module may propagate and be amplified throughout the system, affecting the overall performance.
- **Inconsistency between geometric paths and execution paths**: These systems usually ignore the kinematic constraints of the robot, resulting in generated paths that may be difficult for the low - level controller to execute.
2. **Limitations of pure data - driven methods**:
- **Lack of physical constraints**: Pure data - driven methods usually operate as black boxes, require a large amount of data for training, have low sample efficiency, and have a significant sim - to - real gap. In addition, these methods are difficult to interpret and it is difficult to incorporate explicit physical constraints or prior knowledge into them.
- **Infeasible paths**: Due to ignoring the kinematic constraints of the robot, the generated paths may not be executable, resulting in deadlocks, collisions, or sub - optimal performance.
To solve these problems, the paper proposes iKap (Kinematics - aware Planning with Imperative Learning), which is a new vision - to - planning system that directly integrates the kinematic model of the robot into the learning pipeline. iKap ensures that the network learns collision - free waypoints while satisfying kinematic constraints through a self - supervised bi - level optimization (BLO) framework, thereby achieving end - to - end training. This method not only improves the success rate of paths but also reduces latency and enables the robot to navigate robustly in complex and dynamic environments.
### Main contributions
- **Kinematic model integration**: The kinematic model of the robot is directly integrated into the training pipeline through differentiable model predictive control (differentiable MPC), enabling the network to learn kinematically feasible trajectories.
- **Learning pipeline design**: A learning pipeline that allows gradient back - propagation is developed, from low - level trajectory optimization to high - level network parameters, promoting a kinematics - aware vision - to - planning model.
- **Experimental verification**: The proposed system is evaluated through simulation and real - world experiments, and the results show that iKap has a higher success rate and lower tracking error in various environments.
Through these improvements, iKap addresses the shortcomings of existing methods in generating feasible paths and real - time performance, providing a more robust solution.