Coverage Path Planning for Decomposition Reconfigurable Grid-Maps Using Deep Reinforcement Learning Based Travelling Salesman Problem

Phone Thiha Kyaw,Aung Paing,Theint Theint Thu,Rajesh Elara Mohan,Anh Vu Le,Prabakaran Veerajagadheswar
DOI: https://doi.org/10.1109/access.2020.3045027
IF: 3.9
2020-01-01
IEEE Access
Abstract:Optimizing the coverage path planning (CPP) in robotics has become essential to accomplish efficient coverage applications. This work presents a novel approach to solve the CPP problem in large complex environments based on the Travelling Salesman Problem (TSP) and Deep Reinforcement Learning (DRL) leveraging the grid-based maps. The proposed algorithm applies the cellular decomposition methods to decompose the environment and generate the coverage path by recursively solving each decomposed cell formulated as TSP. A solution to TSP is determined by training Recurrent Neural Network (RNN) with Long Short Term Memory (LSTM) layers using Reinforcement Learning (RL). We validated the proposed method by systematically benchmarked with other conventional methods in terms of path length, execution time, and overlapping rate under four different map layouts with various obstacle density. The results depict that the proposed method outperforms all considered parameters than the conventional schemes. Moreover, simulation experiments demonstrate that the proposed approach is scalable to the larger grid-maps and guarantees complete coverage with efficiently generated coverage paths.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to optimize Coverage Path Planning (CPP) in large and complex environments. Specifically, the author proposes a new method based on the Travelling Salesman Problem (TSP) and Deep Reinforcement Learning (DRL) to generate efficient coverage paths. This method is especially suitable for environments represented by grid maps. By decomposing the environment into multiple cells and recursively solving TSP within each cell, the coverage path is generated. ### Main Problems 1. **Optimizing CPP**: In large and complex environments, how to complete the coverage task efficiently is a challenge. Traditional CPP methods often perform poorly when dealing with large - scale environments due to overly long path lengths, excessive computation times, or high path overlap rates. 2. **NP - hard Problem**: TSP is an NP - hard problem. Directly solving the TSP for all grid points is impractical in large - scale environments. Therefore, an effective method is required to approximately solve TSP to generate efficient coverage paths. ### Solutions 1. **Recursive TSP Method**: The author proposes a recursive method that decomposes the environment into multiple cells, and the path planning problem within each cell is transformed into a TSP problem. By recursively solving the TSP of each cell, the coverage path of the entire environment is finally generated. 2. **Deep Reinforcement Learning**: To solve TSP efficiently, the author uses the Deep Reinforcement Learning (DRL) method. Specifically, they train a Recurrent Neural Network (RNN) containing a Long Short - Term Memory (LSTM) layer and optimize the network parameters through Reinforcement Learning (RL). This method can quickly generate near - optimal TSP paths. 3. **Simulation Verification**: The author verifies the effectiveness of the proposed method through simulation experiments. The experimental results show that, compared with traditional methods, this method performs well in terms of path length, execution time, and overlap rate, especially in large - scale and complex environments. ### Key Contributions 1. **Recursive TSP Method**: A recursive method is proposed, which decomposes the CPP problem into multiple TSP sub - problems, and generates a global coverage path by recursively solving each sub - problem. 2. **DRL Solving TSP**: The TSP problem is solved using the Deep Reinforcement Learning method. By training the RNN model, near - optimal paths can be generated efficiently. 3. **Simulation Study**: Through simulation experiments, the effectiveness and scalability of the proposed method are verified from multiple perspectives. ### Conclusion This paper proposes a new CPP method based on TSP and DRL, which can generate efficient coverage paths in large and complex environments. The experimental results show that this method is superior to traditional methods in terms of path length, execution time, and overlap rate, and has high practical value.