An Active Olfaction Approach Using Deep Reinforcement Learning for Indoor Attenuation Odor Source Localization

Hui Li,Jie Yuan,Hao Yuan

DOI: https://doi.org/10.1109/jsen.2024.3373610

IF: 4.3

2024-01-01

IEEE Sensors Journal

Abstract:The localization of odor sources (e.g., poisonous odor sources) is an important task for the security of the environment and human society. Traditional robot localization methods are sensitive to environmental changes, leading to localization performance degradation in dynamic environments and complex scenes. The time-varying odor sources are not fully taken into account by the traditionally robot localization method in turbulent environments, resulting in low search efficiency and even failure. This paper proposes the odor source localization algorithm using a Proximal Policy Optimization based on Gated Recurrent Unit algorithm (GRU-PPO). Firstly, the odor source localization problem is modeled as a Markov Decision Process, and the state space, action space, and dense rewards are designed to address the sparse reward problem. Secondly, the Gated Recurrent Unit network is applied to the actor-critic framework of the PPO algorithm to extract temporal features from historical data and generate optimal decisions in an end-to-end manner. Finally, in an indoor turbulent environment, the feasibility of the proposed GRU-PPO algorithm for source localization is verified by simulating the leakage process of a decaying plume source using computational fluid dynamics. The algorithm’s effectiveness is demonstrated in diverse and complex environments, with a success rate maintained at a high level (99%).

engineering, electrical & electronic,instruments & instrumentation,physics, applied