Abstract:This paper studies the high-speed railway mobile networks (HSRMN), where multiple railway-side sensors (RSs) are deployed along the track to sense environmental data, and multiple train-mounted sensors (TSs) are deployed on the train to collect train data. Both RSs and TSs are scheduled to transmit their sensed data respectively to the ground base station (BS) in a time division multiple access (TDMA) mode. To keep the data received at the BS from the RSs as fresh as possible and also ensure that the TSs complete the given uploading tasks, an optimization problem is established to minimize the average age of information (AoI) of the data gathered from RSs by jointly optimizing sensors’ scheduling and transmission power control constrained by the maximum transmission power budget of RSs and TSs. Since the problem is non-convex and lacks an explicit expression of the objective function and the prior information about future channel state, we present a deep Q-learning network (DQN)-based method to solve it. Particularly, the BS is viewed as the agent, and the action space is constructed by scheduling policy and power control. To further accelerate the convergence speed of the presented DQN-based solution framework, an action space-reduced (ASR) version of the DQN-based method, i.e., the ASR-DQN-based method, is designed by deriving a closed-form solution to the optimal transmission power for a given sensors’ scheduling policy. Numerical simulations show that, compared to the DQN-based method, the ASR-DQN-based method decreases the number of episodes required for convergence by about 23% and reduces the running time by about 41%. Moreover, compared with three baselines, i.e., the random method, the round-robin method, and the deep-Sarsa method, our presented ASR-DQN-based method achieves the lowest average AoI and has the best robustness among these compared methods.

Development of Machine Learning-Based Energy Management Agent to Control Fine Dust Concentration in Railway Stations

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

The Performance of Reinforcement Learning for Indoor Climate Control Devices according to the Level of Outdoor Air Particulate Matters

Optimization strategy based on deep reinforcement learning for home energy management

Deep Learning-Based Indoor Air Quality Forecasting Framework for Indoor Subway Station Platforms

Performance analysis of particulate matter priority control algorithm for outdoor Air-Introduced ventilation Cleaner using prediction model

A reinforcement learning approach for control of window behavior to reduce indoor PM2.5 concentrations in naturally ventilated buildings

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings

Developing smart air purifier control strategies for better IAQ and energy efficiency using reinforcement learning

Enhancing the sustainable management of fine particulate matter-related health risks at subway stations through sequential forecast and gated probabilistic transformer

Predicting PM10 concentration in Seoul metropolitan subway stations using artificial neural network (ANN)

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Transfer learning driven sequential forecasting and ventilation control of PM2.5 associated health risk levels in underground public facilities

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Application of Deep Learning Algorithm in Optimization Control of Electrostatic Precipitator in Coal-Fired Power Plants

Modelling building HVAC control strategies using a deep reinforcement learning approach

Minimizing AoI in High-Speed Railway Mobile Networks: DQN-Based Methods

DDQN-based data laboratory energy consumption control model

Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning

Deep-AI soft sensor for sustainable health risk monitoring and control of fine particulate matter at sensor devoid underground spaces: A zero-shot transfer learning approach

Energy-efficient Personalized Thermal Comfort Control in Office Buildings Based on Multi-Agent Deep Reinforcement Learning