Abstract:The agile earth observation satellite scheduling problem (AEOSSP) consists of selecting and scheduling a number of tasks from a set of user requests in order to optimize one or multiple criteria. In this paper, we consider a multi-objective version of AEOSSP (called MO-AEOSSP) where the failure rate and the timeliness of scheduled requests are optimized simultaneously. Due to its NP-hardness, traditional iterative problem-tailored heuristic methods are sensitive to problem instances and require massive computational overhead. We thus propose a deep reinforcement learning and parameter transfer based approach (RLPT) to tackle the MO-AEOSSP in a non-iterative manner. RLPT first decomposes the MO-AEOSSP into a number of scalarized sub-problems by a weight sum approach where each sub-problem can be formulated as a Markov Decision Process (MDP). RLPT then applies an encoder–decoder structure neural network (NN) trained by a deep reinforcement learning procedure to producing a high-quality schedule for each sub-problem. The resulting schedules of all scalarized sub-problems form an approximate pareto front for the MO-AEOSSP. Once a NN of a subproblem is trained, RLPT applies a parameter transfer strategy to reducing the training expenses for its neighboring sub-problems. Experimental results on a large set of randomly generated instances show that RLPT outperforms three classical multi-objective evolutionary algorithms (MOEAs) in terms of solution quality, solution distribution and computational efficiency. Results on various-size instances also show that RLPT is highly general and scalable. To the best of our knowledge, this study is the first attempt that applies deep reinforcement learning to a satellite scheduling problem considering multiple objectives.

A Reinforcement-Learning-Driven Bees Algorithm for Large-Scale Earth Observation Satellite Scheduling

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Deep Reinforcement Learning-Based Periodic Earth Observation Scheduling for Agile Satellite Constellation.

Deep Reinforcement Learning for Agile Satellite Scheduling Problem

Deep Reinforcement Learning for the Agile Earth Observation Satellite Scheduling Problem

Revising the Observation Satellite Scheduling Problem Based on Deep Reinforcement Learning

A large-scale mission planning method for agile earth observation satellite

Two-Phase Neural Combinatorial Optimization with Reinforcement Learning for Agile Satellite Scheduling

Deep Reinforcement Learning Method for Satellite Range Scheduling Problem.

Deep reinforcement learning and parameter transfer based approach for the multi-objective agile earth observation satellite scheduling problem

Mission Planning for Distributed Multiple Agile Earth Observing Satellites by Attention-Based Deep Reinforcement Learning Method

Tam-Ech: A Novel Method for Optimizing Multi-Agile Earth Observation Satellite Scheduling Problem Using Deep Q-Learning and Ensemble Heuristics

A DQN-based Hyperheuristic Algorithm for Emergency Scheduling of Earth Observation Satellites

Satellite Observation and Data-Transmission Scheduling Using Imitation Learning Based on Mixed Integer Linear Programming

Learning to Construct a Solution for the Agile Satellite Scheduling Problem With Time-Dependent Transition Times

Large Region Targets Observation Scheduling by Multiple Satellites Using Resampling Particle Swarm Optimization

Scheduling single-satellite observation and transmission tasks by using hybrid Actor-Critic reinforcement learning

Space-Based Sensor Tasking Using Deep Reinforcement Learning

Dual-Population Artificial Bee Colony Algorithm for Joint Observation Satellite Mission Planning Problem

Deep Reinforcement Learning with Local Attention for Single Agile Optical Satellite Scheduling Problem

A Hierarchical Resource Scheduling Method for Satellite Control System Based on Deep Reinforcement Learning