Abstract:Sea freight is one of the most important ways for the transportation and distribution of coal and other bulk cargo. This paper proposes a method for optimizing the scheduling efficiency of the bulk cargo loading process based on deep reinforcement learning. The process includes a large number of states and possible choices that need to be taken into account, which are currently performed by skillful scheduling engineers on site. In terms of modeling, we extracted important information based on actual working data of the terminal to form the state space of the model. The yard information and the demand information of the ship are also considered. The scheduling output of each convey path from the yard to the cabin is the action of the agent. To avoid conflicts of occupying one machine at same time, certain restrictions are placed on whether the action can be executed. Based on Double DQN, an improved deep reinforcement learning method is proposed with a fully connected network structure and selected action sets according to the value of the network and the occupancy status of environment. To make the network converge more quickly, an improved new epsilon-greedy exploration strategy is also proposed, which uses different exploration rates for completely random selection and feasible random selection of actions. After training, an improved scheduling result is obtained when the tasks arrive randomly and the yard state is random. An important contribution of this paper is to integrate the useful features of the working time of the bulk cargo terminal into a state set, divide the scheduling process into discrete actions, and then reduce the scheduling problem into simple inputs and outputs. Another major contribution of this article is the design of a reinforcement learning algorithm for the bulk cargo terminal scheduling problem, and the training efficiency of the proposed algorithm is improved, which provides a practical example for solving bulk cargo terminal scheduling problems using reinforcement learning.

Optimizing container relocation operations by using deep reinforcement learning

Heuristic Algorithms for Routing Yard‐side Equipment for Minimizing Loading Times in Container Terminals

Research on Real-Time Export Container Allocation Considering Loading Efficiency

Container stacking optimization based on Deep Reinforcement Learning

A deep reinforcement learning-based optimization method for long-running applications container deployment

Empty container repositioning problem using a reinforcement learning framework with multi-weight adaptive reward function

Discrete Dynamic Berth Allocation Optimization in Container Terminal Based on Deep Q-Network

Machine Learning-Driven Algorithms for the Container Relocation Problem

Optimizing inland container shipping through reinforcement learning

Integrated scheduling optimization of U-shaped automated container terminal under loading and unloading mode

Container port truck dispatching optimization using Real2Sim based deep reinforcement learning

Developing a Container Ship Loading-Planning Program Using Reinforcement Learning

Deep Reinforcement Learning Assisted Genetic Programming Ensemble Hyper-Heuristics for Dynamic Scheduling of Container Port Trucks

Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning

An Efficient Deep Reinforcement Learning Model for Online 3D Bin Packing Combining Object Rearrangement and Stable Placement

A hybrid algorithm combining ant colony optimization and large neighborhood search based on reinforcement learning for integrated container-truck scheduling problem

Research on Intelligent Dynamic Scheduling Algorithm for Automated Guided Vehicles in Container Terminal Based on Deep Reinforcement Learning

Scheduling of twin automated stacking cranes based on Deep Reinforcement Learning

Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation via Deep Reinforcement Learning

Real-time multi-resource jointed scheduling of container terminals with uncertainties using a reinforcement learning approach

Optimizing Robotic Mobile Fulfillment Systems for Order Picking Based on Deep Reinforcement Learning