Abstract:Delivery of items from the producer to the consumer has experienced significant growth over the past decade and has been greatly fueled by the recent pandemic. Amazon Fresh, Shopify, UberEats, InstaCart, and DoorDash are rapidly growing and are sharing the same business model of consumer items or food delivery. Existing food delivery methods are sub-optimal because each delivery is individually optimized to go directly from the producer to the consumer via the shortest time path. We observe a significant scope for reducing the costs associated with completing deliveries under the current model. We model our food delivery problem as a multi-objective optimization, where consumer satisfaction and delivery costs, both, need to be optimized. Taking inspiration from the success of ride-sharing in the taxi industry, we propose DeliverAI - a reinforcement learning-based path-sharing algorithm. Unlike previous attempts for path-sharing, DeliverAI can provide real-time, time-efficient decision-making using a Reinforcement learning-enabled agent system. Our novel agent interaction scheme leverages path-sharing among deliveries to reduce the total distance traveled while keeping the delivery completion time under check. We generate and test our methodology vigorously on a simulation setup using real data from the city of Chicago. Our results show that DeliverAI can reduce the delivery fleet size by 12\%, the distance traveled by 13%, and achieve 50% higher fleet utilization compared to the baselines.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper attempts to address the efficiency and cost issues in last-mile food delivery. Specifically, existing food delivery methods typically optimize each delivery route individually, leading to low resource utilization and high costs. The authors observed that there is significant room for cost reduction in the current delivery model. To this end, they modeled the food delivery problem as a multi-objective optimization problem, aiming to optimize both consumer satisfaction and delivery costs simultaneously. ### Solution To solve the aforementioned issues, the authors proposed a reinforcement learning-based path-sharing algorithm called DeliverAI. Inspired by the success of the sharing economy in the taxi industry, this algorithm reduces the total driving distance and controls delivery completion time by allowing multiple delivery orders in the same direction to share vehicles. Unlike traditional path optimization algorithms, DeliverAI can make real-time, efficient decisions and adapt to dynamically changing traffic conditions. ### Main Contributions 1. **First proposed reinforcement learning-based path-sharing food delivery network**: DeliverAI utilizes Q-value interactions between reinforcement learning agents to achieve intelligent, real-time, and dynamic decision-making. 2. **Multi-objective optimization model**: The problem is modeled as a Markov Decision Process (MDP), with key performance indicators defined to evaluate DeliverAI's performance. 3. **Large-scale experimental validation**: Extensive experiments were conducted using a real dataset from the city of Chicago, showing that DeliverAI can reduce driving distance by 13%, reduce fleet size by 12%, and improve fleet management and utilization by 50%. ### Method Overview 1. **Path-sharing mechanism**: DeliverAI reduces the total driving distance by allowing multiple delivery orders in the same direction to share vehicles for part of the journey. This is similar to the concept of ride-sharing in taxis, but food deliveries can switch between different vehicles. 2. **Multi-agent system**: Each hotspot location has an agent responsible for navigation and path planning. Agents communicate through Q-values to achieve path-sharing. 3. **Reinforcement learning training**: Agents are trained using the Q-learning algorithm to learn the optimal path selection strategy. During training, agents continuously optimize their action strategies by exploring the environment and receiving rewards. ### Experimental Results Experimental results show that DeliverAI can significantly improve delivery efficiency and reduce costs in practical applications. Specifically, compared to baseline methods, DeliverAI reduced driving distance by 13%, reduced fleet size by 12%, and improved fleet management and utilization by 50%. ### Conclusion By introducing path-sharing mechanisms and reinforcement learning techniques, DeliverAI provides an innovative solution for last-mile food delivery, significantly improving delivery efficiency and reducing operational costs. This approach is not only applicable to food delivery but can also be extended to other logistics and delivery scenarios.

DeliverAI: Reinforcement Learning Based Distributed Path-Sharing Network for Food Deliveries

Optimization of route distance using k-NN algorithm for on-demand food delivery

Modeling and Managing an On-Demand Meal Delivery System with Mixed Autonomy.

Shared lightweight autonomous vehicles for urban food deliveries: A simulation study

A Distributed Model-Free Ride-Sharing Approach for Joint Matching, Pricing, and Dispatching using Deep Reinforcement Learning

Analysis of practices in applying artificial intelligence and machine learning to improve urban route planning results

A Coalition Game for On-demand Multi-modal 3D Automated Delivery System

A Distributed Model-Free Algorithm for Multi-Hop Ride-Sharing Using Deep Reinforcement Learning

Optimizing Same-Day Delivery with Vehicles and Drones: A Hierarchical Deep Reinforcement Learning Approach

Towards Autonomous and Safe Last-mile Deliveries with AI-augmented Self-driving Delivery Robots

Data-driven optimization for last-mile delivery

Rendezvous Delivery: Utilizing Autonomous Electric Vehicles to Improve the Efficiency of Last Mile Parcel Delivery in Urban Areas

A Deep Reinforcement Learning Approach for the Meal Delivery Problem

Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching

FoodMatch: Batching and Matching for Food Delivery in Dynamic Road Networks

Simulation-based algorithm for determining best package delivery alternatives under three criteria: Time, cost and sustainability

Multi-Agent Deep Reinforcement Learning for Efficient Passenger Delivery in Urban Air Mobility

A Deep Reinforcement Learning Approach to Ride-Sharing Vehicle Dispatching in Autonomous Mobility-on-Demand Systems.

Multi-Agent Learning of Efficient Fulfilment and Routing Strategies in E-Commerce

Distributed Adaptive Reinforcement Learning: A Method for Optimal Routing

Optimizing Crowdsourced Delivery Routes Through Concurrent Selection of Pickup Stores and Drivers.