Abstract:Bid optimization for online advertising from single advertiser's perspective has been thoroughly investigated in both academic research and industrial practice. However, existing work typically assume competitors do not change their bids, i.e., the wining price is fixed, leading to poor performance of the derived solution. Although a few studies use multi-agent reinforcement learning to set up a cooperative game, they still suffer the following drawbacks: (1) They fail to avoid collusion solutions where all the advertisers involved in an auction collude to bid an extremely low price on purpose. (2) Previous works cannot well handle the underlying complex bidding environment, leading to poor model convergence. This problem could be amplified when handling multiple objectives of advertisers which are practical demands but not considered by previous work. In this paper, we propose a novel multi-objective cooperative bid optimization formulation called Multi-Agent Cooperative bidding Games (MACG). MACG sets up a carefully designed multi-objective optimization framework where different objectives of advertisers are incorporated. A global objective to maximize the overall profit of all advertisements is added in order to encourage better cooperation and also to protect self-bidding advertisers. To avoid collusion, we also introduce an extra platform revenue constraint. We analyze the optimal functional form of the bidding formula theoretically and design a policy network accordingly to generate auction-level bids. Then we design an efficient multi-agent evolutionary strategy for model optimization. Offline experiments and online A/B tests conducted on the Taobao platform indicate both single advertiser's objective and global profit have been significantly improved compared to state-of-art methods.

Risk-Based Evolutionary Bidding Strategy for Online Multiple Auctions

A Contingency Model of Bidding Strategies in Online Auctions in China.

Applying Opponent Modeling for Automatic Bidding in Online Repeated Auctions

Prior-free Auction Mechanism for Online Supplier with Risk Taking.

Cost-Effective Active Learning for Bid Exploration in Online Advertising.

Reinforcement Learning Model For Optimizing Generation Bidding Strategy With Risk Management

Competitive Risk Analysis of Online Limited Supply Auction

A self-adaptive bidding strategy in continuous double auction market

Strategy Research of Used Cars in Online Sequential Auction Based on Fuzzy Theory.

Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning

An Intelligent Bidding Strategy Based on Model-Free Reinforcement Learning for Real-Time Bidding in Display Advertising

An Iterative Multi-Attribute Auction Mechanism Based on Co-Evolutionary Computation

Risk-Aware Bid Optimization for Online Display Advertisement

Developing a cooperative bidding framework for sponsored search markets – An evolutionary perspective

Multi-Agent Cooperative Bidding Games for Multi-Objective Optimization in e-Commercial Sponsored Search

Risk-tolerance Strategy of Online Reverse Auctions for Infinite Divisible Goods

Coevolutionary Computation Based Iterative Multi-Attribute Auctions

A CHANCE CONSTRAINED PROGRAMMING BASED APPROACH FOR BUILDING OPTIMAL BIDDING STRATEGIES FOR GENERATION COMPANIES WITH RISK MANAGEMENT

An agent negotiation system based on adaptive genetic algorithm

Advertiser-First: A Receding Horizon Bid Optimization Strategy for Online Advertising

Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions