Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments

Abstract

Online advertisers regularly deliver several versions of display ads in a single campaign across many websites in order to acquire customers, but they are uncertain about which ads are most effective. As the campaign progresses, they adapt to intermediate results and allocate more impressions to the better performing ads on each website. But how should they decide what percentage of impressions to allocate to each ad?

This paper answers that question, resolving the classic "explore/exploit" tradeoff using multi-armed bandit (MAB) methods. However, this marketing problem contains challenges, such as hierarchical structure (ads within a website), attributes of actions (creative elements of an ad), and batched decisions (millions of impressions at a time), that are not fully accommodated by existing MAB methods. We address this marketing problem by utilizing a hierarchical generalized linear model with unobserved heterogeneity combined with an algorithm known as Thompson Sampling. Our approach captures how the impact of observable ad attributes on ad effectiveness differs by website in unobserved ways, and our policy generates allocations of impressions that can be used in practice. We implemented this policy in a live field experiment delivering over 700 million ad impressions in an online display campaign with a large retail bank. Over the course of two months, our policy achieved an 8% improvement in the customer acquisition rate, relative to a control policy, without any additional costs to the bank. Beyond the actual experiment, we performed counter-factual simulations to evaluate a range of alternative model specifications and allocation rules in MAB policies.