UCL School of Management is delighted to welcome Professor Tat Chan, Washington St Louis, to host a research seminar discussing ‘Using Algorithm Scores to Measure the Impacts of Targeted Promotional Messages’ 

Abstract:

Targeted promotional messages are a form of advertising directed toward customers based on the characteristics of the products that the firm promotes and customers’ traits including demographics and browsing and purchase history. We propose using matching scores and propensity score matching methods to study the impacts from targeted promotional messages as well as their spillover effects on other types of promotions. Our methods use the algorithm scores that a firm calculates for each customer when deciding who to target to match a customer who receives the message with another who does not from field data. The identification assumption is that, although the treated and untreated individuals can be systematically different, after matched by the algorithm scores, the difference in targeting outcomes within a matched pair is due to exogenous factors. We apply the methods to a customer-level dataset provided from Alibaba. Results show that the promotional messages of Alibaba have significantly increased consumer reads, clicks and purchases; however, they have also increased the likelihood of un-subscription. Contrast to intuition, the messages have also reinforced the consumer responses to other types of promotions. We also find that the consumer responses are moderated in different ways by the number of past exposures to promotional messages. Finally, we run a randomized field experiment on Alibaba’s platform and find similar estimated consumer responses. If we ignore the matching, or match customers by observed demographics only, however, the estimated consumer responses are over-biased. Our proposed methods are useful for firms to repeatedly evaluate the effectiveness of each marketing campaign, without relying on costly field experiments.