Creating Enduring Behavior Change

Megastudies improve the impact of applied behavioural science

A paper introducing BCFG’s pioneering approach to conducting behavioral science research at scale using “megastudies” was recently published in Nature. A megastudy is a massive field experiment in which many interventions are evaluated simultaneously on the same objective outcome in the same population. The megastudy approach allows dozens of ideas proposed by a diverse array of scientists to be tested at once, making direct, “apples-to-apples” comparisons among different strategies for spurring behavior change possible. This approach can accelerate the speed at which knowledge is generated and more effectively and efficiently inform policy decisions than standard social science research methods.

We partnered with 24 Hour Fitness (a large U.S. gym chain) on the megastudy featured in this paper. Our study involved a 4-week digital program aimed at encouraging exercise among 61,293 gym members. Independent teams of 30 scientists from 15 U.S. universities designed 54 different versions of the program, which comprised the different interventions we tested to boost gymgoing. Forty-five percent of the different, 4-week digital programs tested significantly increased weekly gym visits during the 4-week intervention period. The top-performing intervention increased weekly gym visits by 27% by encouraging gym goers not to miss more than two planned gym visits in a row (if they missed one, they could then earn a nine cent bonus for returning to the gym and avoiding a second missed visit). Importantly, impartial judges—public health experts, behavioral science practitioners, and lay people—were unable to predict which program designs would be most effective at spurring physical activity. These findings reinforce the importance of simultaneously testing many ideas to identify which strategies work best for changing behavior.

A citywide experiment testing the impact of geographically targeted, high-pay-off vaccine lotteries 

The answer is No! Our new paper in Nature Human Behaviour features the results of the Philly Vax Sweepstakes, a science-based lottery to encourage COVID-19 vaccination that we designed and executed last summer in partnership with CHIBE, the City of Philadelphia and the Black Doctors COVID-19 Consortium. The Philly Vax Sweepstakes was designed to test the effects of three, high-payoff, geographically-targeted vaccine lotteries. Specifically, in each of the three drawings, residents of a randomly selected zip code received half of the lottery prizes available in the entire city of Philadelphia, which increased their chances of winning the Sweepstakes by 50-100x compared to other residents. We found that massively increasing these residents’ chances of winning a lottery prize of up to $50,000 for being vaccinated did not have any estimated benefit, and we can confidently rule out that it produced even one extra vaccination per $4,690 spent. Sometimes we learn a lot from null results, and this is one such case! The jury is still out, however, on whether regret-style vaccine lotteries in cities can add value. Results from our attempt to assess the overall impact of our lottery on Philadelphians were inconclusive and suggest more testing would be valuable.