American Journal of Health Promotion | October 2022
Mitesh S. Patel, Katherine L. Milkman, Linnea Gandhi, Heather N. Graci, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Jake Rothschild, Modupe Akinola, John Beshears, Jonathan E. Bogard, Alison Buttenheim, Christopher Chabris, Gretchen B. Chapman, James J. Choi, Hengchen Dai, Craig R. Fox, Amir Goren, Matthew D. Hilchey, Jillian Hmurovic, Leslie K. John, Dean Karlan, Melanie Kim, David Laibson, Cait Lamberton, Brigitte C. Madrian, Michelle N. Meyer, Maria Modanu, Jimin Nam, Todd Rogers, Renante Rondina, Silvia Saccardo, Maheen Shermohammed, Dilip Soman, Jehan Sparks, Caleb Warren, Megan Weber, Ron Berman, Chalanda N. Evans, Seung Hyeong Lee, Christopher K. Snider, Eli Tsukayama, Christophe Van den Bulte, Kevin G. Volpp, Angela L. Duckworth
Purpose: To evaluate if nudges delivered by text message prior to an upcoming primary care visit can increase influenza vaccination rates.
Design: Randomized, controlled trial.
Setting: Two health systems in the Northeastern US between September 2020 and March 2021.
Subjects: 74,811 adults.
Interventions: Patients in the 19 intervention arms received 1-2 text messages in the 3 days preceding their appointment that varied in their format, interactivity, and content.
Measures: Influenza vaccination.
Results: Participants had a mean (SD) age of 50.7 (16.2) years; 55.8% (41,771) were female, 70.6% (52,826) were White, and 19.0% (14,222) were Black. Among the interventions, 5 of 19 (26.3%) had a significantly greater vaccination rate than control. On average, the 19 interventions increased vaccination relative to control by 1.8 percentage points or 6.1% (P = .005). The top performing text message described the vaccine to the patient as “reserved for you” and led to a 3.1 percentage point increase (95% CI, 1.3 to 4.9; P < .001) in vaccination relative to control. Three of the top five performing messages described the vaccine as “reserved for you.” None of the interventions performed worse than control.
Conclusions: Text messages encouraging vaccination and delivered prior to an upcoming appointment significantly increased influenza vaccination rates and could be a scalable approach to increase vaccination more broadly.
Nature Human Behaviour | September 2022
Katherine L. Milkman, Linnea Gandhi, Sean F. Ellis, Heather N. Graci, Dena M. Gromet, Rayyan S. Mobarak, Alison M. Buttenheim, Angela L. Duckworth, Devin Pope, Ala Stanford, Richard Thaler, and Kevin G. Volpp
Lotteries have been shown to motivate behaviour change in many settings, but their value as a policy tool is relatively untested. We implemented a pre-registered, citywide experiment to test the effects of three high-pay-off, geographically targeted lotteries designed to motivate adult Philadelphians to get their COVID-19 vaccine. In each drawing, the residents of a randomly selected ‘treatment’ zip code received half the lottery prizes, boosting their chances of winning to 50×–100× those of other Philadelphians. The first treated zip code, which drew considerable media attention, may have experienced a small bump in vaccinations compared with the control zip codes: average weekly vaccinations rose by an estimated 61 per 100,000 people per week (+11%). After pooling the results from all three zip codes treated during our six-week experiment, however, we do not detect evidence of any overall benefits. Furthermore, our 95% confidence interval provides a 9% upper bound on the net benefits of treatment in our study.
Proceedings of the National Academy of Sciences | February 2022
Katherine L. Milkman, Linnea Gandhi, Mitesh S. Patel, Heather N. Graci, Dena M. Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Jake Rothschild, Jonathan E. Bogard, Ilana Brody, Christopher F. Chabris, Edward Chang, Gretchen B . Chapman, Jennifer E. Dannals, Noah J. Goldstein, Amir Goren, Hal Hershfield, Alex Hirsch, Jillian Hmurovic, Samantha Horn, Dean S. Karlan, Ariella S. Kristal, Cait Lamberton, Michelle N. Meyer, Allison H. Oakes, Maurice E. Schweitzer, Maheen Shermohammed, Joachim Talloen, Caleb Warren, Ashley Whillans, Kuldeep N. Yadav, Julian J. Zlatev, Ron Berman, Chalanda N. Evans, Rahul Ladhania, Jens Ludwig, Christophe Van den Bulte, Kevin G. Volpp, and Angela L. Duckworth
Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was “waiting for you.” Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.
Nature | December 2021
Katherine L. Milkman, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Pepi Pandilowski, Yeji Park, Aneesh Rai, Max Bazerman, John Beshears, Lauri Bonacorsi, Colin Camerer, Edward Chang, Gretchen Chapman, Robert Cialdini, Hengchen Dai, Lauren Eskreis-Winkler, Ayelet Fishbach, James J. Gross, Samantha Horn, Alexa Hubbard, Steven J. Jones, Dean Karlan, Tim Kautz, Erika Kirgios, Joowon Klusowski, Ariella Kristal, Rahul Ladhania, George Loewenstein, Jens Ludwig, Barbara Mellers, Sendhil Mullainathan, Silvia Saccardo, Jann Spiess, Gaurav Suri, Joachim H. Talloen, Jamie Taxer, Yaacov Trope, Lyle Ungar, Kevin G. Volpp, Ashley Whillans, Jonathan Zinman, and Angela L. Duckworth
Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens’ decisions and outcomes. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy—a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.
Proceedings of the National Academy of Sciences | April 2021
Katherine L. Milkman, Mitesh S. Patel, Linnea Gandhi, Heather N. Graci, Dena M. Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Modupe Akinola, John Beshears, Jonathan E. Bogard, Alison Buttenheim, Christopher F. Chabris, Gretchen B. Chapman, James J. Choi, Hengchen Dai, Craig R. Fox, Amir Goren, Matthew D. Hilchey, Jillian Hmurovic, Leslie K. John, Dean Karlan, Melanie Kim, David Laibson, Cait Lamberton, Brigitte C. Madrian, Michelle N. Meyer, Maria Modanu, Jimin Nam, Todd Rogers, Renante Rondina, Silvia Saccardo, Maheen Shermohammed, Dilip Soman, Jehan Sparks, Caleb Warren, Megan Weber, Ron Berman, Chalanda N. Evans, Chrostopher K. Snider, Eli Tsukayama, Christophe Van den Bulte, Kevin G. Volpp, and Angela L. Duckworth
Many Americans fail to get life-saving vaccines each year, and the availability of a vaccine for COVID-19 makes the challenge of encouraging vaccination more urgent than ever. We present a large field experiment (N = 47,306) testing 19 nudges delivered to patients via text message and designed to boost adoption of the influenza vaccine. Our findings suggest that text messages sent prior to a primary care visit can boost vaccination rates by an average of 5%. Overall, interventions performed better when they were 1) framed as reminders to get flu shots that were already reserved for the patient and 2) congruent with the sort of communications patients expected to receive from their healthcare provider (i.e., not surprising, casual, or interactive). The best-performing intervention in our study reminded patients twice to get their flu shot at their upcoming doctor’s appointment and indicated it was reserved for them. This successful script could be used as a template for campaigns to encourage the adoption of life-saving vaccines, including against COVID-19.
Organizational Behavior and Human Decision Processes | November 2020
Erika L. Kirgios, Graelin H. Mandel, Yeji Park, Katherine L. Milkman, Dena M. Gromet, Joseph S. Kay, Angela L. Duckworth
Temptation bundling—pairing a pleasurable indulgence with a behavior that provides delayed rewards—combats present bias by making behaviors with delayed benefits more instantly-gratifying. If people are sophisticated and capable of following self-set rules to overcome present bias, they could benefit from learning about temptation bundling. Participants in a four-week exercise-boosting program (N = 6792) received either an audiobook with encouragement to temptation bundle, only an audiobook, or neither an audiobook nor encouragement to temptation bundle. Giving participants audiobooks and encouraging temptation bundling boosted their likelihood of a weekly workout by 10–14% and average weekly workouts by 10–12% during and up to seventeen weeks post-intervention. Relative to giving audiobooks alone, encouraging temptation bundling had a modest positive effect on exercise on the extensive margin. The marginal benefit of encouraging temptation bundling may be small because free audiobooks leak information: Simply providing an audiobook to exercise program participants suggests they should temptation bundle.
Proceedings of the National Academy of Sciences | July 2020
Erika L. Kirgios, Eduard H. Chang, Emma E. Levine, Katherine L. Milkman, and Judd B. Kessler
Policy makers, employers, and insurers often provide financial incentives to encourage citizens, employees, and customers to take actions that are good for them or for society (e.g., energy conservation, healthy living, safe driving). Although financial incentives are often effective at inducing good behavior, they’ve been shown to have self-image costs: Those who receive incentives view their actions less positively due to the perceived incompatibility between financial incentives and intrinsic motives. We test an intervention that allows organizations and individuals to resolve this tension: We use financial rewards to kick-start good behavior and then offer individuals the opportunity to give up some or all of their earned financial rewards in order to boost their self-image. Two preregistered studies—an incentivized online experiment (n = 763) on prosocial behavior and a large field experiment (n = 17,968) on exercise—provide evidence that emphasizing the intrinsic rewards of a past action leads individuals to forgo or donate earned financial rewards. Our intervention allows individuals to retroactively signal that they acted for the right reason, which we call “motivation laundering.” We discuss the implications of motivation laundering for the design of incentive systems and behavioral change.
Journal of the Association for Consumer Research | July 2020
Katie S. Mehr, Amanda E. Geiser, Katherine L. Milkman, and Angela L. Duckworth
Consumers often struggle to achieve self-set, life-improvement goals. We introduce a novel, psychologically wise nudge—the copy-paste prompt—that encourages consumers to seek out and mimic a goal-achievement strategy used by an acquaintance. In a large (N=1,028), preregistered, longitudinal study, participants randomly assigned to receive a copy-paste prompt spend more time exercising the following week than participants assigned to either a quasi-yoked or simple control condition. The benefits of copy-paste prompts are mediated by the usefulness of the adopted exercise strategy, commitment to using it, effort put into finding it, and the frequency of social interaction with people who exercise regularly. These findings suggest that further research on the potential of this virtually costless nudge is warranted.
Proceedings of the National Academy of Sciences | July 2019
Lauren Eskreis-Winkler, Katherine L. Milkman, Dena M. Gromet, and Angela L. Duckworth
Common sense suggests that people struggling to achieve their goals benefit from receiving motivational advice. What if the reverse is true? In a preregistered field experiment, we tested whether giving motivational advice raises academic achievement for the advisor. We randomly assigned n = 1,982 high school students to a treatment condition, in which they gave motivational advice (e.g., how to stop procrastinating) to younger students, or to a control condition. Advice givers earned higher report card grades in both math and a self-selected target class over an academic quarter. This psychologically wise advice-giving nudge, which has relevance for policy and practice, suggests a valuable approach to improving achievement: one that puts people in a position to give.
Psychological Science in the Public Interest | February 2019
Angela Duckworth, Katherine Milkman, and David Laibson
Almost everyone struggles to act in their individual and collective best interests, particularly when doing so requires forgoing a more immediately enjoyable alternative. Other than exhorting decision makers to “do the right thing,” what can policymakers do to reduce overeating, undersaving, procrastination, and other self-defeating behaviors that feel good now but generate larger delayed costs? In this review, we synthesize contemporary research on approaches to reducing failures of self-control. We distinguish between self-deployed and other-deployed strategies and, in addition, between situational and cognitive intervention targets. Collectively, the evidence from both psychological science and economics recommends psychologically informed policies for reducing failures of self-control.
Psychological Science | June 2017
Shlomo Benartzi, John Beshears, Katherine L. Milkman, Cass R. Sunstein, Richard H. Thaler, Maya Shankar, Will Tucker-Ray, William J. Congdon, and Steven Galing
Governments are increasingly adopting behavioral science techniques for changing individual behavior in pursuit of policy objectives. The types of “nudge” interventions that governments are now adopting alter people’s decisions without coercion or significant changes to economic incentives. We calculated ratios of impact to cost for nudge interventions and for traditional policy tools, such as tax incentives and other financial inducements, and we found that nudge interventions often compare favorably with traditional interventions. We conclude that nudging is a valuable approach that should be used more often in conjunction with traditional policies, but more calculations are needed to determine the relative effectiveness of nudging.