The Power and Pitfalls of Megastudies for Studying Health Interventions

Add to Calendar

Health policymakers often turning to social scientists for insights about how to improve citizens’ health decisions and outcomes. However, these insights can only inform policy insofar as they are comparable—and unfortunately, different intervention ideas are typically tested across different samples on different outcomes over different time intervals. In my talk, I will introduce the “megastudy,” a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. I will then share results from four health-related megastudies that my team has conducted over the last several years to illustrate the power and pitfalls of this methodology. I will describe a megastudy encouraging more gym visits (N=61,293 24 Hour Fitness members), a megastudy encouraging flu vaccination at doctor’s appointments (N=47,306 patients of Penn Medicine and Geisinger Health), a megastudy encouraging in-pharmacy flu vaccinations (N=689,693 Walmart pharmacy patients), and a megastudy encouraging in-pharmacy COVID-19 bivalent booster vaccinations (N=3.5 million CVS Pharmacy patients). I will discuss the accuracy of experts’ and laypeople’s forecasts of the performance of different interventions tested in these studies, share best practices for running megastudies, and describe some key limits of this approach to applied behavioral science research.

Speaker:

Katy Milkman is the James G. Dinan Professor at The Wharton School of the University of Pennsylvania, host of Charles Schwab’s popular behavioral economics podcast Choiceology, and the former president of the international Society for Judgment and Decision Making. She is also the co-founder and co-director of the Behavior Change for Good Initiative, a research center with the mission of advancing the science of lasting behavior change. Over the course of her career, Katy has worked with or advised dozens of organizations on how to spur positive change, including Google, the White House, Walmart and the U.S. Department of Defense. She has published over 60 papers in leading academic journals such as Nature, The Journal of Finance, and The Proceedings of the National Academy of Sciences and is the author of the international bestselling book How to Change: The Science of Getting From Where You Are to Where You Want to Be, which was named one of the eight best books for healthy living in 2021 by the New York Times. Katy was also recently named a Top 10 innovator shaping the future of health by Fortune Magazine and won Penn’s highest teaching award, the Provost’s Lindback Award for Excellence in Teaching, in 2022. She writes frequently about behavioral science for major media outlets such as The Washington Post, The New York Times, CNN and The Economist.