Mega-studies improve the impact of applied behavioral science

• Policy-makers are increasingly turning to behavioral science for insights about how to improve citizens’ decisions and outcomes.

• Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals. The lack of comparability of such individual investigations limits their potential to inform policy.

• Recently, to address this limitation and accelerate the pace of discovery, researchers introduced the mega-study—a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration.

• In a mega-study targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programs (or interventions) encouraging exercise.

• Scientists show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered micro-rewards for returning to the gym after a missed workout.

• Only 8% of interventions induced behavior change that was significant and measurable after the four-week intervention.

• Conditioning on the 45% of interventions that increased exercise during the intervention, they detected carry-over effects that were proportionally similar to those measured in previous research.

• Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for mega-studies to improve the evidentiary value of behavioral science.

Published: 08 December 2021

source:

https://www.nature.com/articles/s41586-021-04128-4

https://doi.org/10.1038/s41586-021-04128-4