Abt Associates: Bold thinkers driving real-world impact
Background: Recent reviews suggest that many plausible programs are found to have at best small impacts not commensurate with their cost, and often have no detectable positive impacts at all. Even programs with initial rigorous impact evaluation (RIE) that show them to be effective often fail a second test with an expanded population or at multiple sites.
Objective: This article argues that more rapid movement to RIE is a partial cause of the low success rate of RIE and proposes a constructive response: process evaluations that compare program intermediate outcomes—in the treatment group, during the operation of the program—against a more falsifiable extension of the conventional logic model.
Conclusion: Our examples suggest that such process evaluations would allow funders to deem many programs unlikely to show impacts and therefore not ready for random assignment evaluation—without the high cost and long time lines of an RIE. The article then develops the broader implications of such a process analysis step for broader evaluation strategy.
Abt Associates is a mission-driven, global leader in research, evaluation and program implementation in the fields of health, social and environmental policy, and international development. Known for its rigorous approach to solving complex challenges, Abt Associates is regularly ranked as one of the top 20 global research firms and one of the top 40 international development innovators. The company has multiple offices in the U.S. and program offices in more than 40 countries.