It's a powerful idea and one of those concepts that lead high powered people to ask "Why isn't this happening all over the place already." I'm bullish long term on the idea and I'll probably blog about it more in the future. It funds innovation and the sorts of preventative programs that are often the first to have their budgets cut. It takes the short term funding burden and risk of failure off government since they only pay for demonstrated savings. And, theoretically, it will make huge pools of capital available to service organization that can prove their efficacy, but the devil for now is in the details. For a successful program, you must have:
- A single payer (usually government) that is already incurring a cost that would be reduced if the program is successful. So, keeping children out of expensive special ed programs is good, but improving the air quality that everyone enjoys but no single group is responsible for, that’s difficult to fit in an SIB
- A program that produces results within a reasonable amount of time, say five years. So, keeping people out of prison is good, helping young children develop the skills they need to be happy, successful adults 20 years from now, not so much.
- A clearly quantifiable measure for success. This needs to be clearly agreed before the program starts because but forms the basis for whether and how much the organization (and their investors) get paid.
This brings us to Campbell’s Law, which states:
"The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
Simply put, the higher the stakes are around a single measure of progress, the more likely that groups will try to game the system (or commit outright fraud) an the less useful that measure will be for determining whether or not you are actually accomplishing your goals. The most recent prominent example is outright fraud in Atlanta around high stakes testing, but more broadly you could argue that, as testing becomes the single measure of educational achievement, it crowds out all the other educational outcomes that we were hoping to use testing as a proxy for. Pay for test results, get a generation of optimized test takers (or worse yet, a bunch of cheaters).
Campbell’s Law doesn’t doom SIBs or Pay for Success to failure by any means, but it is something to keep in mind for those designing programs and those considering investing. As we are building these new systems, we need an evolving apparatus that helps us to reward what works without having the measurement tail wag the programmatic dog.