July, 2013:

Last month the Asian Development Bank released the Social Protection Index, which ranks countries based on their welfare programs. The index is mostly based on two parameters- the depth & the breadth of the coverage. While the depth is measured based on the benefit package offered to the beneficiaries, the breadth is based on the number of beneficiaries it covers. It does so using a simple formula of dividing the total program expenditure on welfare schemes by the total number of intended beneficiaries in all the schemes in order to come up with a ratio.

Even though the index is a first step to develop a methodology towards making cross- country comparison on their welfare schemes based on certain standardized measures, the question still remains of the context. Can one country follow another country’s policy with a completely different context? And can they replicate the success of one county in an entirely different setting? This could be similar to an apple and an oranges comparison whereby one country may have different welfare scheme than the other or the time period could be different. So for example, if South Korea started in 1988 its policy to provide health care coverage to its entire population and achieved a certain depth & breadth in a specific time frame, would it be a fair to compare its ratio to another country’s welfare program ratio based on shorter time frame. Well, the ADB has tried to reduce this variation by coming up with index based on region as well as by income. Hence, countries with similar economic growth can compare how well their social protection index fares in comparison to another.

Thinking on similar lines in terms of policy innovation, there is always a question as to how is it some country has been able to deliver a policy successfully and scaled up while some others may not have been? Unlike technological or manufacturing innovation which can be measured in terms of a new product launch or path-breaking model, policy innovation can be in two instances:

1)  When a state truly launches a new program which has not been initiated before.  For example, the first cash transfer program in Latin American or a new way of financing health program or delivering a vaccine.

2)When a country is able to achieve scale that others with similar economic and political structures are finding it hard to achieve.

Considering that there are several research grants in this area and innovation being the buzzword, maybe there is lack of evidence due to which there is such a growing interest in the scholarly and development practitioner community. So if one was to come up with a policy innovation index, then, what would be the parameters to make a standardized comparison of policy or programs:

1)Time Period in which policy was initiated and launched, bringing consensus from all stakeholders

2) The scale at which it was delivered

3)The process which enabled it

One attempt to compile policy innovation in health care is done by the Center for Health Market & Innovation. They have built a huge portfolio of programs in several parts of the world and categorized their innovation in 5 areas: 1)Health financing 2)Organizing delivery 3)Regulating performance 4)Changing behavior 5)Enhancing process

But in each of the category all the programs are different in terms of their approaches. So then how do you make a comparison of which program was more successful, which were the processes followed that made it more successful while another program adopting a similar strategy would have failed.

In other words, how do we build evidence to find out what works and what doesn’t?  As Don Hicks, Professor of Research Design points out, “You really build evidence by taking the same dependent variable and the same main independent variable in different settings and then trying to assess what is leading to variation?”

This would make sense if we would like to know about a successful HIV Aids program or a Tuberculosis prevention and reduction program.  By successful I mean a program or a country that has demonstrated substantial reduction in disease prevalence or in improving overall outcomes.  If we were to take the same dependent variable on outcome measures and have the same intervention as the main independent variable across several different settings, while controlling for certain processes,  to see whether a program could be scaled or not, would be one possible way to understand which innovation in HIV aids really works. And if the same main independent variable fails to produce desired outcome in a different setting, then this would give a clue as to the institutional factors and problems in delivery of the program that made it successful or failure.

While knowing all the good and diverse recipes is a great thing, sometimes it is also useful to know a standardized recipe that works too.  If  “innovation lies in delivery”, and by delivery Dr. Hicks means scaling up, then certainly one needs a more standardized measure to compare similar programs, similar interventions, similar time period and different settings to truly know the missing gaps. Acknowledging that a randomized experiment may not be possible, this kind of comparison could possible answer to not just which innovation works but also the process associated with it.  Just as a standardized Z score makes it possible to compare variables with different units by converting it into a unitless statistic, metaphorically speaking, a standardized policy innovation index could well reduce the variation and serve a similar purpose.