Policy analysis has a broad range of tools that help us better understand the impact of policy proposals. The purpose of policy analysis is to help policymakers better understand the potential outcomes of a proposal based on socially-relevant criteria.
There are many different lenses through which an analyst can measure the impacts of a policy. Among these different lenses, three loom above all the others: effectiveness analysis, efficiency analysis, and equity analysis.
These three criteria help us understand something different about what we expect a policy to do. Understanding the pros and cons of each is vital to performing meaningful analysis of public policy.
Effectiveness
Measuring the effectiveness of a policy is entirely dependent on the criteria an analyst selects. The big question we are trying to answer when we ask how effective a policy is is “what are the outcomes of this policy? Does it accomplish its goals?”
Imagine we are analyzing a policy that is designed to reduce pollution. We could measure its effectiveness by estimating how much carbon emissions are reduced, or perhaps by how we expect climate trends to change. We also could measure how much NOx or PM2.5 pollutants decrease, how many instances of low birthweight could be prevented by reduction of the pollutants, or even how many instances of respiratory illness or death are prevented by these reductions.
This means measuring how effective a policy is at improving both primary and secondary criteria. Along with estimating how much a policy reduces pollution, we can also estimate health benefits or recreation benefits associated with reduction of pollution.
The gold standard of effectiveness is the “randomized controlled trial,” an experiment where participants are exposed to the policy randomly and their outcomes are compared to those of a control group. Not all policies are good candidates for randomized controlled trial, though: often quasi-experimental studies like difference-in-difference or regression discontinuity analysis can provide some evidence of effectiveness. Even lighter evidence can come from pre/post data or point-in-time data.
Efficiency
In contrast to effectiveness–which is only concerned with the outcomes of a policy–efficiency analysis depends on both a policy's inputs as well as its outputs.
The most comprehensive tool policy analysts have to estimate economic efficiency of a public policy is cost-benefit analysis. Cost-benefit analysis is easy to understand, theoretically straightforward to compute, and if done well paints a clear picture of a policy’s efficiency.
Although efficiency and effectiveness are similar to each other, they can sometimes be in tension with one another. One example of this is the example of scaling policy–taking policy from the pilot level to broader application. When we increase the scale of policies, we often find increases in effectiveness and decreases in efficiency.
Whether the most efficient or the most effective policy is preferred by a policymaker depends on the context of that policy. Maybe in the case of a pollution-reducing policy there is some emissions target that needs to be reached, even if it is not the most efficient. On the other hand, a policy may be efficient at a local level and not as desirable at a national level because of decreasing returns to scale. These are ultimately judgment calls that policymakers need to make, but that good analysis can make more clear.
Equity
The equity component of policy analysis is often the most difficult to understand, largely because it is the least well defined. Generally speaking, the goal of equity analysis is to understand how costs and benefits are distributed throughout society.
Currently, methods for equity analysis tend to be less robust than methods for effectiveness or efficiency analysis. Partially this is due to the fact that accurate social/demographic data is often harder to access, and partially due to the fact that equity analysis is often an overlooked step. Equity also exists on multiple dimensions: income, race, gender, age, disability status, national origin, urban/rural etc. While some fields of policy such as tax policy have good standard practice for analyzing equity, examples such of this are few and far between due to the plethora of dimensions equity can be analyzed on.
Still, it is important to understand how equitable the outcomes of any policy are. Good questions for analysts to ask are “what are the demographics of the people who bear the costs of this policy?” and “Does this policy alleviate inequality?”
—
As analysts, it is our responsibility to keep diverse criteria in mind as we explore the possible outcomes of a policy proposal. There are always trade-offs between effectiveness, efficiency, and equity that will determine how a policy impacts society.
However, in many cases analyzing a policy on all three dimensions may not be feasible due to time or resource constraints. Policy analysis is an inherently time-sensitive process – sometimes decisions have to be made quickly and policymaking moves faster than academia.
Under these constraints, it is important to figure out which category of analysis is most useful for policymakers. Sometimes, a policymaker will ask for a specific type of analysis, other times it is up to the analyst to decide which analysis will provide the most useful information.
In any case, it is important to remember that these three criteria tell us different things. Understanding how they are different and communicating it effectively to policymakers looking for information is one of the most important things a policy analyst can do.