At the Society for Benefit-Cost Analysis annual conference earlier this month, the lunch keynote on the first day focused on the difficulty of doing good policy analysis in the face of conflicting data. One topic that Sherry Glied, Dean of NYU’s Wagner School of Public Service spoke on was the role of the policy analyst as opposed to the role of the policymaker. This is a topic we talk about a lot at Scioto Analysis and I thought it would be valuable for me to share a little bit about how we think about this important relationship.
Policymaking is entirely about understanding tradeoffs. As an analyst, my goal is to use the best available data to make those tradeoffs clear to policymakers in a way that is honest and accurate.
It then becomes the role of the policymakers to decide which tradeoffs they want to make. In a well functioning democracy, the policymakers will reflect the view of the people they represent as best they can and allocate whatever scarce resources they have.
This is where policy analysis and policymaking can sometimes conflict. A policymaker might choose to make a decision that does not maximize efficiency, equity, or effectiveness for some reason. Maybe a policy is very efficient only after we count long-term costs, or perhaps it improves equity concerns but would otherwise be inefficient.
It can be hard as a policy analyst to accept the fact that always maximizing efficiency is an impossible goal. On one hand, it is impossible because in order to be certain that we were maximizing efficiency we would need to research every possible alternative which would lead to a never ending cycle of research.
More importantly though, efficiency isn’t always the goal of policymakers. Policymakers have to care about things like elections, political feasibility, and legislative rules.
Often, this separation gets painted as a negative consequence of our political system. “If only the policymakers just did what the research suggests, then we’d have a much better society.” But I don’t necessarily think this is the case.
For one thing, this separation allows policy analysis to (in theory) operate outside of the political discourse. Of course, there are critical assumptions the analyst has to make that can shape the results quite significantly, but good policy analysis is clear about these assumptions and often tests what happens if they do not hold.
Once assumptions have been established, applying the best available data and methods should result in analysts coming to the same conclusions. Keeping an arms length away from debates about politics allows policy analysis to maintain its status as a scientific discipline that deals with the truth.
The other reason this separation is important is because policy analysts are often in the business of predicting the future, which is an inherently difficult proposition. Even the most rigorous analyses can sometimes guess incorrectly about what the future will hold. That doesn’t mean the analysis was bad, just that something unexpected happened.
Statistics give us the tools to measure uncertainty and incorporate it into our estimates, but at the end of the day most policies only get one chance. It might be a more efficient world if the entire political system was centered around maximizing the expected value of our policies, but it might not.
Separating policy analysts and policymakers protects the integrity of the policy analysis process. It keeps the focus of the analysis on the process instead of the results. For the most part, as long as the analysis was done correctly, the analyst will continue to be trusted. Conversely, policymaking is a much more outcome focused business. If an expensive program doesn’t work as expected, usually the elected official is going to be on the hook.
All of this is certainly a glass-half-full take of the dynamics between policy analysts and policymakers. The entire reason Scioto Analysis exists is because we don’t think there is enough good policy analysis happening at the state and local levels of government.
Good policy analysis should be a much larger part of the way our society functions, we’d all be better off for it. However, it can’t be the entirety of our decision making process. Policymaking and politics still play an important role, and they likely always will. And in a democracy, they probably should.