Universities and the looking glass: Taking impact beyond policy-based evidence

 

”In cynical terms, the fable of speaking truth to power is just that: a fable. Evidence-based policy for the most part is a story that politicians tell that helps legitimize their decisions. It has proven to be a very good story: most people find it easy to buy into the narrative.”

Alustus! Down the rabbit hole we have gone as academics, seeking to influence policy and move society with the force of expertise.

But the realm of policymaking turned out to be another country: they do things differently over there. The more academics try to adapt, the further they are stretched from where they started. Can we glance back through the looking glass and ask whether this precise hole is the one to go down? In speaking truth to power, have we really understood what and where power is?

Policy interaction (i.e. engagement with policymakers and related actors to influence policy) has always been an important aspect to scholarship. Policymakers in most countries now actively seek academic advice through parliamentary committees or consultation in ministries and so on. For their part, university academics have typically freely provided advice to policymakers on knotty problems related to their expertise. Universities in Europe and North America, in particular, include such policy interaction as part of the metrics assessing faculty performance.

Driven by a vague desire to “do good,” specific hopes to move policy in directions that will solve outstanding social issues, and to bolster my metrics, I too have offered “expert” advice on social issues I consider important. Mostly, this has been related to what is popularly referred to as “Islamic extremism” or “terrorism,” in an effort to generate more, nuanced understanding of the violence. I have had such occasion in Finland and in Pakistan.

My experiences have been troubling. So much so that I have spent some effort reflecting on them, and have discussed them with social scientists in Finland and elsewhere. Many were kind enough to share their own experiences regarding policy interaction with me as well.

It was a relief to realize I was not alone. It was painful to realize how far we still need to go for social science expertise to impact policy.

Policy and academia: Out of sync?

What emerges from these experiences is that policy interaction is an uphill task and most of what is said is simply thrown out or trumped by ready-fix solutions. That is, crudely put, academic expertise in its proper context is generally not valued. What are valued are de-contextualized solutions that are nominally based on expertise.

Many social scientists point out that policy interaction is characteristically one step forward and two steps back. Yet, few appreciate that over the decades since policymakers started routinely asking academics for advice, that has meant a lot of steps back! Such anecdotal evidence is supported by a quick scan of public accounts by social scientists, which underlines widespread discontent. For instance, in a Guardian article in September 2016, Sir Peter Gluckman (Chief Science Advisor in New Zealand) and James Wilsdon (Professor of Research Policy in Sheffield) pointed out:

… it is easy to feel frustrated by the visible failures of evidence to influence policy in so many areas. On thorny issues like climate change, obesity, biodiversity and migration, the response to overwhelming evidence is often sluggish and incremental.

They further noted that science advice has not managed to keep up with populist rhetoric:

And populist politicians, campaigners or social media warriors can tap into the anxieties caused by globalisation and rising inequalities, and channel these towards resentment, as we saw in the claims during the Brexit referendum that “people in this country have had enough of experts”.

Two points emerge from seemingly rampant discontent among social scientists. First, it is not that certain subjects are more “favored” by policymakers and others are ignored. In fact, physical scientists are found complaining even more than social scientists in most countries that their evidence and analyses are ignored. Rather, the division is between basic and applied science. Basic science in all cases appears to be de-valued by decision-makers, while applied science in all disciplines appears to be more highly valued and incorporated. Applied sciences can provide packaged solutions that the modern culture of decision-making looks for, even if it is for problems that do not exist or are not that pressing.

Second, policymakers, like everyone else, see only as far as they can see in any direction. We all have biases and limited views of some kind due to our psychological or sociological situations: that is a fact of the thinking process. But social scientists (at least, more sensitive ones) recognize these blinkers and do their best to move past them into ever widening views of society. For policymakers, tight blinkers are positively a requirement in order to “zoom in” on the problem at hand or “stay focused” on the progress of their careers. Policymakers are problem-solvers par excellence. For them, (a) problems by definition require solutions, (b) solutions exist for any problem, and (c) their own agenda is necessarily the best one to push that solution.

Naturally, if a solution doesn’t fit the agenda of the time, or the party position or official line, it cannot be the right one and is simply filtered out of the system. More fundamentally, if the “solution” is not a quick-fix yielding immediate results, it isn’t really seen as a solution at all and is likewise filtered out. For example, the “problem” of illiteracy in Pakistan led to consultants advising major budget allocations and quick financing of a legion of state schools by the military dictatorship across the country after 1998. Problem solved: buildings were put up (and dictatorial legitimacy established). By 2009, reports estimated that there were over 30 000 ghost schools throughout Pakistan that could not be staffed, equipped, or monitored. Petitions to focus on quality upgrading and governance to improve schooling over longer periods fell on deaf ears.

Taken together, these two trends impact university policy interaction. For instance, I have had experience in advising policymakers on the threat of Islamic radicalism. However much basic research I might bring to show that the issue is global, is complex, and is interconnected with a multitude of social concerns, this has never worked. Nor have pleas to consider this a long-term agenda in which everybody is implicated. By contrast, consultants who package simple solutions about “deterring foreign fighters” or technical indicators that trigger when “someone gets radicalized” invariably win the day. Most such solutions implicitly rely on demonizing and exteriorizing the issue, contrary to the overwhelming bulk of social and psychological research. None of the solutions has really worked because they are largely based on faulty or one-sided perceptions of the issue. The continuing incidents of religiously motivated terrorism prove that. But policy failures just lead to more solutions of the same kind.

In general, cultural, long-term and indirect solutions are not seen as solutions, and indicator-based, short-term, monitoring tools are always preferred. The issue is that most burning social problems are what we now call “wicked problems.” Not only are they hopelessly entangled, but also any attempts at impacting the system change that system and create new problems. I would say that most societal situations are quantum-entangled systems. Physicists have shown that the smallest bit of matter, the atom, can never be fully described in terms of its position and momentum. In the same way, any attempt at even measuring complex social problems changes them fundamentally, so we have to start again. The major social issues we are all concerned about are “wicked,” including terrorism, climate change, populism, human rights deficits, and so on.

Wickedness of social problems doesn’t go down well in policymaking. So, policymakers choose to see only that which fits into their frame of seeing. We could say that the “field of policymaking” has its own space. In this space the reality of wicked social problems doesn’t have very much to do with policy actions.

Or, we could say that instead of talking about evidence-based policy, we should talk about policy-based evidence: the frame within which (certain) evidence makes sense and counts.

Policy-based evidence

History is replete with examples of policy-based evidence. For instance, at least as far back as 5th century BC Greece, women were considered more prone than men to irrational fits, faintness, nervousness, inexplicable sexual desire or its loss, insomnia, shortness of breath, irritability, and a general tendency to trouble-making. This came to be labelled “hysteria,” or the disease of the “wandering womb.”

All kinds of treatments—from confinement to water pressure on the stomach to forced masturbation—were designed to treat the condition, and in many countries, there were official policies to impose them when (men) deemed necessary. Post-World War II research finally put paid to “hysteria” as a medical condition to be “treated.”

Yet, it is remarkable that women are still prescribed more psychotropic drugs than men everywhere in the world. A survey of insured patients in the United States between 2001 and 2010 noted that 25% of women are on prescribed mental health medication as opposed to 15% of men. Twice as many women are prescribed anti-anxiety medication as men. The list of conditions for prescription are almost the same as they were in 5th century BC Greece: anxiety, depression, irritability, and so on. In most countries, national health agencies ignored the policy basis of evidence. Only in January 2018 have ministers in the UK asked for a review of the situation and the “evidence” that has led to it.

Universities cannot dissociate themselves from the centuries of “rationality” that led to this and similar solutions. Universities produce the knowledge that is considered “standard” and train the policymakers who act on that knowledge. Today, modern research funding infrastructure is complicit. There’s nothing devious or surreptitious about this, nor are there any secret, global governors dictating events behind the scene: this is simply the way people believe things should go.

In cynical terms, the fable of speaking truth to power is just that: a fable. Evidence-based policy for the most part is a story that politicians tell that helps legitimize their decisions. It has proven to be a very good story: most people find it easy to buy into the narrative. No one really questions what scientific advice went in to which particular decision, and how conflicting scientific opinions were negotiated, etc. That, in turn, makes it easier to adjust funding instruments towards universities having a measurable “policy impact,” much like a consultancy wing waiting to be tapped by the state. In the end, universities themselves buy in to the story, setting “policy impact” performance standards and assessment criteria for faculty.

Speaking truth to power

I suggest we can turn the dilemma around. Instead of trying to “get into” a system geared against the university’s basic science, we can re-envision the system. That is, when we think of policy interaction, we can re-imagine what that means. If the university wants to speak truth to power, we must reconsider what we think of as “power.”

It is common to think of decision-making as a pyramid and to focus policy interaction on the top bit of it. To bring adjustments in this top of the pyramid, (1) academics rely on being counted as independent “experts.” The expectation is that (2) affecting a small change there will magnify the impact into a large change in society. Furthermore, (3) we are content to think that each national society is more less a homogenous and isolated pyramid.

This pyramid conception is rooted in a widespread imagery of power organized as a spatial hierarchy, flowing from the top (who have more power) to the bottom. This imagery is common in popular as well as scientific theorizing of the social world. While this imagery captures some parts of social reality, it is readily extended to all of society including parts where it simply doesn’t work.

However, just because it is common to see the world in this way doesn’t make it correct in every case. I contend that all three points of how we conceive policy impact need to be re-imagined. (1) Policy gurus and consultants have already cornered the market on expertise in any field, their business model relying on being commissioned by policymakers and then treating university research more or less like sub-contracts. Anyway, (2) the top of the pyramid in contemporary societies is far less effective than we would like to think. And, (3) far from being an isolated structure, each national society is part of a world society with deep and extensive networks of interdependence: affecting one part of one society may well end up affecting another part of another society altogether (we call this “synchronization” of policymaking in our research group.

So, it is neither accurate nor effective to perceive national governance as a pyramid with policymakers on the top. Instead, we should see it as a complex network in nature. Each society is like a system in which different agents spread throughout the system may have more or less influence. For instance, a forest may both be described as complex systems in itself. Yet, each forest is only loosely bounded, and is deeply impacted by what happens in related ecosystems around it (the lake, mountains, etc.). An ecosystem like a forest is both complex and dynamical.

I suggest that a complex network is a better metaphor for a national society than that of a bounded, pyramidal hierarchy. Just as everything in a natural system is interconnected so is everything in the social world; just as boundaries and borders are hard to define in natural ecosystems, so it is in world society; just as signal feedback, emergence of patterns, and chaotic fluctuations are characteristic in nature, so too should they be recognized in national governance.

Most of all, in any complex dynamical system, understanding all the components of a system does not mean that we have understood the system: the whole is more than the sum of the parts. In other words, just as natural systems are sensitive to initial conditions—varying unpredictably with even the tiniest input in the most remote component—so too are national governance systems. Links of causality are highly dubious, and our focus needs to be on understanding the conditions of possibility for the present situation to arise and sustain.

Affecting policy in complex systems

Instead of trying to “direct” social change by affecting policymakers—which hasn’t worked all that well for social scientists—we should be aiming at constantly generating new and ever-widening knowledge about the network we are in. This necessarily creates a feedback loop that impacts on the network as a whole.

The only problem with this approach (if it is a problem) is that causality is hard to pinpoint. In non-linear natural complex systems, too, it is more often than not very difficult to identify what precise tweak led to what precise effect. Likewise, in any meaningfully complex social system, direct causality is an imagined construct of our order-seeking minds and not necessarily a social fact. So, the only “problem” with this way of conceiving of social change is one we have created ourselves. The issue is with our requirement to impose ordered causality, and our insistence on activities whose outcomes can be directly measured.

If we move away from strict causality, metrics, and direct “policy” effect, universities can free themselves to have a meaningful impact on society. This means that universities need to place more faith in the power of all parts of the social system, and not just what has been considered the decision-making elite. Instead, universities need to concentrate on broader-based and popular communication.

Speaking truth to power should mean speaking truth to people.

Scientific communication is of course fundamental. But to affect the whole system, it must be accompanied by popular communication. Scientific results and writings need more regular dissemination in terms that will be readily understood by the interested, educated public.

The networked nature of society (locally and globally) means that such knowledge will filter into parts of the system where it is called for, where it will be morphed, and where it will impact the system as a whole. We need to integrate such popular science communication into the daily routine of conducting research. Of course the impact of popular communication is much farther down the road than most funding and management decision-makers are prepared to accept. Again, the flaw is with what we consider to be “correct” impact (based on the imagery we have of society and social change) and not with genuine impact that does indeed happen but may not acknowledge this or that specific faculty member.

Valuing and recognizing public communication requires turning away from metrics and “measurable” impact, and toward longer-term, indirect, society-wide impact. This is something only university leaders can ensure and defend.

The point? If we want society to care about the university, then the university must care about society. Politicians and policymakers will follow.