Escaping (from) numbers in education?

To be used effectively to govern social life, numbers require a public that is readily socialized to accept the authority of numbered knowledge. But if these publics possess high levels of numeracy, they become capable of undermining the authority of numbers through questioning and challenge.

Numbers are ubiquitous; their use has spread to global, national and local policy domains as disparate as health, defence, the environment, and education. There is an ever wider and deeper quantification of personal and collective life, engendering a world ‘based on continuous calculation at each and every point along each and every line of movement’ (Thrift, 2004, p. 583).

One of Finland’s major claims to fame on the world stage is its school system, which has been made famous largely by its PISA scores. The Programme for International Student Assessment is an international survey of 15 year-old school students in mathematics, science and reading across OECD countries. PISA also looks at attitudes to learning and learning skills. The annual results, which compare OECD countries, are subject to extensive public reporting and comparative analysis.

From its zenith a decade ago, Finland’s PISA performance is declining, leading to much agonising and, indeed, parental anxiety. This is despite the fact that the country continues to outpace its OECD competitors (see for example).

The PISA story is a good example of how numbers have escaped into our public and private life, shaping our discourses and fuelling pride, dissent and concern. Numbers now appear inescapable – driving and shaping policy making, behaviour and working practices. This article summarises the work that Nelli Piattoeva and Rebecca Boden, of Tampere University’s New Social Research programme, undertook to explore these escaping numbers.

The Great Escape

The presence of numbers in our lives is far from new. Since ancient times states and individuals have utilised accounting systems to control money and other resources (Carmona and Ezzamel, 2007). In medieval times, numbers supported feudal aristocrats’ control over their lands, and in 15th century Italy, the invention of double-entry bookkeeping facilitated vast trading networks that underpinned capitalist economic development. The 18^th century in Europe saw the development of the new science of statistics – literally, the science of the state – as states began collecting demographic data to manage themselves more efficiently. By the early 20th century, numbers were being used to try to achieve greater efficiency in industrial production – exemplified by Taylor’s book Principles of Scientific Management in 1911 (Taylor, 2004 [1911]). Taylor’s work epitomises how numbers can be seen as rational, efficient and objective – providing the ‘right answer’.

Through this history, the role of numbers in public affairs and policy was one of recording for the purposes of probity and information to support decision-making. An instance of the latter was a move to improve public health in Britain following Word War 1 as the extent of the physical incapacity of army conscripts became evident (Winter, 1980). By the middle of the last century, states globally utilised an extensive toolbox of number technologies in accounting and statistics.

As neoliberalism took root globally towards the end of the 20^th century, numbers began escaping from these relatively limited support roles. Neoliberalism can be defined as the resurgence of free market thinking and its adoption and adaptation to the running of the state and of public services. The boundaries between the public and private are dismantled, making the former a source of private profit (Hood, 1989). The justification for this is expressed in the language of economy and efficiency – the lure of lower public expenditure coterminous with better public service delivery as a result of the efficient operation of the market.

With the market came its toolbox of number technologies – financial accounting, management accounting and statistical data – that enables it to do this work. Numbers were increasingly liberated from the confines of the determination of profit, the management of a factory’s inventory or the counting of people. Number technologists refined non-profit oriented number forms such as key performance indicators, audit and rankings for use in these new settings. Numbers had escaped to the social world of the public sphere where, under the guise of Tayloristic scientific management, they comprise a form of epistemic governance (Alasuutari and Qadir, 2019). This facilitates governance by numbers – the reduction of ‘complex processes to simple numerical indicators and rankings for purposes of management and control’ (Shore and Wright, 2015, p. 22).

Numbers in these contexts have been imbued with objectivity and therefore accrued significant power. They beguile with offers to reduce complexity and to render any object manageable, including individuals. Hence, in a complex world of heterogeneous school systems, Finland’s can be judged ‘world leading’. The exalted status of numbers is attained and sustained despite a long history of critique by accounting researchers, who have pointed to their contingent, subjective, fragile and easily manipulated nature. Many other critical scholars, from a panoply of disciplines, have joined accounting researchers in critiquing numbers as they have become embedded in our social lives.

Our reckoning with numbers

Our thinking on this started with a desire to trouble the role of numbers in education– to see them as both powerful and valuable, but also as vulnerable and subjective. We wanted to probe the open, contested and contestable nature of numbers – to understand how they had both escaped and how their worst effects could be escaped from. In 2017, we organized a meeting at Tampere University, bringing together national and international scholars to share their work on this global phenomenon. That fruitful meeting led us to guest-edit a double special issue of the journal International Studies in Sociology of Education. In the rest of this piece, we synthesise the conceptual aspects of the papers from that special issue. At the end, we provide a short summary of each paper so that readers can explore the contributors’ enlightening empirical work and deeper theoretical thinking.

The special issue presents research on governance by numbers that spans national education systems and scales of governance from global to local, and diverse sectors of formal education from early childhood education to higher education. As such, the collection offers contextually rich accounts from around the world. The special issue also includes reviews of three recently published books on this topic, and a provocative afterword from Radhika Gorur – a leading Australian academic in this area.

The centrality and influence of numbers as modern policy instruments and technologies of the governance of education is widely understood. We sought in this international collection of papers and our introductory piece to push thinking further and, in particular, to trouble and to offer alternatives to the recurring determinism and inevitability with which numbers tend to be approached by researchers. To give citizens and policy makers more to think with when dealing with numbers, we sought to query what happens when we do not start with an implicit assumption that numbers are all-powerful. Rather, numbers are shaped by, and in turn shape, practices, and practices are uncertain – they remain open, contested and contestable. Numbers are not neutral, rather they emerge as a result of political work and temporal consensus. Following Law and Ruppert (2013), we also see numbers as potentially internally inconsistent and incoherent – characteristics that enable them to perform different tasks and create collateral realities (Law & Ruppert, 2013), and to travel routes unforeseen by their strategists.

We now synthesise the discussions and thinking in the collected papers around three themes.

The production of numbers is imbued with power

Quantification does more than simply represent reality – by shaping and directing our gaze, numbers also make reality by constituting new objects and issues of concern (Desrosières, 1998). However, the construction of ‘new numbers’ is not a straightforward endeavour. The process may be constrained by the availability of data or conflicting ideas of how complex societal phenomena ought to be represented numerically. In the context of a somewhat limited availability of numbers, those collected for one purpose often escape their initial usage and become part of new indicators or comparative rankings in an effort to objectify and measure new and more complex phenomena.

As escaped numbers travel and become part of new calculations, they act as Trojan horses, carrying a set of ideological assumptions and values in their built-in categories, and the ideas on which their mathematical models are built. Inequalities in monetary and symbolic resources enhance some organizations or countries as data holders and data producers at the expense of others (Merry & Wood, 2015; Rottenburg & Engle Merry, 2015). The incorporation of existing numbers into new calculations carries with it a symbolic component, because existing numbers possess unique ‘epistemic capital’ (Alasuutari, Rautalin & Syväterä, 2015).

Travelling, indeed escaping, numbers therefore carry often unseen power relations with them, helping them to endure and subtly shaping how phenomena are understood and governed (Merry, 2016). The widespread standardization of particular numerical representations excludes possibilities of alternative accounts or indeed of non-measurement. Instead, in a self-referential twist, ‘each organization’s measurement scheme ends up reinforcing the wisdom of its own solution to the problem’ (Merry 2016, p. 378). Because measurement shapes our comprehension of the social world, it also often influences how it is managed. Thus, as Paula Kuusipalo and Marja Alastalo write in their article for the special issue, one must know how data are gathered in order to understand how problems are framed and why certain solutions are proposed in preference to others.

Governance by indictors is dependent substantially on demand for and trust in numbers, and we shouldn’t assume these to be guaranteed – as we now demonstrate.

The ambiguities of trust in and demand for numbers

In order for numbers to circulate and be used to produce knowledge and, in a Foucauldian sense, conduct our conduct, people and organizations must trust them as legitimate representations of a phenomenon. Articles in our special issue show how demand for numbers is powered by diverse and ostensibly rational motivations, such as transparency, a desire to illustrate high quality, ‘value-for-money’, adherence to ‘global standards’, or a desire for exclusive identity. In this manner, numbers – often the same ones – are elastic (Piattoeva, 2015) and serve diverse purposes in a shared culture where they enjoy high status.

However, trust in numbers is paradoxically both strong and fragile, and this reveals something about their contested nature. Whilst frequently considered objective and untouched by human bias, the reductionist nature of numbers is equally well recognized – with actors understanding that they simplify and can lack nuance. The reductionist character of numbers may be raised to challenge and delegitimize them. In particular, when numbers are used in the politics of accountability and high-stakes decision-making on the allocation of funds, they may become increasingly mistrusted. Power plays and a realisation of the contingent nature of numbers often provoke gaming to make numbers ‘look better’ (or worse) – it is not numbers per se that evoke gaming but what they are used for.

The roles of informed publics

Numbers depend, ironically, on both informed and uninformed adherents. The concept of informed publics (Gorur, 2018; cf. Callon, Lascoumes, & Barthe, 2009) helps us to understand this. An informed public is one where actors might unsettle existing accounts and calculations and rearticulate them as controversies instead of stabilized, legitimised answers. Thinking in terms of adherents enables us to move the spotlight from the governing strategist, seeking to exercise power, to the subjectivities and agency of those assumed to be governed by numbers.

To be used effectively to govern social life, numbers require a public that is readily socialized to accept the authority of numbered knowledge. But if these publics possess high levels of numeracy, they become capable of undermining the authority of numbers through questioning and challenge. They may contest numbers in terms of how they reduce or misrepresent the phenomena that they claim to capture and explain. In more extreme cases, the numbers may become a ‘resource for less powerful actors to make [certain] problems visible and politically salient’ (Rottenburg & Engle Merry, 2015, p. 22). Thus, publics may readily escape from number-induced certainty to uncertainty, using the same numbers. In doing so, they might also escape the intentions of the strategist – a challenge to Audrey Lorde’s notion that the master’s tools will never dismantle the master’s house.

Examples of informed publics highlighted in the articles in our special issue are, however, hard to group in a binary way as either submitting to or resisting numbers. Instead, the papers examine the intricate nature of how people relate to, talk about and take numbers into their decision-making processes in particular education and societal contexts. These uses of numbers are often reactive and range between such approaches as gaming, the strategic use of numbers, opting out or refusing to collect data – though the latter two choices remain marginal. But the ways in which people engage with numbers are never effectively contained – they tend to extend to and affect other everyday decisions and practices, introducing new responsibilities or assigning new meanings to the tasks regularly performed.

Way forward: beyond rational explanations of engagement with numbers

To paraphrase Humpty Dumpty in Through the Looking Glass, when those in power use a number, it may mean just what they choose it to mean – neither more nor less. But quantification, counting, accounting, enumeration and numbering are inherently and simultaneously complex socio-cultural and socio-technical practices. Although often portrayed as robust, objective and neutral, numbers are nevertheless inherently interpretive, fluid, amorphous, contested and contestable.

Such numbers ever-present in education policy making and practice, shape thinking and action at every level and across all parts of the sector. This is why, in this special issue, we wanted to illustrate that one generative way to approach the power of numbers and their social lives is to treat them as contingent, and to study both the becoming powerful of numbers and the potentialities for failure and resistance embedded in them.

In presenting the papers published in this special issue, we suggest that the ambiguity of governance by numbers could be rendered visible by examining the dynamics between the development of new numbers and the surprising inertia of existing calculations, the contingent nature of trust in and demand for numbers, and the role of informed publics as key adherents.

This subject calls for yet further empirical research and we suggest that this should go beyond rational explanations of engagement with numbers – for instance, accounts of informed publics usually imply that those publics respond to numbers by way of rational thought and action. In order to extend research we should add an affective component to future analysis (Brøgger & Staunaes, 2016; Sellar, 2015; Staunæs & Pors, 2015). It might seem counterintuitive to talk of affects in the same sentence as seemingly dry and objective numbers, but relations to numbers are evidently also constituted through circulating negative affective states such as fear, anxiety, shame, but also positive ones, such as pride or hope (cf. Sonneveldt and Shahjahan 2020).

Meanwhile, Finnish national angst at a declining ranking in PISA continues…

Papers in the special issue

Steffen Mau’s paper Numbers matter! The society of indicators, scores and ratings challenges the notion that quantification is a neutral way of describing society. Rather, he argues, it is a process of valorisation with three important effects. Firstly, the availability of quantitative data strengthens social comparisons. Secondly, quantitative measurement of social aspects fosters expanded competition. Thirdly, there is a trend towards increased social hierarchisation because representations such as tables, graphs, lists or scores ultimately transform qualitative differences into quantitative inequalities (Mau, 2020, p. 19). Mau argues that this affects our understandings of inequality and queries who gets the power to name the categories of what gets counted.

(Steffan Mau is from Humboldt-Universität zu Berlin Kultur- Sozial- und Bildungswissenschaftliche Fakultat- Institut for Social Sciences, Berlin, Germany)

Tero Erkkilä and Ossi Piironen’s Trapped in university rankings: bridging global competitiveness and local innovation examines empirically the global university rankings that have played an increasing part in the status competition between higher education institutions. The novelty of the paper is that their starting point is the proliferation in rankings of innovation and urbanization. Erkkilä and Piironen show that whilst these are new numbers, they draw heavily on existing rankings and enfold themselves in competitive logics. Local innovation rankings objectify higher education as an element of global competitiveness, using university rankings. Consequently, through the medium of travelling numbers, political imaginaries of global competition are now projected to regional and city level (Erkkilä and Piironen, 2020, p. 38).

(Tero Erkkilä is from the Faculty of Social Sciences, University of Helsinki, Finland and Ossi Piironen works for the Ministry for Foreign Affairs, Government of Finland)

Paula Kuusipalo and Marja Alastalo write about their work on The early school leaver count as a policy instrument in EU governance: the un/intended effects of an indicator. The EU has enthusiastically adopted quantitative indicators to achieve common policy aims. One such indicator concerns the ‘early school leaver’ – students who leave education and training prematurely. This number escaped from being an education policy indicator to being a means of gauging national performance against the targets of the current Europe 2020 strategy. Kuusipalo and Alastalo explore how the number ‘works as a policy instrument at different levels of governance applying the conceptual tools provided by the policy instrumentation approach to unpack the components, pinpoint the political effects, and reveal the power relations they produce’ (p. 61). In querying the sometimes unthinking use of comparison facilitated by numbers, they draw attention away from ‘discussion concerning the role of large-scale comparisons in education towards more productive directions: moving from problematization and deconstruction of comparison to engaging with processes of measurement’ (p. 61).

(Paula Kuusipalo is from the Faculty of Education and Culture, Tampere University in Finland and Marja Alastalo is based in the Department of Social Sciences, University of Eastern Finland).

Steven Lewis’ article Becoming European’? Respatialising the European Schools System through PISA for Schools empirically interrogates the OECD’s ‘PISA for Schools’ – a scheme that translates the better-known national level PISA to a local level, comparing individual school performance in reading, mathematics and science against international schooling systems. Lewis focuses on how the transnational European Schools System has adopted PISA for schools, showing ‘how PISA for Schools reflects contradictory logics within the ESS, in which the inherently context-based goal of ‘becoming European’ is juxtaposed with the desire to employ decontextualised international evidence’ (Lewis, 2020, p. 85). Lewis concludes that the perceived need for such international comparative data can produce a problematic focus on data-driven practices (p. 85).

(Steven Lewis is from the Research for Educational Impact (REDI) Centre, Deakin University, Australia).

Miguel Lim, in Impact case studies: what accounts for the need for numbers in impact evaluation? explores a UK higher education indicator – the impact case study (ICS) – used in the Research Excellence Framework since 2014 to evaluate the beneficial impact of university research on society. His paper examines the evolving roles of the case studies and their relationships with various stakeholders. Lim considers the ‘dysfunctional transformation of indicators into targets’ highlighting ‘the challenges to ICS evaluators and shows how a spiral of mistrust can both undermine but also cause demand for numbers’ (Lim, 2020, p. 106). He concludes that, despite the narrative format, these case studies have not escaped from dominant research evaluation metrics.

Miguel Lim is at The Manchester Institute of Education, The University of Manchester, UK.

Helena Candido, in Datafication in schools: enactments of quality assurance and evaluation policies in Brazil, turns our attention to the Global South, exploring the enactments of quality assurance and evaluation policies in Brazil. She questions ‘how data permeates and changes school environments, school actors’ conduct and their imaginaries’ (Candido, 2020, p. 126). She focuses on policies that include large-scale assessments, indicators, rankings and other steering mechanisms. In particular, these data approaches ‘connect data to quality in education’ (p. 126). Using extensive qualitative data from interviews, Candido reveals how data manifests itself within the schools as a technology of governance, but that alternative courses of action are also proposed.

(Helena Candido works at the Faculty of Educational Sciences, University of Helsinki, Finland.)

Maiju Paananen explores the Fluctuating child–staff ratio: governing by numbers in Finnish early childhood education in her paper, which is a case study of how numbers are used in governing early childhood education. In particular, she looks at how the child:staff ratio is deployed in the everyday life of preschool establishments. Using data from interviews of preschool teachers and city administrators in a Finnish municipality and policy documents, Paananen explores how the ratio steers and shapes the everyday life of preschools, and how teachers are involved in the associated policy assemblages. Paananen demonstrates, by looking at a local level, that teachers are central components in these dynamic policy assemblages – highlighting the centrality of looking at the social processes around numbers.

(Maiju Paananen is in the Faculty of Education and Culture at Tampere University, Finland).

Finally, Radhika Gorur offers up a provocative Afterword: embracing numbers? Gorur argues that ‘while the use of numbers in governance has a long history, the kinds of numbers we now produce enable a range of new possibilities for monitoring, regulation and policy decision-making’ (Gorur, 2020, p. 187). Whilst education policy actors globally are demanding more and more education data, this trust in numbers has been critiqued by researchers who argue that they are reductionist and decontextualising, leading to poor policy. ‘The entry of big data’, she opines, ´poses even more complex epistemological and ontological challenges, many of which we do not fully understand as yet’ (p. 187). Whilst acknowledging these challenges, Gorur asks if big data could offer the possibility of better policy decision-making by enriching our current thin information and turning it into genuine knowledge, or even wisdom.

(Radhika Gorur works at the School of Education, Deakin University, Australia)

 

References

Alusuutari P. & Qadir, A. (2019). Epistemic Governance, London: Palgrave MacMillan

Alasuutari, P., Rautalin, M., & Syväterä, J. (2015). Organisations as epistemic capital: The case of independent children’s rights institutions. International Journal of Politics, Culture, and Society, 29(1), 57–71.

Brøgger, K., & Staunaes, D. (2016). Standards and (Self) Implosion: How the Circulation of Affects Accelerates the Spread of Standards and Intensifies the Embodiment of Colliding, Temporal Ontologies. Theory & Psychology, 26(2), 223-242.

Callon, M., Lascoumes, P., & Barthe, Y. (2009). Acting in an Uncertain World: An Essay on Technical Democracy. Cambridge & London: MIT Press.

Carmona, S., & Ezzamel, M. (2007). Accounting and accountability in ancient civilizations: Mesopotamia and ancient Egypt. Accounting, Auditing and Accountability Journal, 20(2), 179-209.

Desrosières, A. (1998). The Politics of Large Numbers. A History of Statistical Reasoning. Cambridge, MA: Harvard University Press.

Gorur, R. (2018). Escaping Numbers? Intimate Accounting, Informed Publics and the Uncertain Assemblages of Authority and Non-Authority. Science & Technology Studies, 31(4), 89-108.

Hood, C. (1989). Public administration and public policy: Intellectual challenges for the 1990s. Australian Journal of Public Administration, 48(4), 346–58.

Law, J., & Ruppert, E. (2013). The social life of methods: Devices. Journal of Cultural Economy, 6(3), 229-240.

Merry, S. E., & Wood, S. (2015). Quantification and the paradox of measurement. Current Anthropology, 56(2), 205-217.

Merry, S. E. (2016). Cultural dimensions of power/knowledge: the challenges of measuring violence against women. Sociologie du travail, 58(4), 370-380.

Piattoeva, N. (2015). Elastic numbers: national examinations data as a technology of government. Journal of Education Policy, 30(3), 316–334.

Rottenburg, R., & Engle Merry, S. (2015). A world of indicators: The making of governmental knowledge through quantifi cation. In R. Rottenburg, S. E. Merry, S.-J. Park, & J. Mugler (Eds.), The world of indicators: the making of governmental knowledge through quantification (pp. 1–33). Cambridge: Cambridge University Press.

Sellar, S. (2015). A feel for numbers: affect, data and education policy. Critical Studies in Education, 56(1), 131-146.

Shore, C., & Wright, S. (2015). Governing by numbers: Audit culture rankings and the new world order. Social Anthropology/Anthropologie Sociale, 23(1), 22–28.

Staunæs, D., & Pors, J. (2015). Thinking educational policy and management through (frictional) concepts of affects. In K. N. Gulson, M. Sonneveldt, E. L. & Shahjahan, R. A. (2020) Book review on “Engines of anxiety: academic rankings, reputation, and accountability” by Wendy Nelson Espeland and Michael Sauder. International Studies in Sociology of Education, 29(1-2), 177-178.

Taylor, F. W. (2004 [1911]). The principles of scientific management. Whitefish (Mont.):

Thrift, N. (2004). Movement-space: the changing domain of thinking resulting from the development of new kinds of spatial awareness. Economy and Society, 33(4), 582-604.

Winter, J. M. (1980). Military Fitness and Civilian Health in Britain during the First World War. Journal of Contemporary History, 15(2), 211-244.