Guest editorial: Evaluating cities of culture

Citation

Oancă, A., Bianchini, F., Simpson, J., Tommarchi, E. and Wright, D. (2023), "Guest editorial: Evaluating cities of culture", Arts and the Market, Vol. 13 No. 3, pp. 117-120. https://doi.org/10.1108/AAM-10-2023-063

Since the 1980s, culture-led regeneration has gradually become absorbed into mainstream city planning and urban policy, with a growing acceptance of the value of culture in transforming environments, economies and communities. Cultural mega-events (Jones, 2020), like the European Capital of Culture (ECoC), the UK City of Culture (UKCoC) and other national and international City of Culture (CoC) initiatives across the globe, have often been seen as effective catalysts and accelerators for urban regeneration strategies through the delivery of a focused and intensive programme of cultural activities, usually lasting a year.

In our original call for papers, we invited articles that dealt with the ways evaluation has been used – and perhaps in some cases, misused – in policymaking. This Special Issue is one of the outputs of the Cities of Culture Research Network (CCRN), funded by the UK’s Arts and Humanities Research Council and operating from 2019 until 2021. CCRN’s main aim was to create an interdisciplinary space where academics, postgraduate researchers and local, national and international policy-makers could pursue a better collective understanding of CoCs, while specifically exploring the conditions and procedures required to create productive links between evaluation and new policy development. CCRN included ECoC, UKCoC and London Borough of Culture projects delivered by British cities and connected UK researchers with their counterparts in Aarhus (Denmark) and Galway (Ireland).

As part of our work on CCRN, we noted a research gap in critical studies on evaluation (Bianchini et al., 2022). Evaluation studies and impact assessments are often portraying CoCs as producing positive socio-economic effects, improving the image of cities and attracting tourists and inward investment. While there are many isolated studies about the impacts of CoC programmes, they generally neither explore medium and long-term effects (with the possible exception of Garcia and Cox (2013)) nor the often-complicated relationship between evaluation and policy making.

Critical to any discussion around the evaluation of CoC initiatives is taking seriously insights from the sociology of evaluation, critical policy studies and the anthropology of policy that argue that evaluation is not a straightforward data collection exercise but rather a practice of management and governance and a contested process whose effects can be very complex and ambivalent (Belfiore, 2009; Espeland and Sauder, 2016; Lamont, 2012; Porter, 1995; Shore and Wright, 2011, 2015). In studies about the delivery and impacts of CoCs, there is often a lack of clarity about processes of evaluation, the main actors involved, the main practices and organising principles and the effects of evaluation itself.

This issue of the Arts and the Market journal, Issue 3 of Volume 13, contains the first four articles that attempt to deal with these research gaps and to put forward a critical discourse about the conditions, mechanisms and procedures under which CoC evaluations have been produced.

Stephen Crone and Rafaela Ganga’s article is a critical reflection about the authors' experience of “Impacts 18”, a study focusing on the long-term effects of Liverpool ECoC 2008. The paper explores thorny issues in the epistemological foundations and methodological design of Impacts 18, as well as in the management of data and of stakeholder relationships. The authors conclude that the Impacts 18 case is a very good example of the tensions between critical and advocacy-driven evaluation research. The “incentive structures” and the rationales which gave rise to the Impacts 18 initiative were “compelling but incompatible”. They encompassed, on the one hand, the research community’s interest in exploring the longer-term sustainability of the impacts of Liverpool ECoC 2008, and, on the other hand, the “more pragmatic calculation” by local stakeholders that the 2018 anniversary “held significant propagandistic potential to reinforce prevailing, boosterist narratives of ‘city renaissance’”. The authors conclude that Impacts 18 “underscored […] the multi-faceted, pluralistic and often elusive nature of cultural value”. They also recommend that CoC evaluation should “transcend, or at least ‘dial down’, the pervasive rhetoric of impact itself”, embodied in the names of Impacts 08 and Impacts 18. Such rhetoric, they argue “risks undermining the truth and knowledge-seeking functions of evaluation”. However, the authors recognise Impacts 18’s positive contribution to knowledge as a form of “re-study” which helps “uncover and expose event legacy narratives that rest on shaky empirical foundations”.

Michael Howcroft considers the cultural politics of civic pride of the Hull UKCoC 2017. Howcroft’s research, based on interviews with stakeholders and on an analysis of promotional materials and events, critiques the ways in which pride was mobilised by UKCoC organisers and city leaders as an uncomplicated indicator of identity and belonging. Conventional modes of evaluation conceptualise CoC projects as involving culture being done to a place, accompanied by a before and after measurement through which the relative success of the cultural intervention can be measured, including residents' perceptions of their level of civic pride. Howcroft argues that policy-makers mobilised against the perceived pride deficit of the local population while crafting and controlling a singular pride narrative meant to create the feeling of change brought about by Hull 2017 (and not meant to create bottom-up processes of change). This pride narrative served to legitimise Hull2017 and other entrepreneurial initiatives and foreclose dialogue and critical, ambivalent perspectives regarding the UKCoC’s benefits.

Charlie Ingram explores the use of theatre arts and headphone verbatim as part of the evaluation of Coventry UKCoC 2021. In contrast to mainstream modes of evaluation, Ingram’s approach recognises that creative and artistic production itself both involves research and is a way of presenting research, including research which has claims to offer insights into the reality of the lives of participants. The production of inclusive artistic responses in this case is at the same time part of the intervention and part of the evaluation. As such, there is evidence of the value of richer methodological toolkits than purely econometric ones in evaluation practices also to communicate the success of a project to local constituencies. The approach studied by Ingram includes the embedding of qualitative work and of action- and performance-oriented modes of research. These might align with the interests and commitments of researchers from the arts and humanities and from arts organisations themselves.

Jessica Whitfield examines the intangible benefits of Hull UKCoC 2017 Volunteer Programme, a flagship community engagement initiative. Whitfield’s research, based on focus groups and interviews with volunteers which were undertaken respectively in 2019 and 2021, shows the potential transformative outcomes of programmes of this kind as part of a cultural mega event, by highlighting the positive impacts in terms of community engagement, well-being and civic pride witnessed in Hull. Her research also acknowledges critical issues emerging in the longer term from the Hull case, such as the perceived effectiveness of the programme in the city’s deprived areas, as well as the difficulty to maintain momentum and adapt to different circumstances after the event had taken place. By engaging extensively with the experience and perceptions of Hull 2017 volunteers, the paper provides a rare, and yet increasingly necessary in evaluation exercises, longitudinal perspective on a type of community initiative that is more and more often delivered as a component of CoC programmes. Moreover, Whitfield’s paper looks at the role and longer-term legacy of the Hull 2017 Volunteer Programme in the context of the Covid-19 crisis.

The four articles highlight interesting cross-cutting themes. They range from the critique of the dominant quantitative approach to CoC evaluation, by emphasising the value and benefits of qualitative methods (exemplified by Ingram’s article and by research on civic pride and on the benefits of volunteering in the Howcroft and Whitfield papers respectively), to the importance of critical perspectives on CoC evaluation studies and their political uses (in the articles by Howcroft and Crone and Ganga). The articles by Howcroft, Whitfield and Crone and Ganga also stress the need for a critical discussion of what constitutes “impact” in cultural mega events and underline the risk (in the article by Crone and Ganga) of the emergence of “policy-led evidence”, as opposed to evidence-led policy (despite the consensus among researchers and policy-makers about the desirability of the latter).