Rational and symbolic uses of performance measurement: Experiences from Polish universities

2.1 The rational and institutional views on PM in public sector organisations

The classical perspective of the role of accounting in an organisation is a view of accounting within the paradigm of rational choice. The rationality associated with accounting tools and methods “can be seen as an important mechanism through which accounting technologies symbolise and instrumentalise a process of rationalisation of social practices” (Giddens, 1971; Holten and Turner, 1989; Mouritsen 1994). Powell (1991) argues that organisations are truly concerned with task performance and that the difference between formal structure (appearance) and actual operations (reality) is sometimes small enough. In the same vein, accounting changes introduced by new accounting and PM tools can provide relevant and useful information for decision makers (Abernethy and Bouwens, 2005; Moll and Hoque, 2011; Mouritsen, 1994; Odzil and Hoque, 2017; Otley, 2003, 2016). According to such a perspective, the focus on efficiency and performance, in general, requires that organisational activities be monitored with the use of new accounting and PM systems. According to Ansari and Euske (1987, p. 552), accounting information is used as a tool in auditing choices between alternatives defined with present goals, and classified as having a technical rational focus. Rationality is intended as a goal-means consideration in support of a decision (Baxter and Chua, 2003, p. 112).

In contrast, many accounting scholars have highlighted the limitations of the classical Weberian rational choice approach and suggest that accounting, in general, and PM, in particular, may have many complex roles in organisations and society (Burchell et al., 1980). It has also been established that PM tools are adopted to meet organisational needs for institutional, social and political legitimacy (Brignall and Modell, 2000; Modell, 2001; Tucker and Parker, 2015). In general, Mouritsen (1994) proposes that accounting systems contribute to the rational co-ordination of activities, but not always. PM systems often have unintended consequences that “abound partly because accounting operates in complex institutional settings where the location and the context of social interaction is important for explaining and understanding it” (Mouritsen, 1994, p. 196).

The most widely used theory, to date, to understand PM practices and changes better in those practices, is neo-institutionalism. The study of institutions crosses the topics of sociology, politics, economics, and organisational theory, while the core assumption within the study is that organisations are rooted within the broader institutional context (DiMaggio and Powell, 1991). Therefore, organisational practices are either a reflection of, or a response to, rules and structures that are built into a larger environment (Paauwe and Boselie, 2003, cited by Najeeb, 2013, 2014). This environment acts as a source of behaviours, norms, and incentives, but also, from another perspective, the sanctions and limitations of organisational activities. Thus, the organisation does not determine its behaviours on its own, based on autonomous decision-making as a response to problems. Instead, organisations “tend to pattern their strategies on models and procedures that are widely recognised, accredited, and thus ‘institutionalised’ in the societal environment or organisational field” (Schriewer, 2009, p. 33) to increase their chances of survival.

The neo-institutional theory views accounting practices as one of a larger set of features that can legitimise organisations through the construction of an appearance of rationality and efficiency. Ceremonial and symbolic compliance with legitimate norms may have little impact, because a formal organisational structure is decoupled from the actual organisational process (Carruthers, 1995).

PM is often adopted in the public sector to satisfy organisational needs for institutional, social and political legitimacy. These needs can become incoherent with rational motivations (Tucker and Parker, 2015). Several scholars have pointed out that accounting reforms are more rhetorical, and are often symbolic window-dressing, rather than the production of new rational forms of institutional change (Agostino and Arnaboldi, 2017; Mimba et al., 2013; Modell, 2001). Czarniawska-Joerges and Jacobsson (1989), based on examples taken by the Swedish public sector, depicted budgeting as a symbolic or ceremonial performance tool, rather than a decision-making process. They also stated that it is a means of conversation, rather than a means of control, and an expression of values, rather than an instrument for action (Czarniawska-Joerges and Jacobsson, 1989). Past studies show that both, external and internal contexts play a crucial role in explaining how and why the adoption of a new PM tool may (or may not) be translated into practice, assume different features and become institutionalised. ter Bogt and van Helden (2000) have also illustrated that mandatory reform adoption does not necessarily bring about expected results, thus undermining the relevance of studying the institutionalisation of reforms within an organisation (Harun et al., 2012).

In a globalised world, which has been strongly affected by economic and financial crises and has shared the need for reform, exogenous pressures have motivated organisations to adopt a similar reform package through isomorphic behaviours, that is, each organisation tends to resemble the others for the sake of increasing their legitimacy and conforming to socially accepted rules and practices (DiMaggio and Powell, 1983). Hence, the agenda of many universities worldwide converged towards PM systems (Pollitt, 2002). The idea of convergence comes from the institutional theory that has emphasised the process of isomorphism, whereby organisations adopt similar procedures to gain legitimacy as a result of coercive, mimetic and normative pressures (DiMaggio and Powell, 1983).

Mimetic processes are a result of organisations’ responses to uncertainty. Organisations copy successful organisations, in order to gain legitimacy (DiMaggio and Powell, 1983). Specifically, in the higher education context with an increasing level of competition and internationalisation that has shaped the field over the last few decades, universities have tried to model themselves on others operating in similar contexts through mimetic processes. In mimetic isomorphism higher education institution seek accreditations due to competitive pressures rather than to improve their own efficiency.

Further, coercive and normative pressures for meeting the expectations of international organisations, fora, and professional networks, and to respond to economic demands and exogenous pressures (Van Gestel and Teelken, 2006), have motivated universities to try to adopt a textbook model of PM.

Normative isomorphism refers to the relationship between the regulations and employees’ background. This type of isomorphism is associated with professionalisation (Paauwe and Boselie, 2003). Within the higher education context, normative isomorphism considers the role of regulatory bodies and professional organisations as accreditation agencies through their formal requirements, standards and international orientation.

Coercive isomorphism, on the other hand, refers to the influence of the political and governmental regulations on the organisations. It refers to cases where universities are likely to implement changes within their policies in order to adjust to the requirements set by the government, thus, due to coercive pressures (Najeeb, 2013).

The neo-institutional theory is also useful in explaining the role of PM tools in ensuring that internal actors accept an organisation (Grossi et al., 2016; Hoque, 2005). PM tools cannot be forced top-down. Instead, they need to be accepted at the lower levels of the organisations. Employees need to have the potential to impact the performance that is measured (Bouckaert, 1993). At the same time, achieving high legitimacy by internal actors may positively influence the implementation and use of PM tools (Moll and Hoque, 2011; Van Dooren et al., 2010). Such decoupling strategies could be employed by universities to defend their research autonomy (Narayan et al., 2017).

2.2 PM and management in universities

PM has been a relevant issue for public organisations from different areas, including higher education, not only because new public management has brought a stronger focus on output and outcome-oriented performance systems, but also because competitiveness has arisen within these organisations. Based on new public management-doctrines, PM practices arise as a way to “reinvent the government” (Osborne and Gaebler, 1993) and improve efficiency and effectiveness in public sector organisations (Hood, 1995). Accordingly, PM has been considered the main pillar of public management reforms in recent years (Almquist et al., 2013; Johnsen, 2005; Van Helden and Reichard, 2013; Walker et al., 2011). There is a growing critical view among public management and accounting scholars, based on the assumption that some PMs are designed for private sector organisations and are not always suitable for the specific needs of public sector organisations (Broadbent and Laughlin, 1998; Broadbent et al., 2001; Lapsley, 1999; ter Bogt and van Helden, 2000). Public sector organisations provide a variety of activities and set multiple goals to address different basic needs (Ferlie et al., 1996). Therefore, PM will require the consideration of both, quantitative and qualitative profiles, as well as different dimensions within those profiles.

A university is a very interesting context to study. During the past few decades, universities have implemented several changes at the national level that have had several effects at the organisational and individual levels. Since the performance of universities is frequently directly linked to the funds allocated to them, many universities spend considerable resources on the development of specific tools, allowing them to measure and communicate the performance of faculty in terms of research and teaching outputs (Agyemang and Broadbent, 2015; Boitier and Rivière, 2013). As in the case of other public organisations, management control systems enable universities to track how they are performing. Collecting and processing performance data enables organisations to set objectives and work towards meeting those objectives (Broadbent and Laughlin, 2009; Chenhall, 2003; Chenhall et al., 2010; Ferreira and Otley, 2009; Moynihan and Landuyt, 2009).

Control over research activity in universities is primarily based on the new public management approaches of PMM. Osterloh (2010) emphasises that output control relates to the use of output performance indicators (e.g. citations, the number of articles published in peer-reviewed journals and the number of PhDs completed) as a proxy for the output of researchers. The use of such output metrics makes it easier for external stakeholders to understand the PM better (Agyemang and Broadbent, 2015). From the internal stakeholders’ perspective, management-by-results is based on quantitative, rather than qualitative, measures. This management style has negative effects on the motivation to engage in creative, knowledge-intensive work (Kallio and Kallio, 2014).

ter Bogt and Scapens (2012) found that, in the case of Dutch and British universities, the use of judgemental quantitative systems has negative effects on teachers and scholars. This is because judgemental quantitative systems cause uncertainty and anxiety. As a consequence, they may also inhibit creativity and innovation in teaching, thus, limiting potential contributions to the world outside the university. Nevertheless, PM systems seem to remain a dominant force in the public sector, despite their unintended and perverse effects (Van Dooren and Thijs, 2010). When the quantity is valorised, potential perverse outcomes include the “publication game”, in which “the rational response is to produce multiple publications that are small variations on a theme” (Lewis, 2014, p. 424).

The universities’ PM systems are largely shaped by the needs of various external and internal stakeholders (Agyemang and Broadbent, 2015; Elliott and Goh, 2013). Guthrie and Neumann (2007) notice that the establishment of a performance-driven system for higher education is determined by linking the governmental funding of universities with the assessment of the results. Hence, it is directed towards improving the quality of services provided by universities (i.e. in the field of teaching and scientific research).

Greater interest in reliable information on a university’s performance, on the part of public institutions awarding funds for scientific research, stems from the reorientation of government financing of universities. The change in financing especially applies to public universities, which have undergone a shift from being fully funded to partially subsidised (Guthrie and Neumann, 2007). In this way, universities become more open to the market and transform from being academically driven to market driven.

PM is frequently used to create university and journal rankings (Adler and Harzing, 2008; Brooks, 2005; Dill and Soo, 2005; Agyemang and Broadbent, 2015). Research journal rankings indicate the detrimental and dysfunctional impacts of their use in managing research (Macdonald and Kam, 2007, 2009, 2011; Mingers and Willmott, 2013; Parker et al., 1998; Tourish and Willmott, 2015; Willmott, 1995, 2011). Lukka (2010) also points to the fact that the top international journals limit the scope of research by favouring research methods and imposing a narrow subject scope. Despite much criticism, journal rankings are used to manage the performance of academics for promotion and tenure decisions (Agyemang and Broadbent, 2015), but also to reward researchers for their achievements and motivate individuals to reach the desired levels of performance (Ferreira and Otley, 2009; Ryazanova and McNamara, 2016; Townley, 1997).

PM use is essential for improving the quality of teaching (Ammons and Rivenbark, 2008). However, most studies focus on measuring students’ satisfaction, rather than the quality of the teaching (Bedggood and Pollard, 1999; Bedggood and Donovan, 2011; Craig, 1995; Harnash-Glezer and Meyer, 1991; Kleiner, 1989; Lutz and Field, 1998; Sohail and Saeed, 2003). To date, studies in this area demonstrate evidence of such a claim (e.g. high evaluations of chosen tutors may be due to their popularity Ramsden, 1990, expressiveness Ware and Williams, 1980 or lower student requirements Brookfield and Preskill, 1999). Despite the criticism associated with this type of measurement, the results of the analysis are used in the process of university management. They constitute a key source of information while making decisions (e.g. study programmes, scope of the presented material and personnel changes) (Bedggood and Donovan, 2011). Universities also monitor student satisfaction and analyse the professional lives of graduates and their satisfaction with studies (Yorke and Longden, 2004).

The implementation of PM in the university context could easily lead to protests and perverse outcomes. Accordingly, the design of PM systems, without considering the specific characteristics of the university context or without defining the main stakeholders, and without dialogue about legitimate performance indicators, could generate a loose coupling between daily work and formal PM systems, and thus, potentially cause considerable harm for the university community. The success of the implementation phase is strongly affected by the capacity of the organisation to engage all stakeholders, employees and managers. Therefore, a good mix of bottom-up and top-down processes need to be balanced. As such, it is important to stress on the logic of the implementation of PM system at all levels, at the risk of encountering a rejection, or the resistance by academic staff whose values may be contrary to the managerial values involved (Boitier and Rivière, 2013). Moreover, some scholars have argued the relevance of coupling between PM and organisational objectives. In contrast, Johnsen (2005) has argued that the decoupling between PM and organisational objectives could positively determine the effective implementation of PM in political contexts. The most important questions are connected with the use of PM: How is PM used? Who uses PM? What external or internal factors influence the use of PM?

2.3 Theoretical framework

The topic of the uses and users of PM, within the public sector, has recently attracted the growing attention of scholars (Ammons and Rivenbark, 2008; Grossi et al., 2016; Jansen, 2008; Kroll, 2014, 2015; Melkers and Willoughby, 2005; Pollitt, 2006; Taylor, 2007). PM in public organisations is used by different groups of stakeholders, including politicians (Jansen, 2008; Moynihan, 2016; Van Helden et al., 2016), supervising bodies (Taylor, 2007), the management of particular institutions and their departments (Grossi et al., 2016; Jansen, 2008; Kroll, 2014), society (Jansen, 2008), the mass media and individual employees.

Several scholars have focussed on the possible uses of PM in their work (Behn, 2003; De Bruijn, 2002; Van Dooren et al., 2010). Behn (2003) proposes a categorisation of eight managerial uses, namely, to evaluate, control, budget, motivate, promote, celebrate, learn and improve. De Bruijn (2002) stated that PM could fulfil a number of functions, such as creating transparency, learning, appraising and sanctioning. Johnsen (2005) focusses on the “obstacles” (or costs) of using PM, such as the lack of relevant data, measurement errors, low decision relevance, the proliferation of PMs, information overload, indirect lines of responsibility, no ownership of performance, loyalty to professional norms rather than to the management, the manipulation of data and creaming. Van Dooren et al. (2010) focussed on the hard and soft uses of PM. In that study, hard use presupposes a tight coupling between performance information and decision-making. Soft use is related to loose coupling, as a dialogue and an interpretation that mediates the final decision-making process.

Other scholars (Agostino and Arnaboldi, 2017; Brignall and Modell, 2000; Mimba et al., 2013) distinguish between the rational and symbolic (or ritualistic) uses of PM. Rational use, in the case of the effective use for decision-making purposes, is useful for motivation and learning (Johnsen, 2005; Verbeeten, 2008). Where one stakeholder is dominant and the PM system is related to his or her interest, the PM system cascades down the organisational hierarchy. Hence, it can be expected that PM is functionally used for decision-making purposes (Brignall and Modell, 2000).

Symbolic use is associated with ceremonial adoption and is used to ensure external legitimacy (Meyer and Rowan, 1977). When two or more stakeholders with diverging interests are powerful stakeholders, some forms of the decoupling of PM, between the top and lower levels of the organisation, are likely providing the space for the symbolic use of PM and the “decoupling” of PM with their daily jobs (i.e. not using performance information for managerial and/or evaluation purposes) (Brignall and Modell, 2000; Hoque et al., 2004; Johnsen, 2005). This implies that PM has a role in legitimising the organisation to its stakeholders, but is not really used as an input for decision-making purposes (Mimba et al., 2013). PM systems may be adopted by public sector organisations for ceremonial and symbolic reasons, apart from performance-related issues (Tucker and Parker, 2015).

Past studies on universities have also shown similar rational and symbolic uses of PM. Alach’s (2017) study is related to New Zealand’s eight universities and illustrates a relatively high use of PM and a strong alignment of strategy and PM. On the contrary, Melo et al. (2010) studied a high performing English university and showed that, despite the increase in the measurement of performance in several areas (mostly teaching and research), there seems to be a lack of action, especially regarding individual performance. Moll and Hoque’s (2011) study illustrates how the management of an Australian university attempted to change its internal budgetary system to meet the needs of external stakeholders and gain external legitimacy. Narayan et al. (2017) investigate organisational responses to emerging issues on accountability-autonomy tensions that can be managed within the context of university research commercialisation in two universities in New Zealand.

In our study, we decided to build a theoretical model (see Figure 1) that identifies users and uses of PM in universities, as well as exogenous and endogenous pressures (factors) influencing the use. The adoption and use of PM systems can be considered the effect of several exogenous pressures represented by the isomorphic pressures towards performance-oriented reforms at work over the last few decades. As such, its implementation and use are shaped by endogenous pressures and factors at work in each university. While the exogenous factors would support the development of similar reforms, the specific university can respond with convergent or divergent PM systems under the influence of endogenous factors, in terms of decisions, implementation and use of these practices.

We assume that PM tools can be adopted by universities for legitimacy purposes (i.e. external reporting to national authority or accreditation agencies) and be more influenced by ceremonial motivations, symbolic motivations, decision-making purposes (i.e. resource allocation, efficiency, bonus system), and rational motivations. We assume that the use of PM will depend on a number of influential factors, divided into two main categories, namely, exogenous and endogenous factors (Cavalluzzo and Ittner, 2004; Grossi et al., 2016; Saliterer and Korac, 2013; Van Helden and Reichard, 2013).

Exogenous factors are assumed to influence the use of similar PM. Exogenous factors include stakeholder expectations, external relationships, accreditations, rankings and external evaluations. Endogenous factors can influence the adoption and use of similar and also different PM systems and are divided into two types, namely, organisational and individual. Organisational factors are related to the university’s culture (e.g. entrepreneurial vs traditional), traditions, leadership position and organisational structure (e.g. role of departments, PM systems’ maturity and faculty size). Individual factors that affect the uses of a PM system include personal motivation, reward expectations, training, familiarity with the PM system, faculty position, experience, tenure and rationality.

The empirical part of the paper will explore the exogenous and endogenous factors and investigate the manner and extent of the influence of the internal uses and users of PM on four universities in Poland.

4.1 Exogenous factors affecting the PM systems

The higher education market in Poland is rather saturated, with over 400 higher education institutions, out of which almost 70 per cent Polish were non-public (private). The recent demographic changes and price sensitivity of the market participants create tensions and uncertainty in relation to future development and sustainability (Dobija and Hałas-Dej, 2017). The recent years have brought about a wide debate on the ways in which the higher education system should be shaped so that the educational market becomes more internationally competitive, and is able to affect the innovativeness and creativeness of the Polish economy. Further, stakeholders have been involved in the debate and as a result, new regulations on higher education is going to be introduced in October 2018.

The Polish higher education system can be described as highly regulated. Much like other European higher education institutions, Polish universities are part of the European higher education area. At the national level, all universities in Poland, both, public and private ones are subject to regulations included in the Parliamentary Acts. One of the features of the Polish higher education system is that responsibility for the quality and standards of the qualification structure is split between the Act on Higher Education (2005), which regulates the higher education system, organisation, governance, and supervision of the higher education institutions and the Law on Academic Degrees and Title (2003), which regulates matters related to academic qualifications and titles.

Law on Higher Education (2005) stipulates the creation of a Polish Accreditation Committee (PAC) that will be responsible for ensuring the high quality of higher education. The PAC will conduct periodic, compulsory accreditations of all undergraduate and graduate academic programmes as well as institutional accreditations. These evaluations are considered important as the results of the evaluation can have important consequences. Those schools, both, public and private ones, with a positive evaluation with distinctions can be awarded substantial additional financing with the aim of improving the quality of education. A negative evaluation by the PAC would result in the withdrawal of degree granting rights by the MSHE.

The structure of Polish academic qualifications has been modelled like the structure prevailing in France or Germany with two academic degrees of PhD and doctor habilitatus (habilitation) and the title of professor. The Act on Academic Degrees and Titles (2003), also regulates the existence of the Central Commission for Academic Degrees and Titles (CCADT) – a committee to the Prime Minister responsible for ensuring the development of academic qualifications. The committee administers all applications for PhD, habilitations and the academic title of professor. The Act also sets a minimum performance requirement before an individual person can be awarded an academic degree or title. Financing of research activities is directly linked to the outcome of the periodic evaluations of the scientific unit of higher education institutions by the Committee for Evaluation of Scientific Units (CESU) (Law on the Principles of Financing Science, 2010). CESU is a consultative and advisory body to the MSHE. It is responsible for monitoring the scientific excellence of scientific units of higher education institutions. The CESU is also responsible for setting the evaluation criteria as well as for conducting the evaluation. As a result of the evaluation, the scientific units are grouped into different scientific categories. Both, public and private higher education institutions, can be evaluated. The outcome of the evaluation is then linked with different levels of financing of research related activities and faculty development.

The results of both evaluations conducted by SECU and PAC have important impacts on the level of financing granted to higher education institutions. It is, therefore, not surprising that the universities, including business schools, try to adjust their PM systems to the requirements set by the evaluators. This is, however, not an easy task as the systems of evaluation, especially the scientific ones, have been continuously subject to change, and the rules of the game have not been usually known until the evaluation is administered. This creates instability in the internal PM systems and affects the behaviour of individual academics. This is discussed in the section below.

Higher education in Poland has recently been under pressure for internationalisation. In general, the level of internationalisation is considered to be low, as the Polish universities have ranked low in general university rankings. However, some business schools began to join the bandwagon of internationally accredited schools and many more aspire to join the group. This is also possible due to the recent decision made by the MSHE to cover administrative costs of the accreditation process as well as the accreditation fees. This trend, however, puts greater pressure on individual academics as well as the entire institution in terms of performance, and can be also seen as one of the factors affecting the change in the internal PM systems.

4.2 Endogenous factors affecting the PM systems

To understand the functioning of the PM system in the four universities better, interviews were conducted with a focus on the main activities that comprise a university’s mission (i.e. research, teaching and third mission). A summary of the findings is presented in Tables II–V.

Despite the differences in the length of using the PM system as the main motivation for development, the main features of the systems are quite similar, focussing on three aspects of performance, namely, research, teaching and contributions to society. External factors seem to be the main reasons for the implementation and use of PM in all four universities. The PM systems at all universities tend to be driven by the expectation of the main external stakeholders to track and measure strategic goals, targets, and achievements. One major difference is related to the definition of the main stakeholders, as reflected in the strategies of the four analysed universities. In the case of the two public universities, it seems that the Polish MSHE, as the main provider of funding, alongside the state accreditations and evaluation committees, are the main drivers for implementing PM:

We report on the accomplishments and actions that reflect what is considered by the ministry while evaluating higher education institutions […] the system has be regularly adjusted to the criteria imposed by the ministry

(Interviewee P21).

In all four universities, PM is related to research, teaching and a third mission. PM is mainly decided in the interactions between the departmental and central administration, as part of the internally developed PM system. Some of this information has to be prepared and delivered for an external assessment by national actors (i.e. MSHE, PAC, CESU, ACG, CCADT and accreditation agencies). In UniA and UniB, it also needs to be prepared and delivered by international accreditation agencies. In the case of internationally accredited schools, internal accreditation offices often play a mediating role in the design and implementation of teaching performance measures related to individual academics and departments or faculties.

Research performance constitutes an important element of PM systems in all universities. In all four cases, the performance measures of research are connected to individual and organisational outputs, such as the number of publications in journals listed in national rankings, participation in conferences, as well as grants and awards. For the two universities (UniA and UniB) with international accreditation, there is a growing pressure to publish in journals with high impact factor that are listed in international journal rankings (e.g. JCRs, ABS). This is also reflected in PM, especially at the individual level. More specifically, two of the interviewees stated that:

We all participate in this arms’ race to publish the best publications, in the best journals, with the highest points

(Interviewee P5).

The main part of the current individual performance assessment is the publication achievements in highly ranked journals […] if you wish to remain in the place where you are [at the university] you need to get a positive assessment, and it is obvious. We are stressed by this assessment

(Interviewee P18).

The performance measures, in relation to teaching, in all universities, are based on traditional, quantitative metrics with few innovations (e.g. direct teaching hours, number of bachelor’s/master’s/doctoral students). Moreover, concerning teaching, all universities implemented student evaluations using online surveys. They periodically organised peer review observations of classes performed by other staff. Only one university seemed to pay more attention to PM related to teaching and noticed the drawbacks of the current PM, due to the small response rate and the decreasing relevance of the information received through the online system. Consequently, UniA introduced an innovation that measures the teaching quality by using a focus group. This new solution is expected to provide more valuable and accurate information on the teaching performance of the individual, as well as that of the programme.

As for the third mission, the PM system takes into account the different output measures in relation to individual academics (e.g. consulting projects, expert team membership, boards, and associations). However, this PM is mainly considered supplementary information to the performance information on research and teaching:

Whenever I do consulting work […] participate in boards, I report it […] we do not get any additional points for those, but each of us can list what we do

(Interviewee P39).

Similar to research PM, accreditation offices are often playing a mediating role in the implementation of the system, due to the importance of the third mission in the overall evaluation of external evaluations and accreditations.

The management of all institutions declares that the use of PM is provided by the system. However, the major difference between the cases seems to be in understanding how academics use this information at the strategic level. Almost all of the academics at UniA clearly link the PM systems to the school’s strategic objectives and goals. The faculty members also seem to understand how the system functions, especially when it comes to the expectations towards individual performance. In the other three universities, the faculty PM system is less clear for the individual academics. Additionally, most academics have little knowledge of, or familiarity with, the way PM is used at the school level, although the interview results suggest the awareness of the increasing internal pressures to use the PM at the strategic level.

4.3 The uses and users of PM

In the traditional, classical management accounting literature, PMs are considered an important part of the PM systems (Broadbent, 2011; Martin-Sardesai et al., 2017; Otley, 2003, 2016). Universities are not different from other public organisations and can use PM for rational purposes to make operational and strategic decisions internally, with the goal of the efficient use and allocation of the available resources (Ansari and Euske, 1987; Baxter and Chua, 2003). However, PM can be also used in the wider context of the external stakeholders and society. Organisations can also adopt PM for symbolic purposes, aiming at external legitimacy (Tucker and Parker, 2015). This need of the symbolic use is especially important in the case of public organisations, including universities (Moll and Hoque, 2011).

In this section, we provide empirical evidence of how PMs are used in the university context and who is using this information. We use our theoretical model to organise the discussion by first focussing on the different users (i.e. internal and external) of PM. We use the empirical materials collected to provide evidence and examples of the different uses (i.e. rational and symbolic) of PM within the two main groups of users analysed in the paper. A summary and synthesis of the findings for each case are provided in Tables II through V.

A common example of internal and rational use provided by interviewees were decisions connected to employment. In all four universities, research achievements are considered by the university authorities for decisions regarding promotions, as well as the hiring and dismissal of employees. In three universities (UniA, UniB, UniD), the PM results connected with research are also considered during the allocation of internal grants for research projects and international conferences.

In the case of UniB, some of the respondents pointed out that the only rational use of formally reported information on their achievements is reflected in the decisions related to the employees being made by school authorities. According to two respondents, this information is the basis for extending a contract or dismissing an employee:

[…] the evaluation, as such, is used to renew the contract

(Interviewee P17).

If one’s employment agreement is prolonged, it may mean that one has a positive rating. If it is not prolonged, or one is not promoted, reasonable grounds need to be established

(Interviewee P18).

There seems to be some ambiguity about the use of PM, because when asked if they knew of any cases of dismissals based on the formalised PM, both, employees and the school authorities could not provide an answer. It can, therefore, be presumed that the use of PM for decisions related to employees is, to some extent, symbolic:

The management […] draws up the rankings of employees who have obtained a certain number of points, for example, for publications […]. Those who are at the bottom of this ranking become the subject of discussion about what to do with them […], because if someone has not published a single article for six years, has no counselling achievements, and is not particularly liked by students, well, then you ask yourself why this person is still here with us

(Interviewee P22).

Similarly, in the case of the second private university (UniC), empirical materials show that the use of PM is mainly symbolic. Rational choices are focussed on decisions related to the hiring and dismissal of employees. According to one dean:

If one does not actively support the university, does not publish. What is the point of prolonging the contract, right?

(Interviewee P32).

Empirical evidence indicates that annual staff meetings, organised at UniC, have a rational purpose. Interviewees point to its twofold role. On the one hand, during staff meetings, the overview of the skills and achievements of academics is performed. The main purpose of this appears to be motivational, thus serving a rational use. At the same time, the symbolic dimension of those meetings is visible, as specific academics are publicly praised and honoured. The meeting is to celebrate distinguished researchers and lecturers. The following citation from one of the deans illustrates the dual role of annual staff meetings:

Meeting as we do today [annual staff meeting] is a perfect example of how we use that information […] so we meet and talk for several hours about what we have done, specific names are mentioned, and this is all used for internal propaganda. To motivate. It works very well

(Interviewee P26).

Similarly, the deans also indicated that PM is important for motivation purposes:

Whenever there is a person that publishes a lot, we mention him or her and say – you should compete with him or her, look at his or her articles – so we present those most active researchers

(Interviewee P26).

Each employee is discussed during meetings […] the talent is appreciated […] if he/she works for the benefit of the institution, it is praised

(Interviewee P25).

Rational use at the departmental level was rather limited in three (UniB, UniD, UniC) out of four universities. It was emphasised that informal information is mostly used. Employees usually know each other very well and know what their colleagues do. The head of the department can assess employees based on direct contact and observations. In the case of UniB:

At the department level, there exists informal knowledge of what and how one does […]; if someone is the boss for many years, he/she may have a qualitative view of their employees

(Interviewee P17).

At UniA, the empirical data illustrates both, the rational and symbolic use of PM at the departmental level. The rational use of PM is reflected in making relevant decisions by heads of departments, such as those relating to the equal allocation of organisational tasks:

Information on organisational work is considered by the heads of the departments while allocating tasks and university responsibilities […] so there is an equal, proportional distribution between the employees […] we do not want one person to do everything

(Interviewee P3).

In addition, only some interviewees from UniA indicated that the PM system is rationally used, not only to control the staff, but also for learning purposes. Through this system, they are informed about the research and achievements of scholars from other departments. Thanks to this, they can cooperate with each other if they conduct research in similar fields of study or exchange good experiences on their activities. Therefore, they learn how to develop research and improve outputs:

[…] and I use performance information, in the sense that we form some groups and we are working on a topic […] it made me realise that other people have similar scientific interests and they face the same problems as I do. […] Performance system support networking and, basically I found out that it is just great fun to put two or more people in the paper and conference presentations and that good things come out of it

(Interviewee P15).

The research office [at UniA] organises many meetings with employees who won grants [funding for research] and have been published in international journals, to learn from them and to encourage research work

(Interviewee P1).

At the individual level in the four universities, academics use PM for several rational purposes: to plan one’s own academic development (career), to increase their knowledge about their colleague’s work, to be more goal oriented, and to increase their motivation and satisfaction. Academics that choose to conduct research and seek to obtain higher academic degrees are obliged to collect information about their scientific achievements, as their assessment is the basis for awarding habilitation by the external state commission (CCADT). More specifically, one interviewee stated:

[…] the more measurable the scientific contribution, the better, because it is easier to put it later in the documentation for habilitation

(Interviewee P35).

Throughout the interviews, it was determined that the results obtained in teaching and research are often used in both, a symbolic and a rational manner. In all four universities, PM results are used to award outstanding research achievements and from this perspective, it explains the rational use of PM, even if the targets and awards differ among universities. However, the awarding process itself has a highly symbolic dimension, because they are recognised during annual ceremonies. In UniA, awards are given for articles published in journals on the ABS list or with an impact factor. At UniC, each publication is awarded during the aforementioned annual staff meetings. Similarly, the value of the award differs. At UniA, it is a monetary award, which depends on the position and income of the employee. At UniC, there is no monetary incentive. Nonetheless, in each university, there is a public ceremony, with the rector awarding the certificate, serving as the symbolic use of the PM. According to one interviewee:

It is clear that awards are an attempt to promote achievements and to set certain people as role models

(Interviewee P3).

At UniD, the assessment of researchers itself does not have a direct link to their financial bonuses. The motivational system is based on the use of information on obtaining further academic degrees and publications to reward employees. The Rector Award for scientific publications can be awarded to employees who have collected a minimum sum of points for all of the articles that they have published in scientific journals. According to some of the respondents, the Rector Award is purely symbolic. It is not a real attempt to motivate employees to make an effort and prepare high-quality papers that would have a chance of being published in international journals. More specifically, one interviewee said:

[…] satisfactory, good, very good or excellent grades do not significantly affect the level of remuneration of an employee, if there is no scientific promotion connected with it. Of course, employees who have outstanding achievements receive the Rector’s Award, which is perhaps some financial incentive. However, this is not important enough to motivate employees

(Interviewee P40).

At the level of the university authorities, in the case of UniA, respondents emphasised that the PM plays a strong promotional role for the university, both, with external and internal stakeholders. Thus, PM is used for symbolic, marketing purposes through dissemination via social media and internal newsletters. The aim of promoting good research is motivational, on the one hand, and informational, on the other. According to Interviewee P1:

We do it so that employees know who does what […] to motivate others

(Interviewee P1).

In all four universities, the use of PM, in a symbolic manner, was observed. PM is mainly to legitimise themselves in the eyes of various stakeholders, including employers, students, accreditation and ranking agencies, states, and mass media. Additionally, some faculty members are of the opinion that getting an award involves a certain level of prestige:

Of course, all of our accomplishments end up in the grantees

(Interviewee P15).

All the information on publications and research projects are used by the research office to put in upfront to the media (Interviewee P12). Among the people who aspire for a scientific prize, there is definitely some competition in this field […] my colleagues who receive these awards notice who has won the prize during the academic year, […] it is associated with certain prestige, which is of greater importance than any financial prize, for me

(Interviewee P35).

At the departmental level, the empirical evidence points to symbolic use. In the case of UniA, the heads of the departments use information on achievements to honour individuals and present them as role models to the other faculty members. On the other hand, the department leaders revealed, that there is a rational premise to use both, PM and final information, to motivate employees:

Heads of departments always point to outstanding individuals during meetings to appreciate and motivate

(Interviewee P10).

At the individual level, symbolic use was only found at UniA. There, interviewees emphasised on the role of PM in building their own personal brands, recognisability and their area of expertise within the university:

It is quite obvious, that if someone submits an article, and the article is accepted, then they would talk and “brag” about it

(Interviewee P1).

When someone publishes a lot in good journals, then he/she is perceived as great […] I know that I tried to publish in better journals because of this, and not because the university makes me do so

(Interviewee P13).

We have to present ourselves to each other, to exist in the consciousness, that this is what we do, and those are my topics of expertise

(Interviewee P15).

PM on teaching and research are closely analysed, especially in relation to the need to prepare it for external users (e.g. MSHE, national and international accreditation agencies) responsible for monitoring university performance and compiling and publishing the rankings of business schools. One of the respondents at one public university (UniB) highlighted that the use is mainly symbolic:

Maybe in private schools it happens that someone has invented such a system of management […] in order to make rational decisions […] It is not so here; our system has been imposed from the outside and we just have to apply it. It follows some formal regulations

(Interviewee P22).

Similarly, an interviewee at UniC emphasised that PM is mainly used for external reporting purposes:

There are certain indicators that we are obliged to, right? Those that we have to report to external [stakeholders]

(Interviewee P28).

The symbolic use of PM is visible, especially at UniC, as the role of performance information that is mainly limited to external reporting purposes. Respondents from UniC highlighted some criticalities (e.g. lack of standardised information, regulations, and proliferation of bureaucratic requirements) in relation to PM, both, in teaching and research activities. The empirical evidence did not reveal how often the quality of teaching is measured or what the minimum targets for publications are, as the received answers were conflicting:

6 publications, each 10 points minimum

(Interviewee P28).

2 publications, 10 points minimum

(Interviewee P30).

It used to be 20 points annually, but I think it has changed now

(Interviewee P26).

2 publications annually

(Interviewees P27; P32).

4 publications annually

(Interviewee P29).

One of the deans claimed that no target exists for publications:

There is no specified target […]. There is no minimum, no maximum […]. There is no need for them […] because it would not be proper not to publish at all

(Interviewee P25).

Thus, the use of the collection and the use of the PM at UniC seems to be, in large part, symbolic and dedicated to external users.

Despite the evidence of the use of PM internally, for both, rational and symbolic purposes, the empirical data points out the negative consequences of the system (Pop-Vasileva et al., 2011; Kallio and Kallio, 2014). The unintended consequences placed on individual scholars included more stress, conflicts, and time invested for self-assessments. This was also visible in our interviews. Such consequences were especially visible in the university in which the PM system was mostly developed and used both, symbolically and rationally (UniA). In remaining universities, the stress related to the PM system was also visible:

We know that if we stop publishing, then it will not end well for us […] we will not be attractive for the university anymore

(Interviewee P27).

I don’t know if it motives or demotivates, it for sure stresses, I can say that with confidence

(Interviewee P37).

The respondents presented a sceptical view towards certain measurements. Some academics believed that neither the number of publications, nor the rank of the journal, reflected performance. The actual contribution to science cannot be measured. Similarly, the rating of journals does not indicate how much effort the employees put into the preparation of a certain research article:

I work on an article, which is very difficult and it takes me a lot of time, and someone in the same time may publish 5 articles and it would take the same amount of time as this one difficult article took me. So, it is not comparable. I am not a fan of those journal points, either, It is imperfect, but at least it showcases whether the journal is demanding or not

(Interviewee P15).

When it comes to the students’ evaluation of the quality of teaching, a major concern of the interviewees was towards the unequal, incomparable number and structure of the students involved, as well as the lack of reliability:

I wouldn’t say that it [students’ surveys] measures my performance or quality of teaching […], rather, it measures their emotional attitude towards the lecturer

(Interviewee P11).

It still requires a lot of work, so, for example, the student’s surveys are reliable, it has to be ensured that not only the number of evaluators is adequate, but also the structure

(Interviewee P1).

PM was used across different levels in UniA in both, a symbolic and a rational way. Hence, interviewees were able to notice the potential drawbacks of the system and were able to question its adequacy. In the remaining researched universities (UniB, UniC and UniD), the interviewees did not raise any objections related to the adequacy of PMs and their use.

In conclusion, the implementation and use of PM systems in the four universities were clearly positive. There were also critical effects observed, both, at the organisational and the individual levels. The positive issues were related more to transparency and legitimacy, credibility, and image towards external users (e.g. state, accreditation agencies, employers, students, and mass media). It should be emphasised that interviewees had trouble distinguishing among specific public bodies that supervise and assess the academic activities (e.g. CCADT, ACG and CESU were referred to as the MSHE).

The use of PM has also had a positive impact on internal decision makers (e.g. rectors, deans, heads of departments, individuals and academics) in increasing the rationalisation of resources, learning about work activities, individual motivation, goal orientation and the introduction of incentive systems. Interviews also revealed critical effects (e.g. symbolic use of PM, data errors, data manipulations, proliferation of bureaucracy, more stress, myopia, less motivation, less collaboration and several critics of the system).