Citation
Sandes-Guimarães, L.V.d. and Hourneaux, F. (2024), "Editorial: Post-graduate programmes and their societal impact: reflections on the business and management field in Brazil", RAUSP Management Journal, Vol. 59 No. 1, pp. 2-6. https://doi.org/10.1108/RAUSP-03-2024-278
Publisher
:Emerald Publishing Limited
Copyright © 2024, Luisa Veras de Sandes-Guimarães and Flavio Hourneaux Junior.
License
Published in RAUSP Management Journal. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence maybe seen at http://creativecommons.org/licences/by/4.0/legalcode
Not everything that can be counted counts, and not everything that counts can be counted.
Cameron, W.B. (1969, p. 12)
Post-graduate programmes are the primary locus where research is produced in Brazil. These programmes have historically contributed to the advancement of knowledge and the training of highly skilled researchers. According to data from the Coordination for the Improvement of Higher Education Personnel (CAPES), collected on their platform Geocapes [1], the fields of Administration, Accounting, and Tourism conferred a total of 3,431 doctoral and 8,799 master’s degrees in academic programmes between 2017 and 2022. Regarding scientific production, researchers, faculty and students from these programmes have published roughly 50,000 articles in scholarly journals in the same period (Dados Abertos Capes [2]).
Although these outcomes cover the first and second missions of the university system (teaching and research), there is a missing link between these activities and their practical application for society. Bridging this gap would enhance the relevance and impact of post-graduate programmes, ensuring that research outcomes are effectively translated into real-world solutions and innovations. Hence, this editorial discusses the societal impact of post-graduate programmes, focusing on the field of Business and Management (B&M).
Post-graduate evaluation in Brazil
In an institutionalised format, post-graduate programmes in Brazil date back to the 1960s. Nevertheless, the post-graduate assessment system in Brazil was formed only in 1976, aiming to cultivate and establish high-quality standards for graduate programmes and scientific research within the nation. The first evaluation of post-graduate programmes carried out by peers, with the work of committees of consultants, took place in 1978, establishing the practice of peer evaluation as a fundamental component of the whole evaluation system (Marques, Veiga, & Borges, 2020).
Since its creation, the post-graduate evaluation process has undergone multiple revisions to enhance the evaluation procedures. In 1996–1997, CAPES sought the assistance of 18 foreign specialists to improve its evaluation system by obtaining recommendations for modifications. The uncovered challenges included local pressures from the scientific community, which led to inflated assessment marks. These facts ultimately showed that the system could no longer effectively differentiate between different levels of performance (Balbachevsky, 2005). Starting in 1998, CAPES implemented a notable modification to its evaluation approach by incorporating quality indicators to ensure comparability across programmes and areas, ultimately leading to the creation of the Qualis system for journal ranking.
The system evolved continuously, improving the evaluation methodology to promote the quality and excellence of post-graduate programmes in Brazil, not without criticisms from the scientific community. For a long time, the model raised the pressure to publish – worldly known as the “publish or perish” dilemma – making researchers less likely to generate research that could transform reality and people’s lives and more likely to develop research with incremental or less innovative theoretical contributions (Costa, Machado, & Câmara, 2022).
The current format, initiated in 2021, aims not only to evaluate the performance of post-graduate programmes in terms of the training of master’s and doctor’s and intellectual products but also includes an assessment of their impact on society [Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), 2021]. This aspect did not really occupy a significant portion of the programmes’ assessment until this last update (now 1/3 of the programmes’ score), which calls for more insights on concepts and methods for evaluation. Furthermore, as stressed by Costa et al. (2022), this shift in evaluation processes will pose difficulties for researchers who have mainly concentrated on scientific relevance throughout their careers.
To assist programme coordinators in understanding societal impact, a working group at CAPES produced a report on the subject, highlighting definitions, types of impact and evaluation formats based on a literature review [Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), 2019]. The assessment agency kept some evaluation fields open (without defining specific indicators or guidelines regarding what counts as impact) so that coordinators of post-graduate programmes could report relevant information that best represented the impact of their programmes in society. This perspective highlights the significance of recognising the programmes’ societal contributions, as emphasised in the evaluation report for CAPES’ Administration, Accounting, and Tourism area: “It is advisable to consider the trajectories that programs aim to pursue, surpassing the limited focus on scientific output guided by Qualis and emphasising excellence in people training, knowledge generation, and societal impact” [Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), 2022, p. 81].
Given the growing significance of societal impact in the CAPES evaluation process, we must discuss how to evaluate this societal contribution and collaboratively develop a model involving the academic community, CAPES and their several societal stakeholders.
The societal impact
Societal impact or research’s impact on society is not a “set in stone” concept. As highlighted in a previous editorial (Sandes-Guimarães & Hourneaux Junior, 2020), there are several definitions of research impact in the literature, reflecting different assumptions about what science is, scientific knowledge and what is its role in society (Greenhalgh, Jackson, Shaw, & Janamian, 2016). Overall, the whole concept of research impact on society revolves around the benefits, effects, influences or contributions that research (both its process and its result) can have for the different domains of society (Sandes-Guimarães et al., 2022). Recent studies in B&M highlight the importance of the research process, which also leads to changes in how people think about business practices and how they do them (Hughes, Davis, Robinson, & McKay, 2021; Śliwa & Kellard, 2022).
Although the emergence of societal impact in national evaluation systems is recent, with the UK’s Research Excellence Framework as a key precursor since 2011, the societal impact of research was already a concern of programmes and professors long before. In the context of the Brazilian national research assessment conducted by CAPES, Dantas (2004) long had already expressed concerns about the distancing of the academic programmes and societal needs, highlighting the role of the assessment process: “The areas’ evaluation reports remain silent about the association of lines of investigation with relevance for social and economic development, their alignment with national strategic objectives, or even about the innovation capacity of the programs, and non-traditional academic paths.” (p. 165).
Assessment systems play a huge role in defining a country’s academic research direction. Frishtak (2019), while talking about innovation (one of the ways academic research can reach society), states that Brazilian universities and research institutions have a relatively rigid and narrow incentive structure that does not reward collaboration with external actors, adopting an inward-looking system that hampers collaborative innovation. Despite laws and incentives designed to strengthen relationships between universities and the productive sector, research is mostly isolated from the wider society, even though much of it has practical and applied purposes (Schwartzman, 2022).
As emphasised before, CAPES is now starting to evaluate the societal impact of post-graduate programmes, which means, in the near future, selecting or developing a framework for this assessment. In another editorial (Hourneaux Junior & Sandes-Guimarães, 2020), we already discussed existing models and frameworks and their limitations. It is essential to mention that at least 20 frameworks are available for evaluating projects’ societal impact. Many countries, such as the UK, France, Netherlands, Australia and Norway, are implementing national assessment exercises that assess societal impact, drawing from these project-level frameworks and other experiences. All models, especially those associated with funding, have been criticised and improved with the collaboration of the scientific community.
In essence, these experiences demonstrate that evaluating the societal impact is a complex task, especially considering social fields (like B&M itself) that deal with people and organisation-based changes. Challenges to pursuing this societal impact assessment include the time lag between research and the outcome/impact; disciplinary differences, that is, the nature of knowledge in different areas, especially when evaluating interdisciplinary research; and the focus on comparability for funding allocation (McKenna, 2020). One of the most crucial challenges is evidencing (or measuring) this type of impact.
Two primary approaches have been devised to deal with this assessment dilemma. One is to devise elaborate measurement methods, which may not be suitable for every evaluation case; the other, more simplistic, emphasises collecting basic numerical information and a case study (Morton & Cook, 2022). While the measurement approach is attractive as it enables comparability, concentrating only on numerical data conceals the fundamental element of research activity that gives it its unique value – the individuals involved. As highlighted by Morton and Cook (2022), “It’s not possible to understand the difference you are making with numbers alone” (p. 104). Thus, models for societal impact assessment should incorporate both qualitative and quantitative data suitable for the given context to provide a comprehensive understanding of the effects of research activities on individuals and communities. Incorporating qualitative data allows for a more holistic evaluation of the research impact, capturing the human experiences and perspectives, something that may not be achieved through numerical data alone.
Notes
References
Balbachevsky, E. (2005). A pós-graduação no Brasil: novos desafios Para uma política bem-sucedida. In C. Brock & S. Schwartzman (Orgs), Os desafios da educação no Brasil, Rio de Janeiro: Nova Fronteira.
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) (2019). GT Impacto e Relevância Econômica e Social: Relatório Final de Atividades. Retrieved from www.gov.br/capes/pt-br/centrais-de-conteudo/2020-01-03-relatorio-gt-impacto-e-relevancia-economica-e-social-pdf
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) (2021). Portaria no 122, de 5 de Agosto de 2021. Consolida os parâmetros e os procedimentos gerais da Avaliação Quadrienal de Permanência da pós-graduação stricto sensu no Brasil. Retrieved from http://cad.capes.gov.br/ato-administrativo-detalhar?idAtoAdmElastic=6742
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) (2022). Relatório de Avaliação: Administração Pública e de Empresas, Ciências Contábeis e Turismo. Retrieved from www.gov.br/capes/pt-br/centrais-de-conteudo/documentos/avaliacao/19122022_RELATORIO_AVALIACAO_QUADRIENAL_comnotaAdministrao.pdf
Costa, F. J. D., Machado, M. A. V., & Câmara, S. F. (2022). Por uma orientação ao impacto societal da pós-graduação em administração no Brasil. Cadernos EBAPE.BR, 20(6), 823–835, 10.1590/1679-395120210222.
Dantas, F. (2004). Responsabilidade social e pós-graduação no Brasil: ideias Para (avali)ação. Revista Brasileira De Pós-Graduação, 1(2), 160–173, 10.21713/2358-2332.2004.v1.46.
Frischtak, C. (2019). Science and innovation in Brazil - where to now?. In E. B. Reynolds, B. R. Schneider & E. Zylberberg, (Eds.). Innovation in Brazil: Advancing Development in the 21st Century, London: Routledge. pp. 93–119.
Greenhalgh, T., Jackson, C., Shaw, S., & Janamian, T. (2016). Achieving research impact through co-creation in community-based health services: literature review and case study. The Milbank Quarterly, 94(2), 392–429, 10.1111/14680009.12197.
Hourneaux Junior, F., & Sandes-Guimarães, L. (2020). Editorial. RAUSP Management Journal, 55(4), 427–433, 10.1108/RAUSP-10-2020-227.
Hughes, H. P., Davis, M. C., Robinson, M. A., & McKay, A. (2021). The pursuit of organizational impact: hits, misses, and bouncing back. European Journal of Work and Organizational Psychology, 30(3), 339–354, 10.1080/1359432X.2020.1722219.
Marques, C., Veiga, A., & Borges, L. (2020). A avaliação da pós-graduação no Brasil: resultados e determinantes da avaliação da Capes (2013-2016). Revista Meta: Avaliação, 12(37), 979–1003, 10.22347/2175-2753v12i37.2825.
McKenna, H. P. (2020). Research Impact: Guidance on Advancement, Achievement and Assessment, Cham: Springer.
Morton, S., & Cook, A. (2022). How Do You Know If You Are Making a Difference? A Practical Handbook for Public Service Organisations, Bristol: Policy Press.
Sandes-Guimarães, L., & Hourneaux Junior, F. (2020). Research impact – what is it, after all? Editorial impact series part 1. RAUSP Management Journal, 55(3), 283–287, 10.1108/RAUSP-07-2020-202.
Schwartzman, S. (2022). Pesquisa e Pós-Graduação no Brasil: duas faces da mesma moeda? Estudos Avançados, 36(104), 227–254, 10.1590/s0103-4014.2022.36104.011.
Śliwa, M., & Kellard, N. (2022). The Research Impact Agenda: Navigating the Impact of Impact, Routledge.