Evaluating the impact of cross-institutional teaching enhancement collaborations using a professional capital framework

Abstract

Purpose

This study aimed to evaluate the impact of four cross-institutional teaching enhancement projects (TEPs), a relatively new form of professional collaboration. The focus is on the impact at departmental, institutional and cross-institutional levels because such impact is the main reason for establishing cross-institutional TEPs.

Design/methodology/approach

A professional capital framework guided the examination of decisional and social capitals at departmental, institutional and cross-institutional levels. A theory-of-change method was adopted to collect data from 35 sets of documents, 22 project members and 65 stakeholders.

Findings

The authors found five forms of impact, showing the development of decisional and social capitals mostly at institutional and cross-institutional levels, whilst signaling the relatively weak impact at departmental levels. Therefore, the values of cross-institutional TEPs have not been fully realized and future endeavors need to better utilize the capitals in programs.

Originality/value

Few studies evaluated the impact of large-scale, cross-institutional TEPs. The authors offered new contributions by gauging the impact of these under-explored forms of complex professional collaborations.

Citation

Zou, T.X.P., Hounsell, D., Parker, Q.A. and Chan, B.Y.B. (2024), "Evaluating the impact of cross-institutional teaching enhancement collaborations using a professional capital framework", Journal of Professional Capital and Community, Vol. 9 No. 1, pp. 68-82. https://doi.org/10.1108/JPCC-03-2023-0021

Introduction

With a growing demand for high-quality teaching in higher education, more academics undertake large-scale, cross-institutional projects to improve their teaching and create resources to support their colleagues within a university or across several (Davison et al., 2014). Following Bamber et al. (2009), teaching enhancement involves an intentional plan and actions to improve teaching. Cross-institutional teaching enhancement projects (TEPs) involve academics from more than one institution working on teaching enhancement (Turner et al., 2022) and tend to aim at fundamental changes rather than small improvement (Kember et al., 2019).

Cross-institutional collaborations have emerged as a strategy for solving complex problems in schools and universities (Azorín, 2019; Feys and Devos, 2014; Treleaven et al., 2012) because they potentially generate significant benefits that cannot be achieved by one institution (Azorín, 2019). Muijs et al. (2010) identify three such benefits: collective school improvement through data sharing and joint projects, broadening opportunities for the collaborating schools to serve the community and develop curricula, and resource sharing in professional development.

While cross-institutional collaborations are potentially valuable, they also encounter many barriers, including complicated coordination required among institutions, lack of incentives for teachers, and extra time and resources needed (Azorín, 2019). Azorín (2019) further points out that a specific challenge of cross-institutional collaboration is to effectively connect social capital (e.g. relationships between institutions) with human and decisional capitals (e.g. collective teacher development and curriculum design).

Feys and Devos (2014) describe cross-institutional collaborations as “the new orthodoxy” (p. 738), implying that institutions are sometimes pressured into collaborations by policymakers. In the higher education sector, policymakers expect cross-institutional TEPs to exert systemic influences on teaching and learning (Kember et al., 2019). The continuous investment by the Hong Kong government on TEPs in the past ten years reflects such expectations (UGC, 2022). However, Hargreaves (2019) cautions that not all collaborations are fruitful and that “contrived collegiality”, denoting unnecessary bureaucratic arrangements forcing teachers into meetings, is counter-productive.

Given the potential value of cross-institutional TEPs, we argue it would be worthwhile investigating their impact, especially the impact beyond the scope of a project. This is because broad, systemic impact is the main reason for establishing cross-institutional TEPs. However, connecting TEPs to tangible outcomes, such as improvement of teaching practice or student learning, is difficult (Miller-Young and Poth, 2022). Such connections are particularly challenging in universities where academics enjoy a high level of autonomy, making it difficult to reach a consensus on teaching matters (Bromage, 2007). Moreover, participating in TEPs seldom contributes directly to one's career progression because TEPs usually result in course development rather than research outputs; the latter are more valued (Willcoxson et al., 2011). What adds to the complexity is that cross-institutional TEPs involve negotiations across institutional boundaries (e.g. Zou et al., 2022; Treleaven et al., 2012). Leaders of these TEPs have to resolve the tensions between academics' autonomy and accountability to the funding body (Willcoxson et al., 2011).

This study investigated the impact of four government-funded, cross-institutional TEPs, aiming to help bridge the gap between an increasing amount of investment in these projects and the hitherto under-explored dimension of their impact. It is part of a larger study examining the motivations, processes and outcomes of TEPs (e.g. Zou et al., 2022). The four projects under study focused on educational themes that align broadly with promoting student-centered learning. Our focus is on the impact at departmental, institutional and cross-institutional levels because it is less researched and more significant compared with individual-level impact.

Literature review

The professional capital framework

Scholars frame professional collaborations with various concepts, such as communities of practice (Scanlan et al., 2016; Wenger, 1998), professional learning communities (Vescio et al., 2008) and collaborative inquiry networks (Pino-Yancovic et al., 2022). Despite operational differences, all these concepts involve teachers interacting with one another to enhance their teaching practices and build a collaborative culture (Washington and O'Connor, 2020). The term “collaborative professionalism”, coined by the Ontario Ministry of Education (2016), captures the essence of collaborations as “professionals–at all levels of the education system–working together, sharing knowledge, skills and experience to improve student achievement and well-being of both students and staff” (as cited in Hargreaves, 2019).

The professional capital framework (Hargreaves and Fullan, 2013) is useful for evaluating professional collaborations (e.g. Osmond-Johnson and Fuhrmann, 2022; Pino-Yancovic et al., 2022). Specifically, human capital depicts individuals' quality, comprising their knowledge, skills, qualifications and commitment. Decisional capital refers to teachers' professional judgment developed through experiences and learning. Social capital, positioned as the most critical capital among the three, is possessed by a group of teachers as a result of their networking, mutual support, shared learning and trust (Hargreaves and Fullan, 2013).

While human capital resides within an individual, decisional capital can be located within and between individuals as teachers develop solutions to complex issues (Chapman et al., 2016). Decisional capital strengthens individual capital as individuals learn from the decision-making process and enhance their competencies and confidence. Social capital exists in relationships between individuals, which bind the other two capitals together and create conditions for their development (Osmond-Johnson and Fuhrmann, 2022).

Cross-institutional TEPs as professional collaborations

Cross-institutional TEPs as a relatively new form of professional collaboration involve academics in different institutions working on teaching enhancement (Willcoxson et al., 2011). Examples include the projects funded by the Australian government office for learning and teaching (Department of Education, 2001) and by the Hong Kong government (from which, the projects under study were selected). Kember et al. (2019) identified the Hong Kong teaching and learning project scheme as the largest scheme they were aware of. The recent schemes emphasized collaboration and the funding amount for a project increased with more institutions involved. The funding cap for a single-institution project is HK$2.5m (∼US$0.32m), whilst it goes up to HK$15m (∼US$1.9m) when four or more institutions are involved (UGC, 2022).

Cross-institutional TEPs are expected to impact more broadly on teaching and learning practices at departmental, institutional and cross-institutional levels (Willcoxson et al., 2011). Possible impact includes systemic changes in curriculum design (Kember et al., 2019; Turner et al., 2022) and teaching innovations that potentially transform existing approaches (Treleaven et al., 2012). Such impact might be achieved through synergies between faculty members learning together in communities of practice and networks (Wenger, 1998; Zou et al., 2022), experimentation of new pedagogies across contexts (Pharo et al., 2012), collective reflective discussions (Kember et al., 2019) and co-development of new curricula and resources (Willcoxson et al., 2011).

Only a few studies examined the impact of cross-institutional TEPs. Green and Whitsed (2013) studied an Australian national teaching fellowship project involving two universities. Discussions on course designs and teaching practices were launched. In Green and Whitsed's findings, human capital increased as academics developed a multifaceted understanding of the pedagogy. The development of decisional capital was shown in the work-in-progress of its implementation in courses. Social capital was most evident as many small-scale communities of practice had been formed around the new pedagogy.

Pharo et al. (2012) evaluated the impact of a cross-institutional TEP on building capacity for interdisciplinary teaching through bringing teachers together to discuss their practices. The impact included improving an existing climate change course in one university and developing a new course in another, showing evidence of human and decisional capital. Social capital, including trust, tolerance and mutual respect, was accumulated through meetings, developing teaching materials, co-teaching and working with students (Pharo et al., 2012).

Despite the positive results, it is noteworthy that systemic changes in curriculum design, expected as desirable outcomes of cross-institutional TEPs, are seldom reported. One main reason is that sustaining collaborations needed to incorporate the project outcomes into curriculum design is difficult for cross-institutional TEPs. Willcoxson et al. (2011) investigated a six-university teaching collaboration and found that the partnering universities hesitated to allocate additional funding and make more commitments to building new structures (Willcoxson et al., 2011). In contrast, sustainability seems to be achievable for some small-scale TEPs. Amundsen and D'Amico (2019) use the “ripple effect” to denote the impact of small-scale TEPs beyond the pre-determined project scope and timeframe. For example, leaders of TEPs mentored colleagues informally after the project completion (Amundsen and D'Amico, 2019).

Methodology

Research design

This study aimed to investigate the impact of four cross-institutional TEPs based on a professional capital framework. Our focus was on the impact beyond individual development or the projects themselves. The research questions we attempted to address were:

RQ1.

What decisional and social capitals at departmental, institutional and cross-institutional levels have been generated through government-funded, cross-institutional TEPs?

RQ2.

How are the capitals at departmental, institutional and cross-institutional levels generated and how do they contribute to teaching enhancement?

Recognizing that the concept of “impact” is contingent upon interpretations, this study was situated in the interpretivist paradigm that views reality as constructed by meanings individuals and groups attribute to events (Cohen et al., 2011). Thus, we sought to understand how project members and stakeholders made sense of their project experiences.

Study context

In Hong Kong, publicly-funded universities are required to provide high-quality teaching as their most important mission (UGC, 2010). Accordingly, University Grants Committee (UGC) invests in teaching development grants, traditionally small-scale projects within an institution, to support teaching innovations. However, academics in Hong Kong increasingly find teaching less rewarding than research, because the latter often leads to better career progression (Thomas et al., 2011). In view of this, UGC launched large-scale, cross-institutional TEPs to incentivize universities and academics to promote teaching innovations and sector-wise improvement for three triennia: 2012–15, 2016–19 and 2022–25.

The projects

The four projects under study were selected from 38 projects on the 2016–19 scheme. We attempted to select projects we could access, yet we also included one that none of us were involved in to obtain an outsider's perspective. Specifically, three of the authors were involved in the first three projects, respectively. The fourth project was not related to any of us. Our reflexibility is explained in the sub-section “Researchers' reflexivity”. The four projects focused on different themes aligned broadly with the government's vision of a shift from an examination-oriented culture towards student-centered learning. The amount of funding for these projects ranged from HK$3.3m (∼US$0.38m) to HK$8.9m (∼US$1.1m).

We considered literal replication and theoretical replication principles (cf. Yin, 2014). A literal replication means selecting similar cases to collect confirmatory evidence (Yin, 2014). In our study, the four projects are similar in the sense that they adhered to the same funding rules and adopted similar processes, which meant that we would find similar types of impact across projects. A theoretical replication means selecting cases with contrasting features so that the findings will be predictably different (Yin, 2014). In our study, each project had an enhancement theme, meaning that we would find different examples to illustrate the impact accordingly.

• Project 1 (P1) promoted the internationalization of the curriculum by engaging academics in incorporating international, intercultural and global perspectives into their courses. The main project activities were designing and implementing professional development activities, hosting small-scale action research projects and cultivating a community of practice.

• Project 2 (P2) promoted interdisciplinary teaching through building an educational program that applied esthetic and scientific principles to explore the cultural meanings associated with Chinese arts, artifacts and antiquities residing in and passing through Hong Kong. It involved the development of a common core course (i.e. an interdisciplinary course open to all undergraduates) and subsequently, the expansion of the approach to several programs. The main project activities included course development and adapting the pedagogical design to a wider pool of curricula.

• Project 3 (P3) aimed to develop students' innovation mindset through providing a physical collaborative working space for students to create and test prototypes. A first-year cornerstone course was developed that integrated the “makerspace”, the subject learning and innovative mindset development. The main project activities included designing the course and applying the approach to other related courses.

• Project 4 (P4) promoted the design and application of gamification to enhance students' global perspectives and intercultural competence. It engaged students from different backgrounds to tackle real-world challenges through tournaments. The main project activities included designing the pedagogy, organizing professional development and supporting academics to apply the approach in their courses.

Researchers' reflexivity

With three authors' dual roles as practitioners and researchers in the first three projects, we applied reflexivity following the guidance of Berger (2015); being insiders provided a better access to the project documents, whilst switching between insider and outsider perspectives helped enhance the credibility. We also avoided conflict of interest by dedicating an author not involved in a particular project to conduct the interviews related to that project. Only when we conducted cross-project analyses did all authors get involved. Meanwhile, we were vigilant about the tendency to look for positive impact by constantly searching for disconfirming evidence in the data, including concerns and difficulties expressed by interviewees, as a validity check procedure (Cohen et al., 2011).

Data collection

A theory-of-change method investigates how a change happens because of the intervention (Amundsen and D'Amico, 2019). It has been used in researching the impact of interventions in educational contexts where the relationships between the interventions and outcomes are nonlinear (e.g. Amundsen and D'Amico, 2019; Miller-Young and Poth, 2022).

We found a theory-of-change method suitable because it accommodates multiple pathways that link TEPs to the outcomes and allows the emergence of unanticipated outcomes (cf. Miller-Young and Poth, 2022). The four phases (Amundsen and D'Amico, 2019) are explained as follows. Ethical approval was obtained from the institution where the first author was based (No.: EA1704009). Informed consent was collected from all participants.

1. Phase 1. Articulating the underpinning theories (Figure 1): We conceptualized cross-institutional TEPs as professional collaboration and recognized the embedded action research approach. Specifically, the projects align with the institution's teaching vision and are conducted by a team with regular meetings. They potentially benefit student learning, stimulate changes in teaching practices and generate reflections (cf. Kember and McKay, 1996). We also considered the unique elements in cross-institutional TEPs and expected the impact would be generated at departmental, institutional and cross-institutional levels (cf. Kember et al., 2019).

2. Phase 2. Identifying sources of evidence: Following the pathways, we suggest the sources of evidence include document reviews and interviews with project leaders, team members, and teacher and student stakeholders.

3. Phase 3. Collecting evidence: we collected and analyzed the following data (Table 1):

   • Project documents: meeting minutes, progress and final reports, and curriculum documentation (35 documents in total). The project leaders were invited to provide documents that contained rich information about the project impact. Progress and final reports were standard documents of every project, whilst the curriculum documentation and project dissemination documents varied according to the project processes.

   • Interviews with project leaders and members (n = 22) and stakeholders (n = 65). Project leaders and members were those who were on the project team, whilst stakeholders included academics and students that potentially benefited from the project but were not involved in the project execution. All project leaders and members in the four teams, 33 people in total, were invited to participate in one-on-one, semistructured interviews and 22 agreed (response rate: 67.7%). The contact of teacher stakeholders was obtained from the project leaders. The student stakeholders were randomly selected from those who enrolled in the courses developed as a part of the project outcomes.

4. Phase 4. Examined the impact: The analysis is presented in the following subsection.

Data analysis

Data analysis followed “template analysis” (King, 2004) a deductive analysis method. We drafted an initial coding template based on our identification of possible forms of decisional capital (e.g. teaching innovations) and social capital (e.g. awareness of the value of collaboration), which were applied to data segments across documents and interview transcripts. Where meanings in the data could not be embraced by the template, we updated the template by adding new codes or refining the existing ones. Five forms of impact denoting different capitals were outlined and the new forms of impact distilled from the data included increased interest in teaching, ripple effect and societal impact. At the same time, we did not have sufficient evidence for the expected enhanced status of teaching or systemic changes in curriculum design. Finally, we examined how different forms of capital were generated.

To ensure the analysis quality, we performed an inter-rater reliability check. Two researchers independently coded the impact by assigning each relevant data segment to one form of impact. The inter-rater reliability was 79%, meaning the two researchers made the same judgment on 60 out of 76 occasions. Disagreement was resolved by discussion.

Findings

Five forms of impact, denoting different capitals, were identified at the institutional and cross-institutional levels (Table 2). Each form of the impact is explained in the following subsections supported by documentary evidence and interview excerpts.

1. Raised awareness of the value of collaboration

All the project members and teacher stakeholders interviewed expressed a high level of appreciation for collaborations brought by the projects and showed enthusiasm for building a collaborative culture, as revealed in the following excerpt.

It is all to do with the collaborative spirit. How you convince people; how you engage them; how you build a community of collaborators. It is through sharing of great ideas. (Team leader, P2)

According to several project members, the raised awareness of the value of collaboration was shared among project stakeholders.

Since the project leader exercised good leadership that brought people together, different participants in our partnering institutions are able to see the value of this collaboration. Some of them apply the approach in their own teaching. (Team member, P4)

The interviews showed that the raised awareness was largely a result of the project leaders' intentional cultivation of social capital. Both the project leaders and members affirmed the importance of establishing meaningful collaborations that are valuable to faculty members who were occupied by heavy research and teaching duties.

The raised awareness of collaboration was positioned at institutional and cross-institutional levels because such collaborations were reflected on multiple occasions as collaborations among institutions rather than units within them. We observed through the document review that logos representing the partnering institutions were put in eye-catching places on the project websites. When teacher stakeholders mentioned the project, they also tended to refer to it as collaboration between universities.

Meanwhile, it is noteworthy that there were many counter-arguments in the interviews of team members that did not reflect a high spirit of cross-institutional collaboration. One P3 team member mentioned:

We have multiple collaborating institutions. For [University X], the interaction is not frequent and we are not that close. I am not sure how they have been doing. (Team member, P3)

Another interviewee pointed out that collaborations between departments within one institution were difficult.

At university, unfortunately, we have specialized deeply in certain areas and collaborations between departments have become much less positively accepted by departments and faculties. (Team member, P2)

Teacher stakeholders also acknowledged the contradiction between the project being honored as cross-institutional collaborations and the impact being limited to a small group. The quote below illustrates this point:

What we do is not shared by others in this university. It would be difficult for anyone outside this community to appreciate or even understand our mission. It is good to get to know someone in another university also working on this, but still, we are just a small community. (Teacher stakeholder, P1)

1. Teaching innovations and enhancement

In all projects, evidence was collected on teaching innovations and enhancement achieved through the project teams creating new courses or working with teacher stakeholders. Here, we only present a few examples beyond an individual, meaning that they influenced teaching and learning across the university.

The most prominent example was found in P3, where the new course developed was offered to the entire cohort of approximately 800 first-year students annually. Student stakeholders exhibited appreciation for the new course:

In other courses, you learn something, but you cannot apply it. For makerspace, I learn something and I can use it in future. It is a safe space and I can access different materials and make experiments …. I have never thought about this before. (Student stakeholder, P3)

The documentary evidence from Project 2 showed that a new course was developed based on the theme of interdisciplinary learning, which was offered to students from all disciplines and further recognized as an exemplar for interdisciplinary teaching. Students of different backgrounds recognized the distinctiveness of the course, illustrated by the excerpt below:

The course provides a different angle to study familiar subjects, and as a mix of arts and science as an interconnected entity. I have not experienced this in any other courses. (Student stakeholder, P2)

In P4, the gamification design was extended to an e-learning platform for developing intercultural capabilities for all students:

Students from different cultures have used the platforms …. Some teachers not involved the project also tried this approach. (Team member, P4)

These innovations and enhancement were achieved through the development of a model or a platform applied to multiple disciplines. It is noteworthy that only the innovations in P3 were incorporated into the curriculum. Others remained project-based during this research, meaning there is no evidence that they could lead to any systemic changes in the long term.

1. Increased interest in teaching

Increased interest in teaching was found as a type of decisional capital at institutional and cross-institutional levels. For example, a P2 member who previously focused on research made the following comment.

We brought students [of the new course] to the laboratory. A colleague devoted her expertise to teaching them what materials they could use to do analysis … I feel the passion from students. They ask questions and bring ideas.… This is the way we learn from them. I really like working with students. (Team member, P2)

Similar narratives were collected from teacher stakeholders across the four projects. A P1 teacher stakeholder highlighted that her increased interest came from connecting with other teachers.

I see the value in a teacher from the faculty of medicine hearing what is happening in business. The same applies to those from the same discipline getting connected from different institutions. We need more different perspectives in our teaching. (Teacher stakeholder, P1)

The increased interest in teaching was also observed among those who simply attended project events. Documents in all cases recorded the feedback of participants in the project events. An example reads: “The conference was well organized. It provided an excellent community for discussing teaching and learning ….Many of the ideas I got can be applied to my courses.” (Teacher stakeholder, P1).

The increased interest in teaching seemed to occur because of the accumulated visibility of the social capital from the public events. These events made some academics curious about what had happened. As evidenced from the document review, all four projects had a total of eighteen public events, four of which were online because of the social distance measures during the COVID-19 pandemic. To take one example, the international conference organized in P1 had recorded over 400 attendees from 28 countries. A P1 teacher stakeholder was surprised to find colleagues discussing teaching:

It was surprising to me that many academic colleagues are thinking about improving their teaching. Nowadays, the pressure to publish is enormous, but we cannot ignore students (Teacher stakeholder, P1)

However, it would be overly optimistic to assume that an increased interest in teaching would lead an enhanced status of teaching. The expected enhanced status of teaching was not evident mainly because of the lack of sustainability. Several teacher stakeholders involved in the projects admitted in the interviews that they might not continue the new teaching approach after the project due to a lack of resources and uncertainties about students' reactions.

Another limitation of this impact is that it only applied to those who attended the project events, while many academics seldom attend any teaching-related activities. The interviews with teacher stakeholders helped reveal the resistance to teaching enhancement:

I found I am the only person working on it in my department.… Am I right that we are supposed to do this? It is on the university website that our education is global. (Teacher stakeholder, P1)

1. Ripple effect

Ripple effect describes the outcomes amplified in new practices, cultures and structures beyond the pre-determined scope. One form of ripple effect was new collaborations invoked based on the project achievements. In P3 where the new course developed became part of the undergraduate curriculum, new collaborations were invoked:

We delivered the concept [of makerspace] at several conferences and some Korean schools became interested. Now, we have formed a partnership and bring our students to join their innovation festival. (Team leader, P3)

In other projects, the ripple effect was found through project teams initiating new projects and extending the project ideas, which encompassed the development of both decisional and social capitals beyond the project scope. For example,

The findings we got from interdisciplinary teaching and learning were actually put into new teaching development grants.… One of the ideas was to get scientists and engineers to come together. There are a lot of pedagogical innovations starting from there. (Team leader, P2)

Ripple effect was also developed through new collaborators attracted by the materials developed in the project as evident in the document review. In some cases, these collaborations resulted in an exchange of ideas; in others, new grants were secured by the project team and the collaborators.

Another form of ripple effect was identified through teacher stakeholders generating new projects, embedding decisional and social capitals. This is evident by more teachers assuming leadership in teaching and learning partially as a result of participating in cross-institutional TEPs. One teacher stakeholder of P1 started a new teaching project:

Participating in this project gives me concrete ideas of what we can achieve. I use this experience as a reference for my own project. But my project is different. We only invite people within one discipline.… It [note: the community developed in P1] was too large and people might not have a sense of belonging. In my project, I tried to build a tighter community. (Teacher stakeholder, P1)

This example illustrated a reinforcing relationship among capitals. The social capital nurtured the human capital among teacher stakeholders, who then transformed it into tangible outcomes as a form of decisional capital. The outcomes further result in more substantial social capital. However, the data showed that most new collaborations were project-based, leaving questions about their impact on departmental and institutional teaching development unanswered.

1. Societal impact

In P4, the team collaborated with an international nongovernment organization to promote students' global learning. Analysis of the project reports showed that one learning activity, an international contest, attracted 416 students participating online from 42 countries, which went beyond the original scope of the project targeting Hong Kong's higher education. Moreover, this activity also covered several of the United Nations' sustainable development goals. One conference presentation file showed positive feedback from student participants. This societal impact was highly appreciated among Project 4 team members and described by the project leader as “something definitely heart-warming” (Team leader, P4). Although societal impact was not yet identified in other projects that did not involve stakeholders outside the university, Projects 2 and 3 showed an intention and some plans to extend the models to secondary schools.

Discussion

Main findings

This study has delineated the impact of four cross-institutional TEPs at institutional and cross-institutional levels. Regarding our first research question on what capitals have been generated, we found decisional and social capitals in five forms of impact: (1) raised awareness of the value of collaboration; (2) teaching innovations and enhancement; (3) increased interest; (4) ripple effect and (5) societal impact. Our main contribution includes featuring impact within and across institutions of a kind not commonly found in within-institutional projects while also signaling the weak departmental-level impact. Moreover, we found limited systemic changes and identified the under-explored impact such as ripple effect and societal impact.

Regarding the second research question on how the capitals are generated, we found a prominent role of social capital as many other capitals were developed through relationship building. The central role of social capital has been emphasized (e.g. Hargreaves and Fullan, 2013), but most studies frame social capital as the relationships between individuals in a community (e.g. peer interactions in Osmond-Johnson and Fuhrmann, 2022). In the context of cross-institutional TEPs, we found that social capital can be amplified through ripple effect between collaborating teams (e.g. new collaborations formed between the original TEP team and other teams). In the process, the decisional capital was also developed as these teams engaged in new teaching enhancement.

However, systemic changes in curricula were limited, with only one example in P3. An even greater concern was the relatively weak impact discernible at a departmental level. For example, there were few instances documented of departmental teaching and learning committees leveraging the teaching innovations or enhancement resulting from the projects. Not being able to engage the majority of academics in teaching enhancement has been a long-standing issue. Our findings indicated that cross-institutional projects offered partial solutions to this problem, but they are unlikely to prove attractive to those who do not participate in teaching enhancement events.

The findings exhibit mixed results in terms of whether “collaborative professionalism” has been established. Solidarity and solidity were identified, but it would be difficult to suggest how sustainable they are. “Collaborative professionalism” will lead to a stronger professional (Hargreaves and O'Connor, 2018). While increased interest in teaching and raised awareness of the value of collaboration were evident, the relatively weak impact generated at a departmental level implies that many capitals have not been valued or transformed into new structures, practices or cultures.

In addition, societal impact was rarely discussed in the teaching enhancement literature. This might be because teaching is largely perceived as classroom activities on the campus. Given an increasing need to connect university learning with the community, this finding might add additional insights, for example, regarding the involvement of community partners (cf. Hinojosa-Pareja and García-Cano, 2023).

While discussing how worthwhile cross-institutional TEPs are seems crucial, our capacity to offer a straightforward answer is necessarily limited. First, the complex interrelationships between the project activities and the impact do not lend themselves to the application of conventional linear yardsticks. Nonetheless, the evidence gathered concerning the value of the underpinning theories are useful outcomes in themselves. Secondly, although we did not carry out a longitudinal study to validate any such claim, it seems reasonable to suggest that cross-institutional TEPs offer a welcome reminder of the ever-present need to support efforts to improve the quality of curricula, teaching and learning.

Implications

This study has three main implications. First, we provide implications on what constitutes as evidence for the impact of cross-institutional TEPs. The ambiguities and contrasting narratives found in our study imply that future impact studies need to triangulate multiple data sources and attend to possibly different interpretations. Second, cross-institutional TEPs seem meaningful in view of such impacts as ripple effect and societal impact. While some of these impacts are also found in within-institutional projects, certain impacts are more likely achieved through cross-institutional TEPs through public events. However, these projects did not seem to overcome the institutional rigidity reported in Hinojosa-Pareja and García-Cano (2023) since their outcomes are not fully incorporated into the institutional structure or transformed into practices more broadly. More pathways are needed for the project to impact on the departmental system. For example, it might be helpful to promote the project outcomes to departmental leaders and program directors alongside cross-institutional activities. Third, some contrasting evidence about collaboration at cross-institutional levels we observed implies that project leaders of cross-institutional TEPs need to better synergize collaboration at different levels.

Limitations

Several limitations need to be acknowledged. One is related to social desirability and selection biases given a natural tendency to focus on the positives. To minimize these biases, we triangulated our findings and allocated the data collection and analysis to the one without any conflict of interest in the author team. Nonetheless, we admit that we could not tell whether nonrespondents might have answered differently. Finally, we have no intention to claim that the findings could be straightforwardly extrapolated across the larger pool of 38 projects funded through the same scheme. Rather, we hope to show the opportunities and challenges for cross-institutional TEPs to be meaningful professional collaborations.

Conclusions

This study evaluated the impact of four large-scale, cross-institutional TEPs as a form of professional collaboration. We found many encouraging instances that these projects had “made a difference” in helping to advance the quality of curricula, teaching and learning. We also gathered evidence of the pivotal role of cross-institutional collaborations that draw participants together. Nonetheless, we also recorded the relatively weak take-up of the fruits of the projects' work by the wider pools of departmental colleagues at least within the necessarily time-frame available to us and similarly, a lack of strategic engagement at an institutional level with sustainability. While contextual factors can never be entirely ruled out, the formidable challenges of the status of teaching relative to research, as of the readiness of many academics to engage with pedagogical change, are not confined to a specific milieu but systemic and global. What our findings perhaps also suggest, then, is that enhancement of teaching quality, if it is to succeed at scale, will need much greater targeted resourcing by all the stakeholders.

Figures

Figure 1

A theory-of-change model featuring impact generation pathways and sources of evidence

Table 1

Data collection

Projecta	Number of universities in the collaboration	Data
		Document (the number in brackets indicates the number of documents)	Semistructured interview (the number in brackets indicates the number of interviews)
P1: Promoting internationalization of the curriculum	4	- Project reports (3)	- Project members (6)
		- Project disseminationsb (10)	- Teacher stakeholders (25)
P2: Promoting interdisciplinary teaching and learning	2	- Project reports (1) - Project disseminationsb (1) - Curricular documentsc (6)	- Project members (8) - Student stakeholdersd (30)
P3: Developing students' innovation mindsets	3	- Project reports (1) - Project disseminationsb (3) - Curricular documentsc (1)	- Project members (5) - Student stakeholders (8)
P4: Enhancing students' global perspectives and intercultural competence	4	- Project reports (3)	- Project members (3)
		- Project disseminationsb (6)	- Teacher stakeholders (2)

Note(s): ^aThe projects were re-named to ensure confidentiality. ^bProject disseminations refer to such documents as conference presentations, websites and leaflets. ^cCurricular documents include course syllabi showing teaching innovation/enhancement. ^dBecause of the COVID-19 social distancing regulations, the student stakeholder data in P2 were collected electronically, resulting in more responses than expected

Source(s): Table 1 is created by the authors

Table 2

Findings: decisional and social capitals at multiple levels

	Decisional capital	Social capital
Departmental	Possible areas for future development
Institutional	(1) Raised awareness of the value of collaboration	(3) Increased interest in teaching
Cross-institutional	(2) Teaching innovations and enhancement	(5) Societal impact
	(4) Ripple effect

Source(s): Table 2 is created by the authors