Holistic evaluation of a team-lecture hybrid (TLH) instructional design applied in a public affairs course

Abstract

Purpose

This uses quantitative and qualitative methods in assessing performance and process outcomes in a team lecture hybrid (TLH) instructional design applied in a public affairs course.

Design/methodology/approach

Within a non-experimental prospective design, individual and team outcomes were assessed as follows: survey of student perceptions of learning outcomes; comparison of individual and group scores on in-class case analyses using paired t-tests; external reviewers' observations of traditional lecture versus TLH activities; and analysis of students' reflections on team dynamics using a team process reflection tool adapted from four team development stages.

Findings

The following student learning outcomes results were observed: increased use of critical thinking; higher student interaction with other students and the instructor; higher student engagement in initiating or contributing to content or other learning activities; higher student enthusiasm; increased use of problem-solving skills; improved performance evidenced by quality of individual versus group products; evidence suggesting improvements in student learning outcomes when active learners and an active instructor interact in a learning environment.

Practical implications

Instructor practice tips were provided in the following areas: use of assessment methods; student engagement as an active instructor; motivational tips for classes with students from a variety of disciplines; and individual team member accountability.

Originality/value

This paper contributes to the scholarship of teaching and learning (SOTL) by addressing limitations in both traditional and collaborative learning models and expanding holistic evaluations in SOTL.

Citation

Tataw, D.B. (2023), "Holistic evaluation of a team-lecture hybrid (TLH) instructional design applied in a public affairs course", Journal of Research in Innovative Teaching & Learning, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/JRIT-01-2023-0007

Introduction

The past half century has seen the growth and evolution in social learning models such as small groups and team based learning (TBL with a focus on overcoming the learning challenges in traditional models such as lectures. Growing out of constructivists and social learning models (Bruner, 1966; Gold, 2001; Hodgson and Watland, 2004; Piaget, 1971; Vygotsky, 1978); collaborative learning strategies, depart from objectivist approaches dominant in traditional learning environments (Jonnassen, 1991; Gokhan, 2012). Traditional learning models have been mainly faulted for placing limitations on the learner's participation in the learning process, while collaborative models, particularly TBL, have been principally criticized for lack of learner accountability and for limiting the role of the teacher (Feichtner and Davis, 1985; Fink, 2002; Tataw, 2014; Ahmed, 2021). This paper addresses limitations in both traditional and collaborative models of learning mentioned above through the team lecture hybrid (TLH) instructional strategy; and reduces the dearth of holistic evaluations in the scholarship of teaching and learning (SOTL) by combing a variety of process and outcomes evaluation strategies.

The TLH instructional strategy contributes to the SOTL, by integrating traditional learning strategies and collaborative learning strategies; bringing together the active learner and the active instructor in one learning community; and making both team-based work and lectures the primary and permanent activities of the semester (Tataw, 2014). TLH was implemented in 15 health administration, leadership and public affairs courses from 2011 to 2016 (Tataw, 2014, 2022, 2023).

This paper also adds to the body of SOTL evidence from holistic evaluation of instructional designs. Four distinct quantitative and qualitative methods are used to holistically assess process and performance outcomes in a TLH instructional strategy applied in a public affairs course at a Mid-Western university in the USA in Fall 2011. This is the only TLH courses in which a holistic evaluation was implemented.

Performance and process outcomes were evaluated as follows: assessment of student perceptions of learning outcomes using an end of course outcomes survey; comparison of traditional lecture versus team-lecture activities for the same material and same students using external reviewers' observations; comparison of individual and group performance scores for in-class case analysis using paired t-tests; and a review of students' reflections on team dynamics during the semester using a team process reflection tool adapted from team development stages. The following student outcomes were assessed: increased use of critical thinking; higher student interaction with other students and the instructor; higher student engagement in initiating or contributing to content or other learning activities; higher student enthusiasm; increased use of problem-solving skills; and improved performance evidenced by quality of individual versus group products.

The emerging patterns of evidence across methods, measures and instruments should clarify the impact of TLH activities on expected student learning outcomes and the aspired or experienced impact of the active instructor.

Literature review

TLH student learning communities

The main learning units in the TLH instructional strategy are permanent teams (Tataw, 2014), which maintain stable membership throughout the semester. As student learning communities (SLCs), permanent teams in TLH instructional strategies are important structures in which collaborative learning occurs. SLCs provide students with a structured way to solve problems, share insight and help one another to develop new skills and expertise and advance dynamic collaborative learning (Fisher et al., 2021). A basic learning community includes people, resources, rituals, norms, dependency and ties, as well as nodes and holes within different communities (Ozturk and Ozcinar, 2013; Vesely et al., 2007), which could exist as a single community (Becket et al., 2012); or multiple/sub-set of communities in a network structure (Maddix, 2013). In the course evaluated for this study, students belong to two learning communities, the permanent team of 5–7 members and the entire class of 36 students.

Objectivist and collaborative learning strategies

TLH combines collaborative and traditional learning approaches. Collaborative learning strategies such as team learning, small groups, and TBL are traceable to constructivists and social learning models (Bruner, 1966; Gold, 2001; Hodgson and Watland, 2004; Piaget, 1971; Vygotsky, 1978), which are improvements from objectivist approaches dominant in traditional learning environments such as lectures. In traditional objectivist approaches, the instructor prescribes the boundaries of knowledge and the learner is a passive recipient (Jonnassen, 1991; Gokhan, 2012). Constructivists see an active learner and the teacher as a mediator and an organizer of the learning environment (Vytgotsky's, 1978; Mingyuan and Fanhua, 2001). Social learning models including small groups, see learning as a social activity (Gold, 2001; Hodgson and Watland, 2004).

As a traditional learning strategy, lectures enable learning through the physical presence of an experienced and skilled expert, who engages the audience in the lecture, and captures their curiosity (Pale, 2013; Mccabe and O'Connor, 2014). However, some aspects of lectures impede the learning and teaching process when compared to collaborative approaches and TLH as follows: synchronicity or schedule conflicts (Ghenghesh and Nakhla, 2011; Petrovic and Pale, 2015); limited ability to accommodate variability in learning styles (Coffield et al., 2004); use of prior knowledge (Finley, 1985); and limited knowledge assessment strategies which exclude formative assessment (Roskos and Neuman, 2012), self-assessment (Taras, 2010) and peer assessment (Lu and Law, 2012).

TBL, as an advanced collaborative strategy, makes small group work the primary in-class activity and calls for activities that support the transformation of newly formed groups into high-performance learning teams, and usually requires a change in the structure of the team and transforms teamwork from a technique to a strategy (Fink, 2002; Michaelsen and Sweet, 2008; Michaelsen et al., 2008; Parmelee et al., 2012). TBL and other collaborative learning environments advance student learning outcomes such as cognitive, inter-professional and team skills (Michaelsen et al., 1993; Haidet et al., 2002); positive student attitudes and motivation (Gomez et al., 2010; McInerney and Fink, 2003); as well as critical thinking, creativity, and innovation (Almuqrin et al., 2020). In general, team learning contributes to overall improved academic performance and relationships in the learning community when compared to traditional learning (Carpenter et al., 2022; Cremerius et al., 2021; Eskeen, 2019; Venter, 2019).

Some drawbacks in TBL designs and other small group learning environments have been noted in the literature (Whillier et al., 2021), including lack of individual student accountability. and scheduling problems particularly with working adults (Feichtner and Davis, 1985; Fink, 2002; Ahmed, 2021). Other students desire active leadership from the teacher such as clarifying content, mentoring, and class management; in addition to greater accountability from other team members (Tataw, 2014).

The team-lecture hybrid (TLH) instructional strategy in in-class permanent teams

TLH builds on collaborative learning strategies such as TBL and objectivist approaches such as lectures and relies on an integrated framework that combines elements of objectivist, constructivists, and social learning approaches in in-class permanent teams. In TBL designs, team activities are the main class activities, whereas, in TLH designs, the lecture, and the instructor's engagement with students, occupy as much space as team-activities. Also, the team membership is permanent throughout the semester. Team members work together on weekly class activities and on semester long projects.

TLH relies on the active teacher, who departs from the dominant role of the traditional teacher in objectivist approaches yet is more engaged than the often-passive facilitator role in constructivists and social learning models. The active instructor retains a leadership role in the instructional environment (Ausubel, 1978; Vygotsky, 1978; Pale, 2013) even as he/she enlarges the empowerment zone for students. The active instructor is the student resource person for individuals and teams, and the facilitator, mentor, and guardian of both the team process, team learning, and active learning (Tataw, 2014).

In response to students concerns about individual member accountability in team-oriented strategies such as TBL (Feichtner and Davis, 1985; Fink, 2002; Ahmed, 2021), TLH designs provide for student accountability through graded individual assignments that precede in-class team activities and intra-group assessments which contribute to individual scores in semester long group projects. The individual assignments cover the same subject matter as the lecture and in-class permanent team activities (Tataw, 2014).

Program intervention elements

Instructional components

In TLH, learning and assessment flexibility and the use of prior knowledge were met through four instructional components including preparation, application, assessment and lecture adapted with modifications from Team-based Learning literature (Fink, 2002). Full details of the intervention and variations in implementation are described elsewhere (Tataw, 2014). A summary of the intervention elements is presented below.

The preparation component was made up of two activities: (1) Team formation where the instructor assigned students to different groups to minimize barriers to group cohesiveness and distribute member resources evenly. Teams stay together and work together on weekly activities and semester projects throughout the semester. (2) Student preparation which required students to individually complete weekly readings and assignments before weekly class activities and semester long group term project activities.

Application occurred during weekly in-class team activities covering weekly group assignments and term-project activities as follows: (1) Use of individual responses from the preparation phase above to create a group response to the weekly assignments in the class; (2) Collectively work on group term projects in the class; (3) Inter-group peer review of parts or entire group projects as they are completed or become due.

Assessment was usually ongoing throughout the semester and involved students, instructors, and external reviewers. Student roles included the following: (1) Inter-group peer review of group projects as described in the preparation section above, (2) Intra-group peer review of group member participation in group projects and/or weekly group work by completing an evaluation of every other team member of the permanent teams at the end of the semester; (3) Completing an end of course survey instrument assessing student perceptions of the impact of the TLH design on their learning; (4) Participation in individual and team reflections on the team dynamics of the TLH experience; (5) External reviewers' observations which compared TLH learning sessions and traditional lectures where applicable.

The instructor role included the following activities: facilitate and assess class participation; grade exams; grade group projects; review portfolios; grade weekly group assignments as needed; grade student weekly individual assignments; and facilitate process evaluation using nominal techniques.

The lecture components came before or after in-class teamwork depending on the nature of the subject matter covered or the specific needs of the learners during a specific class session. Pre-teamwork lectures occurred after the students have completed their individual weekly assignments but before they engaged in teamwork activities. Post-teamwork lectures happened after the team-work, and integrated matters arising from team discussions. During the lecture the teacher presents some basic course information that might not have been emphasized or properly explained in the student self-directed learning activities and ties loose ends that might not be obvious during teamwork sessions.

The course evaluated in this study, is the only course of all fifteen courses implemented in TLH, in which a holistic evaluation was implemented using four of the assessment strategies mentioned above. Other classes were evaluated using one or two of the assessment strategies depending on context or opportunities to implement the evaluation strategy. The paper reports the results of the analysis of survey data, team process assessment, external reviewers' observations, and individual versus team performance scores.

Expected learner outcomes in the TLH instructional strategy

Participants in the TLH program were expected to achieve the following student learning outcomes:

1. Increased use of critical thinking,

2. Higher student interaction with other students and the instructor,

3. Higher student engagement in initiating or contributing to content or other learning activities,

4. Higher student enthusiasm,

5. Increased use of problem-solving skills and

6. Improved performance evidenced by quality of individual versus group products (Tataw, 2014).

Hypotheses

H1.

Participants in permanent teams will report improvements in all six TLH expected student learning outcomes

H2.

External reviewers will report improvements in TLH expected student learning outcomes when performance in a traditional setting is compared to team lecture setting.

H3.

Emerging themes from team process reflections will align with TLH expected student learning outcomes

H4.

Team scores will be higher than average individual scores in weekly in-class case analysis

Methods

Design

This study analyzes data from a non-experimental prospective evaluation of individual and team performance in a TLH in-class permanent teams using a TLH student end of course outcomes survey instrument, individual and group scores on in-class case analyses, external reviewers' observations and student reflections on the team processes. The study relied on qualitative and quantitative data, and a combination of inductive and deductive analysis.

The implementation and evaluation of TLH design was approved by the Institutional Review Board, a Mid-Western US institution as an expedited study in 2011.

Sample selection for evaluation purposes

All students in each class under study were recruited to participate in course evaluation activities. Students were recruited directly from the class by representatives of the instructor and provided a written or verbal consent in the absence of the instructor. Program participation was obligatory for all students since program activities were part of the course. Participation in evaluation activities was voluntary.

Data collection and measures

Evaluation activities included the following measures and data collection activities: assessment of overall student outcomes using a post only end of course outcomes survey; comparison of traditional lecture versus team-based activities for the same material and same students using external reviewers' observations; comparison of individual and group performance scores on in-class case analysis; and a review of student team process reflections.

1. The students end of course outcomes survey instrument was a ten-item survey covering five expected student learning outcomes areas including use of critical thinking, student interaction with other students and the instructor, student engagement in initiating or contributing to content or other learning activities, student enthusiasm, problem solving skills, and better use of the course subject matter to provide answers to real life problems. In five questions, students were asked to rate their experience in team-learning compared to traditional sessions on a Likert scale of 1–5. (1) being I strongly disagree and (5) being I strongly agree. Three of the questions were open ended allowing the students to comment on their experiences in in-class permanent teams as a whole and during inter-group peer review of term projects in particular. Demographic data was also collected including gender, ethnicity, age, and discipline.

2. Excel spread sheets were used to collect data on individual and group performance which was used to compare individual and group scores in 13 in-class case analyses.

3. Three external reviewers visited both the traditional and team-based learning sessions of each class under study and conducted a guided review of both the traditional and team-lecture sessions of the class. The external reviewers' observations questionnaire was an open-ended set of guidelines based on five expected TLH outcome areas including critical thinking, student interaction, student engagement, student enthusiasm, and student use of problem-solving skills. There was also one unstructured question that allowed reviewers to document any other relevant observation.

4. A Team Process Reflection Tool adapted from the four team development stages was used to facilitate team process evaluation. Students reflected individually, then as a team on the team development process including forming, storming, norming, and performing. Three open-ended questions asked participants what the team did well during the semester, what they could have done better, what additional support the instructor could have given the team process or groups.

Data analysis

Quantitative analysis

Responses to end of course student outcomes survey and student grades on 13 in-class case analysis were entered in an excel spreadsheet and descriptive statistics were calculated for all demographic variables, for survey questions, and for student individual and group grades on weekly case analysis. A paired t-test was also conducted to see if there is a significant difference between the individual and group scores on weekly case analyses.

Qualitative analysis

External reviewers' observations and student reflections on team dynamics were qualitatively analyzed through interpretive thematic analysis using content, categorical, and frame analysis of student perceptions of team dynamics and external reviewers' observations of student performance during traditional versus TLH. Interpretive thematic analysis together with inductive and deductive analysis were used to categorize and interpret responses to structured open-ended questions on the team process reflection tool and the external reviewers' guidelines. Here emerging themes and frequency of responses from external reviewers' and student responses were structured around assessment instrument topics. Responses that did not fall under any of the instrument topics were summarized and grouped into other emerging themes. Emerging thematic categories from both team process reflections and external reviewers' observations were matched with expected TLH student learning outcomes to determine if they aligned with, or advanced program outcomes.

Results

Mixed data results

Survey results included responses to open ended questions and descriptive statistics of responses to structured survey questions. Both descriptive and paired t-test statistics were reported on comparisons of individual and group scores on weekly case analysis.

Survey data

Table 1 below presents detailed quantitative results of the TLH end of course outcomes survey which is tied to five TLH student expected outcomes. Of 36 enrolled students, twenty-three students completed end of course outcomes survey. The survey sample had the following gender and racial/ethnic distribution: female (65%), male (35%), African American (8.7%), Asian (0%), Latino (0%), White (91.3%). Age distribution was as follows: 18–24 (60.9%), 25–35 (26.1%), 36–46(13%), and 47–64(0%). In addition, the disciplinary distribution by declared majors was as follows: criminal justice (56.5%), health administration (8.7%), informatics (8.7%), psychology (4.4%), public management (12.11%), and sociology (8.7%).

In their responses to survey questions asking them to compare traditional to TLH learning sessions, students reported having invariably better outcomes during TLH learning sessions than in traditional session.

Three responses to one open ended question asking students to share any other thoughts they have about their experience in TLH during this semester showed that students both enjoyed and learned from TLH as s follows:

1. It was a valuable learning experience

2. Really enjoyed group work

3. Liked it

Table 2 below presents descriptive statistics of individual scores compared to teams in 13 in-class case analysis made up of 36 students and six teams of five–seven students. The results show that team scores consistently exceeded individual scores and no individual score exceeded team scores.

The results for the paired-t test are shown in Table 3 below. There is evidence of a difference between the individual and group test scores (p-value < 0.001). Within about 95% confidence it is estimated that average scores are between 2.2 and 3.5 points higher when assessed in groups.

Qualitative data

External reviewer observations

This section describes the consolidated results from three external reviewers' observations of student activities on Tuesdays, when the traditional/individual sessions were held and Thursday, when the TLH sessions were held. The common themes and statements which emerged are listed according to questions on the external reviewer guidelines. Overall, reviewers observed better student outcomes on Thursdays when compared to Tuesdays.

1. Critical thinking. How the student engages in critical thinking?

Common themes: On Thursdays, students exhibited higher levels of critical thinking through synthesis of material, practical application of content, thoughtful discussions, and interactions.

Statements:

• ∙

  Limited engagement in critical thinking on Tuesday versus Thursday when students combined individual answers in coherent and thoughtful manner

• ∙

  Tuesday students appeared unable to absorb the material versus Thursday were there was better understanding after meeting in groups

• ∙

  I did not observe students asking questions on Tuesday but they did respond to questions posed by instructor on Thursday

• ∙

  On Thursday students assessed and applied course content to current events

• ∙

  On Tuesday, some students responded to questions without thought to their implicit assumptions

• ∙

  On Thursday, some students presented thoughtful opposing views though they did not explain the basis for their theory

• ∙

  On Tuesday there was little student engagement, while on Thursday there were interactions and discussions with more analysis, though discussion were brief.

1. Interaction. How the student participates in the learning process by interacting with other students and the instructor.

Common themes: On Thursdays students interacted more among themselves and with the instructor probably due to a more structured interactive environment and being more prepared.

Statements:

• ∙

  Tuesday students were reluctant to interact with the instructor and each other and did not seem attentive versus Thursday when students cooperated with each other, interaction was structured and desired responses were easily forthcoming

• ∙

  Tuesday students had a difficult time interacting with instructor versus Thursday when students interacted more with the instructor and each other in groups perhaps because of better preparation

• ∙

  On Tuesdays, fewer students interacted with instructor and other students than on Thursday

1. Student engagement. How the student initiates or contributes to content or other learning activity.

Common themes: More student engagement on Thursdays with greater student-initiated contributions and responsiveness to instructor, with a demonstration of greater understanding of material.

Statements:

• ∙

  Tuesday, student-initiated contributions were limited and when required, minimum response was given, versus Thursday when answers to instructor's queries were more readily volunteered and answers begin to connect conceptual models to personal experiences

• ∙

  Less engagement on Tuesday than Thursday. Students who engaged among themselves and with instructor on Thursday, and those who engaged with instructor on Tuesday showed a good understanding of the material.

• ∙

  On Tuesday, compared to Thursday, few students contributed to the discussion and there was very limited engagement

1. Student enthusiasm. How are the students interested in the subject matter or learning process?

Common themes: Students showed greater enthusiasm on Thursday perhaps due to higher expectations of participation and hands on application of material.

Statements:

• ∙

  Tuesday students appeared disinterested and engaged in laptops and cell phones versus Thursdays when student participation significantly increased, perhaps due to higher expectations of participation

• ∙

  Enthusiasm on Thursday was greater than Tuesday, in part because of the hands-on application of material

• ∙

  On Tuesday, compared to Thursday, very few students were interested in the subject matter, and some engaged in personal conversations

1. Other. Any other relevant observation.

Common themes: Some students lacked an understanding of the benefit of the course to their education and future careers. Explaining the benefit of the course to the different disciplines represented in the class would be helpful.

Statements:

• ∙

  On both Tuesday and Thursday, some students and groups seem disengaged and did not think the material was important. Perplexing that a student planning to be a probation officer said the class was a waste of time.

• ∙

  Classroom was too large for the number of students

• ∙

  Course relevance to some majors. Some students need to understand the benefit of this course to their education and future

Team process reflection

Participants in team process reflections included 36 students in six teams. Fifty-three statements were recorded, and the frequencies ranged from 7% for forming and storming domains to 26% for what did your group do well domain.

Domains and emergent themes

A detailed description of domains and emergent themes in the team process reflection responses is provided in this section.

1. Forming, included four statements or 7% of total statements recorded for the study, including two thematic categories distributed as follows: team formation and rationale (75%), and social process of team formation (25%). The sub-categories in each major thematic category were as follows:

Group A. Team Formation Rationale −3.

• ∙

  Worked together or acquainted before team formed −2

• ∙

  Individuals took the lead and invited other members −1

Group B. Social Process of Team Formation −1.

• ∙

  We set operational roles 1

1. Storming, encompassing four statements or 7% of total statements recorded in the study. Two thematic categories emerging from four statements were distributed as follows: easily resolved conflict (75%), and communication (25%). The sub-categories of each thematic category are presented below:

Group A. Easily Resolved Conflict−3.

• ∙

  Sort differences quickly 2

• ∙

  Had similar ideas 1

Group B. Communication−1.

• ∙

  Read each other's papers 1

1. Norm, included 13 or 25.0% of all statements recorded in the study. One thematic category emerging from 13 statements was distributed as follows: set rules and standards (100%). The sub-categories of each thematic category are presented below:

Group A. Set Rules and Standards−13.

• ∙

  Set routine roles, responsibilities, and procedures 7

• ∙

  No rules or regulations 6

1. Performing, included five statements or 9% of total statements collected for the study.

One thematic category emerged from five statements and was distributed as follows: Good Performance (100%). The sub-categories of each thematic category are presented below:

Group A. Good Performance−5.

• ∙

  Like-minded performed efficiently−1

• ∙

  All people's ideas were considered−1

• ∙

  Assigned roles and leadership-moderator, secretary, editor etc.−1

• ∙

  Answered all questions 1

• ∙

  Finished group work quickly−1

• 5. What did your group do well? Responses included- 14 statements or 26% of all statements recorded for the study. Three thematic categories emerging from 14 statements were as follows: Democratic Participation (29%), Communication (42%), and Collaboration and commitment (29%). The sub-categories of these thematic categories are presented below:

Group A. Democratic Participation−4.

• ∙

  Everyone took part in the discussion 3

• ∙

  Looked at every point of view 1

Group B. Communication−6.

• ∙

  Brainstormed on all questions 2

• ∙

  Communicated Well −4

Group C. Collaboration and Commitment−4.

• ∙

  Worked well together−2

• ∙

  Performed well when called upon−2

• 6. What could your group have done better? This included- five statements or 9% of all statements recorded for the study. Three thematic categories emerging from five statements were as follows: Democratic Participation (40%), Commitment, preparation, and readiness (40%), and other (20%). The sub-categories of these thematic categories are presented below:

Group A. Commitment, Preparation, Readiness−2.

• ∙

  Better attendance 1

• ∙

  Could be more prepared, read chapters 1

Group B. Democratic Participation−2.

• ∙

  Greater diversity for more discussion

• ∙

  Discuss answers a litter longer

Group C. Other.

• ∙

  No improvement needed

• 7. What additional support could your instructor have given the team process or groups? This included eight statements or 15% of all statements recorded for the study. Three thematic categories emerging from eight statements or 15% of total statements recorded for the study were as follows: Productive resource and support (12%), Communication (50%), and Preparation and lecture (38%). The sub-categories of these thematic categories are presented below:

Group A. Productive Resource and Support−1.

• ∙

  The support we received was great

Group B. Communications−4.

• ∙

  Instructor told us exactly what he wanted done for case studies

• ∙

  Should specify and remind us on chapter due one class before

• ∙

  More information early

• ∙

  Consistent with instructions

Group C. Preparedness and lecture 3.

• ∙

  Have lecture on computer much earlier

• ∙

  Be more toward center of the room during lecture

• ∙

  Post resources much earlier, like weeks in advance

Discussion

Summary

In this paper, quantitative and qualitative methods were used to understand process and performance outcomes among permanent team members in a TLH instructional strategy applied in a Fall 2011 public affairs class at a US Mid-Western university. Results of both qualitative and quantitative evaluations supported study hypotheses and confirmed the achievement of TLH expected student learning outcomes. Results showed students doing better in TLH learning environments when compared to traditional learning environments in all measures of the TLH design expected outcomes. Results also seem to suggest improved student performance when active learners are combined in the same learning community as an active instructor.

Convergence of patterns

There is a convergence of positive findings across evaluation methods, strategies, and instruments in support of the study hypotheses and TLH expected student learning outcomes. Participants invariably performed better in TLH contexts compared to traditional contexts of learning. There was also congruence within results in the strategy such as quantitative and open-ended responses pointing to improved student outcomes in student learning environments. In addition, there was inter-rater agreement among external reviewers; and between all external reviewers' observations and student assessments of team dynamics which found some teams having only limited engagements and some academic majors not understanding the benefit of the public affairs course to their education or career. These findings suggest reliability in the measures, methods and results and a likelihood that the TLH intervention elements contributed to observed variations between traditional class sessions and TLH learning sessions.

Achievement of TLH outcomes

Both qualitative and quantitative findings showed or suggested that TLH expected outcomes where achieved.

Quantitative findings and support of TLH outcomes

Survey data and individual versus team performance scores comparisons supported both the study hypothesis and TLH expected outcomes. Teams scores consistently exceeded individual scores and no individual scores exceeded team scores, and students invariably reported better outcomes during TLH learning sessions than in traditional sessions.

Qualitative findings and alignment with TLH outcomes

External observers findings and alignment to TLH outcomes

Summaries of emerging themes from external reviewers' observations, align with TLH expected outcomes showing improvements on Thursday (TLH sessions) compared to Tuesday (traditional sessions) in the following four expected TLH outcomes: critical thinking; interactions with other students and the instructor, student initiates or contributes to content; greater enthusiasm. In addition, taken together, external reviewers' observations showed higher group performance when compared to individual performance in every comparative category tied to TLH outcomes. Reviewers also noted a more structured environment on Thursdays than Tuesdays, perhaps due to the active instructor.

Emerging themes from team process assessment and alignment to TLH student learning outcomes

The thematic categories emerging from student reflections on team dynamics align with TLH expected student learning outcomes as follows:

In the forming domain, both team formation rationale and social process of team formation thematic categories represent readiness to perform and are aligned with two TLH outcomes including: (1) interactions with other students as students worked together, and (2) higher enthusiasm observed in factors such as students taking the lead and inviting other members to get to work.

In the storming domain, elements in two thematic categories including easily resolved conflict and communication, aligned with one expected TLH outcome including increased problem-solving skills seen in factors related to the quick resolution of differences and having similar ideas.

In the norming domain, factors in the thematic category set rules and standards, aligned with one expected TLH outcome including increased use of problem-solving skills which is observed in the setting of norms which create a good context for problem solving and the setting of clear expectations.

Elements in the thematic category of good performance in the performing domain, are aligned to the following TLH outcomes: (1) improved performance seen in the timely completion of tasks; and (2) increased interactions with other students observed in all questions being answered and the accommodation of other people's ideas.

Emergent themes from responses to the question What did your team do well, encompass factors in three thematic categories including democratic participation, communication, collaboration and commitment, which are aligned to the following TLH expected outcomes: (1) increased use of critical thinking observed in robust discussions and looking at every point of view; (2) higher interactions with other students observed in brainstorming activities; and (3) use of problem-solving skills evident in reports of working well together.

Responses to the question what your group could had done better, had four thematic categories including commitment, preparation, communication and readiness; and democratic participation related to factors that were barriers to problem solving observed in the lack of diversity in discussions; and barriers to performance seen in poor attendance at meetings and unpreparedness.

Responses to the question what additional support your instructor could provide, had three thematic categories including productive resource and supporter, communication, preparedness and lecture which relate to matters of clarity, consistency, support and early sharing of resources that affect student performance on all expected TLH outcomes.

The suggested and demonstrated links between TLH activities and expected student learning outcomes in this study, are consistent with findings in other studies which suggest associations between student learning outcomes and collaborative learning such as cognitive, inter-professional, and team skills (Michaelsen et al., 1993; Haidet et al., 2002); improved positive student attitudes and motivation (Gomez et al., 2010; McInerney and Fink, 2003); critical thinking, creativity and innovation (Almuqrin et al., 2020); and overall improved academic performance and relationships in the learning community when compared to traditional learning (Carpenter et al., 2022; Cremerius et al., 2021; Eskeen, 2019; Venter, 2019).

Increased instructor engagement is recognized and aspired across data collection points. Responses to survey questions show perceived improvements in interactions with other students and instructor when TLH learning environments are compared to traditional learning environments. Students' reflections on team dynamics revealing reticence and lack of understanding from students, as well celebration of instructor support, suggest recognition and aspiration for an active instructor. In addition, the reports of external observers affirm greater instructor engagement and a more structured environment in TLH learning sessions on Thursdays when compared to traditional learning sessions on Tuesdays. These findings appear to support the TLH design theory anticipating improved performance when we combine active learners and an active instructor in the same learning community (Tataw, 2014). This also partly reflects the expert teacher who captures his/her audience during lectures (Pale, 2013; Mccabe and O'Connor, 2014).

Some limited dissatisfaction is observed in this study, in relation to levels of team member commitments and students being disengaged during classes. These findings are consistent with complains of lack of student accountability and conflicting schedules in the literature on lectures (Ghenghesh and Nakhla, 2011; Petrovic and Pale, 2015), and team learning (Feichtner and Davis, 1985; Fink, 2002; Ahmed, 2021).

Study limitations and future research

The main limitation to the findings of this study is the reliance on self-report in the end of course survey which usually comes with subjectivity. There is also, the absence of baseline data and comparison or control groups. However, these weaknesses are mitigated by the following: the diversity of the contexts in which micro learning took place; the inherent control in the design as traditional lectures occurred alongside TLH design activities, and the diversity of participants who reported perceptions of improvements in student learning outcomes as both participants and external reviewers. The results are further strengthened by consistency of findings across methods, measures and raters. Besides, voices of students should still be included in student outcomes assessments (Kareen, 2021).

Nevertheless, future iterations of the TLH should include control or comparison groups and should include baseline data in the assessment process.

Conclusion and practice implications

This study assessed the impact of a TLH instructional design on student learning outcomes in a public affairs course. The study findings support the study hypothesis and confirm the achievement of six expected learning outcomes. Further, the evaluation results appear to support TLH design theory anticipating an active instructor contributing to improvements in student learning outcomes.

Practice implications

Four lessons for practice are noted:

1. The success of the four assessment methods in this study in measuring the expected outcomes of the TLH design, and the fact that data from each assessment method validated results from all other three methods suggest that instructors can successfully use any of the assessment methods in isolation or in combination with the others.

2. Further, instructors should consider increasing the magnitude of their engagement with students even in TLH environments, because the active instructor did not diminish team performance and the study results suggest that the active instructor might have contributed to improved individual and team performance.

3. In addition, when instructors are teaching general education courses or courses with students from a variety of disciplines, it might be beneficial to explain to students how the courses are related to their declared majors or careers. That might increase interest in the class activities.

4. Explain factors enhancing individual student accountability such as the use of intra-group assessments scores to determine individual scores in team projects which usually carry a common score for all students. This should diminish perceptions of free loading on the part of other team members.

Table 1

Descriptive statistics (count and percentage) for outcomes survey questions

Variable	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
Q6: When compared to the traditional lecture session, I experienced more critical thinking during Team-Lecture Hybrid (TLH) Learning sessions	1 (4.4%)	0 (0%)	3 (13%)	8 (34.8%)	11 (47.8%)
Q7: Compared to the traditional lecture session, I had more interactions with other students and the instructor during team-lecture hybrid (TLH) Learning sessions	1 (4.4%)	0 (0%)	3 (13%)	4 (18.2%)	14 (63.6%)
Q8: Compared to the traditional lecture session, I contributed more to class learning content and other learning during team-lecture hybrid learning sessions	1 (4.4%)	0 (0%)	2 (8.7%)	12 (52.2%)	8 (34.8%)
Q9: Compared to the traditional lecture sessions, I was enthusiastic in the subject matter or learning process during Team-Lecture Hybrid learning sessions	1 (4.4%)	0 (0%)	2 (8.7%)	12 (52.2%)	8 (34.8%)
Q10: Compared to the traditional lecture sessions, I made better use of the course subject matter to provide answers to real life problems during the Team-Lecture Hybrid learning sessions	0 (0%)	0 (0%)	0 (0%)	10 (43.5%)	13 (56.5%)

Source(s): Table by author

Table 2

Individual versus team scores in in-class case analyses

Weekly individual versus team performance on in-class case analyses for 36 students in 6 teams (group A to F)
Student no.	Individual average score	Group average score	Student no.	Individual average score	Group average score
1	18	20	21	14	17
2	16	20	22	18	17
3	15	20	23	15	17
4	15	20	24	13	17
5	14	20	25	14	17
6	13	20	26	11	17
Group. A average score	15	20	27	17	17
7	16	18	Group. D average score	15	17
8	16	18	28	18	20
9	15	18	29	16	20
10	17	18	30	16	20
11	12	18	31	16	20
12	12	18	32	18	20
13	13	18	Group E average score	17	20
Group. B average score	14	18	31	14	18
14	17	17	32	15	18
15	17	17	33	15	18
16	16	17	34	16	18
17	15	17	35	16	18
18	15	17	36	16	18
19	15	17	Group. F average score	15	18
20	18	17
Group. C average score	16	17

Source(s): Table by author

Table 3

Paired t-test results for individual versus group performance

Test
Null Hypothesis	H0: μ -difference = 0
Alternative hypothesis	H1: μ - difference = 0
T-value	8.83
p-value	0.000
Estimation of paired differences
CI	95%
Mean	2.868
St. Dev	2.002
Mean	0.325
μ _ difference	(2.210,3.527)

Note(s): μ _ difference population mean of (group-individual)

Source(s): Table by author