Exploratory study of academic excellence associated with persistence in ODL setting

Introduction

This paper is the reinforcement of previous study on the relatively similar structure with quite different respondents and also with modified attributes primarily in the seven foremost variables as previously introduced by Sembiring (2016). The issues however are still tightly relatable to the students’ persistence and attrition by pondering to Tinto (1997) and Bean (1983). Students’ persistence, including loyalty (Kunanusorn and Puttawong, 2015), is an integral part of satisfaction as identified by Brown (2006). Satisfaction in academic excellence is also frequently attached to the service quality (Parasuraman et al., 1988; Arokiasamy and Abdullah, 2012). These conceptions, including in educational sectors, and more specifically in higher education context, are widely adopted (Petruzzellis et al., 2006; Rojaz-Mendez et al., 2009; Bharwana et al., 2013). These efforts are essential since many students strived to pursuing their degree and in fact mostly ineffectual to persist (Roberts and Styron, 2009). It just happens so as the service provided and delivered are understandably below the standard and also below of the students’ expectation. These perceptions are plausibly relevant in the open distance learning (ODL) environment (Sembiring, 2014).

All credentials explained previously are firmly valid to the environment of Universitas Terbuka as well (Sembiring, 2015). The university is now insistent of sustaining the size and growth of the student body reflecting on the gradual decrease of student numbers in recent years (for illustration, take a look the following number of students for 2011-2015: 446,326, 415,030, 353,193, 333,501, and 309,508 consecutively). If no quantifiable leap forward is cautiously put into operations, it is envisaged that the student body for 2016-2020/2021 would predictably be decreasing as 297,372, 277,814, 251,095, 220,743, and 193,099, respectively (Universitas Terbuka, 2016). These figures are clearly in disharmony with the strategic plan of the university (Universitas Terbuka, 2014). The student body in 2020/2021 is expected not less than 250 thousand students in total to sustain the existence.

This situation therefore leads us to probe the more fundamental inquiries: is it because of many students graduated, is it a result of less new student registered or is it due to the vast majority of students do not re-register in a consecutive semester? If the latter is the case, it is then a question of students’ persistence that associated with satisfaction. In this study, it is affiliated with the so-called academic excellence questions.

The main objective of the study is therefore to assess academic excellence, by pondering to the work of Moore and Kearsly (2015), through all potential related variables and dimensions as they were expected and experienced by the students of Universitas Terbuka registered in 2016 and especially for those are domiciled overseas. It is also of interest to reveal crossing details between satisfaction along with persistence, loyalty and future careers in ODL perspectives. Relevant answers to these questions are related to an effort of maintaining the size and growth of the student body so that services provided by the university will converge to as many students’ expectation as possible. Having disclosed these answers, the university will then be able to envisage the associated efforts with respect to assuring better and more appropriate services perceived from student outlooks and for the sake of students’ successes.

Conceptually, the exploratory framework of the study starts with the general perspective of ODL outlook. This is the basis of Universitas Terbuka to provide services in relations to academic excellence that was associated with satisfaction and finally leads to persistence, loyalty and career advancement behold by overseas students (Figure 1).

The operational framework

The conceptual or exploratory framework (Figure 1) is then used as a tool of weighing up satisfaction and its inference noticed from academic excellence perspectives. This would let the university to modify the important aspects of operations to accommodate students’ need and expectation. It might focus on the institutional directions to accomplish the students’ need so that the university is able to maintain and make progress on the size and growth of student body as previously projected and officially stated in the operational plan document of the university. In other words, this is the way on how the university is searching for proper and adequate approach and orientation to maintain the student body.

Before introducing an operational framework, it is worth perceiving that academic excellence was determined by the academic quality; to certain extent, referring to Moore and Kearsly (2015). In Universitas Terbuka milieu and especially for this study (Table I), academic excellence was determined by the seven main factors (student orientation, academic counseling, learning materials, tutorial supports, evaluation systems, feedback mechanisms and referral schemes).

Each factor is further elaborated into dimensions and/or attributes. Besides, academic excellence is an indicator to persistence, loyalty and future careers of students. To ease the naming, variables engaged with related dimensions are prearranged as illustrated in Table I.

Methodology and the design

This study utilizes mixed-methods approach, i.e. exploratory design (Creswell and Clark, 2011). It is conducted under qualitative approach first and then followed by quantitative sequence. Two instruments are established – they are list of questions for interviews and/or focus-group discussions (for qualitative purpose) and questionnaires (for quantitative purpose). Table I is utilized as a basis to develop required and relevant instruments and sent to 600 out of 1,954 overseas students to be completed. All questions (X₁₁-X₇₂) were simultaneously answered two times by the respondents. The first and second answers are to measure excellence level and its importance degree, respectively. The rests (Y₁₁-Y₄₂) were answered once by the respondents to view the impact of academic excellence related to the students’ persistence, loyalty and their expected future careers.

Variables engrossed are explored through questionnaire (Tjiptono and Chandra, 2011; Shahsavar and Tan, 2012). Survey is implemented to accumulate data from the respondents (Fowler, 2014). Purposive (for qualitative purpose) and simple random (for quantitative purpose) sampling techniques are chosen to select eligible respondents (Cochran, 1977). Importance-performance analysis (IPA) and customer-satisfaction index (CSI) are utilized to measure academic excellence (academic satisfaction) along with their importance degree (Kitcharoen, 2004; Silva and Fernandes, 2010; Nugraha et al., 2014). The operational framework for quantitative purpose is shown in Figure 2. Structural equation modeling (SEM) is finally used to identify plausible relations among the involved variables (Wijayanto, 2008; Hair et al., 2009).

Figure 2 describes the features that affect the academic excellence (Y₁) leading to student’s persistence (Y₂), loyalty (Y₃) and future career (Y₄). Academic excellence included GPA (Y₁₁), length of study (Y₁₂), relevance (Y₁₃), accreditation (Y₁₄) and civil effect (Y₁₅). Academic excellence (Y₁) was assessed by perceiving the attributes of students’ orientation (X₁), academic counseling (X₂), learning materials (X₃), tutorial supports (X₄), evaluation systems (X₅), feedback mechanisms (X₆) and referral schemes (X₇). The instrument (questionnaire) consisted of 2×26 questions related to academic excellence and the level of its importance plus seven additional questions to validate persistence, loyalty and the future careers; whether they were affected by and relatable to academic excellence or not. There were 59 questions in total on the questionnaire and 184 out of 600 distributed questionnaires were finally completed and analyzed. Serially, these results will be unified comprehensively with earlier qualitative results.

This approach then examines ten hypotheses: (refer to Figure 2), they are: academic excellence is directly influenced by students’ orientation, academic counseling, learning materials, tutorial supports, evaluation systems, feedback mechanisms and referral schemes. Besides, students’ persistence, loyalty and future careers are also directly influenced by academic excellence.

Results and arguments

Before conversing to the end results, it is valuable to represent the respondent characteristics as illustrated in Table II. This will enrich the perspectives on the outcomes obtained later. Other elaborative analyses are detailed in the next clarification, including Tables III-IV and Figures 3-4.

At this stage, 184 out of 600 distributed questionnaires to 1,954 students were returned and completed. It is then worth to note that most of students domiciled in South Korea and then followed by Taiwan, Hong Kong, Malaysia and Singapore; none of the questionnaires were completed and returned by the students domiciled in the Middle East, Europe, USA and Australia. Besides, most of them have had experiences in ODL mode of learning for more than two years. They can be categorized as good students as only 11 percent of them got the GPA<2.00. In terms of age and gender, most of them are categorized as “youth” (only 13 percent of them with the age of above 35 years old) and highly motivated women as students. To a certain extent, they are all considered as matured, independent and responsible students.

Now let us observe the hypothesis analysis and the loading factors outcomes from the examined model (Figure 3). This is to witness the real interrelations amongst variables and dimensions involved as well as the power of their relations.

Figure 3 clearly shows that two out of ten of the hypotheses examined were statistically not validated by the analysis. They are: H4=1.39 (tutorial supports to academic excellence) and H6=−1.18 (feedback mechanisms to academic excellence). This is so as the t-value⩽1.96 for α=0.05.

Inversely, the rests were all positively validated by the analysis. They are: H1=2.27 (students’ orientation to academic excellence), H2=2.67 (academic counseling to academic excellence), H3=4.61 (learning materials to academic excellence), H5=4.73 (evaluation systems to academic excellence), H7=3.00 (referral schemes to academic excellence), H8=8.49 (academic excellence to persistence), H9=7.73 (academic excellence to loyalty) and H10=7.64 (academic excellence to future career). This is so as the t-value⩾1.96 for α=0.05.

Before describing the end results on the power of relations amongst variables engaged, it must be revealed that the level of academic excellence and its importance degree resulted from IPA-CSI chart. The analysis generates spots of excellence components related to the quadrants (Q) to comprehend the degree of their importance (Figure 4). It is clear that Figure 4 has four quadrants: Q₁ (concentrate here), Q₂ (maintain performance), Q₃ (low priority) and Q₄ (possible overkill); following the research of Wong et al. (2011).

Q₁ has a single attribute that should be carefully noted by the university: X₆₁ (standardized feedback). Q₁ indicates that satisfaction (excellence) is at a low level whereas the degree of its importance is at a high level. Here, the university must pay attention to this evidence and put them at the top of priority so that students’ expectation might be fulfilled and they are more likely to re-register regularly on the semester basis and finally complete their study.

Q₂ includes seven points that should be cautiously recognized by the management: Y₁₅ (civil effect), X₄₁ (face-to-face tutorial), Y₁₄ (accreditation), X₂₁ (access for the counseling), X₂₃ (the value of counseling), X₂₂ (the quality of counseling) and X₃₁ (written learning materials). Q₂ is a symptom of satisfaction and importance degree as both are being placed at a high level. The university must take care of these seven points so that more students get advantages and will pursue their study with intent. All attributes falling into this quadrant are the strength and pillar of academic excellence which is the pride of the university.

Q₃ has 11 points should be seriously remarked by the university: X₃₂ (digital learning materials), X₆₂ (conclusive feedback), X₅₂ (online exams), X₇₃ (referral validity), X₇₁ (referral availability), Y₁₁ (GPA), X₃₃ (learning material supplement), Y₁₂ (length of study), Y₁₃ (relevance), X₄₂ (tutorial via media) and X₆₃ (feedback usage). Q₃ is an indication of both satisfaction and degree of its importance that are in a low category. The university should classify them all as the next focus after concentrating on the critical points especially found in Q₁ and maintaining all points in Q₂. Any attribute falls into this quadrant is not so important and poses no direct and quick threats.

Finally, Q₄ has seven points, they are: X₄₃ (on-demand tutorial), X₁₃ (orientation delivery), X₇₂ (referral flexibility), X₅₁ (classroom exams), X₅₃ (assignments), X₁₂ (orientation content) and X₁₁ (orientation schedule). Q₄ indicates that the service provided is considered much less important but most respondents considered them as high in satisfaction. Here, again, attention to attributes included in this quadrant might be less focused. So, the university can save costs by redirecting them to take up crucial point in Q₁ and again maintaining all fundamental aspects found in Q₂.

Having positioned all factors in appropriate quadrants, we are now in the position of relating the loading factors outcomes of the tested model. This is to observe the power of relations of each variable involved under SEM to positively work out the end results. Figure 3 openly reveals five prime remarks:

1. The first is related to the five variables that directly influence the academic excellence. They are rank in the order as: evaluation systems (0.37), referral schemes (0.28), academic counseling (0.28), learning materials (0.23) and students’ orientation (0.14). Note cautiously that the most critical aspect here is on the evaluation systems.

2. The second is relatable to the rank of dimensions in the evaluation systems, they are: assignments (X₅₃=0.99), classroom exams (X₅₁=0.98) and online exams (X₅₂=0.88). It implies that the assignments are the most important aspect according to the students as compared to the exam itself. The order in referral schemes are as follows: flexibility (X₇₂=0.86), validity (X₇₃=0.68) and availability (X₆₁=0.66). Here, flexibility is the most critical aspect behold by the students. The position in academic counseling: access (X₂₁=0.93), value (X₂₃=0.88) and quality (X₂₂=0.82). Here, access is very important in terms of counseling provision. The order in learning materials: written (X₃₁=0.88), supplement (X₃₃=0.88) and digital (X₃₂=0.77). In this part, the written material is still the most valuable one according to the respondents. The order in student orientation is as follows: schedule/time (X₁₁=0.90), delivery mode (X₁₃=0.88) and content (X₁₂=0.81). Timing in providing orientation is critical.

3. The third is concerning the order of academic excellence provision viewed from the academic service outlooks: relevance (Y₁₃=0.77), civil effect (Y₁₅=0.74), length of study (Y₁₂=0.73), accreditation (Y₁₄=0.71) and GPA (Y₁₁=0.64). It is good to see that the GPA is not a big problem according to the students who are domiciled overseas. What is crucial for them is on the relevance and the civil effect of the program they are undertaking. In other words, they are more interested in knowing that the program they take not only relevance for they future but also has wider opportunity to have a better future after going back home.

4. The fourth is on the relation powers of academic excellence toward the dependent variables. Figure 3 obviously confirmed that academic excellence has significant effects on: students’ persistence and loyalty (0.31) and followed by future careers (0.30) This implies that student overseas is strongly confident that apart from re-registering on a semester basis, they also eager to finish their degree; this is indeed positive.

5. The fifth is on the rank of persistence (they are the same): re-register regularly and study up to finish (Y₂₁=Y₂₂=0.66). Loyalty: further study (Y₃₁=0.99) and endorse to others (Y₃₂=0.31). Future career: progression (Y₄₁=0.82) and contribution (Y₄₂=0.78). What is important here that students overseas are willing to endorse Universitas Terbuka to others. This is a good sign for the university in relations to increase the participation rate of Indonesian communities entering education in the university level.

Before integrating the qualitative-quantitative results and the previous results, it is worth considering the analysis of goodness of fit of the model. The analysis shows that they are all in good fit category (Table III). It implies that statistically the quantitative result is reliable to be used as a point of reference to draw the inferential closing.

Having collected and aggregated outcomes accomplished by quantitative and qualitative inquiries, three major validities need to be noticed and elaborated thoughtfully. The first is on the conceptual and operational framework (Figure 1 and Figure 2, plus Table I). The second is on the IPA-CSI chart (Figure 4). The third is on the chosen methodology property.

It was quantitatively understood that persistence and loyalty are confirmed as the primary aspects and then followed by future careers as a result of academic excellence or student satisfaction academically. This result is clearly in harmony with the previously obtained qualitative inquiry. These factors are found from literatures, interviews and also focus-group discussions. Besides, in terms of its order, selected experts (qualitative respondents) preferred to express that academic excellence leads to Students’ persistence, loyalty and future careers as well. Moreover, in the attributes level, the ranks are clearly in harmony with each other. This is a good sign; the results obtained under qualitative and quantitative approaches are the same and affably supported each other.

Quantitative outcomes here partially excluded tutorial supports and referral schemes from the qualitative framework that obtained earlier; supplementary explanation is clearly needed for this differences. From Table II, it was detected that all respondents are domiciled overseas. It is implied that most of them have more access and opportunities to use electronic media devices compared to other students residing in Indonesia. It also implies that the vast majority of them are more capable of searching for learning resources such as open educational resources to support their study as they already had experiences at least four semesters in ODL mode of learning in Universitas Terbuka settings. They are moreover categorized as middle and upper levels in terms of the GPA. In short, these are the explanation why the tutorial supports and referral schemes are no longer the case or problem for most of students residing overseas. The rest of quantitative outcomes are relatively consistent with the qualitative marks.

As this is a follow-up study, it is also important to disclose significant differences found between these results as compared to the previous upshot (Sembiring, 2016). They are all summarized in Table IV. In the previous study (Sembiring, 2016), where the respondents are all graduates and most of them are teachers as their profession, it was concluded that four out of ten hypotheses are excluded (Table IV). They are: academic counseling, learning materials, feedback mechanisms and referral schemes. In contrast, tutorial supports and feedback mechanisms are excluded from the framework in this study. All the same, here the evaluation systems is the most crucial one for academic excellence (tutorial support is even excluded), whereas in the previous study tutorial supports is the most critical one (Table IV). It seems that deeper study is important to be implemented to find the reasons behind this dissonance.

From IPA-CSI charts, it seems that the differences between the previous vs the current results only in GPA aspect. In the previous results (Sembiring, 2016), GPA is in the first quadrant (Q₁); together with the feedback mechanisms. It implies that according to the respondents of this study (students domiciled overseas), GPA is not a problem. It entails that most of students found that the GPA is important and they are satisfied with their achievements.

Looking up to the third effect, it appears that the mixed method used in this study is reliable despite the slight and minor difference on the end results still did take place. The differences in terms of the end results took place in the level of hypotheses testing; not in the conceptual outlooks within the dependent variables. Despite the difference, it does not indicate they are in high contradictory intensity. It can then be inferred that the difference took place are basically to amplify our perspectives on the context.

From methodological direction, IPA-CSI approach was moreover able to distinctively display what things should be placed within the top priority to be controlled prudently, i.e. feedback mechanisms (Q₁). The approach is proficient enough to classify what are the things should be persistently maintained (Q₂), i.e. the seven crucial aspects; what are the things classified as the next priority (Q₃) and considered less important so there is no need to be rush (Q₄).

IPA-CSI chart effects are reinforced quantitatively by the SEM results. By combining these upshots, it will objectively direct the university to formulate alternative course of actions for future needs in anticipating the students’ expectations accordingly. It is fortunate that qualitative inquiry to certain extent was inline with the quantitative conclusion. It has been phenomenon that most universities are limited by tangible resources, 5M (man, money, material, machine and method). By considering this constraint, according to Sembiring (2016), it is just right to re-formulate “new” ideas on how to effectively re-direct resources such that sufficient efforts and supports are available to deal with aspects in Q₁ and maintaining critical aspects in Q₂ as also indicated by Tileng et al. (2013).

This result will be incredibly useful to re-formulate on the things that should be put as the top priority by the university to fulfill students’ expectations in conjunction with satisfying the needs for those that are still studying. The one attribute dropped into Q₁ should be brilliantly controlled (X₆₁, standardized feedback). Additionally, seven aspects dropped into Q₂ should also be repeatedly preserved as they are the pillar and pride of the university in assuring academic excellence; by all means, aspects from Q₁ can be moved onto Q₂. It will improve the number of students getting satisfied. The more students were satisfied, the more likely they stayed; as persistence here is defined as students re-register regularly in each semester. This implies that the university will be able to maintain the size and growth of student body as it was initially projected in the strategic plan.

At the end, the respondents were asked a closing question: what is your perception on the GPA and the length of study from existing services?

Amazingly, the answers provide a quite robust acceptance that in the future the university will be able to accomplish the initial plan in terms of retaining the size and growth of student body. Why? Since the answers to the last question is quite convincing. They are: completely unsatisfied=2 percent, unsatisfied=14 percent, satisfied=58 percent, very satisfied=21 percent and extremely satisfied=5 percent. At least up to 84 percent of the respondents were satisfied with their GPA and the length of study. This is a truly good indication for the University.

Remarks for future actions

The research has created both qualitative and quantitative frameworks of academic excellence and its dimensions in Universitas Terbuka milieu with respect to their links extended from a comprehensive analysis of educational perspective and student behavior. The framework was validated using SEM through assessing empirical data by the survey of 184 respondents (Universitas Terbuka students domiciled overseas). The study ascertains that academic excellence leads to persistence and loyalty and then followed by future careers. Additionally, academic excellence is affected by evaluation systems, referral schemes, academic counseling, learning materials and student orientation. Under the IPA-CSI procedure, one aspect that should be cautiously noticed in anticipating and fulfilling students’ needs and expectations is on the feedback mechanisms.

Further research, however, is crucial and it should explore the excellence level beyond attributes included in the dimensions assessed in this inquiry, for example, both the use of new and more appropriate media for learning and enhanced mode of interaction, synchronous and/or asynchronous, in relations to searching for better approaches to maintain student satisfaction and persistence. The scope of the study should also be broadened beyond the students domiciled overseas only by including all students of the university in all levels. It would put forward more comprehensive perspectives especially on persistence, loyalty and future careers with reference to academic excellence to meeting students’ needs as ODL learners. This in turn will improve the students’ persistence rate.

In short, this will provide an opportunity for the university to be more contributive in supporting the Government of Indonesia to eradicate the restraints access to higher quality education. If this awareness is emblematical worldwide, management and academic elsewhere are well-advised to ruminate on the academic excellence explored here to prolonged continued existence of their institution. For Universitas Terbuka, persistence and loyalty can be conquered through the provision of academic excellence. This means that the university is on the right path to encourage its righteous mission of making higher education open to all with respect to protecting the nation through flexible quality education. The university will be in harmony to reorganize the vision of becoming the world’s quality institution in preparing the world’s quality graduates (Universitas Terbuka, 2014; Sembiring, 2015, 2016).