Abstract
Purpose
The purpose of this paper is to examine how the Japanese and Taiwanese national quality assurance (QA) agencies, National Institution for Academic Degrees and Quality Enhancement (NIAD-QE) and Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT), transform their respective frameworks in response to social demands, and analyze and compare the respective approaches for the key concepts of autonomy, accountability, improvement and transparency.
Design/methodology/approach
Using a qualitative document analysis approach, this paper initially examines the higher education system, major policies and QA developments, after which the methods associated with the QA restructuring transformations are outlined in terms of motivations, expectations and challenges. Finally, the NIAD-QE and HEEACT evaluation policies and frameworks are compared to assess how each has prepared to respond to emerging challenges.
Findings
During the QA framework restructuring, both the NIAD-QE and HEEACT struggled to achieve autonomy, accountability, improvements and transparency. While the new internal Japanese QA policy is assured through the external QA, the Taiwanese internal QA, which has a self-accreditation policy, is internally embedded with university autonomy emphasized. The QA policies in both the NIAD-QE and HEEACT have moved from general compliance to overall improvement, and both emphasize that accountability should be achieved through improvements. Finally, both agencies sought transparency through the disclosure of the QA process and/or results to the public and the enhancement of public communication.
Originality/value
This study gives valuable insights into the QA framework in Asian higher education institutions and how QA has been transformed to respond to social needs.
Keywords
Citation
Noda, A., Hou, A.Y.C., Shibui, S. and Chou, H.-C. (2018), "Restructuring quality assurance frameworks: A comparative study between NIAD-QE in Japan and HEEACT in Taiwan", Higher Education Evaluation and Development, Vol. 12 No. 1, pp. 2-18. https://doi.org/10.1108/HEED-12-2017-0008
Publisher
:Emerald Publishing Limited
Copyright © 2018, Ayaka Noda, Angela Yung Chi Hou, Susumu Shibui and Hua-Chi Chou
License
Published in Higher Education Evaluation and Development. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Introduction
The dramatic economic, political and social changes over the past few decades have led to significant changes in higher education in terms of expansion, massification, competition, innovation, deregulation and commercialization. Further, because of the reductions in public funds, universities are now required to be accountable and transparent in terms of quality. However, with these funding changes, higher education institutions are expected to be more autonomous in the management of their financial and personnel resources and in their academic teaching and learning decisions (Costes et al., 2010), which has required them to be more accountable for their service quality and performances through regular quality assurance (QA). Therefore, to assure the quality of higher education providers, there has been an increased demand for accurate, efficient evaluation tools that can measure the success of the university programs and identify and measure the competencies graduates will have at the end of their education programs.
The most important change driver in higher education in the past decades has been the emergence of QA, with many QA agencies opening and others being expanded, as evaluation and/or accreditation tools have become a powerful way to regulate and rationalize interventions to define quality-based objectives and the associated criteria (Normand, 2016). QA agencies are now expected to assure university compliance and enhance student learning outcomes. Therefore, this tension between “accountability” and “improvement” has been widely discussed in associated research (Banta and Palomba, 2015; Harvey and Williams, 2010; Harvey and Newton, 2007) and is reflected in many QA methodologies. Although QA approaches vary greatly by country and region, most external QA systems include accreditation, evaluation, audit, review, recognition and approval. External QA focuses on institutions and programs, with accreditation approaches stressing compliance and quality enhancements (Smidt, 2015). While initial focus has been on the introduction of external QA, there has also been a gradual shift toward internal QA in the past decade (Gover and Loukkola, 2015). The Berlin Communiqué (2003) claimed that internal QA emphasizes how higher education QA is consistent with the principles of institutional autonomy (Ministerial Conferences, 2003). Costes et al. (2010) also defined internal QA as focusing on teaching and learning quality improvements, while external QA is involved with external accountability to the public and the multiple stakeholders for quality improvements. Geven and Maricut (2015) defined internal QA as evaluation initiatives by people inside universities, and external QA as evaluations undertaken by the government or other actors “external” to the university. However, the relationship between internal and external QA varies by country.
To enhance accountability and social responsibility, higher education and QA agency’s transparency is paramount. That is, to connect universities with the society, the universities and QA agencies need to be prepared to disclose information about university activities, intended learning outcomes, assessment procedures, learning opportunities and graduate employment.
The four dynamic concepts – autonomy, accountability, improvement and transparency – highlighted in this research are interrelated, supplementary, and sometimes contradicted with each other. As discussed above, reduced funding has led universities to be more autonomous, which requires them to be more accountable to the society at the same time (Costes et al., 2010). Bernhard (2012) discussed that “accreditation was considered accountability-oriented” while, at the same time, “institutional autonomy can be ensured” (EURASHE, 2005, as cited in Bernhard, 2012 p. 177). Existing literature in the USA, however, addressed the criticism that accreditation seeking for accountability ends up undermining the autonomy of universities: accreditation should not diminish the autonomy of higher education institution (Gaston, 2014). Transparency has also become a key principle for QA and has interacted with accountability. As a current global trend, QA is always geared to improvement, even if it also functions for accountability.
Overall, however, this shift to more accountable and transparent QA frameworks in Asian countries have been relatively slow compared to western countries, most of which have had higher education compliance and QA bodies for several decades. In Japan, for example, the term “internal quality assurance” was first mentioned in line with higher education in a government proposal in 2008 referencing the 2005 European Standards and Guidelines. The proposal stated that the initial responsibility for improving the quality of university education lay with the universities themselves (MEXT, 2008), emphasized self-assessment and underlined that third-party evaluations were needed to ensure universities developed adequate internal QA systems based on self-assessment and a Plan-Do-Check-Act (PDCA) cycle (Hayata and Mochizuki, 2012). In Taiwan, the issue of “quality” higher education provision was first discussed in the 1980s, at which time the government encouraged higher education institutions to conduct their own performance assessments. Although the term “internal quality assurance” had not yet been officially enshrined, these early government announcements encouraged higher education institutions to implement QA systems. In 2005, the Ministry of Education revised the “University Law,” which included a stipulation that all “universities periodically self-assess their teaching, research, services, counseling, administration, and student engagement and that evaluation guidelines be developed by each university” (Hou, 2011; Ministry of Education (MOE), 2005), which was the first time the term “self-assessment” had been used in official documents in relation to higher education institution internal QA developments. Based on the law, the Ministry of Education borrowed an “accreditation system” from the USA and implemented mandatory higher education evaluations.
Japan and Taiwan have both developed higher education systems to respond to international competition and technological innovations. As both countries have social issues related to low birthrates and aging societies, higher education institutions have been required to improve their teaching, learning and research quality, with QA becoming a national issue in both Japan and Taiwan. Consequently, Japan and Taiwan established national QA agencies, the National Institution for Academic Degrees and Quality Enhancement (NIAD-QE) in Japan and the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT) in Taiwan, to assist universities improve their teaching, research and governance. Despite national QA systems operating for several years, there is now a need to transform QA concepts and frameworks to respond to changing social demands.
Overall, however, there has been little research on actual QA conditions, practices, challenges and transformations in Japan and Taiwan, even though there have been ongoing developments in higher education QA mechanisms and frameworks. Further, as far as we are aware, there have been no comparative QA studies on Japan and Taiwan.
Purpose of the study
This study examined the methods used by the Japanese and Taiwanese national QA agencies, NIAD-QE and HEEACT, to develop new frameworks in response to social demands. In particular, this paper compared the two agencies’ approaches to issues associated with autonomy, accountability, improvement and transparency, and examined the balance between external and internal QA and institutional and program-based QA. First, background information on the higher education system and QA system development is given, after which the challenges in the QA systems are examined. Finally, the QA policy and framework restructuring at NIAD-QE and HEEACT in response to new demands are analyzed and compared.
The following research questions informed this research:
How did NIAD-QE and HEEACT restructure their respective QA frameworks to respond to emerging challenges?
In particular:
How did the QA agencies balance external and internal QA and institutional and program accreditation to promote university autonomy?
How did the QA agencies find a balance between accountability and improvement?
How did the QA agencies enhance the transparency of the QA mechanism?
Methodology
A qualitative document analysis approach was adopted for this research. The initial focus was to review previous research and studies on higher education systems and policy, and QA system and framework development in Japan and Taiwan. Then, a document analysis was conducted on relevant government QA policy reports, guidelines, proposals, NIAD-QE and HEEACT evaluation guidelines, standards and indicators. Document analysis involves the gathering and reviewing of written documentation to extract information in a systematic manner (Institute of Development Study, 2013). In the third stage, the emerging issues NIAD-QE and HEEACT are facing and QA framework and mechanism restructuring were examined to allow for a comprehensive comparison of NIAD-QE and HEEACT in achieving higher education autonomy, accountability, improvement and transparency. The analysis also included a review of the ways that QA methods (i.e. external and internal, and institutional and program QA) have been applied to enhance these concepts.
Overview of higher education systems and policies, and the development of QA systems in Japan and Taiwan
This section gives an overview of the higher education systems and the major policy, QA system developments and the original QA frameworks in Japan and Taiwan.
Higher education systems and policy
Higher education system and policy in Japan
From pre-school to university, the Japanese education system is regulated by the Ministry of Education, Culture, Sports, Science and Technology (MEXT). At the tertiary level, there are four types of institutions: Universities, junior colleges, colleges of technology and professional training colleges (specialized courses). In 2017, there were 780 universities (86 national, 90 prefectural, and 604 private), 337 junior colleges, 57 colleges of technology and 2,822 professional training colleges (specialized courses). In Japan, the private higher education sector has the majority of student enrollments with 2,128,000 students, followed by national universities with around 609,000 students and prefectural universities with around 153,000 students (MEXT, 2017).
There has been a significant expansion in the Japanese university sector in the past half century; in 1965, there were only 317 universities, but by 2001, this had almost doubled to 649, and by 2009, there were 773. To enhance university flexibility and autonomy, the Deregulation of University Act was promulgated in 1991, with the aim of decreasing government intervention in such areas as curricula design, credit hour allocations and academic degree titles. Universities were made responsible for periodic self-assessments to encourage quality and governance improvements. Deregulation brought significant reforms to the national universities, with the status of national universities changing to national university corporations (NUCs) in 2004. The introduction of these market mechanisms gave the NUCs autonomy in terms of university governance and financial and personnel management. Universities were expected to focus on some of the seven core functions: acting as a global research and education center, developing highly professional or wide-ranging human resources, providing a comprehensive liberal arts education, conducting education and research in specific professional fields, serving as community centers for lifelong learning or making social contributions. In 2013, MEXT asked the national universities to redefine their missions based on objective research, educational evidence and defined functions (MEXT, 2013). It was hoped that by clearly redefining their missions and goals, universities would be encouraged to be more competitive.
Higher education system and policies in Taiwan
Taiwan has a dual higher education system that is administered at the ministerial level by the Department of Higher Education and the Department of Technological and Vocational Education, both of which are under the Ministry of Education. After the 1980s, as access to higher education expanded, the number of higher education institutions increased from 39 in 1988 to 158 in 2017, with 1.15m students enrolled. Universities vary significantly in size, with the largest having around 30,000 students and the smallest having around 100 students, and all offer a wide range of programs. In recent years, the low birth rate in Taiwan of less than 200,000 per year is expected to result in a drastic drop in college enrollments over the next 15 years and the closure of smaller colleges and technical colleges (MOE, 2013).
The increased competition among top Asian universities has applied substantial pressure on the economies that had developed earlier. Over the past ten years, the Ministry of Education has launched several excellence initiatives: the Development Plan for World Class Universities and Research Centers of Excellence (2005-2016), the Teaching Excellence Initiative (2005-2014) and the Technological University Paradigms (2013) (Department of Higher Education, 2011), each of which has different foci.
As the Taiwan Government adopted a selection policy for its three excellence programs, most resources and funding have been granted to a limited number of institutions. Starting in 2017, the Ministry of Education launched a new initiative, “Deep Plowing in Higher Education,” which is aimed at enhancing higher education quality and excellence through improved social accountability and responsibility, teaching quality and learning outcome accessibility, affordability for underprivileged students, and internationalization and global competitiveness (MOE, 2017; Hou et al., 2012).
QA systems and development
QA framework development in Japan
Japanese first accreditation agency, the Japan University Accreditation Association (JUAA), was established in 1947. As a voluntary and membership-based accreditation agency, the JUAA developed its own university quality standards; however, most universities at that time did not apply this voluntary-based accreditation. At this time, Japanese university quality control relied mostly on this approval system. In the 1990s, when approval systems were deregulated to give universities more freedom, flexibility and diversity and to encourage public openness and competition, a self-evaluation system was implemented. The 2000s saw a drastic transformation in university management systems, with the national university status being changed to autonomous corporations in 2004 under the expectations that this would allow them more freedom and autonomy, and would better promote competition. During this time also, university QA systems were developed and a third-party evaluation system implemented. As of 2017, Japan has four QA legally required frameworks: an approval system for university establishment, self-evaluation, National University Corporation Evaluation (NUCE) and certified evaluation and accreditation (CEA). The government (MEXT) approval system for the establishment of universities is based on the “Standards for the Establishment of Universities” as the minimum quality standards (e.g. student-staff ratios, required space per student, curricula, faculty and university buildings) for awarding degrees. All universities, junior colleges and colleges of technology are required by the School Education Law to conduct self-evaluations and publish the results to assess progress toward their own goals. Every six years, the NUCE conducts a performance-based evaluation on the universities’ mid-term plans, and depending on the progress and achievements, uses the evaluation results when allocating university funding. The CEA, which this paper focuses on, promotes quality enhancements and compliance. In Japan, the CEA is based within the universities, junior college, and colleges of technology, with program-based CEA used only for professional graduate programs. In Japan, all universities must conduct an institutional CEA every seven years using one of the three official QA agencies certified by the MEXT: the JUAA, NIAD-QE and the Japan Institution for Higher Education Evaluation.
The NIAD-QE (which was the NIAD-UE: National Institution for Academic Degrees and University Evaluation until spring 2016) mainly targets national and prefectural universities and emphasizes compliance, with the evaluation standards in the first cycle (2005-2011) being: university mission; education and research structure; academic staff and education support staff; student admissions; academic programs; institutional performance; student support; facilities; internal QA systems; finance; and management. There were several CEA evaluation standard revisions in for the second cycle (2012-2018) in response to international trends in QA systems and accountability, with the revised CEA focusing more on learning outcomes, internal QA system in teaching and learning, and public information disclosure. The new standards for the second cycle were: university mission; teaching and research structure (organizations); academic staff and teaching support staff; student admissions; academic programs (content and methods); learning outcomes; facilities and student support; internal teaching and learning QA system; finance and management; and public information disclosure on teaching and learning (NIAD-UE, 2011).
QA framework development in Taiwan
Because of the rapid increase in higher education institutions in Taiwan since the 1980s, the desire to increase both “quantity” and “quality” has placed tremendous pressure on the Taiwanese Government. Besides encouraging the institutions to conduct their own assessments, in the 1980s, there were few professional associations chartered by the Ministry of Education to conduct program-based academic assessments; the Chinese Management Association, the Chemical Society and the Physical Association of the Republic of China. Because of public pressure, in the 1990s, the government implemented a wide range of comprehensive institutional evaluations with the goal of establishing a non-government professional evaluation agency to conduct higher education institutional evaluations (Hou, 2011).
In 1994, Taiwan’s Congress, the Legislative Yuan, passed the “University Law” which clearly stated that the national government was entitled to conduct university evaluations to ensure the maintenance of higher education quality. The Ministry of Education then revised the “University Law” to “establish evaluation committees or support professional accrediting agencies to periodically conduct university evaluations and publish the results as reference for government for subsidy allocations and to allow the institutions to adjust their future development plans” (Hou, 2011; MOE, 2005). According to this law, the MOE then funded the establishment of the HEEACT in 2005. Before this time, several local accreditors such as the Taiwan Assessment and Evaluation Association (TWAEA), which mainly undertook institutional assessments of Taiwan’s technology universities, had been providing QA services to Taiwan’s institutions, and there were three other professional accreditors focused on medicine, nursing and engineering programs. As the oldest professional accreditor, the Taiwan Medical Accreditation Council (TMAC), which was established by the National Health Research Institute in 1999, assesses all medical schools. The other professional accreditor, the Taiwan Nursing Accreditation Council (TNAC) was established by the Ministry of Education in May 2006 to conduct nursing program evaluations. After the establishment of HEEACT in 2005, the TMAC and TNAC were officially moved to the HEEACT office; however, due to the unique features of medical and nursing education, these two agencies have remained independent accrediting agencies. Founded in 2003, the Institute of Engineering Education Taiwan (IEET) is an independent, non-government, not for profit organization committed to the accreditation of engineering and technology education programs in Taiwan. The difference between these local accreditors and HEEACT is that these accreditors are self-funded institutions that offer services on a voluntary basis; therefore, institutions which voluntarily apply for accreditation by a local accreditor must pay their own fees.
To eliminate any duplication across the various accrediting agencies and to lessen institutional burdens, in 2009, the Taiwanese Ministry of Education announced an “exemption policy.” If a program had been accredited by international and local accreditors recognized by the MOE’s task force of “Local and International Accreditors’ Recognition,” it did not need to be assessed or re-assessed through HEEACT. Up to 2017, the task force had recognized three local accreditors, and one US accreditor; TWAEA, IEET, the Association of Chinese Collegiate Schools of Business, Advance Collegiate Schools of Business and the Council on Education for Public Health.
As the national accreditation agency in Taiwan, HEEACT has been conducting compulsory institutional and program-based accreditation since 2006. The three external review components are an institutional self-assessment report, an on-site visit and a peer review. The external review costs are covered by the MOE and detailed final reports are published on HEEACT’s official website. In 2006, HEEACT began a five-year program-based nation-wide accreditation scheme, with the standards developed in the first program accreditation cycle being: goals, features and self-enhancement mechanisms; curriculum design and teaching; learning and student affairs; research and professional performance; and graduate performance Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT, 2012). From 2011, HEEACT has been conducting a comprehensive assessment of the 81, four-year national and private universities as well as second cycle program accreditations. HEEACT evaluates institutions based on the PDCA model and evidence, and have five review standards; self-positioning, government and management, teaching and learning, accountability and continuous quality improvements. The second program accreditation cycle emphasized the development and operation of student learning outcome evaluation mechanisms within all programs and disciplines, with the HEEACT assisting universities to analyze their strengths and weaknesses in facilitating successful student learning and in relation to the new standards: educational goals, features and curriculum design; teaching quality and learning assessment; student guidance and learning resources; academic and professional performance; and alumni performance and self-improvement mechanisms (HEEACT, 2012).
In addition to institutional and program accreditation, a system was implemented to promote university self-improvement. As a result of requests from the universities, the MOE launched a “self-accreditation” policy in 2012 with the aim of increasing university autonomy and developing a higher education quality culture (MOE, 2013). The universities selected as the self-accrediting universities were expected to have internal QA mechanisms in place and had to be able to demonstrate academic accountability; therefore, in 2012, only a few institutions were eligible for self-accreditation as they had to meet one of the following rigorous requirements: awardees of the MOE Development Plan for World Class Universities and Research Centers of Excellence; awardees of the MOE Top University Project; or awardees of the MOE Teaching Excellence Project and had been granted at least US$6.7m over four consecutive years. Universities were encouraged to develop measurable learning outcomes; to design a variety of assessment tools at the course, program and institutional level; and to establish measures for assessing learning outcomes.
Emerging challenges for QA frameworks
It has been more than ten years since Japan and Taiwan implemented their respective QA systems. This section examines the challenges NIAD-QE and HEEACT have faced during this period.
QA system challenges in Japan
MEXT reported that the CEA has highlighted the importance of reviewing university teaching and research (Central Council of Education, 2016). Therefore, the CEA has assisted higher education institutions develop a QA culture and improve quality; however, some issues still require improvement.
Overemphasis on compliance matters
The existing CEA framework has been criticized for its focus on numerical requirements such as graduation rates, student-staff ratios, students per faculty, and building and facilities rather than focusing on the enhancement of education and research quality. A recent study (Shibui, Takahashi and Noda, 2017) found that universities had improved in terms of compliance issues that could be fixed short term, rather than focusing on teaching and learning improvements that require more time.
Demands for effectiveness in quality enhancement
The second issue is closely related to the above challenge. NIAD-QE’s second cycle CEA examined the areas that universities believed had substantially improved since the first cycle. Most universities felt that their compliance had improved (e.g. student enrollments rates, number of faculty members, facilities, library and resources); however, few mentioned that they had improved teaching and learning matters (Shibui, Takahashi and Noda, 2017). Because the Japanese CEA system was designed to be an institutional evaluation, there is little detailed information about each education program (Noda, 2017); as a result, the demand for program reviews as well as internal QA has been increasing.
Evaluation “exhaustion”
The government reports (Central Council of Education, 2016; KPMG AZSA LLC, 2014; Science Council of Japan, 2012), industry (Japan Association of Corporate Executives, 2013) and the media (Recruit, 2017) have repeatedly warned that universities and peer reviewers have been overwhelmed with the evaluation workload, which was described as “evaluation exhaustion.” NIAD-QE (2016) simplified evaluation procedures by reducing the evaluation viewpoints from 99 in the first cycle to 81 in the second cycle in 2012, after which a follow-up survey indicated that the universities found this second cycle easier to manage as there was less duplication or overlaps. However, most universities were feeling burdened when preparing evidence for peer reviewers because of the need to collect data or resources from each unit (e.g. school, faculty, department, program or course) within the three weeks before the site visit. The government’s and NIAD-QE’s recent emphasis on concrete and detailed evidence therefore has added to the QA administrative burden, with many universities expressing concern about the difficulties associated with the type and volume of evidence needed.
Enhancing public awareness
Japan Association of Corporate Executives (2013) released a report that complained about inadequate system of sharing information about the CEA. The sharing of the QA system and evaluation results with the public is necessary for social responsibility and accountability. Even though QA agencies have fully shared their mechanisms and procedures and published the CEA results, the Central Council of Education (2016) and Japan Association of Corporate Executives (2013) insist the CEA system is still not fully understood by the labor market, high schools, prospective students or their parents. It was also observed that many people within the universities were not conversant with the CEA. Therefore, universities need to promote a common awareness of the CEA goals by widely distributing user friendly information about distinctive university initiatives and good practice.
QA system challenges in Taiwan
As in Japan, the Taiwan QA exercise has attracted criticism from both the universities and the general public (Liberty Time Net, 2018). There have been several major concerns about the HEEACT’s accreditation such as the increased workloads, the use of the accreditation results by the institutions, the reviewers’ quality and qualifications, the building of public trust and the enhanced use of the evaluations by employers and students.
Heavy workloads on universities
Taiwanese universities complained most about the increased staff and faculty workloads associated with HEEACT accreditation (Hou, 2015). This increased burden appears to have resulted from a lack of investment into administrative human and financial resources, which has led to a certain level of resistance within universities going through a HEEACT review.
Reviewer quality and qualifications
While the self-accreditation policy of 2012 has resulted in the development of a quality culture at most universities, there have also been several problems such as inappropriate reviewer selection processes, arbitrary standards and indicator elimination, and inconsistent cross review decisions (Hou et al., 2017). External review validity relies on the reviewer professionalism, with an INQAAHE report finding that the professionalism of the QA process and procedures reviewers was very important to the universities (Hou et al., 2017). Therefore, the QA agency needs to develop more rigorous recruitment policies and training programs focused on international capacity building for the reviewers.
Program survival or closure affected by accreditation result
The accreditation results have significantly influenced institutional governance and management. First, as the external QA system requires that universities clearly identify their missions and objectives, this has assisted the institutions in recognizing their distinctive features; has strengthened governance and resource allocation management; and led to program revitalizations, curriculum reform and improved staff recruitment. A recent report by HEEACT (2015) reported that 92.2 percent of all accredited programs in the first cycle reviews had been retained compared to only 41.2 percent of non-accredited programs. The accreditation results have been used for organizational restructuring, staff recruitment policy adjustments, program merges and closures rather than for the establishment of internal QA and self-improvement measures.
Building public trust and enhanced evaluation use by employers and students
There was a strong demand that the QA needed to embrace the society’s needs and build public trust. However, in general, the accreditation was less important to Taiwanese employers and students than global rankings. Regardless of the problems associated with rankings, many employers use global rankings as one of major selection criteria for new staff recruitment, and students and parents use the global rankings as a vital reference when choosing institutions both in Japan and in other countries. Several international QA networks have advised that national QA agencies need to “make their reviews and evaluations transparent and available to society and provide information regarding the performance of higher education institutions” (Hénard, 2016, p. 26), which could improve communication between QA agencies and society and raise public confidence.
New reforms: trends and challenges
In response to social needs and the criticisms of the current QA mechanisms in Japan and Taiwan, NIAD-QE and HEEACT both launched new reforms to enhance QA efficiency.
New reform in Japan: emphasis on self-improvement and program review
The NIAD-QE will start its new cycle of CEA in 2019 referring to the Article 17 of Ordinance of MEXT amended in 2016 (MEXT, 2016b), with the key discussions now focusing on enhancing connectivity with society and improving the efficiency of the QA mechanism.
Defining a new evaluation standard for the universities’ “three policies”
The Article 16 of Ordinance of MEXT (2016a) required that Japanese universities define and publish “three policies,” namely, a diploma policy (DP), a curriculum policy (CP) and an admissions policy (AP), to clarify their educational goals, processes and expected learning outcomes in degree programs so that stakeholders could understand the program goals and objectives. The DP requires that the universities clearly define the expected graduation competencies. The CP requires that the universities clearly define the curriculum structures, course content, teaching methodologies and student assessment methods they use to achieve the DP. The AP requires that the universities clearly define the admissions requirements for each program/degree. The idea of three policies is not new; since MEXT (2005) report emphasized the importance of defining three policies to have closer alignment and smooth transition between secondary and tertiary education, many universities have already developed the policies but in a very broad and unclear way. The amended Ordinance of MEXT (2016a) asked universities to demonstrate relevance and consistency among the three policies emphasizing student competences and learning outcomes.
These three policies are expected to be implemented at the unit level each university defines (e.g. institution, faculty and department); however, in principle, they are aimed at the program level that confers the degree (e.g. school/faculty, department or other programs the universities design). The Central Council of Education (2016) requires universities to build their internal QA systems around these three policies. Therefore, the NIAD-QE’s new CEA focuses on how consistently these three policies (at least DP and CP) have been implemented, encourages universities to have regular program monitoring and reviews and evaluates how academic programs manage the curriculum, assess performance and ensure student learning outcomes.
Mandating universities to evaluate internal QA
While always recognizing compliance, the next CEA needs to also emphasize quality improvements through internal QA systems. In the new cycle (2019), NIAD-QE will remain external to the institutions but will stress the need for internal QA evaluation standards at the degree and program level. NIAD-QE developed guidelines in 2017 for internal QA systems for education and research and therefore will be examining the extent to which faculty and staff have built, operated, monitored and reviewed their own programs (NIAD-QE, 2017). Universities are now expected to regularly monitor and review the effectiveness of their degree programs and continually improve quality. The “three policies” are key to successful internal QA, including curriculum mapping, learning environment support and student learning outcome improvements. The NIAD-QE also plans to implement follow-up evaluations one year after universities receive evaluations that recommend improvements.
Evaluation efficiency
The “Evaluation exhaustion” associated with the heavy workload problem has been resolved through the simplification of the evaluation methodology. Previously, universities were required to provide detailed descriptions, rationales and evidence; however, the new CEA will only review selected essential evidence, and also plans to include collaboration with other existing third-party program evaluation schemes to reduce duplication. Based on past experiences, the NIAD-QE reframed the evaluation standards from ten standards with 81 points to six standards (renamed “domains”) with 27 points (newly called “standards”).
Information disclosure and transparency
The demand for QA agency and university transparency has been increasing. To date, the NIAD-QE has sought to be as transparent as possible in its evaluation standards, procedures and results. However, to further expand its ambit, the NIAD-QE is seeking public comment and university advice about the evaluation frameworks and standards, and also plans to include employers and practitioners on the peer review teams. The MEXT has stated that evaluation information be available both domestically and internationally, and that the evaluation process involves a range of stakeholders such as students, employers and high schools (Central Council of Education, 2016). NIAD-QE released an English language summary translation of the evaluation results, university good practice and areas that needed improvement, and has also developed a national database platform to accommodate and share basic university information about the three policies, admission systems and employment.
New reforms in Taiwan: development of self-accreditation
The core QA value is continuous self-improvement; therefore, institutions are expected to become learning organizations through well-established internal QA mechanisms, to allow universities to become self-accrediting. To achieve this, each university’s QA office needs to balance these dual responsibilities to ensure self-accreditation value. Self-accreditation tends to apply with a “fitness for purpose” approach only, inspecting if a university’s performance fulfills its specific missions. Within a well-developed internal QA system, institutional capacity will be also enhanced in order to deal with more complicated quality issues, such as program restructuring, faculty development, etc. (Stensaker et al., 2011). With an emphasis on self-enhancement, self-accreditation focuses more on development of internal QA rather than external review. In other words, self-accreditation means that the universities need to clearly identify the roles and responsibilities of the external QA agencies and must learn to partner with other institutions to guarantee accreditation quality.
Since the MOE announcement of the new QA policy in early 2017, the self-accreditation policy by HEEACT (2017) has been revised. First, the self-accreditation eligibility has been expanded beyond the selected institutions, with all institutions that achieved a higher than 80 percent program pass rate in the previous program accreditation cycle now being able to apply for self-accreditation status. The other minor changes were in processes and procedures. While the two phases of proposal approvals and final results recognition are the same, the final results recognition is now to be conducted at the program rather than the institutional level; in other words, the QA mechanism proposal review is based on the whole university, while the second stage final results recognition is now assessed at the individual program level by HEEACT’s self-accreditation decision committee. The reason for this change was to bring the compliance in line with international program quality standards. The MOE also announced that HEEACT was to be the only QA organization allowed to oversee the self-accreditation policy and that all review costs are to be covered by the institutions. The second cycle for institutional accreditation was undertaken in 2017, for which the accreditation standards had been reduced from five to four: governance and management; resources and support systems; institutional effectiveness; and self-improvement and sustainability. The institutional review focus is on the university policy development using objective data and evidence.
In 2017, as well as the change in program accreditation from mandatory to voluntary, and the simplification of the institutional reviews, there were three major QA policies that affected Taiwan’s higher education.
Emphasis on academic autonomy and QA culture embedded
To reduce the university’s burden and increase institutional autonomy, in early 2017, the MOE announced that program accreditation in Taiwanese higher education institutions was now voluntary and that there were now several external review alternatives. First, under the new policy, over 80 percent of Taiwan’s institutions could choose to become self-accrediting universities. Second, universities are now able to select any recognized QA agency to review their programs; however, they can continue to invite HEEACT to be their external examiner. Third, universities can choose not to undertake external program reviews; however, internal QA mechanisms must be established at the institutional level.
Reduction in and sophistication of institutional review procedures and process
In response to requests to reduce the administrative workloads, the second institutional accreditation cycle was simplified; all quantitative data will now be collected from the Higher Institutions Database rather than being provided by the institutions, social accountability and community engagement are key features of the review standards, and universities are expected to demonstrate local engagement and contribute to Taiwanese society.
Transparency and public responsibility
Transparency and social responsibility are two major issues in the second institutional review cycle; universities are now expected to provide institutional data for society and higher education stakeholders on the university website, and must provide underprivileged students with sufficient learning and financial support and make this information public. Through this increased information disclosure and through the provision of equal accessibility, it is hoped that the value and achievements of QA can be more widely known by the public.
The new MOE QA policy has slightly changed the QA ecosystem in Taiwan. Government-funded accreditors or self-funded QA agencies no longer have the mandate to undertake program accreditation, which has pressured them to think of multi-functions as an external QA agency. Currently, the public demand to prove the effectiveness of Taiwan’s accreditors is getting stronger and stronger. Indeed, it is time for Taiwan’s accreditors to transform themselves from a traditional role – a quality regulator, basic quality gatekeeper or project convener – into new multi-roles – quality improvement instigator, capacity developer and even future thinker (Wolff, 2011).
Conclusion and discussion
Influenced by the QA systems in Europe and the USA, both Japan and Taiwan developed QA mechanisms and frameworks, particularly based on central government-led approach. Faced with low birthrates, aging societies and international economic pressure for competition, innovation and high quality manpower, both countries sought to develop efficient systems to improve higher education quality. However, the respective national accreditors, NIAD-QE (Japan) and HEEACT (Taiwan), have struggled to balance between higher education regulation and autonomy, accountability and improvement, and protection and transparency. As in other countries, QA methodology (e.g. internal QA vs external QA, institutional QA vs program Q, or a combination) has also been a key issue when seeking to enhance the quality of higher education institutions.
This comparative study examined the methods used by NIAD-QE and HEEACT to transform their QA frameworks in response to social demands, and particularly focused on an analysis of their respective approaches to the common key concepts of autonomy, accountability, improvement and transparency. The research questions were as follows:
How did NIAD-QE and HEEACT restructure their respective QA frameworks to respond to emerging challenges?
How did the QA agencies balance external and internal QA and institutional and program accreditation to promote university autonomy?
How did the QA agencies find a balance between accountability and improvement?
How did the QA agencies enhance the transparency of the QA mechanism?
First, an overview of the higher education systems, the major policies and the QA system development histories in both countries was given. Then, the QA restructuring, the challenges and the transformations were examined. Based on this information, an assessment was given on the ways NIAD-QE and HEEACT had focused on the key concepts of autonomy, accountability, improvement and transparency, which is summarized in the following. Table I gives a comparative analysis of NIAD-QE and HEEACT.
Autonomy
Initially, NIAD-QE and HEEACT took different approaches to university autonomy. NIAD-QE’s new QA standards emphasized the construction, operation, and monitoring of the internal QA system and also provided guidelines to assist programs build their internal QA mechanisms: therefore, the internal QA was assessed through the lens of the external QA. However, in Taiwan, the internal QA is embedded within the self-accreditation system; therefore, there is reduced involvement of external QA agencies. Further, under the new QA policy, Taiwanese universities have more choices and flexibility when selecting their internal QA approaches, which include self-accreditation, HEEACT program accreditation and third-party program evaluation. Therefore, Taiwanese universities have greater QA autonomy than Japan.
The other way to enhance university autonomy was to reduce the administrative burdens related to evaluation. Both NIAD-QE and HEEACT received complaints from the universities about the heavy evaluation workloads; consequently, under the new reforms, both decided to reduce the number of evaluation standards and indicators. NIAD-QE now only requires selected essential data and evidence for their university analyses, and the HEEACT has introduced improvements in institutional accreditation and quantitative data reporting, all of which has supported their aim or improving QA framework efficiency.
It has been challenging to ensure that the external regulations (external QA) and university autonomy (internal QA) are compatible. While the goal of Taiwan’s 2012 self-accreditation policy was to encourage university autonomy, some universities have struggled with developing objective evaluation standards, selecting peer reviewers and ensuring consistent judgements across reviews. In Japan, while the external regulations for promoting university self-improvement can enhance objectivity and fairness, they may also discourage university autonomy. Therefore, the balance between autonomy and regulation continues to be an issue in both Japan and Taiwan.
Accountability and improvement
What is most important – meeting a minimum standard or improving quality? This question has continued to be a controversial debate about the role of QA. In Japan, although NIAD-QE’s original motivation to launch the CEA in 2005 was to achieve both accountability and improvement, initially, the CEA focused too much on compliance matters to ensure minimum standards; however, in response to public and institutional criticism, QA policy in Japan has shifted to an emphasis on self-improvement based on internal QA, while continuing to ensure compliance. In Taiwan, the HEEACT encouraged universities to develop their own strengths through the launch of the 2012 and 2017 self-accreditation policies; therefore, standards compliance is also no longer the main focus of the new HEEACT accreditation system, with internal QA and quality culture building being regarded as the key elements in the new QA system. Accountability and improvement have often been discussed as dichotomous concepts (Banta and Palomba, 2015; Harvey and Williams, 2010; Harvey and Newton, 2007); however, the recent higher education QA policies in both Japan and Taiwan have seen accountability to be a demonstration of continuous improvement; therefore, accountability is related to proven (and visible) improvements.
In terms of the QA approaches, the NIAD-QE has attempted to achieve continuous improvements based on the universities’ internal QA mechanisms but under the umbrella of external QA standards, indicators, and guidelines. Geven and Maricut (2015) claimed that both “internal” and “external” evaluations were seen as being imposed by “others.” It is also important to note that “institutional” external evaluation (CEA) now requires internal “program” QA (Figure 1). In Taiwan, the HEEACT has updated the self-accreditation policy to program level, and continues to operate external evaluations at both the institutional and program levels. However, unlike Japan, the new QA policy gives the universities a greater range of choices and flexibility for their internal QA approaches at the program level, and have also been given them the autonomy to select from self-accreditation, HEEACT external program evaluation or third-party evaluations.
Transparency
The transparency of QA agencies and universities is important for social responsibility. This study found that the accreditation systems in both Japan and Taiwan have suffered from insufficient social awareness; therefore, both NIAD-QE and HEEACT have recognized that they need to improve communication with the society and stakeholders. NIAD-QE has developed and released their evaluation standards, processes and results based on public comments, stakeholder consultations and a questionnaire, and has developed detailed guidelines to assist universities self-evaluate, and have involved multiple stakeholders (e.g. employers and practitioners) in the peer review teams so as to gain wider opinions from the society. Universities are also required to disclose their self-evaluation results in public and are encouraged to release their information in a detailed portrait. The new Japanese QA mechanism has been based around the mandatory development of three policies (i.e. diploma, curriculum and admission policies) to encourage the universities and programs to be more coherent and transparent. In Taiwan, HEEACT has increased the release of public information through making the final report available on the HEEACT website and releasing review outcomes to the general public at a press conference. Self-accrediting institutions are required to publish all related self-accreditation documents such as QA regulations, review results and reports in specific areas of the official university website.
Nonetheless, accreditation in both Japan and Taiwan has received less attention from society than expected. Therefore, the QA agencies, the universities or both need to develop mass media strategies that share the universities’ distinctive initiatives and unique practices, or need to develop user friendly reports. Further, both QA systems need to involve more stakeholders to reflect a wider range of voices in their evaluation procedures, frameworks and standards.
Figures
Comparative analysis of QA restructuring in NIAD-QE and HEEACT
 |
References
Banta, T.W. and Palomba, C.A. (2015), Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education, Jossey-Bass, San Francisco, CA.
Bernhard, A. (2012), “Quality assurance in an international higher education area: a case study approach and comparative analysis” (Vs Research).
Central Council of Education (2016), Ninshouhyouka no Jyujitsu ni Mukete, MEXT, Tokyo.
Costes, N., Hopbach, A., Kekäläinen, H., Ijperen, R.V. and Walsh, P. (2010), Quality Assurance and Transparency Tools, European Association for Quality Assurance in Higher Education, Helsinki.
Department of Higher Education (2011), “Development plan for world class universities and research centers of excellence”, available at: http://140.113.40.88/edutop/index_1.phpw.edu.tw/high/itemize.aspx?itemize_sn=3520&pages=1&site_content_sn=1234 (accessed December 28, 2013).
Gaston, P.L. (2014), Higher Education Accreditation: How It’s Changing, Why It Must, Stylus Publishing, Sterling, VA.
Geven, K. and Maricut, A. (2015), “A merry-go-around of evaluations moving from administrative burden to reflection on education and research in Romania”, in Curaj, A., Matei, L., Pricopie, R., Salmi, J. and Scott, P. (Eds), The European Higher Education Area: Between Critical Reflections and Future Policies: Part II, Springer, pp. 665-684.
Gover, A. and Loukkola, T. (2015), “Eureqa moments! Top tips for internal quality assurance”, available at: www.eua.be/Libraries/publications-homepage-list/eua_eureqa_moments_web_highq (accessed December 13, 2017).
Harvey, L. and Newton, J. (2007), “Transforming quality evaluation: moving on”, in Westerheiden, D.D., Stensaker, B. and Rosa, M.J. (Eds), Quality Assurance in Higher Education: Trends in Regulation, Translation, and Transformation, Springer, Dordrecht, pp. 225-246.
Harvey, L. and Williams, J. (2010), “Fifteen years of quality in higher education”, Quality in Higher Education, Vol. 16 No. 1, pp. 3-36.
Hayata, Y. and Mochizuki, T. (2012), Daigaku no Globalka to Naibu-Shitsuhosyou, Kouyou-shobou, Kyoto.
Hénard, F. (2016), “Principle 3: quality and society”, in Uvalić-Trumbić, S. (Ed.), CIQG International Quality Principles: Toward a Shared Understanding of Quality, CHEA, Washington, DC, pp. 23-30.
Higher Education Evaluation & Accreditation Council of Taiwan (HEEACT) (2012), “2011 annual report”, HEEACT, Taipei.
Higher Education Evaluation & Accreditation Council of Taiwan (HEEACT) (2015), Quality Assurance and Its Use in Taiwan Higher Education: Implications for Fully Accredited and Non-Fully Accredited Institutions, HEEACT, Taipei.
Higher Education Evaluation & Accreditation Council of Taiwan (HEEACT) (2017), Self Accreditation Handbook, HEEACT, Taipei.
Hou, A.Y.C. (2011), “Quality assurance at a distance: international accreditation in Taiwan higher education”, Higher Education, Vol. 61 No. 2, pp. 179-191.
Hou, A.Y.C. (2015), “Is the Asian quality assurance system for higher education going glonacal?: assessing the impact of three types of program accreditation on Taiwanese universities”, Studies in Higher Education, Vol. 40 No. 1, pp. 83-105.
Hou, A.Y.C., Ince, M. and Chiang, C.L. (2012), “A Reassessment of Asian excellence programs in higher education – the Taiwan experience”, Scientometrics, Vol. 92 No. 1, pp. 23-42.
Hou, A.Y.C., Lin., S.R., Kuo, C.Y., Chen, C.H.J., Chih, J.C.C. and Chou, H.C. (2017), “Self-accreditation policy, quality culture building and role of quality assurance agencies in Taiwan higher education: diversity, autonomy and accountability”, INQAAHE research report, unpublished.
Institute of Development Study (2013), “Learning about qualitative document analysis”, available at: https://opendocs.ids.ac.uk/opendocs/bitstream/handle/123456789/2989/PP%20InBrief%2013%20QDA%20FINAL2.pdf?sequence=4 (accessed December 11, 2017).
Japan Association of Corporate Executives (2013), “Daigakuhyoukaseido no shindankai”, Japan Association of Corporate Executives, Tokyo.
KPMG AZSA LLC (2014), “Gakushu seika no haaku to gakushu seika no hyoka ni tuite no gutaiteki housaku ni kansuru chousa kenkyu houkokusho”, available at: http://www.mext.go.jp/a_menu/koutou/itaku/__icsFiles/afieldfile/2014/07/17/1347643_01.pdf (accessed May 18, 2018).
Liberty Time Net (2018), “50 percentage of evaluation workloads have been reduced: suspension of program evaluation”, available at: http://news.ltn.com.tw/news/life/breakingnews/1968503 (accessed May 20, 2018).
MEXT (2005), “Wagakuni no koutoukyouiku syouraizou”, available at: www.mext.go.jp/b_menu/shingi/chukyo/chukyo0/toushin/05013101.htm (accessed May 18, 2018).
MEXT (2008), “Gakushikatei kyouiku no kouchiku ni mukete”, available at: www.mext.go.jp/component/b_menu/shingi/toushin/__icsFiles/afieldfile/2008/12/26/1217067_001.pdf (accessed December 11, 2017).
MEXT (2013), “Kokuritsu daigaku kaikaku plan”, available at: www.mext.go.jp/a_menu/koutou/houjin/1341970.htm (accessed May 18, 2018).
MEXT (2016a), “Article 16 of Ordinance of MEXT”, available at: www.mext.go.jp/b_menu/hakusho/nc/__icsFiles/afieldfile/2016/04/26/1369884_1.pdf (accessed May 15, 2018).
MEXT (2016b), “Article 17 of ordinance of MEXT”, available at: www.mext.go.jp/b_menu/shingi/chukyo/chukyo4/038/siryo/__icsFiles/afieldfile/2016/04/25/1369683_03.pdf (accessed May 15, 2018).
MEXT (2017), “Gakkou kihon chousa (FY2017 School Basic Survey)”, available at: www.mext.go.jp/component/b_menu/other/__icsFiles/afieldfile/2018/02/05/1388639_1.pdf (accessed June 16, 2018).
Ministerial Conferences (2003), “Bologna process: Berlin 2003”, available at https://media.ehea.info/file/2003_Berlin/28/4/2003_Berlin_Communique_English_577284.pdf (accessed December 20, 2017).
Ministry of Education (MOE) (2005), Revised University Law, Ministry of Education, Taipei.
Ministry of Education (MOE) (2013), Study in Taiwan, Ministry of Education, Taipei.
Ministry of Education (MOE) (2017), MOE Higher Education Sprout Project, Ministry of Education, Taipei.
NIAD-QE (2017), “Kyouiku no naibushitsuhosyo ni kansuru guideline”, available at: www.niad.ac.jp/n_shuppan/project/__icsFiles/afieldfile/2017/06/08/guideline.pdf (accessed December 11, 2017).
NIAD-UE (2011), “Institutional certified evaluation and accreditation of universities standards for evaluation and accreditation of universities: 2012-2019”, available at: www.niad.ac.jp/n_shuppan/package/no9_Standards2012-2019.pdf (accessed December 11, 2017).
NIAD-UE (2016), “Daigaku kikanbetsu ninsho hyoka ni kansuru dai2 cycle no chuukan kekka houkokusho”, available at: www.niad.ac.jp/n_hyouka/jouhou/__icsFiles/afieldfile/2016/06/13/no6_12_dai2chukanmatome.daigaku.pdf (accessed December 11, 2017).
Noda, A. (2017), “Gakusyuseika to hyoka no arikata”, in Tanigawa, H. (Ed.), Clues for Adapting American Learning Assistance into Japanese Higher Education: from Historical and Practical Perspectives, Nakanishiya, Kyoto, pp. 146-153.
Normand, R. (2016), “The politics of standards and quality”, in Normand, R. (Ed.), The Changing Epidemic Governance of European Education, Educational Governance Research 3, Springer, New York, NY, pp. 63-94.
Recruit (2017), “Ninsyou-hyouka: daisan saikuru ni mukete (CEA for the third cycle)”, Recruit College Management, Vol. 204, pp. 1-74.
Science Council of Japan (2012), “Wagakuni no hyoka system no arikata kenkyu-sha wo ikusei shien suru hyoka system eno tenkan”, available at: www.scj.go.jp/ja/info/kohyo/pdf/kohyo-22-t163-1.pdf (accessed May 18, 2018).
Shibui, S., Takahashi, N. and Noda, A. (2017), “A longitudinal study of effectiveness, impact, and challenges in the Japanese quality assurance system”, International Journal of Institutional Research and Management, Vol. 1 No. 1, pp. 83-102.
Smidt, H. (2015), “European quality assurance: a European higher education area success story (overview paper)”, in Curaj, A., Matei, L., Pricopie, R., Salmi, J. and Scott, P. (Eds), The European Higher Education area: Between Critical Reflections and Future Policies, Part II, Springer, pp. 625-637.
Stensaker, B., Langfeldt, L., Harvey, L., Huisman, J. and Westerheijden, D. (2011), “An in‐depth study on the impact of external quality assurance”, Assessment & Evaluation in Higher Education, Vol. 36 No. 4, pp. 465-478.
Wolff, R. (2011), “Conference summary and reflections”, in Neubauer, D. and Hawkins, J. (Eds), Quality in Higher Education: Identifying, Developing and Sustaining Best Practices in the APEC Region, APEC Secretariat, Singapore, pp. 274-283.