Certification bodies’ interpretation and application of the ISO 19011 audit process guidelines

Abstract

Purpose

The purpose of this paper is to show that third-party quality audits (TPQAs) facilitate performance improvement and give confidence to organisations concerning the process quality of services and products. However, because of inconsistencies and unethical practices often observed in the industry, organisations question the significance of TPQA. A perception exists that its initial purpose as an impartial tool ensuring quality of deliverables is no longer upheld. Hence, the need to determine and explain the influence of the ISO 19011 standard interpretation on the application of the audit guidelines in performing TPQA, to promote consistency in the audit process.

Design/methodology/approach

The study employed document analysis of the ISO 19011 standard, followed by semi-structured interviews with certification managers (CBs) to gain insight related to their interpretation and application of the ISO 19011 guidelines.

Findings

The CBs interpret the ISO 19011 guidelines differently; hence, their application of the standard to compile their audit documents differ. Adherence to the principles of auditing particularly, integrity and independence were found as the core of the audit process while their disregard reflects failure of the real intent of auditing. The inconsistencies in the audit procedures and documents developed for auditors are ascribed to some CBs’ personal interpretations.

Originality/value

The study explores how the different interpretations of the ISO 19011 standard prevail and are perceived by the CBs and auditors. The findings aim to support standardisation and reduce the variations across and amongst the different CBs and auditors.

Citation

Sepeng, T.D., Lourens, A., Van der Merwe, K. and Gerber, R. (2024), "Certification bodies’ interpretation and application of the ISO 19011 audit process guidelines", International Journal of Quality & Reliability Management, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/IJQRM-10-2023-0339

1. Introduction

Organisations demonstrate their ability to provide quality services or products meeting customer requirements by obtaining ISO 9001 certification through third-party quality audits (TPQAs) conducted by certification bodies (CBs) (Castelló et al., 2017; Kuceja, 2017; Johannesen and Wiig, 2020). In this study “CBs” refers to people competent to establish processes and procedures to manage the audit programme, and assist the auditors in compiling their own audit documents. ISO 9001 certification gained popularity as a requirement in business tender criteria (Becker and Van Dyk, 2018; Kartha, 2022) to provide assurance to the customers concerning the quality of services and products. The demand for ISO 9001 certification resulted in an increase in the number of CBs, stimulating competition, which resulted in apparent flaws in the audit process (Prajogo and Castka, 2015). The CBs saw this demand as an opportunity to embark on consultancy services which besides certification services assisted organisations in implementing quality management systems (QMS), and ultimately getting these certified (Heras-Saizarbitoria and Boiral, 2018). However, certification is criticised internationally for no longer accomplishing the industry needs (Heras-Saizarbitoria and Boiral, 2018; Ab Wahid, 2021; Kartha, 2022; Saida and Taibi, 2021). Generally, the audit process is criticised for failing to identify problems in QMS (Simons et al., 2017). The current study explored the case locally in South Africa.

Organisations started moving from one CB to another searching for reliability and value; however, no difference and improvement were observed. Reportedly, confidence diminished due to the ineffectiveness of the audit process, and lack of auditor competency, integrity and commercialisation (Castelló et al., 2016; Castka et al., 2015; Castka et al., 2020). Such challenges, which are ongoing, suggest the need to extend the TPQA beyond compliance, to add value in auditing (Lenning, 2019; Ab Wahid and Tan, 2021). Situations have occurred that many organisations in South Africa started questioning the significance of these audits (Piek, 2019; Tayo Tene et al., 2018).

Perceived weaknesses of the audit process such as those mentioned raised alarms, as they seemed to undermine the ISO 19011 guidelines, and there were similar and ongoing concerns in several countries. For example, although Quality Council of India (QCI) (2008) and (Castka et al., 2015; Quality Council of India, 2008) reported concerns by the quality professionals about the uncertainties surrounding the TPQA, the causes were not removed.

To assure the quality of audits, certification bodies implement systems based on the ISO 17021-1: 2015 standard for conformity assessment consisting of requirements for bodies providing audit and certification of MS (International Organization for Standardization, 2015). ISO 17021-1 specifies the principles and general requirements for certification bodies to consistently and impartially determine and assess conformity of MS using competent auditors, with adequate resources.

Besides being audited against the ISO 17021-1 standard, CBs also follow the guidelines provided in the ISO 19011 standard (Johannesen and Wiig, 2020; Piek, 2019). The ISO 19011 standard consists of guidelines for auditing MS (ISO, 2018). It corresponds to ISO 17021-1 as the foundational standard that formed a basis for setting the requirements for the third-party certification of MS contained in ISO 17021-1 (ISO, 2011; ISO, 2018; ISO 9001 Auditing Practices Group, 2020). However, ISO 19011 is explicit in that it was prepared to provide guidelines and not requirements for conformity assessments (ISO, 2011; ISO, 2018).

As described in ISO 19011, process effectiveness and reliability are reliant on knowing, understanding and applying of the audit principles (Godfrey et al., 2021); and the establishment of an audit programme (Pereira, 2018). Although not drawn as a specification the ISO 19011 standard is meant to provide guidance, which could lead to the effectiveness of the audit process. Studies assert that problems of the audit process do not receive adequate attention (Castelló et al., 2017; Johannesen and Wiig, 2020).

Whilst research has addressed the various aspects of the quality audit process, few academic studies were undertaken to determine the practitioners’ perceptions of the TPQA (Castka et al., 2015; Domingues et al., 2019; Johannesen et al., 2020). Evidently, not much attention is given to how the ISO 19011 standard is interpreted and applied in conducting TPQA (Prytulska et al., 2019). According to Pereira (2018) the success of an audit is dependent on the correct interpretation of the ISO 19011 guidelines. Therefore, this study aimed to determine and explain how the interpretation of the ISO 19011 standard influences the application of the audit guidelines in conducting TPQA, to promote uniformity and consistency in the audit process. This study sought to fill this gap in the literature by addressing the main research question, and sub-questions:

How does the ISO 19011 standard interpretation influence the application of the audit guidelines during the TPQA process?

Research sub-questions.

1. What are the factors influencing the application of the audit guidelines in the TPQA process?

2. What are the perceptions of the CBs related to their interpretation and application of the ISO 19011 standard when developing their audit process guideline documents?

It is anticipated that formal consultations and reviews would be initiated among practitioners to reflect on the interpretation of the ISO 19011 standard to eliminate root causes and practices affecting the TPQA process, improve the management of audit programmes and delivery capability by integrating the outcomes into auditor training courses in South Africa.

The remainder of the paper is structured as follows: The second section presents a brief review of existing literature on quality auditing, followed by the description of the methods employed, namely, document analysis of the ISO 19011: 2018 standard and semi-structured interviews with the CBs. The fourth section organised analysed and interpreted the data, leading to the findings discussed in the fifth section. The summary and conclusions of the findings are presented in the sixth section, and finally, the recommendations and contributions which include social, academia and practical implications are made.

2. Literature

Since the inception of the ISO 9001 standard, organisations globally are pressing forward their efforts to improve competitiveness by certifying their QMS. The organisations demand that their suppliers achieve ISO certification for systematic ordering of measurement of supplier capability and resources (Castka et al., 2015; Kartha, 2022; Simon and Kafel, 2018). Subsequently, certification became a marketing tool for business tenders, because it presented organisations with competitive benefits and disadvantaged the uncertified. In most South African organisations, ISO 9001 certification forms part of the critical supplier criteria (Becker and Van Dyk, 2018).

When attention was drawn to the certification practice, numerous CBs came into the market (Prajogo and Castka, 2015). The need for certification influenced the CBs to become too commercial. Since they became a marketing tool the variations of certifications increased as the years progressed and the integrity of certification and the audit process started to be questioned (Prajogo and Castka, 2015; Castka et al., 2020), particularly its necessity and effectiveness (Simons et al., 2017). Moreover, Clougherty and Grajek (2023) found in their recent study that organisations withdraw or decertify their QMS due to the costs of recertification being greater than the benefits.

Organisations shared disappointments as expectations for implementing and certifying their QMS were not realised (Alolayan, 2014). The TPQA became added cost activities (Castelló et al., 2016), which often failed to improve the organisation’s financial performance (Simon and Kafel, 2018). According to Prytulska et al. (2019), ISO 19011 underpins all MS audits. However, Castka (2011) states that there was a lack of worldwide acceptance of the ISO 19011 standard.

Pereira (2018) asserted that the guidelines need to be interpreted correctly to have a successful plan, audit and report. The TPQA should be conducted by competent and independent CBs using external auditors to rule out subjectivity. Kuceja (2017), and Johannesen et al. (2020) argue that quality audits are indeed appropriate for evaluating conformance of the QMS for industry standards and for measuring performance and improvement. It must be recognised however, that audits are not different from other tools and can fail to achieve set objectives, subsequently failing in its effectiveness. According to Prajogo and Castka (2015) and Godfrey et al. (2021), audit effectiveness is reliant on the credibility of the audit process, which is also dependent on the people involved (Johannesen and Wiig, 2020; Ab Wahid and Tan, 2021).

Previous research recognises the significance of the quality audit process (Lenning, 2019), and includes understanding the weaknesses and determining perceptions of the audit process (Heras-Saizarbitoria and Boiral, 2018; Domingues et al., 2019), the need for improvement of the QMS auditing process (Lenning and Gremyr, 2022). In addition, Liao (2014) and Ab Wahid et al. (2019) identified a gap in the audit performance and customer expectations, and reinforced the need for external auditor education; while according to Kluse (2012), the people involved in registration schemes are not practising auditors and are detached from the audit profession. Castka and Balzarova (2018), and Johannesen and Wiig (2020) pointed at variations in the quality of service and audits by certification bodies, and “fake certification” was highlighted by (Heras-Saizarbitoria and Boiral, 2018).

Research indicates that TPQAs no longer fulfil the objective of finding out whether the QMS enables organisations to provide products or services with persistent quality as agreed by conforming to the ISO 9001 standard requirements, and continuously improving the QMS for customer satisfaction (Dalmau et al., 2016; Castka and Balzarova, 2018; Prajogo et al., 2018). It appears that the purpose of serving as an impartial tool to ensure reliability and quality of deliverables in an objective manner is no longer upheld (Brooks and Gunning, 2022). Variations are noted in the conduct of the audit process, inconsistencies in the audit outcomes (Simons et al., 2017; Castka and Balzarova, 2018; Johannesen et al., 2020; Johannesen and Wiig, 2020) and poor conduct of auditors (Boiral et al., 2019). Brooks and Gunning (2022) remarked on the inconsistencies and how less codified the requirements for audits were in training and expertise of auditors.

Castka et al. (2020) pointed out the credibility concerns about the ISO standards certification as comparable to the voluntary sustainability standards’, and relate them to perceived inconsistencies in auditing. Despite existing studies, CBs reportedly dispute inconsistencies in the audit quality of any of their clients. Impartiality or lack of independence facilitates conflicts of interest between certification bodies and the organisation being audited (Castka et al., 2020). Despite the studies above, the value of the audit process continues to be questioned (Castelló et al., 2017; Ab Wahid and Tan, 2021).

There has been little research on audit process improvement and organisations are losing confidence because of the inconsistencies observed in the audit process, and failure to meet expectations (Simons et al., 2017; Ab Wahid, 2021). The case that TPQA process needs improvement has been made before (Lenning and Gremyr, 2017).

Consistently, audits must be adaptable to change since the QMS and the overall business are evolving (Kafel and Rogala, 2022). Studies on the impact of the pandemic on CB functioning found that the CBs have vastly applied remote auditing by changing their audit policies to achieve their obligations (Kafel and Rogala, 2022). Remarkably, analysis of the on-site and the remote auditing methods showed no statistically significant differences between them. The possibility that carrying over of some of the inconsistencies perceived in on-site auditing exists.

While earlier research has investigated MS audits, there still remains a poor understanding of what and how the audit process is affected (Castelló et al., 2017). Correspondingly, a few, if any, empirical studies have been conducted, particularly in South Africa, to determine and explore the shortfalls in the TPQA processes (Prajogo and Castka, 2015; Abuazza et al., 2019; Ab Wahid, 2021).

Understanding of the ISO 19011 standard plays a pivotal role because its application in the audit process is dependent on how it is interpreted, which is what the current study will determine.

3. Method

A limited number of studies were found about the TPQA process (Prajogo and Castka, 2015) particularly in South Africa, this lack indicated a need for a qualitative, exploratory research (Patton, 2001). Document analysis of the ISO 19011 standard was first conducted to determine the factors that can influence its application in the audit process.

3.1 Data collection: document analysis

A deductive approach was followed using Braun and Clarke’s thematic analysis where a priori codes from the ISO 19011 standard were imposed on the data at category level (Braun and Clarke, 2006). The ISO 19011 standard was analysed to establish how these codes manifested in the document. By systematically working through and organising the data, additional codes were generated to identify sections of text that fit under each a priori category. From the relationships found between codes, it was necessary to elevate one of the initial a priori categories to become a theme. Some categories fitted under this theme, while the other categories were combined to form the second main theme.

The themes were reviewed checking whether they formed a coherent pattern at the level of extracts, and then at the whole dataset by checking if individual themes were valid. The themes were then defined and refined to inform what they were about, the characteristics of the data; and were then properly named principles of auditing and managing an audit programme. The findings of the analysis were reported with the themes supported by data extracts identifying the relationships among codes and themes. An interview protocol (Table 1) to interview the CBs was then developed based on the document analysis findings of the ISO 19011 standard.

3.2 Data collection: semi-structured interviews

Qualitative exploratory research was conducted owing to the negligible amount of research previously done on the TPQA process (Schutt, 2012). Patton (2001) reports that qualitative research is more useful in researching topics with limited existing knowledge and that are not well-defined. For its flexibility and the freedom to generate questions based on the context of the discussions, semi-structured interviews were employed to gain insight from the CBs about their interpretation and application of the ISO 19011 guidelines when developing their audit process documents and procedures (Rubin and Rubin, 2005).

The interview protocol in Table 1 was used to collect the data from the CBs. The CBs were allowed to formulate answers and the researcher to elaborate on the questions and intervene only when seeking clarification, until satisfactory responses were obtained (Rubin and Rubin, 2005, 88). The protocol confined the interviews to the aim of the study through research questions, and allowed probing with relevance (Creswell, 2009, chapter 7).

The invitation to participate in the study was accompanied by an information sheet about the interview, the study background, and guidelines on how the interview would be conducted. The procedure consisted of the interview set-up, an introduction which included informed consent approved by the university’s Ethics Committee, assurance regarding confidentiality and the actual interview process. Consistency between the interviews and reliability of the findings were maintained by use of a single interviewer, the researcher (Boyce and Neale, 2006, 5). The interviews were audio recorded, with permission from the CBs. The data was transcribed verbatim and stored as text.

3.3 Participants

Participants were selected purposefully based on their expertise as persons who manage the audit programme (CBs), and who have knowledge and experience enabling them to provide rich information about issues of importance on the TPQA process (Babbie and Mouton, 2001). The CBs had experience in TPQA, and had audited ISO 9001 certified organisations in South Africa. They were knowledgeable and experienced in the ISO 19011 standard.

Owing to the exploratory nature of the study, an in-depth investigation of a small population of CBs was conducted (Schutt, 2012). The sample was obtained from different CBs in South Africa through an Internet search. Of the identified ten accredited CBs, only three showed interest and willingness to participate; two were not interested, four could apparently not obtain approval from the organisations’ management based abroad, and the remaining one did not have time to spare. Hence, the study proceeded with three CBs because it aimed to gain understanding rather than finding representative data.

4. Organising the data

Data from the CBs interviews were uniquely identified to ensure that they are not traceable and to maintain anonymity. The researcher listened to the recordings and studied the transcripts, to facilitate data analysis. A semantic approach to deductive thematic analysis was employed (Braun and Clarke, 2006). The data identified aspects that formed the basis of repeated patterns, and gave meaning to explain observed patterns (Braun and Clarke, 2006).

A working document was prepared consisting of pre-determined codes from the ISO 19011 standard (Creswell, 2014), namely principles of auditing, managing an audit programme, establishing the audit programme, implementing the audit programme, monitoring the audit programme and reviewing and improving the audit programme. These codes were imposed on the data at category level to enable a description of how they manifest in the data. The code descriptions assisted in the interpretation of data.

The coding process entailed reading through the interview transcripts, highlighting and labelling segments of the verbatim textual data (Braun and Clarke, 2006). The codes were matched, sorted, and organised into categories. Throughout the process, codes were assigned and reassigned to categories that best represented them (Braun and Clarke, 2006).

4.1 Credibility, dependability and confirmability of data

Credibility was ensured through persistent engagement with the interview data recordings, notes and transcripts to demonstrate the connection between the data and interpretations (Tobin and Begley, 2004). The verbatim responses revealed the range and tone of the data validated by the audio-recordings. This was assured by consistencies given by the credibility and accuracy of data obtained from the semi-structured interviews (Neuman, 2007).

For dependability and confirmability electronic files, audio recordings and hard copies of documentation are maintained, and for ease of retrieval and examination (Tobin and Begley, 2004). Since the interviews and transcriptions were conducted by the researcher, data collection and analysis happened simultaneously. Consistency was maintained and the findings demonstrated that they were derived from data. Also, the interview protocol, informed consent, ethics statement and the information sheet authenticated the interviews.

5. Findings

The following sections report on the factors derived from the document analysis of the ISO 19011: 2018 standard and are considered to be influencing the application of the audit guidelines in the TPQA process.

5.1 Findings of the ISO 19011 guidelines document analysis

Based on the document analysis of the ISO 19011: 2018 standard, two main themes manifested, namely, the principles of auditing and managing the audit programmes. The principles of auditing highlight integrity, fair presentation, due professional care and confidentiality, independence, evidence-based and risk-based approach as important factors that can influence the application of the audit guidelines (Prytulska et al., 2019). The analysis imply that upholding ethical practices by being honest and truthful can foster strong moral principles that ensure that the audit responsibilities are performed in an impartial manner, with fair judgement and without threats of CBs and auditors compromising their integrity (Serrano and Wellbrock, 2021).

Due professional care not applied in the audit process can hinder the auditee’s confidence in the CBs and auditors. Unprofessional conduct can cast doubt in reliability, and competence of the CBs and auditors to deliver on audit requirements without compromising value systems (Castka et al., 2020; Ab Wahid and Tan, 2021). Understanding of confidentiality and appropriate use and protection of auditee information can also influence the application of the audit guidelines facilitating ease of the auditees to share organisational documentation for establishing and managing the audit programme.

The findings further reveal independence as another critical factor that can influence objectivity of the findings based on the evidence gathered during the audit process, and the relationship with the organisation being audited (Castka et al., 2020; Johannesen and Wiig, 2020). Auditing an organisation for which work was previously performed or once consulted can create conflict of interest and produce a biased audit report. Furthermore, in agreement with Ab Wahid et al. (2019) it appears that the type of audit approach which is not evidence-based, cannot provide proof of occurrence of the audits as activities are sampled within the limited audit periods. The audit findings that cannot be verifiable cannot assure confidence in the audit conclusions (Godfrey et al., 2021). Understanding of risks and opportunities throughout the audit programme, is also observed as a factor that can influence the application of the audit guidelines in determining the scopes and applicable resources to ensure achievement of the objectives (Serrano and Wellbrock, 2021).

Furthermore, a number of factors that can influence how the guidelines are applied were also identified in managing the audit programme and they are included in the establishment, the implementation, the objectives, the monitoring, reviewing and improvement of the audit programme. Observed in the studies by Ab Wahid and Tan (2021), it appears that the success of an audit programme depends on the competency of a person managing the programme (CB). Competency promotes the understanding of the guidelines, and can affect how roles and responsibilities of the CBs are defined.

The findings indicate that facilitation of effective planning and the conduct of audits can be influenced by the audit programme objectives and should be established and implemented (Chiromo and Msibi, 2023). These objectives should be measurable to direct the audit programme which would otherwise be random. Establishment of the audit programme can be affected by not fully understanding and defining the extent of the audit programme. The extent that is poorly defined tends to influence identification and provision of resources which includes the selection of competent auditors and team members, finances and audit methods (Serrano and Wellbrock, 2021). These resources should prioritise high risks to matters of significance so that control measures can be applied to achieve objectives.

Implementation of the audit programme provides for clearly defined objectives, scope and criteria for an individual audit, and determination of the auditing methods (Johannesen and Wiig, 2020). A properly defined scope can facilitate planning and audit approaches to be employed. Domingues et al. (2019) asserted that the selection of the audit team members should be performed with care to consider the competencies and skills required to achieve the objectives of the audit, within its scope and set criteria. Technical experts can supplement the audit team if skills shortages are identified.

Establishing procedures for the audit programme can influence reliability to manage the programme in a systematic and consistent manner to effectively plan, schedule, conduct audits and review performance of the audit programme (Johannesen et al., 2020; Serrano and Wellbrock, 2021). Hence, underlying challenges can be discovered and improvement suggested for subsequent audit plans.

Managing and maintaining the audit programme outcome and records can provide objective evidence to demonstrate achievement of the audit programme objectives (Chiromo and Msibi, 2023). Review of the audit reports can evaluate the correctness of audit findings and verify if they are acceptable. Furthermore, the findings indicates that monitoring, reviewing and improving the audit programme can promote ongoing effective accomplishment of the objectives and facilitate risk assessments that can suggest preventive actions to realise effectiveness (Serrano and Wellbrock, 2021). This indicates the impact that evaluation of performance of the audit team and the feedback from the interested parties can have by revealing whether the planning, conduct, and reporting conformed to the set objectives (Ab Wahid and Tan, 2021). The gaps would otherwise not be known and hence no improvement measures would be introduced.

Overall, the analysis provided insight into the influencing factors, and what is expected of the CBs when establishing the audit documents to assist the third-party quality auditors.

The next section, reports on the perceptions of the CBs related to their interpretation and application of the ISO 19011 standard when developing their audit process guideline documents.

5.2 Findings of the interviews with the CBs

The factors identified in the document analysis informed the basis of an interview protocol used to address the second sub-question by exploring the perceptions of the CBs related to their interpretation and application of ISO 19011. The focal themes, namely the principles of auditing and managing an audit programme, supported by the interview data showed the relationships among codes and themes. The CBs perceptions and application of the ISO 19011 guidelines are summarised in Table 2.

6. Summary and conclusion

The guidelines for achieving effective audits as provided in the ISO 19011 standard are that the CB would have established an audit programme, ensured its implementation, and have had it monitored, reviewed and improved (Prytulska et al., 2019; Serrano and Wellbrock, 2021). These would be within the confines of the prescribed audit principles (Godfrey et al., 2021). However, it has been found that the interpretation of the ISO 19011 guidelines is subject to several influences that could limit its effectiveness.

Common understanding of the audit process was observed amongst the CBs in relation to the ISO 19011 standard. However, the study revealed that some clauses were interpreted differently and hence applied differently when CBs compiled documents for the auditors. The differences created potential conflicts since most auditors are contracted across the CBs, and have to work according to the CBs’ processes.

The findings showed that the audit process is dependent on adherence of the audit principles by both the CBs and the auditors (Godfrey et al., 2021); also, that they need to be independent of the MS audited, follow an evidence-based approach and act professionally with integrity exercising confidentiality throughout the audit process (Castka et al., 2020; Serrano and Wellbrock, 2021). However often the CBs tend to be biased despite the impartiality statement they commit to and have conflicts of interest due to the multiple roles they play in their organisations, which factors bring about a loss of integrity.

Management of the audit programme was also discussed. Competence featured in most sub-themes so that lack of it was seen as the main hindrance to the process (Lenning and Gremyr, 2022). To ameliorate this problem the CBs made suggestions for improving auditors’ training. The CBs also believed the problems are not with the selection process but with the auditors’ competence in relevant technical fields (Johannesen and Wiig, 2020; Brooks and Gunning, 2022). Furthermore, it is believed that most auditors do not have the experience to manage people or work environments, and that their competences fall short of certain management skills (Lenning and Gremyr, 2022).

Some CBs have different interpretations which compromise reliability and repeatability of the audit process. Differences in interpretations could be ascribed to the use of the audit procedures and documents developed for the auditors by different CBs (Johannesen et al., 2020). The prevalence of the inconsistencies could be perpetuated by the in-house training of auditors by CBs, as also remarked by Brooks and Gunning (2022) about the audit training requirements being less codified and not well-developed. Sub-contracting of freelance auditors could also be contributing to the varied interpretations as these auditors are shared across the industry to supplement the shortage of resources. The auditors are required to apply different audit guidelines and procedures compiled by the CBs they work for at a specific point in time. Discrepancies arise when one CB interprets the standard differently from another. Different adaptation to suit the CBs MS inevitably affects the audit process, and if not corrected proliferates across the CBs. Deriving from the standard, certain terms and statements could be understood differently and only certain guidelines adopted for application as the ISO 19011 standard is not mandatory.

It can be concluded that the application of the ISO 19011 guidelines can be affected by several factors (Godfrey et al., 2021; Serrano and Wellbrock, 2021). The factors identified in this study have the potential to hinder the development of the audit documents if not well-interpreted (Prytulska et al., 2019). The influence of adherence to the principles of auditing particularly, integrity and independence were highlighted as the core of the audit process while their disregard reflected failure of the real intent of auditing.

In their perceptions, the participants highlighted that different CBs have different interpretations of ISO 19011 guidelines, and based on that, their application of the standard to compile the audit guidelines differs. Considering that the CBs attended the same training in ISO 19011 standard, and conduct internal training for their auditors, the question arises whether training may not be the source of misinterpretation of the ISO 19011 guidelines.

7. Recommendations and contribution

It is recommended that consultations be initiated among practitioners and experts to reflect on these interpretations towards improving the management of audit programmes and controls, and to begin eliminating root causes and practices affecting the TPQA process. A review of the competence requirements in the study could present ground for establishing a unit standard training programme that would provide standardisation of TPQA training. Linking competence in QMS auditing to professional competence should be investigated to discourage lack of industry-specific knowledge or technical skills. Human attributes should be given sufficient attention considering the impact they have in the audit process. An assessment tool can be developed to be used in auditor selection and evaluation. Furthermore, TPQA should be regulated to eliminate unethical conduct of CBs and auditors. Clear segregation of auditing business from consulting business should be made.

Further studies, not reported in this article, have developed an instrument for collecting quantitative data to verify the empirical framework against the perceptions of practitioners and experts relating to the interpretation and application of the ISO 19011 guidelines in the field of quality auditing.

7.1 Practical applications and implications for social and academia

The findings can create awareness of challenges and enable a new approach to deal with the interpretation of the ISO 19011: 2018 standard by learning about its influence in the present, and implementing changes henceforth. The findings of the study are furthermore anticipated to contribute towards improvement of the delivery capability of TPQA by integrating the outcomes into auditor training courses in South Africa. Since auditors receive training based on the ISO 19011: 2018 standard, knowledge of the influence of its interpretation and its impact would be highlighted as a measure to control and discontinue the perceived irregularities before trainees qualify as practicing auditors. The training would also support standardisation and reduce the variations across and amongst the different CBs and auditors. The CBs will be guided on what should be emphasised when compiling documents for the audit process to improve audit performance.

Considering that quality plays a vital role in different industries as observed through organisations mandating MS certification as a requirement for sourcing business, implementation of the recommendations of the study will revive the failing confidence in the credibility of TPQA. As a consequence, trust and credibility of TPQA as a tool for QMS improvement can be restored. The demand for TPQA justifies the need for more credible, performance-changing audit process to assure service and manufacturing deliverables.

The influence of interpretation of the ISO 19011: 2018 standard is a critical area in the audit process that many researchers have not explored. Using different research paradigms would be helpful for future discussions that would lead to a more in-depth analysis, and propelling new knowledge and theories on the interpretation of the ISO 19011: 2018 standard that may be arrived at. Expansion and refining of the empirical framework using larger and more varied groups of participants would be helpful to develop a useful tool to identify factors that hamper the auditing process.

Table 1

Interview protocol used to collect data from interviewing the CBs

Interview protocol

Interview Title: The influence of the interpretation of ISO 19011 guidelines on conducting TPQA at ISO 9001 certified organisations in South Africa

Introduction

Biographical Information

a) Institution/Organisation

b) Interviewee (Title and Name)

c) What role do you play in your organisation?

d) How long have you worked for the organisation?

e) Which CBs have you worked for before?

f) Are your experiences working for CBs the same?

g) How long have you been in a quality position?

Instructions to the interviewee (opening statements)

a) Risks, data collection for research purposes only

b) Consent

c) Confidentiality: code names and organisation

d) Research Ethics Committee (Human) of the University

e) Audio recorded

1. What elements do you think people look for in audits?

2. How are these elements presented in the audit programme guidelines?

3. How do you ensure audits are effective and reliable?

4. Describe how you apply the guidelines for auditing management systems in third-party quality audits

5. How do certification bodies influence the conduct of the audits?

6. In what ways can audit quality be compromised?

7. How free are programme managers to exercise their judgement in interpreting the audit guidelines?

8. What documents are you using together with ISO 17021/19011 in compiling audit guidelines?

9. How do you interpret the extent of the audit?

10. How do you ensure accomplishment of the objectives is assessed? And what performance indicators are used to determine the accomplishment degree?

11. To what extent are audit guidelines you develop more detailed to be understood by auditors to promote audit effectiveness?

12. In your opinion how auditee-specific are audit programme procedures?

13. Do you think the audit guidelines should be more industry-specific or custom-made or generic?

14. What information do you collect from the auditee to guide in the development of the audit programme?

15. Based on interpretation of the audit objectives, how can an auditor change the audit methods?

16. How do you orientate yourself to the auditee’s site/documentation when setting audit objectives?

17. What audit methodologies do you employ to enhance the auditor’s ability to challenge the auditee’s management system?

18. What in the guidelines for auditing MS should be enhanced to improve the application in the audit process guidelines?

19. To what extent are the risks identified and mitigated?

20. How does resource provision impact on the effectiveness of the audit?

21. To what extent are technical skills, personal qualities and practical experiences key drivers to audit quality?

22. How do you ensure compliance with audit principles?

23. How do you measure auditor performance? What techniques are used?

24. How do performance reviews within your organisation enhance auditors’ technical expertise?

25. How adequately do the programme managers discharge their responsibilities? What might be added or considered to enhance effectiveness?

26. Can you take me through the process of declaring programme managers competent to manage the audit programme?

27. How often are there disputes from your customers (auditees) regarding the way audits are conducted?

28. What suggestions would you make on assessing the audit programme effectiveness?

29. How do you maintain audit documents and records?

Closing

Transition messages for the interviewer: Is there anything to add that would add to the body of knowledge?

Source(s): Author’s own creation

Table 2

Findings relating to the CBs perception, interpretation and application of the ISO 19011 standard

Themes	Factors	Findings from the CBs interviews
Principles of auditing	Integrity	Some CBs are becoming inconsistent in terms of moral character, honesty and truthfulness. They succumb to the influences of the auditees to issue certification in exchange for business opportunities. Inducements are offered to the CBs and the auditors in exchange for certification
		Having a relationship with the auditee exonerates unethical practices, making it acceptable to issue certification even if the management system does not conform to the requirements
		Unaccredited, incompetent CBs prey on gullible organisations, offering certification at low cost, thereby luring organisations to contract them
		A focus on profit margins causes the CBs to accept more work with inadequate capacity at the expense of quality
	Fair presentation	The auditors have two different personalities: the one for which they are ordinarily known, and the other for auditing
		The auditors cannot handle what the ISO 19011 standard requires, so they assume a hostile auditing personality
		The audit tools are used to hide and overcome the lack of interpersonal skills
		The auditors’ different interpretations of the standards cause unresolved diverging opinions between them and the CBs
	Due professional care	Personality and interpersonal skills are not as comprehensively addressed in the ISO 19011 standard compared to the technical requirements
		The lack of interpersonal skills prevents the auditors from engaging the auditees and interrogating the management system in fear of being challenged by the auditees. They intimidate the auditees, and do not afford them opportunities for interaction or debate
	Confidentiality	Even though the CBs commit to non-disclosure, the auditees remain reluctant to submit their organisational documentation to them
	Independence	Despite commitment to impartiality, not all the CBs and auditors are independent of the management system they audit. Close relationships with the auditees are in some cases not perceived as a threat to the audit independence. Moreover, helping the auditees with both implementing and auditing the system is not always perceived as a conflict of interest
	Evidence-based approach	Observations and works, rather than training certificates, are regarded as evidence demonstrating competence
	Risk-based approach	The CBs are prepared for responding to potential audit programme risks
Managing an audit programme	Managing an audit programme	The audit programmes are managed according to the ISO 19011 standard and the requirements of the International Accreditation Forum
	Roles and responsibilities of the person managing the programme	The CBs are the owners of the certification bodies, programme managers and lead auditors
	Competence of the person managing the programme	The CBs have extensive experience to conduct third-party quality audits and to manage the audit programme
		Technical knowledge is not necessarily relevant when auditing ISO 9001 management systems as the focus should be on the management of a quality system, the mechanisms used and the processes; not on what the business is about
	Establishing the extent of the audit programme	While onsite reviews are performed to physically verify the accuracy of the information initially supplied by the auditees, familiarity with the various organisations sometimes is used to exempt the CBs from performing onsite reviews of those auditees’ processes; furthermore, only the implementation of the ISO 9001 framework is verified
		The extent of the audit programme is sometimes established before the stage one audit
	Identifying and evaluating audit programme risks	The CBs are prepared for responding to potential audit programme risks
	Establishing procedures for the audit programme	Procedures for the audit programme are established based on the ISO 19011 and ISO 17021 standards
	Identifying audit programme resources	Many auditors have no management experience, knowledge or skills
		There is a shortage of auditors in senior professional positions because they are not paid well
		South Africa seems not to have auditors of the same high-quality standard as certain countries overseas, which have doctoral engineers and scientists working for the CBs on a full-time basis
		Competence criteria do not include personality, interpersonal skills or self-confidence
	Defining the objectives for an individual audit	The audit objectives are defined and achieved irrespective of how activities are planned
	Defining the scope for an individual audit	The auditees often disagree with the predefined plans on the day of the audit despite prior acceptance, thus affecting the audit scope
		The stage one audit is usually used to collect information to define the audit scope
	Defining the criteria for an individual audit	The audit criteria are defined using information obtained from the auditees and are sometimes limited to tools such as checklists containing only the ISO 9001 system requirements and no technical questions
		The auditors seem to be prescriptive and do not audit according to the set criteria, but have their own requirements, not based on the standard
		Technical/process-related questions are often avoided and not considered to form an essential part of the audit criteria
	Selecting the audit methods	The remote auditing method is not popular and seldom used
		Some auditors might use checklists to hide their incompetence
		While implementation of procedures is reviewed, outcome verification and correctness checks are not always performed
	Selecting the audit team members	Problems with the audit teams are often due to a lack of competence and not necessarily due to the selection process
		The CBs subcontract auditors to supplement inadequacies in the competence requirements
		Many auditors are not of age, not registered professionals, do not have management skills, or experience, and hold no senior positions at their full-time employment
		The CBs are under pressure to fulfil the needs of the customers which leads to signing auditors off and engaging them to audit without their having the necessary competence
		Unlike the technical skills, people’s personalities and interpersonal skills are not given enough attention when selecting auditors
	Assigning responsibility for an individual audit to the audit team leader	The CBs are also lead auditors and share certain audit responsibilities with the other lead auditors
	Managing the audit programme outcome	The CBs are responsible for both the audit results and approval of the audit reports
	Managing and maintaining the audit programme records	The audit records and documents exchanged with the auditees are managed as contractual documents
	Monitoring the audit programme	The ISO 19011 standard is generic and gives no guidance on how to monitor implementation and evaluation of conformity; hence the CBs compile their own procedures aligned with ISO 17021 standard
		The interpersonal skills are difficult to measure formally as the evaluations are not defined; but only proof of competence from a qualification and work-related experience
	Reviewing and improving the audit programme	The review and achievement of objectives is demonstrated by different measures which include customer satisfaction, yearly evaluations, witnessing of audits and issuing of certification to the auditees

Source(s): Author’s own creation