Learning via assistance systems in industrial manufacturing. An experimental study in an Industry 4.0 environment

Abstract

Purpose

The purpose of this paper is to investigate how learning solely via an assistance system influences work performance compared with learning with a combination of an assistance system and additional training. While the training literature has widely emphasised the positive role of on-the-job training, particularly for groups that are often underrepresented in formalised learning situations, organisational studies have stressed the risks that emerge when holistic process knowledge is lacking and how this negatively affects work performance. This study aims at testing these negative effects within an experimental design.

Design/methodology/approach

This paper uses a laboratory experimental design to investigate how assistance-system-guided learning influences the individuals’ work performance and work satisfaction compared with assistance-system-guided learning combined with theoretical learning of holistic process knowledge. Subjects were divided into two groups and assigned to two different settings. In the first setting, the participants used the assistance systems as an orientation and support tool right at the beginning and learned the production steps exclusively in this way. In the second setting, subjects received an additional 10-min introduction (treatment) at the beginning of the experiment, including detailed information regarding the entire work process.

Findings

This study provides evidence that learners provided with prior process knowledge achieve a better understanding of the work process leading to higher levels of productivity, quality and work satisfaction. At the same time, the authors found evidence for differences among workers’ ability to process and apply this additional information. Subjects with lower productivity levels faced more difficulties processing and applying additional process information.

Research limitations/implications

Methodologically, this study goes beyond existing research on assistance systems by using a laboratory experimental design. Though the external validity of this method is limited by the artificial setting, it is a solid way of studying the impact of different usages of digital assistance systems in terms of training. Further research is required, however, including laboratory experiments with larger case numbers, company-level case studies and analyses of survey data, to further confirm the external validity of the findings of this study for the workplace.

Practical implications

This study provides some first evidence that holistic process knowledge, even in low-skill tasks, has an added value for the production process. This study contributes to firms' training policies by exploring new, digitalised ways of guided on-the-job training and demonstrates possible training benefits for people with lower levels of (initial) abilities and motivation.

Social implications

This study indicates the advantage for companies and societies to invest in additional skills and training and points at the limitations of assistance systems. This paper also contributes to training policies by exploring new, digitalised ways of guided on-the-job training and demonstrates possible training benefits for people with lower levels of (initial) abilities and motivation.

Originality/value

This study extends existing research on digital assistance systems by investigating their role in job-related-training. This paper contributes to labour sociology and organisational research by confirming the importance of holistic process knowledge as opposed to a solely task-oriented digital introduction.

Citation

Wotschack, P., Vladova, G., de Paiva Lareiro, P. and Thim, C. (2023), "Learning via assistance systems in industrial manufacturing. An experimental study in an Industry 4.0 environment", Journal of Workplace Learning, Vol. 35 No. 9, pp. 235-258. https://doi.org/10.1108/JWL-09-2022-0119

1. Introduction

The current phase of economic digitalisation is characterised by an increased use of autonomous technological systems in many areas. A well-known example of this is navigation systems in cars, which display the optimal route to a destination and directions through a city or country and help in finding the right route even in a completely unknown environment. In a similar way, digital assistance systems in industrial production are intended to instruct and support employees in operating machines, even when they lack prior knowledge (Oestreich et al., 2020). However, when using these systems, workers rely on fragmented advice broken down into small tasks by automated technology instead of applying holistic knowledge. Consequently, we observe a shift in the nature of training, from abstract theoretical learning, usually based on holistic information communicated by trainers, colleagues or supervisors or received from independently compiled information, to practical learning based on the repetition of predefined tasks. Previous research has shown that both types of learning have advantages and disadvantages (Kuhlmann et al., 2018; Falkenberg, 2018; Oestreich et al., 2020; Tang et al., 2004; Warnhoff and de Paiva Lareiro, 2019). However, the role of digital assistance systems in learning is yet to receive much attention. The question of how practical task related learning with a digital assistance system influences the work performance and learning experience of workers who lack prior holistic process knowledge remains an open one. Moreover, the role of varying capabilities and learning preferences among individuals is yet to be addressed.

Our study addresses the following two research questions:

RQ1.

How does additional holistic process knowledge influence work performance and work satisfaction, compared with training solely with digital assistance systems?

RQ2.

Does the effect of additional process knowledge vary because of differences in individual productivity?

We follow Fischer and Boreham’s definition of work process knowledge as:

[…] an understanding of the work processes in an organization as a whole, as opposed to the more restricted knowledge base which underpins an individual task or narrowly defined job within one department of that organization (2009, p. 466).

This holistic form of knowledge is particularly important when unpredictable situations or mistakes occur, which can be very harmful in highly interdependent work settings.

Theoretically, we refer to the concept of constructivist learning (Loyens and Gijbels, 2008). We expect that if training is based on step-by-step instructions from a digital assistant system (instructivist learning), then it is limited to (factual) task knowledge and does not provide a deeper understanding of the work process. This will negatively influence productivity and work satisfaction in the long term. Constructivist learning, in contrast, entails the active integration of new information (step-by-step-introduction) with prior knowledge of the work process. We expect that learners provided with prior process knowledge will gain a better understanding of the work process and increased motivation through constructivist learning, leading to higher levels of productivity, quality and work satisfaction.

We used an experimental design in a simulated realistic factory environment to study how training based (solely) on an assistance system influences work performance and work experience compared with a system combined with process information via initial training. The experiment assumed a highly automated work process in which workers are responsible for operating, equipping and monitoring machines. The test subjects’ productivity and quality of work was assessed digitally in real-time during the experiment. In addition, each subject filled out two online questionnaires (one before and one after the experiment).

In the following sections, we will discuss the background and objectives of our study (Section 2), outline the theoretical framework (Section 3) and experimental design (Section 4) and discuss and summarise the main findings (Sections 5–7).

2. Research context and objectives

The review of existing studies in the field of assistance systems and training reveals two research lines. The first one – coming from labour sociology – focuses on the role of assistance systems in industrial work processes regarding working conditions and productivity (Section 2.1). The second one – coming from training research – stresses the importance of work-related forms of learning and possible improvements because of advances in information and communication technologies (Section 2.2).

2.1 Digital assistance systems in industrial work processes

Digital assistance systems are implemented in production processes to create synergetic effects, combining a lively workforce with technical resources to optimise work processes. Unlike classical automation processes, assistance systems do not usually substitute for manual labour but supplement it. Depending on the sector and profession they are applied in, the systems aim at different aspects of the labour process, providing physical, sensory or cognitive assistance to compensate for, maintain or extend the labour force of workers (Apt et al., 2018).

In assembly and logistics, most jobs are characterised as low- or semi-skilled work (Abel et al., 2014). Different kinds of digital assistance systems can often be found in these sectors, guiding workers throughout the assembly process or in the selection of parts. While engineering literature usually emphasises the positive effects of assistance systems on productivity – increased worker speed and fewer errors (Reif and Günthner, 2009; Tang et al., 2004) – other studies do not find an increase in productivity after assistance systems are implemented (Klippert et al., 2018). Productivity goals are often reflected in the design of systems, which usually provide workers with rather restrictive work instructions to reduce independent decisions or intuitive deviations from the predetermined work process (Niehaus, 2017). The fixation on a predefined workflow found in most systems for low skilled work limits workers’ control over the process and can be problematic if unexpected situations arise (Müller et al., 2019). A lack of flexibility could cancel out productivity gains from the process standardisation described above, reduce workers’ ability to optimise their workflow (Mark et al., 2020) and lead to a feeling of alienation and deskilling (Möncks et al., 2020; Warnhoff and de Paiva Lareiro, 2019).

In contrast, other studies have shown that assistance systems for manufacturing processes can enable workers to perform a higher variety of tasks and relieve stress (Kuhlmann et al., 2018; Stockinger et al., 2020). However, transferring the responsibility for the correct process flow to digital assistance systems can also lead to a loss of process knowledge and accountability in the long term. Fink and Weyer (2011) showed that through the “involvement” of technology in a process, users ascribe a responsibility to the technology for action in the work process. This could lead to employees concentrating more on the area of action assigned to them (manual activity) and, thus, also relieving themselves of responsibility for the actions ascribed to technology.

2.2 Digital assistance systems in work-related training

Because of the advances in information and communication technology and human–computer interaction technology, many scholars and policymakers expect substantial improvements regarding work-related training (Gorecky et al., 2013; Oestreich et al., 2020). Digital assistance systems are one example. However, their role in training has not yet been sufficiently explored, and the questions of how these systems might change our way of learning, shape our knowledge and view of the production process, or can be usefully combined with traditional types of training, have yet to be answered.

Regarding lifelong learning and continuing vocational training, policymakers and researchers have increasingly stressed the importance of alternative forms of learning that do not take place in courses or seminars (Baethge et al., 2003; Moraal et al., 2009). Among these alternative learning activities such as job rotation, learning and quality circles or self-directed learning, guided on-the-job-training plays a prominent role (Cedefop, 2015a). While courses are “typically separated from the active workplace” and “exhibit a high degree of organisation (…) by a trainer or a training institution”, guided on-the-job training “is characterised by planned periods of training, instruction or practical experience in the workplace using the normal tools of work, either at the immediate place of work or in the work situation” (Cedefop, 2015a, p. 124; see also Cedefop, 2015b). Though there is still a discussion on how these different types of training relate to each other (as substitutes or complements), most studies agree on the advantages of alternative types of learning. As these approaches are easier to adapt to the training needs of employees (Baethge et al., 2003; Moraal et al., 2009) and less expensive and easier to organise than courses (Brussig and Leber, 2005), they have the potential to foster training participation by companies (such as small and medium-sized firms) or groups of employees (such as low-skilled workers) who are significantly underrepresented and often turn down the option of formalised, theoretical types of training like courses or seminars (Cedefop, 2015b). Distance to formalised learning processes and being out of the habit of learning were identified as major reasons for non-participation in continuing training. Once the distance to organised learning has become a barrier, informal learning close to the workplace may build bridges. In a German study (Schröder et al., 2004), almost two of three non-participants stated that they would like to learn from concrete application examples and directly practice what they have learned, ideally under the guidance of an instructor.

Few studies have focused on the role of digital assistance systems in training processes and learning. Mark et al. (2020) compared the use of digital assistance systems for training purposes in industrial work settings to paper-based instructions. Their study showed that while the users of the assistance system performed tasks faster at the start of the experiment, they were outperformed by the group with paper-based instructions in the long run. While the participants with paper-based task instructions optimised their work processes after a few cycles, the participants using the assistance systems lacked the flexibility to do so and had a slower production speed at the end of the experiment. The importance of adaptivity in the design of digital assistance systems for efficiently supporting learning processes and work performance was stressed by other studies (Meier and Vernim, 2021; Oestreich et al., 2020).

Company-level case studies in labour sociology show that experience-based process knowledge plays an important role in advanced production processes, even in low-skilled jobs that do not require a professional qualification (Hirsch-Kreinsen, 2016). When workers lack holistic knowledge on the entire work process and its product, their ability to cope with unexpected situations and problems is limited (Bainbridge, 1983; Fischer and Boreham, 2009; Warnhoff and de Paiva Lareiro, 2019). The role of holistic process knowledge in assistance-system-guided learning has not been considered by previous research. Our study aims to shed a light on possible limitations to productivity when learning is solely based on a digital step-by-step introduction and not combined with holistic process information.

3. Theoretical framework

Our study builds on established learning theories, particularly behaviourist and constructivist approaches towards development and design of learning environments, with a special focus on experiential learning. As it is our aim to explain how the use of assistance systems for (initial) training shapes learning processes and learning outcomes, we first introduce our theoretical framework, explaining in detail the central ideas behind these theories.

As a theory of learning, behaviourism focuses on observable behaviours, discounting any mental activity. Learning is defined simply as the acquisition of new behaviour (Pritchard, 2008, p. 6), organised according to the stimulus–response scheme. Knowledge is seen as an external object, not bound to the learner. This theoretical approach applies to the development of simple factual knowledge rather than problem-solving skills or individual learning paths. So-called “programmed learning” (Skinner, 1958) describes managing human learning under controlled conditions. Behaviourism is bound to the idea of teaching the learner to do or not do something (instructivist learning) rather than transferring additional knowledge and understanding. Suitable behaviouristic learning modules are question–answer sequences (e.g. vocabulary training) often with the use of “drill and practice”, in which a direct positive or negative response is given to the learner depending on their answer (Pritchard, 2008, p. 11).

In contrast to this, constructivist learning goes beyond the rather passive accumulation of information on how to perform a certain task. The constructivist approach assumes that “knowledge is actively constructed by the learner” (Loyens and Gijbels, 2008). In this theoretical view, individuals interpret new information by using prior knowledge to achieve a better understanding of the subject matter (Blumenfeld, 1992). Thus, knowledge is constructed internally by the individual by linking new and prior knowledge (Gardner and Thielen, 2015, pp. 54–55). Especially in the context of the experiential learning theory as one of the most often used constructivist models, the role of the learners and their subjective experience within the learning process as well as the high learner activity and the opportunity to create an appropriate learning environment are emphasised (Kluge, 2007; Kolb and Kolb, 2005).

Moreover, the constructivist approach stresses the advantages of real-life learning situations and emphasises the positive effects of constructivist learning regarding the learner’s capabilities (including problem-solving skills), interest and motivation (Forbes et al., 2001; Loyens and Gijbels, 2008). Learners observe, process and interpret information and then convert the information into personal knowledge (Cooper, 1993). They learn best when they can contextualise what they learn, both for immediate application and to acquire personal meaning (Ally, 2005).

We address the perspective of these two different theories of learning in our experimental design. For the development of our experimental learning environment, we assume that training solely via the step-by-step instructions of the digital assistant system leads to a passive, behaviouristic form of learning (Instructivist learning), because of the nature of the tasks and the fact that the participants do not have access to other information or knowledge on the (overall) work process. According to this, the participants (“workers”) learn how to perform a certain task by repeatedly executing predefined work steps, guided by the assistant system. Information on how to perform this task optimally is accumulated in a passive way and workers cannot relate to previous knowledge on the overall process to contextualise the task knowledge provided by the system. Though this type of learning is easier to perform and control than constructivist learning, it is limited to (factual) task knowledge and does not lead to a deeper understanding of these tasks or their role in the entire work process. The second training design is affected by the premises of constructivist learning. This learning, in contrast, requires the active effort of the learner in achieving a better understanding of the subject matter and the acquisition of prior knowledge to interpret the new information provided by the digital device. In this respect, the availability of prior knowledge on the overall work process (process knowledge) should make a considerable difference, enabling and activating constructivist learning. Though this type of learning entails more time and effort in the beginning, it leads to greater understanding of the work process and fosters the active involvement, interest and motivation of learners.

Summing up, we derive the following three theoretical expectations regarding the role of digital assistance systems in guided on-the-job-training:

• We expect that the advantages of a step-by-step introduction based solely on a digital assistance system (instructive learning) are only short term.

As this type of learning requires less time and effort from learners and leads to quick experiences of practical success, there will be a positive impact on productivity, quality and work satisfaction in the early phase of production:

• We expect that the lack of prior process knowledge hinders constructivist learning processes, leading to long-term negative effects in terms of lower productivity, quality of work and work satisfaction.

Learners provided with prior process knowledge are able to integrate this knowledge with information from the step-by-step instruction from the assistance systems (constructivist learning). This will contribute to a more comprehensive understanding of the work process and more active involvement and motivation for learners. In the long term, this will lead to higher levels of productivity, quality and satisfaction:

• We expect that individuals with lower productivity levels are more likely to benefit from training solely with a step-by-step introduction based on a digital assistance system (instructive learning).

As this group of workers may be less capable at processing the additional process knowledge required for constructivist learning (at least in the beginning), these learners may achieve better outcomes and a higher level of work satisfaction by solely following the instructions of the assistance systems.

4. Research design

Previous research on digital assistance systems has mainly been based on observational studies (often company-level case studies), investigating their role for workers (Kuhlmann et al., 2018; Warnhoff and de Paiva Lareiro, 2019). However, a proper case-study design to study our question would require an intervention study to identify causal effects of process knowledge – and in the absence of that, a selection of two cases providing two similar work settings (including similar assistance systems and similar types of workers) that only differ in terms of additional process knowledge. It is obvious that this is hard to achieve. For that reason, we have chosen a laboratory experimental design. By keeping all restrictions in the working setting constant (except the treatment variable), it provides a method to study causal effects over time. Laboratory experiments provide a high degree of control over the treatment and experimental conditions (Falk and Heckman, 2009; Santhanam, 2002). “The internal validity afforded by laboratory experiments is seen to be their greatest strength […], enabling precise and clear tests of social scientific theories” (Jackson and Cox, 2013, p. 37). There are at the same time limitations. Experimental settings run the risk of creating an artificial situation without much in common with the real situation. Moreover, possible bias in the sample of experimental subjects can restrict the generalisability of the results. Both problems are addressed under the heading of (lacking) external validity, a standard objection to laboratory experiments (Falk and Heckman, 2009; Jackson and Cox, 2013, p. 37). Given these limitations of laboratory experiments, our study aims to test predicted causal relationships (as derived from constructivist learning theory), rather than generating generalisable empirical findings. In addition, we carefully designed the experimental setting to be as close as possible to the real work situation (Appendix, Figure A1) by incorporating insights from in-depth company-level case studies on the application of digital assistance systems in low-skill manufacturing jobs and designing the experiment in close collaboration with researchers from these studies.

Eventually, we used a learning factory as a special lab to ensure that modern manufacturing work was simulated accurately as far as real labour processes are concerned (name of the learning factory has been removed from the manuscript for reasons of anonymity). Learning factories are learning environments in which participants are trained using simulated real production processes that are as realistic and authentic as possible (Enke et al., 2015; Tisch et al., 2013).

4.1 Experimental setting and sample

Our study focuses on the initial training phase of a production process. The experiment participants had the task of producing parts (artificial knee joints) by operating the machine interface according to strictly prescribed steps. The production environment in the experiment included both digital elements to be operated (machines, workpieces and production line) and spatial conditions (such as the warehouse). The necessary steps/tasks of the work process were mapped in an assistance system developed explicitly for the experiment. For each step of the instruction displayed in the assistance systems, the test subjects had the option of either following the displayed instructions or pressing a “help button” at any time to obtain additional information and instructions (Appendix, Figure A2).

Subjects were divided into two groups and assigned to two different settings. In the first setting, the participants used the assistance systems as an orientation and support tool right at the beginning and learned the production steps exclusively in this way. The participants in this setting were only provided with instructional task knowledge via the assistance system.

In the second setting, subjects received an additional ten-minute introduction (treatment) from the experiment leaders at the beginning of the experiment. This introduction included detailed information regarding the process flow and the relationships between different tasks within it. It included more general knowledge and information, not explicitly necessary for performing well on a task level. This information helped the subjects to understand the function of the systems and the structure of the whole simulated production process. Test subjects’ comprehension of the contents of this introductory training was then directly checked via a questionnaire.

The combination of the assistance system and this introduction aimed at providing the required holistic production knowledge needed to initiate constructivist learning processes; the task knowledge provided by the assistance systems could, thus, be integrated with broader knowledge about the work process provided by the introduction.

The total duration of the experiment was two hours, including brief instructions at the beginning on how to use the assistance system, one trial sequence and 10 min at the end to complete a standardised questionnaire. The remaining time for production was 90 min, including three successive learning attempts with the assistance system and a subsequent production sequence without assistance system support (Appendix, Figure A3). The number of completed workpieces was counted as a measure for productivity, the extent of errors as a measure for quality.

Participation in the experiment was voluntary. Test subjects were approached within one university and received financial compensation. From the original sample of 83 students (mostly from business informatics) who agreed to participate in the experiment, 59 subjects entered the analyses; 35 of them (7 women and 28 men) were assigned to the treatment group (receiving additional training), while 24 (11 women and 13 men) did not receive the treatment. Regarding the missing 24 cases, students either failed to show up, dropped out during the experiment or had to be removed from the sample because of technical problems, language barriers or prior experience or knowledge related to the experiment [1].

After the production sequence, all test subjects filled in a standardised questionnaire using the LimeSurvey online survey tool. The survey was structured into five blocks: demographic information, perceived satisfaction and stress, overall evaluation of the assistance system, evaluation of specific aspects of the assistance system and self-efficacy.

4.2 Indicators and variables

Our experimental study generated two types of data: First, machine data have been generated and recorded in real time during the experiment. These data gave us information on the number of workpieces (first dependent variable DV1) completed by each person over time, the extent of errors (second dependent variable DV2) that occurred when the participants carried out the requested tasks and the frequency with which additional information was requested (third dependent variable DV3) from the assistance system (by touching the virtual “help button” on the device). The errors were calculated on the basis of optimal matching distances (Abbott and Tsay, 2000). A standard sequence was defined for the optimal execution of the work steps. Deviations from this standard sequence in the experimental process were recorded and linked to costs. These costs reflected the extent of the errors. The larger the distance between work performance and standard sequence, the higher the costs. When actions were carried out that were not provided for in the process, maximum costs were allocated.

Regarding the survey questions (standardised questionnaire), data was collected in five dimensions (Appendix, Table A1) including items on work enjoyment (questions regarding stress, pressure and excitement during the experiment), the perceived role of the assistance system (helpful, disturbing and easy or difficult to use) and the evaluation of the production phase without assistance system (difficulties, desire for more information and desire for personal contact). All questions were evaluated descriptively with a Likert scale from 5 (agree completely) to 1 (disagree completely) and with regard to group differences (Appendix, Table A2).

As our study is interested in not only the overall effects of additional training (treatment) (corresponding RQ1) but also possible differences because of (varying) productivity levels (corresponding RQ2), we distinguish between three groups of similar size according to their overall level of productivity: low achievers (LAs) who produced a relatively small number (2–6) of workpieces in the given time of 90 min, medium achievers (MAs) who produced a medium number (7–9) of workpieces, high achievers (HAs) who produced a large number (10–15) of workpieces.

5. Findings

Our study aims to explore the role of process knowledge within an experimental design. The focus is on differences between the treatment and non-treatment group (as formulated in our theoretical expectations).

5.1 Perceived role of the assistance system

The survey shows clear differences between the groups (treatment and non-treatment) in two dimensions: desire for more information and the wish to have a contact person to talk to. Both are higher in the group that received theoretical training. Following the predictions from our learning model (constructivist learning), this might indicate both a stronger desire for additional information and/or difficulties to process and integrate two sources of information (process knowledge and step-by-step introduction).

The majority of participants in both groups indicated that they found the assistance system support helpful. However, 82% of the participants in the treatment group (with additional training) as well as 75% in the non-treatment group also found the AS annoying. Of subjects in the treatment group, 42% wanted a personal contact person during the experiment; in the non-treatment group, the share was 29%. The majority of participants in the treatment group found the theoretical content unhelpful to the learning process, with some even finding the information confusing. At the same time, there were explicit requests from the non-treatment group, stating that an introduction would have been helpful to better understand the entire situation in the laboratory and the context.

5.2 Effects on work performance

Overall mean values in the machine data do not display a significant pattern. In line with our expectation, the treatment group (with additional training) used the help button less often, indicating a lower need for assistance-system-based advice in the initial phase of production. Moreover, there is evidence that the treatment group worked more productively: On average, each participant in the treatment group produced 8.82 workpieces, while participants who worked solely with the assistance system only produced 8.33 workpieces (Appendix, Table A3). However, the difference is rather small. When we look at the extent of errors (distance between optimal and actual work performance), the treatment group shows worse outcomes (higher distance: 0.25 versus 0.22 in the group without treatment).

Interestingly, we find significant gender differences. Women in both groups performed better and achieved higher outcomes. For that reason, the following analyses were controlled for possible gender differences.

As shown in Figures 1 and 2, the reported overall difference (mean values) regarding productivity and quality is related to differences over time and achievement levels. While members of the treatment group show worse outcomes when producing workpieces 2–9, they show better outcomes when producing workpieces 10–15. This pattern also applies when separate models for men and women are compared (Figures 1.1 and 1.2). However, regarding the extent of errors (Figures 2.1 and 2.2), differences between treatment and non-treatment group are less pronounced among men. Overall and in line with our expectations, the analyses confirm that while the constructivist learning process – combining training via an assistance system and personal training – requires more time at the beginning, it leads to better outcomes over time.

5.3 Differences between high, medium and low achievers

To explore the impact of additional training (treatment) on groups with different levels of productivity (RQ2), we distinguished between three groups: In the given time of 90 mins, LAs produced 2–6 workpieces, MAs produced 7–9 workpieces and HAs produced 10–15 workpieces.

The composition of the three achievement groups does not indicate a clear pattern in terms of treatment (yes or no), age or prior experience (regarding experimental studies) (Appendix, Table A4). Regarding gender differences, women are overrepresented in the group of HAs, indicating an overall higher productivity level of women in this experimental study.

We expected that the effect of additional training (treatment) on the participants’ work performance would vary with their overall level of productivity. As we can see in Figure 3, this expectation is largely confirmed by the analyses (Appendix, Figure A4 and Table A5).

Within the group of LAs (two to six workpieces), the treatment group had a significantly lower level of productivity compared to the non-treatment group. Both groups made a similar number of errors (Appendix, Table A5). However, when we compare the errors (mean values) for different workpieces (Appendix, Figure A4), there is also evidence of slightly more errors in the treatment group in the beginning (workpiece 2) but fewer errors in the long run.

Among the MAs (seven to nine workpieces), the treatment group performs slightly better in terms of productivity but worse in terms of quality, indicating a possible trade-off. On average, both groups (treatment and non-treatment) show a considerable learning effect regarding the extent of errors (Appendix, Table A5), but the difference between both groups remains.

HAs who received training (treatment) show the best outcomes regarding both productivity and (small) extent of errors. Compared to the non-treatment group, they performed stronger in the first two phases of the experiment and produced the highest number of workpieces. Only in the last phase of the experiment (10–15 workpieces) did the average number of errors (distances) exceed the values of the non-treatment group (Appendix, Table A5). This was probably because of fatigue, as the treatment group produced a significantly higher number of workpieces on average.

6. Discussion and conclusions

Previous research on digital cognitive assistance systems has mainly focused on changes to the work situation and work experience. This paper investigated how learning solely via an assistance system influences work performance compared with learning with a combination of an assistance system and additional training.

While existing studies underlining the opportunities and risks of digital assistance are mainly based on firm-level case studies, we explored their effects using an experimental design. Though laboratory experiments are limited in terms of their external validity and generalisability, the method provides a high degree of internal validity and enables precise testing of scientific theories (Falk and Heckman, 2009; Jackson and Cox, 2013; Santhanam, 2002).

Theoretically, we referred to the idea and concept of constructivist learning (Loyens and Gijbels, 2008), emphasising the importance of the active integration of new information (step-by-step introduction) with prior knowledge. Crucially, we expected that learners provided with prior process knowledge (treatment group) would achieve a better understanding of the work process and higher motivation through constructivist learning, leading to higher levels of productivity, quality and work satisfaction. At the same time, we expected differences among workers’ ability to process and apply this additional information. Our findings widely confirm this. While overall group differences (mean values) remain rather low, demonstrating the slightly better performance of participants who received additional training (treatment), we found evidence for considerable variation between groups of different achievement levels.

Among the HAs (those showing the best results in terms of productivity), the treatment group performed significantly better, indicating a positive effect from additional holistic knowledge for highly capable participants. In the group of LAs (lowest results in terms of productivity), additional holistic knowledge was related to significant losses in productivity, while quality was not affected. Please note that LAs have been overrepresented in the treatment group by 6%, while HAs have been underrepresented by 7%. This might influence the results, but – as these differences are rather small – most likely not in a significant way.

Summing up, our study confirms evidence from existing research pointing at the limitations of a solely task-knowledge oriented way of training (Fischer and Boreham, 2009; Hirsch-Kreinsen, 2016; Oestreicher et al.; 2020; Warnhoff and de Paiva Lareiro, 2019) by underlining the advantages of additional process knowledge. Moreover, there is evidence for a significant desire for more information and direct contact pointing to an overall need to ensure functioning communication in work processes.

Given the outlined limitations and demands for future research, our explorative study provides evidence that extended process information, even in low-skill tasks, has an added value for the production process. This might contribute new insights for firms' training policies and motivate researchers, companies and training providers to further explore digitalised ways of guided on the-job training in combination with theoretical schooling. Recent studies have underlined the importance of adaptivity of digital assistance system regarding learning types and user state along the learning path (Meier and Vernim, 2021; Oestereich et al., 2020). In line with this, our experimental study provides evidence for group differences regarding the combination of theoretical knowledge and learning via a digital assistance system. Workers’ initial skill and competency levels, individual learning types and firms’ HR and training strategies – aiming at temporary or permanent workers – shape the role and effects of digital assistance systems and process knowledge in guided-on-the-job-training. Future research is needed, however, to further explore these differences and to extend our knowledge on the optimal fit of assistance-system based and other forms of training to meet the diverse needs of different employees along the learning path.

Laboratory experiments are limited in terms of their external validity – even more so when the sample is rather small and limited to students, as it was the case in our study. In this respect, the findings of this experimental study cannot be directly applied to real work situations and the real working population. The experiment was conducted within a rather short period (90 min) of initial learning and performing. This short time period also limited the design possibilities for the additional training for the treatment group. Therefore, the identified differences between the two groups – with and without treatment – should initially be treated with caution. Further research is required, including on the one hand laboratory experiments with larger case numbers and more elaborated schooling (in contrast to a rather short 10-min introduction), on the other hand intervention studies, exploring long-term effects in real work situations to further confirm the validity of our findings for the workplace.

Eventually, the interpretation of group differences must consider correlations with the subjects' ability to perform (LA vs HA). We argue that the additional information is an influencing variable, but individual differences will also have intervening effects. Further research is also required in this respect, building on theories that investigate the influence of information on thought processes. One possible theoretical base is provided by the cognitive load theory (Sweller, 1988; Sweller et al., 2019). It points at restrictions in the human ability to process novel information at a given period because of the working memory's limited capacity and acknowledges the importance of pre-existing expertise, knowledge and external stimuli, like the complexity of learning material or tasks (Mayer, 2015; Sweller, 2010).

Regarding the contribution of this study to the existing research and policies in the field, we find clear evidence that more knowledge at the work process level (in contrast to task knowledge provided by the digital assistance systems) contribute to productivity and work satisfaction. Longer production periods and higher levels of capabilities and motivation have proven to be important preconditions for this. This finding complements evidence from existing research based on firm-level case studies (Fischer and Boreham, 2009; Hirsch-Kreinsen, 2016; Warnhoff and de Paiva Lareiro, 2019) by isolating and testing the underlying theoretical mechanism. At the same time, our study goes beyond existing research by showing differences because of workers’ capability and motivation levels: Process information in the initial phase can have counterproductive effects for groups with lower capability and motivation levels, while a practical, task-oriented learning (guided by digital assistance) gains importance.

The use and application of the mechanisms, evidenced by our study, in real production processes remains the challenge for practitioners and experts in the field of organisations and training. The huge variety of given production processes, work settings and workers’ characteristics (including skills and competencies) will require tailor-made training measures that combine and provide process and task knowledge in efficient ways by using different tools and sources of knowledge. Our study provides clear evidence that a one-size fits all approaches – even in an experimental setting – is not productive.

Figures

Figure 1.

Share of participants (in %) by number of completed workpieces over time (90 min)

Figure 2.

Extent of errors (distances) per workpiece (mean values) overt time (90 min)

Figure 3.

Number of workpieces by achievement level and group (treatment/non-treatment)

Figure A1.

The centre for Industry 4.0 at the University of Potsdam

Figure A2.

Assistance system display with indication of the next activity

Figure A3.

Experimental sequence in settings 1 and 2

Figure A4.

Extent of errors by achievement level and group (treatment/non-treatment)

Table A1.

Survey at the end of the experiment

Classes	No. of items	Specification
Demographic information and influencing variables (relevant experience)	13	Age, gender, study subject, work or training experience, experiment experience, language, semester of study and degree
Satisfaction and stress	6	Perceived enjoyment, physical pressure, psychological pressure, excitement and time pressure (Source: own consideration)
Overall evaluation of the assistance system	4	The system is perceived as: helpful, superfluous, annoying and coping well (Source: own consideration)
Evaluation of specific aspects of the assistance system	5 closed questions and one open question	Difficulty to produce without assistance system; satisfaction regarding the information from the system; Need of more information from the system desired; Need of more process knowledge; and Need of personal contact (Source: own consideration)
Self-efficacy	10	General self-efficacy expectation (Source: Schwarzer and Jerusalem M, 1995)

Source: Authors’ own data, 2020

Table A2.

Descriptives

Machine data	Mean	SD	Minimum	Maximum
Number of workpieces	8.63	3.31	2	15
Errors (distance)	0.24	0.11	0.05	0.58
Frequency help button	5.27	5.86	0	28
Survey data	Dimension: Enjoyment
Items
I enjoyed the activities	2.67	1.04	1	5
I found the activities physically stressful	1.89	1.00	1	5
I found the activities mentally stressful	2.30	1.26	1	5
I found the tasks exciting	2.21	0.99	1	5
I felt time pressure	2.49	0.99	1	5
I felt stressed	2.00	0.95	1	5
Survey data	Dimension: Assistance system use
Items
The assistance system was helpful	4.26	0.874	1	5
The assistance system was superfluous	1.87	0.974	1	5
The assistance system was annoying	4.05	0.884	1	5
I got along well with the assistance system	4.05	0.865	1	5
Survey data	Dimension: Working in the simulation
Items
I felt part of my work environment	3.16	0.92	1	5
The simulation triggered emotions (e.g. anger, sadness, satisfaction) in me	2.34	1.05	1	5
Working in the simulation was satisfying for me	2.72	1.04	1	5
Working in the simulation was boring for me	3.14	1.07	1	5
While I was operating the equipment, I forgot that I was participating in a study	2.59	1.15	1	5
While I was operating the system, I was mentally immersed in the simulation	2.82	1.04	1	5
Survey data	Dimension: Enjoyment
Items
I found the phase without the assistance system difficult	1.97	0.97	1	5
The information was sufficient to cope with the tasks	3.54	0.89	1	5
I would have liked more information to master the tasks	2.95	1.18	1	5
I would have liked to know more about the entire manufacturing process	3.02	1.26	1	5
I would have liked to have a personal contact person	2.89	1.23	1	5

Source: Authors’ own data, 2020

Table A3.

Work performance with/without treatment – mean values

Dependent variable	Treatment (training) (n = 35)	No treatment (n = 24)
Number of workpieces	8.82	8.33
Errors (distance)	0.25	0.22
Usage of help button (frequency)	5.00	5.67

Source: Authors’ own data, 2020

Table A4.

Characteristics of low, medium and high achievers (mean values)

Achievement group	Women	Age	Treatment	Prior experience
Low achiever (n = 17)	0.24	23.88	0.65	1.18
Medium achiever (n = 20)	0.30	23.55	0.50	1.00
High achiever (n = 22)	0.37	23.77	0.64	1.05

Source: Authors’ own data, 2020

Table A5.

Work performance of low, medium and high achievers – mean values

	Workpieces 2–6		Workpieces 7–9		Workpieces 10–15
Achievement level	WPs	Distance	WPs	Distance	WPs	Distance
Low achievers (n = 17)	4.65	0.275	/	/	/	/
Treatment (n = 11)	4.00	0.276	/	/	/	/
No treatment (n = 6)	5.83	0.275	/	/	/	/
Medium achievers (n = 20)	6.00	0.170	2.25	0.244	/	/
Treatment (n = 10)	6.00	0.213	2.50	0.305	/	/
No treatment (n = 10)	6.00	0.127	2.00	0.183	/	/
High achievers (n = 22)	6.00	0.103	3.00	0.103	2.27	0.193
Treatment (n = 14)	6.00	0.088	3,00	0.083	2.71	0.174
No treatment (n = 8)	6.00	0.129	3,00	0.139	1.50	0.226

Source: Authors’ own data, 2020

Note

1. Respondents who indicated in the standardised questionnaire that they had already participated in an experimental study in our lab (learning factory) or worked in an industrial context were removed from the sample.