Abstract
Purpose
Introducing technology at work presents a special challenge as learning is tightly integrated with workplace practices. Current design-based research (DBR) methods are focused on formal learning context and often questioned for a lack of yielding traceable research insights. This paper aims to propose a method that extends DBR by understanding tools as sociocultural artefacts, co-designing affordances and systematically studying their adoption in practice.
Design/methodology/approach
The iterative practice-centred method allows the co-design of cognitive tools in DBR, makes assumptions and design decisions traceable and builds convergent evidence by consistently analysing how affordances are appropriated. This is demonstrated in the context of health-care professionals’ informal learning, and how they make sense of their experiences. The authors report an 18-month DBR case study of using various prototypes and testing the designs with practitioners through various data collection means.
Findings
By considering the cognitive level in the analysis of appropriation, the authors came to an understanding of how professionals cope with pressure in the health-care domain (domain insight); a prototype with concrete design decisions (design insight); and an understanding of how memory and sensemaking processes interact when cognitive tools are used to elaborate representations of informal learning needs (theory insight).
Research limitations/implications
The method is validated in one long-term and in-depth case study. While this was necessary to gain an understanding of stakeholder concerns, build trust and apply methods over several iterations, it also potentially limits this.
Originality/value
Besides generating traceable research insights, the proposed DBR method allows to design technology-enhanced learning support for working domains and practices. The method is applicable in other domains and in formal learning.
Keywords
Citation
Dennerlein, S.M., Tomberg, V., Treasure-Jones, T., Theiler, D., Lindstaedt, S. and Ley, T. (2020), "Co-designing tools for workplace learning: A method for analysing and tracing the appropriation of affordances in design-based research", Information and Learning Sciences, Vol. 121 No. 3/4, pp. 175-205. https://doi.org/10.1108/ILS-09-2019-0093
Publisher
:Emerald Publishing Limited
Copyright © 2020, Sebastian Maximilian Dennerlein, Vladimir Tomberg, Tamsin Treasure-Jones, Dieter Theiler, Stefanie Lindstaedt and Tobias Ley.
License
Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Introduction
Information and communication technology is now a part of many professions, and therefore a natural part of workplace learning (Billett and Choy, 2013). Many features of this technology offer new opportunities for workplace learning that takes place across a number of learning contexts in relation with work, such as formally organized and curriculum-based learning programmes (e.g. trainings), but it more importantly includes informal and self-regulated learning alongside work (Eraut, 2000). For example, this technology can help to bridge learning across different formal and informal learning contexts (Eraut, 2000), it can help to connect to teams or communities (Thorpe and Gordon, 2012; Ley et al., 2014; Renner et al., 2014; Treasure-Jones et al., 2019) and it can help to learn and reflect on a just in time basis by supporting self-regulated learning at work (Eraut, 2004; Littlejohn et al., 2012; Fessl et al., 2017).
Informed by a long tradition of research into the nature of human activity as a social and situated practice (Wenger, 1998; Engeström, 2001), we understand the technology people are using in this context as offering cultural tools that scaffold individual learning (Vygotsky, 1978), mediate individual and collaborative knowledge building (Paavola and Hakkarainen, 2014) and inform corresponding practices. For example, a particular technology and its features can make available cultural knowledge that may extend the capabilities of the individual for sensemaking and problem solving (Stahl, 2013). This becomes apparent when a tool empowers the employee to perform certain professional practices such as the patient record system affording a general practitioner (GP) to explore medical records for understanding the course of disease.
Taken together, these technologies should guide professionals’ meaningful sensemaking in authentic work contexts. However, the increasing digitization of work life also makes this kind of learning more challenging. Billett and Choy (2013) suggest that in current workplaces, a much stronger reliance on conceptual and symbolic knowledge is a result of the introduction of digital technology, and experience of information overload is reported in many professions today (Joynes et al., 2017; Ashley et al., 2018). For these reasons, we believe a much more careful consideration of the cognitive affordances is necessary when designing workplace-learning technology.
Such focus can be achieved with a perspective that conceptualizes workplace-learning technology as “cognitive tools”. Cognitive tools afford distributed and situated cognition as an interaction between internal and external representations, between members of a group as well as over time (Hollan et al., 2000). They are tightly integrated in working and learning practices and offer latent action possibilities, i.e. cognitive affordances that support social and situated practices. In that case, the tools are meaningfully embedded in affordance networks (Barab and Roth, 2006) and they align with the intentions of the activity system (Engeström, 2001) as they are useful in fulfilling relevant goals.
Designing cognitive tools and introducing them in workplace-learning contexts need to recognize the specificities of workplace learning and help to embed tools in authentic learning and working practices. For example, the health-care domain is characterized by the constant introduction of new research or new evidence-based guidelines. Official bodies are not able to keep up the pace and provide formal trainings, which require professionals to keep up to date themselves to make well-informed decisions, and review and improve practice. The massive amount of heterogeneous data in the information rich system means that professionals need to be able to find, retrieve, understand and act on relevant information in a quick and self-regulated way. Introducing cognitive tools in such workplace settings is a challenging undertaking, as it means to take into account existing practices and introduce changes to them, while not overloading but supporting busy professionals.
Currently, different types of design methods that iteratively and collaboratively design (co-design) technology are considered standard methods to integrate learning research, practice and technology development. Design-based research (DBR), for example, has been focused on formal education (DBRC, 2003). In such settings, learning is the main activity and usually follows a particular learning design that guides students with comparable prior knowledge to learn. In workplace settings, however, introducing learning technology is difficult because of heterogeneous existing practices and work processes (Ruiz-Calleja et al., 2017). Informal learning settings are less controlled and include different approaches to learning as well as a more diverse set of learning gaps, often in different professional fields. For these reasons, a more diverse set of stakeholders in terms of disciplines, roles and approaches to learning must be included in the design process to fully understand the relevant working and learning practices.
Co-designing cognitive tools for work settings has been addressed with methods based on activity theory. Activity theory has been proposed to gain a holistic understanding of human activity in multiple working domains, and it also differentiates three levels of analysis to describe human behaviour: the operational, the action and the activity level. However, the abundance of analytical tools based on activity theory contrasts with a certain lack of methods that offer guidance for design (Clemmensen et al., 2016), especially when it comes to the cognitive level of analysis. In this respect, theories of “technology design” (Norman, 1988) inform the evolution of cognitive perceptible and usable tools and theories of “technology adoption” guide the study of general utility in the design process (Carroll, 2004), but they miss a concrete and traceable method for co-designing cognitive tools for the workplace.
To summarize, we see a particular need to extend DBR methods targeted at working domains with a focus on the cognitive level of analysis that helps to understand and design affordances of cognitive tools for workplace learning. In our view, such a focus would also address an additional challenge connected to DBR methods, namely, it would contribute to making design decisions evidence-based, inspectable and traceable (Barab and Squire, 2004; Svihla, 2014). As DBR is inherently complex because of its iterative, problem-driven and highly contextualized process, it often uses small samples and has a considerable involvement of researchers in the design process, and traceability of design insights presents a special challenge. Previous attempts into this direction, such as “conjecture mapping” (Sandoval, 2014), have primarily focused on formal educational settings. As we have argued above, contextual factors are even more important in workplace settings, and hence we see a particular need to extend these approaches towards application in workplace contexts.
In this paper, we present a co-design method for workplace learning, which is in the tradition of DBR by taking into account characteristics of work and its context to attain impact on practice when introducing technologies. It draws upon theoretical models of workplace learning that focus on the evolution of situated cognitive processes afforded by cognitive tools. Its key defining characteristic is to iteratively design affordances and evaluate their appropriation at the workplace. We present the application of this method in the context of a long-term case study in which we specified sensemaking as part of experiential learning and developed “Bits and Pieces” (B&P), a sensemaking tool for health-care practitioners’ informal learning at the workplace.
Related work: design-based research for co-designing workplace learning tools
In several disciplines, suitable research strategies geared towards improving practices have been conceived: e.g. design science in information science (e.g. Hevner et al., 2004), co-design in computer science (e.g. Kvan, 2000), DBR in pedagogical science (e.g. DBRC, 2003) and experience-based co-design in health care (e.g. Robert et al., 2015). Common to these views is the inclusion of important stakeholders (e.g. researchers, developers and users) into the design research, the problem-driven character and its contextualization in actual practices, and a focus on the design artefact, which is evolved in an iterative way. We will subsume these approaches under the term DBR. When using the term co-design, we particularly refer to the design process following the interpretation from Durall et al. (2020). They understand that co-design stresses the centring of each iteration on the users and their needs, and the importance of collaboration with all relevant stakeholders, building shared understanding and making sense together to create and mature design ideas capable of impacting practice.
Design-based research as an iterative exploration of domain, theory and design
Following Wang and Hannafin’s (2005) understanding of DBR, researchers work in collaboration with end users to systematically, pragmatically and iteratively generate, analyse and improve the design and the underlying theory to result in impact on practice (Van den Akker et al., 2006). In formal educational settings, this means that DBR contributes to learning theory as well as to changing learning and teaching practices. In research on workplace learning, an additional focus is on the understanding of the domain itself (in our case the health-care domain). The workplace does not follow a uniform pedagogical schema like formal learning and only understanding its circumstances, stakeholder, practices, etc. builds the ground for theory and design.
In line with several well-known DBR approaches, DBR in workplace learning allows for contributing to three different, but interdependent research perspectives: domain, theory and design. The domain inquiry (e.g. Hevner et al., 2004; Goggins and Jahnke, 2010; Fischer, 2013; Fishman et al., 2013; Ley et al., 2014; Sandoval, 2014) allows for insights about the people (e.g. roles, tasks and needs), organization (e.g. processes, constraints and opportunities), technology (e.g. infrastructure and applications) and existing practices. In our case, these would be insights about the health-care domain, such as the differentiation into clinical and administrative tasks, or the fact that health-care GPs work under strict time pressure. An outcome of the domain inquiry could be what Mor et al. (2012) call practice patterns.
The theory inquiry (e.g. March and Smith, 1995; Barab and Squire, 2004; Wang and Hannafin, 2005; Goggins and Jahnke, 2010; Fishman et al., 2013; Ley et al., 2014) allows for insights in the form of theoretical propositions elaborating existing theoretical concepts or frameworks. In our case, theoretical insights would include findings about professionals’ workplace learning, such as a form of learning where learners revisit and reflect about prior experiences made during work. Other DBR studies might choose a different theoretical focus. Van den Akker et al. (2006) stress that “DBR aims at making both practical and scientific contribution”. In analogy to Mor et al. (2012), the scientific contribution could emerge from so-called theory patterns in the sense of repetitive theoretical insights that evolved and gained validity alongside DBR iterations.
Finally, insights about tool characteristics can be gained when taking the perspective of design and practice (e.g. March and Smith, 1995; Barab and Squire, 2004; Wang and Hannafin, 2005; Kali, 2006; Goggins and Jahnke, 2010; Fishman et al., 2013; Ley et al., 2014), i.e. characteristics that can trigger particular cognitive and social processes and that can take influence on the current practice. Examples may be that browsing personal notes on a timeline triggers recalling of prior experiences. Designs commonly trigger particular cognitive and social processes and take influence on the current practice and can be documented as design patterns (Mor et al., 2012).
To advance in the inquiry into the design, theory and domain, the design process asks for as many concretizing prototyping stages as necessary to arrive at a solution that is compliant with the professional practice and implemented in a stable technical solution with high applicability (Fishman et al., 2013; Ley et al., 2014). These prototyping stages represent the systematic iterations of the build-and-evaluate loop (March and Smith, 1995; Carroll, 2004). In parallel, the understanding of the three perspectives develops in the form of a coevolution, i.e. insights in one perspective lead to questions, progress and finally insights in another perspective. For example, knowing about the limitations of a work context (domain insight) allows for respecting them in tool designs (design insight). DBR, hence, looks at the process of adopting technology as an iterative and consecutive development through the three research perspectives.
Design and appropriation of affordances in design-based research
Through the iterative co-design, an interdependency between technology and its users is created which leads to a mutual shaping of user-constructed, dynamic meaning of technology and to a technology constrained perception- and action-range of the user (DeSanctis and Poole, 1994).
At the interface of this dynamic interplay are latent action possibilities called affordances (Gaver, 1991; McGrenere and Ho, 2000; Barab and Roth, 2006; Gibson, 2014). These affordances represent the “bridge” linking the user, his/her actions and the designed technological artefact, i.e. they are created when the user interacts with a cognitive tool and depend on attributes of both (Gaver, 1991; Gibson, 2014). When new features are created in the prototype as part of a DBR study, then these features provide affordances (Maier and Fadel, 2009; Seidel et al., 2018) that are potentially perceived in dependence of the experience, knowledge or culture of the user (Norman, 1988). These latent action possibilities can be in line with existing practices, change existing practices or trigger new ones. Out of this reason, affordances “guide” the practice of the user. This practice can range from simple operations to actions to higher-order learning activities (activity theory levels: operation, action and activity) (Kuutti, 1996). These theories of “technological design” do explain the coevolution of professionals with their tools in communities of practice (Lave and Wenger, 1991; Wenger, 1998), but they do not provide a concrete method for designing of cognitive affordances and analysing their use.
The mutual shaping of the designed tool and the user’s practice happens via appropriation (Carroll, 2004); non-appropriation, where the user does not take possession of the capabilities of the technology; appropriation, where the user decides to adopt the technology and, hence, applies or manipulates it; and dis-appropriation, where the user decides to stop the application of the technology. According to Carroll, appropriation can be a result of two decisions (Figure 1). In a first decision, “the technology as designed” is either adopted or not. In a second decision, the “technology in use” is explored and evaluated before being either appropriated or dis-appropriated. This appropriation can be faithful (DeSanctis and Poole, 1994), in case the application of technology is in line with the designer’s intent. Unfaithful appropriation refers to an unexpected application. Similarly, Lonchamp (2012) studies unexpected application in his interpretation of “instrumental genesis” (Rabardel and Bourmaud, 2003), but focuses on the design of collaborative systems in formal learning contexts. Jarrahi and Sawyer (2015), finally, also study tool appropriation but focus on the reciprocity of practices and social structures. These theories of “technology adoption” allow a macroscopic analysis of utilization schemes, but they are limited to appropriation on tool level and miss the analytical depth required for designing for the cognitive level.
While theories of technology design and appropriation have significantly contributed to our understanding of the dynamics of human–computer interaction, and how to study and design it, there is up to now, no practical method that allows a systematic study of appropriation of cognitive affordances, especially in the context of a DBR study for workplace learning. In the next section, we present Co-design by Appropriation of Affordances (Co-DAA), a DBR method for iteratively co-designing cognitive tools for workplace learning that addresses this gap.
Co-design by Appropriation of Affordances: a design-based research method for the traceable design of cognitive tools for workplace learning
In the tradition of much of the previous work in DBR, Co-DAA includes a practical approach to the inquiry into domain, theory and design by iterative prototyping. It introduces the concept of affordances into the build-and-evaluate loops to study and enhance tool appropriation. The insights gained via the appropriation of affordances in the evaluate-part of the loop determine the hypothesis about affordances for the next build-part. In the B&P case study that we will later demonstrate, we detail the Co-DAA approach of systematically designing affordances and evaluating their appropriation in each of the cycles to make design decisions evidence-based and traceable.
Co-DAA foresees a scheme of iteration, in which insights of a finished iteration trigger the next iteration. The top part of Figure 2, illustrates this scheme, which constitutes one build-and-evaluate loop and consists of four steps:
Infer hypotheses: Derive hypotheses about domain, theory and/or design for the unresolved research questions.
(Re-)Design and implementation: Make design decisions to create or improve an interactive prototype that includes affordances to test hypotheses.
Introduce design into practice: Confront stakeholders with the prototype based on real use cases and tasks in the field to allow for the emergence of affordances, to trigger non-, dis-, expected or unexpected appropriation and to evaluate this interaction in manifold ways.
Analyse results: Examine collected user interactions with the design to understand the appropriation of affordances and conclude on the stated hypotheses and research perspectives, which can result in new research questions.
Both first and second steps, “Infer hypothesis” and “(Re-)Design and implementation”, represent the build-part of on DBR loop. Hypotheses are inferred during the iterations that focus on different research perspectives in DBR. The second step requires the design of interactive prototypes offering various ways of expression to the user and including the intended affordances, e.g. conceptual prototypes that can be sketched on or functional prototypes that log user actions. For the third and the fourth step, “Introduce design into practice” and “Analyse results” (evaluate part of the loop), we followed a structured approach, i.e. plan study designs and methods to test assumptions and answer research questions, run the study in the field and systematically analyse results with respect to domain, theory and design, which leads to resolution or specification of research questions.
This approach allows us to evaluate the appropriation of affordances on the cognitive level, continuously becoming more specific and aggregating more evidence in each iteration. While qualitative methods are helpful in the initial, exploratory design research, quantitative methods are more feasible in the later explanatory phases. To triangulate the rich contextual data and converge in the direction of the optimal design (Sandoval and Reiser, 2004), we used a variety of methods, e.g. (non-)participant observation, thinking aloud and different kinds of prototyping and variety of data sources, e.g. interviews and researcher notes.
With the concretization of the design artefact alongside the DBR study, the understanding of the three research perspectives (domain, design and theory) coevolves alongside an increasing degree of implementation into the workplace. This ranges from one-time workshops for the co-design of conceptual and paper prototypes, over limited workplace trials for low-fidelity prototypes (e.g. time or feature-wise) to long-term workplace integrations for high-fidelity prototypes.
Studying the appropriation of affordances with the Co-design by Appropriation of Affordances method
Throughout the Co-DAA process, the DBR team collaboratively identifies potential affordances for the desired cognitive tool and translates them into prototype features. The features create latent action possibilities, and their appropriation is analysed. When introducing the design into practice (3rd step) in each iteration, we can observe the appropriation of affordances in the interaction of the user with the features of the designed artefact in the work context and find three types of evidence:
Appropriation:
Expected appropriation: Evidence that the affordance created by the designed feature worked and was appropriated in the intended way and used respectively.
Unexpected appropriation: Evidence that the designed affordance materialized, but it was appropriated in a creative, unintended way; this offers valuable information about necessary changes in the domain understanding, design or underlying theory.
Dis-appropriation: Evidence that the affordance materialized, i.e. the feature did create the intended affordance, appropriation was stopped as it did not add any additional value, or it was not understood.
Non-appropriation: Evidence that the affordance did not materialize, i.e. the feature did not create the intended affordances and was never used.
Understanding the reasoning behind appropriation, hence, reveals the socio-cultural context of the professionals in terms of habits, mindset and needs. In particular, the unexpected appropriation offers valuable information about necessary changes in the domain understanding, the design or the underlying theory. For example, if tags provide the affordance of freely describing the content of collected learning resources with keywords, an expected appropriation would mean that professionals were using them in an intended way such as for reflection. If professionals were observed using predefined categories as tags that refer to their professional revalidation, then this would constitute an unexpected appropriation, which could point towards the need for including a corresponding ontology. Non- and dis-appropriation differ with respect to the general perception of the affordance, i.e. tagging could be stopped and the affordance dis-appropriated, because users do not see a value in tagging for description and searching; a “not materialized” affordance, on the contrary, could not be appropriated, because users do not recognize the action possibility of the tagging feature. This detailed analysis of the appropriation of affordances allows to derive several different insights on the domain (e.g. the importance of professional revalidation in the health-care domain), theory (e.g. how semantic categories underlie sensemaking processes) and design (e.g. usability of the tagging feature or the need for inclusion of certain predefined categories).
Evolving the knowledge base in Co-design by Appropriation of Affordances
Alongside the scheme of iteration that is shown in Figure 2, Co-DAA requires that knowledge of the three research perspectives that has been gained in the appropriation of affordances is continuously and systematically recorded in a knowledge base (Figure 2, lower part). This evolving knowledge base is used to formulate and update domain, theory and design hypotheses in step one of an iteration, and record research insights in step four. The building of the knowledge base follows three steps. Firstly, research insights are recorded individually along each of the three research perspectives (such as in the tagging example mentioned in the previous section). The dependency of those individual insights is then recorded across the iterations in the form of traceability chains, usually from domain, over theory to design insights. Finally, the elaborated traceability chains are synthesized into key research insights by aggregating evidence across iterations and developing stable sets of mutually corroborating research insights. These steps must be continuously implemented along the DBR study to evolve domain, theory and design insights. Recording and maturing research insights in all three research perspectives in this way leads to a systematic curation of the knowledge base, which contributes to the traceability of a DBR study and, hence, its validity.
Building a knowledge base through DBR represents a known approach in the community as it allows to collaboratively collect and mature abstracted insights (Kali, 2006; Gaß et al., 2012), and aggregate evidence for them over time. Our focus on traceability chains is related to the idea of conjecture maps that has been introduced by Sandoval (2014) in an attempt to create a clear link between theoretical hypotheses and their embodiment in design. Similar to what Sandoval calls “design conjectures”, components of design and their mediating roles in learning, we analyse these as affordances, which lets us consider design conjectures on a cognitive level. This is our attempt to decompose the learning processes into more fine-grained interactions.
Sandoval (2014) also argues for the importance of context in the design of learning technology. We concur with this view and contend that in the case of workplace learning, context may even be more important. In Co-DAA, we therefore propose the domain inquiry as a third perspective (besides theory and design) to capture part of this context that is critical for the insights gained. With Co-DAA, we present how the appropriation of affordances (from the perspective of domain, theory and design) contributes to the knowledge base in a systematic way.
Case study: description of the “bits and pieces” design-based research study
Overview over the bits and pieces case study
As a reaction to challenges in the health-care domain, especially the need to better support informal learning at the workplace, the DBR team “B&P” was set up. The B&P case study was conducted in close collaboration with practitioners from the UK health-care domain. Specifically, we worked with health-care professionals in primary care, e.g. medical centres, referred to as GP practices in the UK. This work took place over four years (2013-2017), from which the first one and a half years are reported here. The study resulted in a prototype for sensemaking of experiences, as well as insights about design challenges in the health-care domain, and about informal professional learning more generally.
The study is presented to illustrate and to validate the Co-DAA method we used. The design of Co-DAA itself was informed by previously suggested DBR methods and then iteratively refined in collaboration with practitioners and researchers. In our opinion, such validation through one in-depth case over a prolonged period of time offers distinct benefits such as gaining sufficient understanding of the needs and interest of the involved stakeholders, building trust with them and being able to apply similar techniques in several iterations in different design phases. Limitations of this approach will be discussed later. Here we present the B&P case in more detail, so as to allow for the replication of the method in other contexts.
Forming the design-based research team
To gather all necessary perspectives for a co-design study (Durall et al., 2020), the multidisciplinary DBR team comprised the research team with learning theory, design, software development and domain experts as well as our stakeholders, the health-care professionals (Figure 3). While the professionals represented our actual end users, the domain experts coordinated the field activities and included researchers in the field of medical education, contact persons to the English health-care system and senior practitioners.
A total of 40 health-care professionals from several GP practices were included in the domain inquiry (Thalmann et al., 2013; Figure 2) to take into account a wide set of stakeholders and perspectives (Edelson, 2002). We continued this multiple perspective approach during the co-design phase (Figure 4) and relied upon 11 health-care staff members with clinical, administrative and management roles from 2 GP practices as part of the DBR team (Table I): 2 GPs, 6 nurses, 2 assistants and 1 manager.
Design-based research phases
The B&P case study included a contextual inquiry, a concept creation phase and a co-design phase with three iterations (Figure 4). The purpose of the contextual inquiry (Leinonen et al., 2008) was to gain an initial understanding of the health-care domain, the current practices, challenges, constraints and opportunities. Based on this acquired understanding, we derived the design problem together with the professionals. To put this challenge into perspective, an in-depth inquiry into the design and theory was undertaken. Each took the form of a review of workplace-learning theories and of current technologies available for support. This way, the design concept was backed by research that led to a theoretical understanding of the issue, and research that led to several existing tools. After this inquiry, the design concept (“making sense of bits and pieces”) was explicated for the cognitive tool, and a common understanding was formed in the team about the objectives of the design.
The preparatory phases of contextual inquiry and concept creation, therefore, include a domain, theory and design inquiry that built the ground for the DBR study and subsequent co-design iterations. Recording a basic understanding of the three research perspectives in the knowledge base allowed us to infer hypotheses about domain, design and theory and, in parallel, identify a design problem and create a design concept that incorporates first affordances to be evaluated with prototypes in the field.
Following the concept creation, three co-design iterations took place that iteratively built three different, more and more concrete prototypes to implement and validate the concept of the cognitive tool, especially in terms of compliance with the professional practice. Each of them was administered at the workplace of health-care professionals in realistic settings, i.e. at the GP practices, on actual work tasks and under usual conditions. All iterations included partial simulations and did not represent a full workplace integration with unconstrained usage by the user. While we implemented the first two iterations in workshop style to co-design the initial conceptual and first interactive paper prototype, we designed a more controlled field study with actual usage of the software prototype in the third iteration. Depending on the study design and practical considerations, we conducted these iterations either in groups or individually.
Contextual inquiry: building a domain understanding and identifying the design problem
To gain a more in-depth understanding of the health-care domain and identify the design problem, a contextual inquiry was conducted to guide and inform later DBR phases. It began with a field visit in February 2013 to two GP practices in the UK. In the following months, 47 interviews, 4 focus group discussions and accompanying observations were produced (Thalmann et al., 2013). Several design artefacts were also created, for example, context cards showing pictures of some of the context factors (e.g. “time pressures” and “doctors’ revalidation”). These cards were later used in the concept creation and co-design phases to remind participants of important context factors. This initial contextual inquiry established the importance of self-regulated experiential workplace learning in health care.
The wealth of experiences is increasingly recognized in UK health care and professionals must undertake continuous professional development (GMC, 2012), which is monitored in all staffs’ yearly appraisal and the doctors’ five yearly revalidation. All valuable informal learning experiences, ranging from picking up a “to-read” during a short chat with a colleague to profound experiences connected to a certain professional activity (e.g. “patient unmet needs” that lead to “doctor’s educational needs”) can be reported as learning and considered in appraisal and revalidation. This is reflected in the professionals’ practice of collecting bits of information in a paper box file. The key thing that professionals are expected to demonstrate is that the learning has led to improvements in practice. This is not learning for the sake of learning – it is driven by a need to improve practice and care.
We identified that the lack of time and support to record, elaborate, document and share informal learning experiences during work gets in the way of self-regulated professional development and group learning, i.e. while there is the need to quickly record experiences (so called B&P of information) during “working time”, there is the need to efficiently remember, elaborate and document them during “learning time”. This constitutes the design problem of the present study.
Concept creation: inferring the research gap and creating the design concept
After a first formulation of the design problem, the DBR team conducted a state-of-the-art analysis of informal learning to situate the design problem in the wider context of learning theory. In particular, the question is how experienced events are to influence future behaviour. This may happen if experiences are memorized, understood, generalized and used to improve practice (Eraut, 2004). Turning information into action-oriented knowledge is called sensemaking (Pirolli and Card, 2005). It consists of two interacting sub-processes; the information foraging loop for searching, browsing and narrowing down data to subsets, which the sensemaking loop builds upon for organizing data, introducing concepts and producing final representations. Yet, sensemaking has not been studied in experiential learning. These insights about the way informal learning takes place at work and the role of different memory and sensemaking processes were informing the design inquiry. We found that none of the explored tools for sensemaking and reflection (Gotz, 2007; van Arnhem, 2013; Renner et al., 2014; DeWitt et al., 2015) helps professionals to contextually remember experiences and to meaningfully relate experiences to each other. Following our idea of a cognitive tool that would externalize representations, we could clearly perceive a gap in the available technology that is adapted to the needs of health care.
The identified theory gap of understanding the sensemaking process in informal learning as well as the missing technological support led us to consider a tool for professionals’ self-regulated sensemaking of experiences that affords the required cognitive processes and fits to the professionals’ daily working life. Therefore, we reviewed the results of contextual inquiry in a design workshop and externalized the acquired understanding into explicit concepts (Nonaka, 1991), i.e. articulate the tacit knowledge about the domain into a design idea by drawing upon related theories and design ideas.
The workshop was held as an open and free design activity. For example, we executed the box file practice as a fictitious simulation and explored its role in the organization of information. Each member took the role of a certain clinician and collected B&P of informal learning experiences on post-it(s) in a box file, which was represented by several sheets of paper. The categorization was reminiscent of an organization of notes on a corkboard and led to the creation of three groups of bits: patients (e.g. support for timing pills – identified need), research (e.g. telephone call to patient – method) and team (e.g. send new guideline – to-do’s). By making the activity of health-care professionals explicit, the design exercise raised the attention to the on-going process of remembering and applying learning experiences, which is grounded in action and happens every day of our life. We see this as a process in which external representations are created to aid memory retrieval and sensemaking that entails these central affordances.
The design workshop resulted in the creation of the design concept “making sense of bits and pieces” (metaphor for sensemaking in informal learning) and the tool called B&P. To illustrate relevant affordances, our design concept aimed to highlight the importance of moving from chaotic unstructured raw data, which is created as records of experiences, to meaningful learning that can be used to guide one’s own professional actions and practices.
Co-design iterations: an iterative exploration of theory, domain and design
After concept creation, the following steps now include the iterative, systematic and collaborative production of design solutions and parallel observation of appropriation of affordances. Below we describe the first three iterations that the B&P DBR team performed to go from a conceptual to a high-fidelity software prototype of a cognitive tool ready to be tested with a larger number of users. After giving an overview of these iterations, we explain in more detail the way in which we undertook the analysis of the appropriation of affordances and tracked the design decisions between iterations with the Co-DAA method to infer and evolve research insights about domain, theory and design.
Design iteration I: scenarios and wireframes – conceptual prototype.
Build loop.
In the first design iteration, we implemented a conceptual prototype that explicated the design concept. During working time, health-care professionals mainly collect (anonymized) records of their experiences such as photos of a journal article on deflection, training video on bandaging, notes on a discussion or a bookmark of a guideline. Hence, they need recording facilities with the option to add notes and tags. These B&P of information later function as memory aids to reconstruct experiences during sensemaking or learning time. Besides remembering experiences, professionals then need to make sense of them by sorting and organizing corresponding B&P into topics in corkboard style (Figure 5).
Evaluate loop.
The conceptual prototype represented the basis for the first co-design iteration. In individual meetings, the research team first explained the design idea with the help of the scenarios and wireframes (Figure 5) printed on big sheets of paper. After clarifying question and acquiring a shared understanding of the design idea, the professionals were encouraged to directly modify scenarios and wireframes by sketching imaginary tool support, for example. In a subsequent group setting, the box file exercise was conducted analogous to the activity during concept creation. The method of graphical elicitation “allow[ed our professionals] taking part in the research process according to [their] own preferred modalities of expression” (Bagnoli, 2009). To build upon results of contextual inquiry, professionals were provided with the context cards. This way, we initiated the dialogue about the domain, theory and design hypotheses underlying the conceptual prototype and non-, dis- or appropriated the affordances.
Design iteration II: low-fidelity prototyping – paper prototype.
Build loop.
In the second design iteration, we deepened the theoretical understanding, revised the hypotheses and transformed them into a paper-based prototype. As the collection of B&P of experiences is already sufficiently supported with a multitude of systems, we focused on recalling, narrowing down (second part of information foraging) and sensemaking of informal learning experiences. Therefore, we decided to split the user interface into two parts, so called canvases, with each of them having its specific purpose (Figure 6), i.e. while one should support browsing and recalling of experiences to create a meaningfully related subset during information foraging, the other should facilitate their organization to generate and instantiate representations during sensemaking. These canvases were vertically arranged with an upper browse-canvas providing different presentations of episodic and semantic cues in form of B&P and a lower organize-canvas for different iterative and interactive visual manipulation opportunities. For example [Figure 6(a)], the timeline styled browse-canvas allowed for recalling experiences with time-based cues and the Venn-diagram styled organize-canvas allowed for categorizing bits in circles. Dragging and dropping of bits from the upper to the lower canvas enabled building a subset of related experiences.
Evaluate loop.
The research team introduced the mock-ups using oversized paper prototypes (Bødker and Grønbæk, 1991; Ley et al., 2009), i.e. a paper-based user interface simulating the canvases, separate small paper snippets depicting different user interface elements (e.g. contextual menus, drop-down boxes, arrows or maps) and paper icons representing the collected bits of experiences (Figure 6). After, we explained the paper prototype and stopped for discussion at the intended affordances, i.e. the general architecture of the interface and the browse (timeline, maps and tag cloud) as well as the organise views (circles, layers and links). During each break, the users had the opportunity to interact with the paper prototype. We asked them to make use out of their prepared “bits”, play around and explore, what they would do with them in the different views and what they would share and when. This paper prototype already allowed for a more thorough user interaction, which we traced via graphical elicitation. The analysis of the affordance appropriation evolved the threefold DBR insights.
Design iteration III: high-fidelity prototyping – functional software prototype.
Build loop.
In the third design iteration, we reduced the design ideas from paper prototyping and streamlined them to merge into a first functional software prototype. Because the design inquiry had resulted in the understanding that it was the sensemaking loop that was currently weakly supported, a decision was made to use an existing tool for foraging and focus our co-design efforts on supporting the sensemaking activity. Hence, an import mechanism was implemented for B&P, i.e. Evernote (van Arnhem, 2013) was used for recording experiences (busy working time) and the B&P prototype to make sense of them (flexible working time). This allowed us to offer a comprehensive set of multimedia recording functionalities and to concentrate on the missing cognitive support of browsing, recalling and sensemaking in informal learning. For browsing and recalling, the timeline was implemented in the upper canvas (Figure 7). The bits were depicted as notes in the timeline at the date of their collection with Evernote. Users could drag and drop the timeline to adjust the period shown as well as zoom in and out via the mouse wheel to browse through the gathered bits. While clicking a bit opened a pop-up with information on title, resource, author and time stamp, double-clicking linked to the full content of the bit in Evernote (Figure 7: (1) click on user action). This navigation should allow professionals to find and recall experiences during information foraging they were searching for. Dragging bits from upper to lower canvas enabled the creation of a meaningfully related subset of experiences and complemented information foraging support (Figure 7: (2) drag record entity). In the lower canvas, sensemaking was implemented via a circle-based categorization (Figure 7: (3) drop record entity). Users could create circles by double-clicking and move bits into these categories or their overlap to indicate shared structures. The emerged circles could then be named by simply clicking on the initial placeholder (e.g. “concept X”).
Evaluate loop.
To investigate our hypotheses, we set up the following procedure for individual co-design interactions; a set-up meeting to introduce the task and give an Evernote tutorial, followed by a period of three weeks to record experiences and a half day meeting to use the B&P prototype and make sense of the collected bits. Each participant first chose a realistic learning need for the collection phase in the set-up meeting, e.g. update understanding around vitamin D deficiency (GP2) or new diabetes diagnostics (DSN1) and clarify implications for the GP practice. The definition of a real learning need was crucial to enable meaningful achievements for the professionals’ work and to increase their motivation at the same time. This way, the study acted as a realistic test of affordances and did not add to the workload of HC staff. Afterwards, participants received a 30 min introduction to Evernote. The participants were then left to use Evernote during the collection phase. In the sensemaking phase, the research team then introduced the B&P prototype and acquainted the professionals with its functionalities. Their Evernote material was already imported into B&P and participants had time to browse, recall and organize bits. The software prototype enabled real-user interaction with working materials embedded into actual learning needs, which further deepened the understanding of the three research perspectives.
Application of the co-design by appropriation of affordances method in the bits and pieces case and obtained results
Studying the appropriation of affordances with co-design by appropriation of affordances
Recording the appropriation of affordances.
The key method that was applied throughout all iterations was an analysis of which of the intended affordances were appropriated in the field and how, and which were not appropriated and why. This appropriation was analysed from the research team by observing the usage of artefacts and prototypes and deducting the emergence of affordances from these past user interactions. All three iterations relied on a qualitative study of appropriation, i.e. we applied different means to trace the recognition of affordances during the interaction with the B&P prototypes, ranging from direct measurements such as recording the annotation of prototypes to indirect measurements such as writing observational notes. In the second and third iteration, we additionally asked the professionals to think aloud when interacting with the prototypes (Lewis, 1982) and conducted subsequent interviews to record all traces of appropriation for analysing affordances and to better explicate confusions, concerns, ideas, etc. for understanding their reasoning.
Figure 8 shows one example of an annotated design artefact, namely, a conceptual poster prototype, that resulted from the graphical elicitation, by one of the GPs involved in the DBR team, during the first design iteration. After a common understanding had been established within the DBR team, GP1 used post-it(s), stuck them to the relevant scenario or depicted wireframe, added a rating or provided further contextual information. For example, he noted that he “could see a photo acting as a trigger”, which confirmed our understanding of the importance of contextual cues when learning experiences are recalled. This insight aggregated evidence for the deducted affordance of “cue-based retrieval of experiences”.
The appropriation process resulted in the following data for the three iterations:
Design iteration I: Eight professionals including one general practitioner (GP1), four practice nurses (PN1, PN2, PN3 and PN4), two health-care assistants (HCA1 and HCA2) and one practice manager (PM1) created 12 annotated poster prototypes as well as one annotated paper-based box file, and the research team created 16 A4-pages of observational notes.
Design iteration II: Six professionals including one general practitioner (GP2), one practice nurse (PN1), two health-care assistants (HCA1 and HCA2) and two diabetes specialist nurses (DSN1 and DSN2) created six annotated paper prototypes and the research team, again, created 36 A4-pages of observational notes.
Design iteration III: Four professionals’ including one general practitioner (GP1), one health-care assistant (HCA2) and two diabetes specialist nurses (DSN1 and DSN2) – thinking aloud and follow-up interviews resulted in 3 h of audio records that were transcribed to 71 pages and 17 additional pages of researchers’ observation notes.
Analysing the appropriation of affordances.
In each co-design iteration, we conducted deductive qualitative content analysis (QCA) (Mayring, 2000) based on this data to determine the efficacy of the affordances, i.e. audio recordings were first transcribed literally, paraphrased and then iteratively analysed together with the other records of appropriation, whereby the content was further structured and summarized in categories in each iteration, to finally discover the appropriation of emerged affordances in the form of unexpected, expected, non- or dis-appropriation. Additionally, we iterated the QCAs across co-design iterations. When an insight was discovered in a later iteration, for example, we had to return to the QCAs of preceding iterations and search for validating or rejecting evidence in them. This qualitative approach was demanding as it required reopening and reanalysing already administered QCAs in terms of new clues, again and again, i.e. many assumptions are emerging and need to be followed-up in such an iterative qualitative endeavour, but only some will aggregate enough evidence to be considered as valid and taken forward.
To give an example, we list below instances of actual appropriation of emerged affordances (formatted in italic font) that happened alongside the third iteration in the context of the tagging functionality intended for immediate reflection of an experience as mentioned above:
Expected appropriation: Direct evaluation of an experience, i.e. professionals were observed to create a title and tags of a note in Evernote when there is time during recording an experience (early reflection).
Unexpected appropriation: Re-evaluation of recorded experiences after recall, i.e. professionals were observed to manipulate the title and tags of a note in Evernote after recalling the experience (late reflection), and requested the manipulation of tags in B&P.
Dis-appropriation: Discontinuation of tagging, i.e. some individuals were discontinuing tagging in Evernote because of the limited number of notes they had recorded and because they saw limited use of tagging for description and reflection.
Non-appropriation: Lack of awareness of the tagging functionality, i.e. some professionals did not perceive the possibility for tagging in Evernote.
Figure 9 provides the traceability graph, i.e. an overview of all affordances, how they were motivated from insights into domain, theory and/or design decisions and how they were appropriated throughout all iterations. These affordances are depicted in the top row of Figure 9 and they are listed in the below in italic formatting:
The possibility to re-find notes and recall the respective experiences in one system (one place – no loss of recorded experiences and ideas).
The possibility to record experiences on the run and making sense of them when workload allows for (collection and sensemaking/“working and learning time” → busy and flexible working time).
The possibility to reflect on the relevance of a collected note and annotate it when recording or recalling an experience ([re-]evaluation).
The possibility to recall reflected notes based on semantic cue-based retrieval and not reflected notes based on temporal cue-based retrieval (time and topic recalling – information foraging: retrieval).
The possibility to build a subset of collected notes for sensemaking (information foraging: narrowing down information to a meaningful subset).
The possibility to visually organize collected notes to relate experiences to each other and to the prior knowledge (card sorting → interactive sensemaking visualization → unconstrained interactive sensemaking visualization).
The possibility to enable hands-on reflection for conscious cognitive processes (auto import and manual sensemaking).
The possibility to collaboratively make sense of experiences (meaning making).
The graph includes several traceability chains of creating and appropriating specific affordances of the cognitive tool that followed a vertical or diagonal pattern, starting from a new domain insight, i.e. a chain comprises identification of a design problem, continues over theoretically inspired design of a feature and results in the creation of an applied action affordance for appropriation. A traceability chain, hence, represents a concertation of insights from all three research perspectives and respective assumptions of the single iterations. These traceability chains give an account of the emergence of cognitive processes which happens alongside the socio-cultural development of professionals and their practices. Understanding and recording the theoretical, contextual and technical genesis of affordances in a traceable way, allows leveraging these insights in similar application scenarios.
Below we report the traceability chain of the poster prototype “busy and flexible working time” (Figure 5) and its unexpected appropriation of the “collection and sensemaking” (“working and learning time”) affordance; all relevant domain theory and design insights (Figure 9) are formatted in italic font (see two further examples in Appendix 1 – one expected and one unexpected appropriation):
Domain insight: Health-care staff faces a wealth of experiences during their working time. For example, when there is a new official diabetes guideline, they search the Web to understand the best way of implementing it in practice. Because of time pressure, they cannot immediately address those experiences, engage in self-regulated learning and gain action-oriented knowledge that is needed for the implementation of the guideline and their continuous professional development. Hence, they need to quickly record their experiences with whatever means at hand (e.g. digital – email and analogue – box file) to come back to them when time permits it during learning time.
Theory insight: Eraut (2004) sheds light on informal learning and the required cognitive processes of memorization and retrieval. He describes the process with the help of the two “memory systems”, episodic and semantic memory. On the one hand, personal experiences enter the episodic memory and get transferred into the semantic memory by training and gaining expertise. On the other hand, received knowledge can directly enter the semantic memory system by learning facts. The sensemaking theory (Russell et al.., 1993) then allows to understand the process of professionals gaining action-oriented knowledge in more detail.
Design insight: We created a poster prototype to reflect the domain insights about the two-fold health-care professional life (working and learning time) as well as theory insights on memorization, retrieval and sensemaking. While recording pictures or making notes of experiences supports the information foraging, a corkboard indicated sensemaking support.
Appropriation: The conceptual prototype is in line with the identified practice of collecting bits, first, for making sense out of them at a later date: “chuck it in there, I’ve no time to do this today, and then I’ll do it when I’ve got the [chance] once a month time or something” (PN1). The professionals, however, stressed that learning time needs to be part of work because “working time” indicated that workplace learning needs to happen in free time. Instead, they suggested busy and flexible working time, which represents an unambiguous label and reflects the right timing. This reflects an unexpected appropriation.
Because of the bottom-up nature of the traceability chains, the traceability graph in Figure 9 also evolved diagonally from the bottom left to the top right alongside the phases of a DBR study (as shown in Appendix 2). However, the increased domain understanding is at the same time a result of appropriation, which closes the design cycle. It can also be noted that affordances of cognitive tools evolve together with the related practices across iterations. For example, a sensemaking affordance identified in the contextual inquiry (box file) was used in a wireframe (corkboard – conceptual prototype) and then developed as an interactive sensemaking visualization (circles view – paper and software prototype; see enumeration of affordances above traceability graph in Figure 9 for more examples of evolved affordances).
Synthesizing domain, theory and design insight into key research insights with co-design by appropriation of affordances
The third step in building the knowledge base alongside the design iterations is the synthesis of the research insights about learning theory, domain and design, and their relationship (as recorded in the traceability graph) into what we call key research insights. Next, we report two of these insights to illustrate how these were obtained.
Key insight 1 – the process of sensemaking in informal learning.
In the case study, we extended our understanding of the role of sensemaking in informal learning. Starting from domain insights, it became evident that the practice of informal learning is aligned to the professionals’ busy working life, while they quickly record experiences in a box file to not forget about them during busy working time, they recall and elaborate them to gain action-oriented knowledge during flexible working time. This allowed us to derive some new insights about informal learning, namely, that the raw “data” an individual is searching, evaluating and filtering during foraging originates from a person’s own experiences. These experiences are recalled, re-evaluated, browsed and condensed to a meaningful subset for semantically categorizing them during sensemaking. The separation of collection and elaboration, hence, accounts for the information foraging and sensemaking loop (Pirolli and Card, 2005). While the collection of experiences is already well supported (e.g. through tools like Evernote), we focussed the design on sensemaking in informal learning with B&P, i.e. to recover collected bits (notes) based on temporal cues, recall experiences and develop an understanding by categorizing bits for changing practice as well as reporting for professional revalidation. The sensemaking support is administered via the canvas-based separation of the user interface with respect to recalling and browsing in the browse-canvas (foraging loop) as well as relating and organizing experiences in the organize-canvas (sensemaking loop). This includes features such as contextual information on notes for cue-based retrieval in the timeline, possibilities to edit the title and tags of notes in the browse-canvas for re-evaluation and unconstrained interactive organization of notes for sensemaking via Venn-diagram styled categorization in circles of the organize-canvas.
The first key insight comprises the following affordances (see enumeration of affordances above) and respective key insights (Figure 9: 1, 2, 3, 4, 5 and 6). The described sensemaking process as part of experiential learning in health care contributes to the understanding of individual problem solving of ill-defined problems in organizations as well as respective knowledge creation processes (Goggins and Jahnke, 2010). Furthermore, the cognitive process can be used to specify macroscopic processes of managing and making sense of information in personal knowledge management (Jarrahi et al., 2019).
Key insight 2 – past- and future-focused experiences in workplace learning and their role in sensemaking.
The case study helped us identify a two-fold learning focus in making sense of experiences at work. With respect to the domain, we found health-care professionals collecting informal learning experiences with past and future focus. While past-focused experiences represent what has been done or learned, future-focused experiences represent quick notes of tasks or things one wants to come back to later and learn more about (i.e. to-do’s). These domain insights informed the theory inquiry and led to the insight that the two-fold learning focus is in line with Eraut’s (2004) differentiation between reactive and deliberate informal learning. While reactive learning means opportunistic learning in terms of facing a chance to learn, deliberate learning describes an intentionally planned behaviour. In deliberate learning, a certain prior understanding, i.e. a pre-existing cognitive representation, is instantiated for the chosen learning need. This also means an individual has reflected on some experiences, knows about learning gaps and has ideas of how to take the next steps in terms of future-focused experiences. In contrast, in reactive learning, experiences are actually made in the moment because of a given learning opportunity. This leads to past-focused experiences and the generation of an initial representation to be developed in sensemaking (Russell et al., 1993). The focus of experiences evolves dynamically as past-focused experiences reveal knowledge gaps and lead to future-focused experiences (from “so what” to “what next”). Thus, sensemaking establishes meaningful relations between past and future experiences. These two ways of using B&P were promoted. On the one hand, reactive learning in sensemaking is supported by the possibility to record past-focused experiences (e.g. audio-recording a discussion with colleagues) on the run and then return to them during more flexible working time. In that case, bits are collected first and then iteratively categorized in new circles and a sensemaking representation is initiated. Deliberate learning in sensemaking, on the other hand, is supported by creating the circles in the organize-canvas first and then collecting bits to fill these categories (e.g. saving a document “to-read”).
The second key insight comprises the following affordances (see enumeration of affordances above) and respective key insights (Figure 9: 1, 3, 4 and 7). The specification of informal sensemaking process in terms of its intentionality also contributes to the understanding of organizational knowledge creation in ill-defined problems (Goggins and Jahnke, 2010).
Discussion of the co-design by appropriation of affordances design-based research method
In this paper, we have suggested an extension of DBR and demonstrated its application in developing a cognitive tool for health-care professionals’ informal learning. Below, we discuss how the application of this Co-DAA method produces insights about learning theory, domain and design, and we establish its value by discussing its advantage of yielding traceable key insights about sensemaking in informal workplace learning. We close with a reflection about the wider applicability of the method for creating cognitive tools for workplace learning beyond health-care.
Benefits of the method for co-designing cognitive tools embedded in workplace practice
We have claimed that our proposed extension of DBR is more suitable for the design of cognitive tools for workplace learning than more traditional methods. Following our argument of cognitive tools as socio-cognitive artefacts, we assume that cognitive tools create affordance networks (Barab and Roth, 2006) in the sense that they are embedded in the socio-cultural development of practice, and make certain behaviours more likely than others. If designed in the right way, these affordances are in line with the intentions of the activity system and guide respective operations, actions and activity. The Co-DAA method we have suggested emulates this socio-cultural development process and has an impact on it via the appropriation of co-designed affordances. This means an iterative and integrative approach to development of cognitive tools, which includes a in depth domain inquiry, involves key stakeholders in the design process and impacts their practice. Next, we discuss the methods’ ability to evolve traceable and valid insights about domain, theory and design as well as synthesized key research insights through systematic analysis of appropriation.
Co-design by appropriation of affordances develops cognitive tools with working practices through an iterative and integrative approach.
The iterative long-term approach allowed us to stay in constant exchange with all stakeholders. Continuously focussing on their design problems over prolonged period also helped to build trust and ownership with our participants along the case study. As researchers, we did not approach them as experts to teach them about how to do their job better, but we took them seriously as true participants in the research and co-developed potential design solutions together. We found this trust and ownership building to be critical for professional’s motivation in co-designing cognitive tools in less controlled workplace settings and in developing the openness for changes in practices that technology adoption entails. The integrative co-design study turned into a common learning journey, and, hence, the appropriation of technology was built into the research process.
In line with Sandoval’s (2014) pointer to the importance of “context” in DBR, we focused on the domain alongside the theory and design inquiry as understanding the domain with its practices is even more important in DBR for workplace learning. Out of this reason, the inquiry into the domain was continuously implemented throughout the B&P case study. The domain perspective was first addressed in-depth in the contextual inquiry as part of the preparatory phase to identify the design problem and build the ground for concept creation. Afterwards, the domain inquiry was continued in each of the design iterations. This way, the Co-DAA method respects workplace practices and integrates tools in them as well as co-evolves tools with them over time in a DBR study.
There is now some evidence that the iterative and integrative DBR approach we have presented has not only led to the design of particular cognitive tools, but in fact practices of health-care professionals involved in the DBR team and beyond have changed (Treasure-Jones, 2017; Treasure-Jones et al., 2019). We do not claim that these changes have been a result of our intervention, exclusively. The argument we want to bring forward here is that the Co-DAA method was able to go alongside the regular and natural changes of how cognitive tools are appropriated in practice and how practices are changed in a co-evolving manner. The evidence we can bring forward for this claim is how the developed tools were appropriated over time, and how the health-care practitioners felt ownership of the tools and the process.
Co-design by appropriation of affordances contributes to traceability and validation in design-based research through systematic analysis of appropriation.
We have claimed that DBR often lacks methods to systematically establish traceability and validation about the knowledge that is developed in the design process, especially when it comes to the messier workplace contexts and the cognitive level. We introduced the appropriation of affordances in Co-DAA as a general analysis principle, and implemented it in various data collection methods, such as observations, graphical elicitation, thinking aloud and structured interviews across all design iterations. Although affordances are usually discussed only with regards to a running system (where they offer latent action possibilities), we found it useful to take a similar perspective with the paper and even conceptual prototypes of cognitive tools. In these cases, the “use” of the system often consisted of simulated behaviour or even just mental simulation to check assumptions.
Traceability is documented in the traceability chains (Figure 9; Appendix 1), which depicts concerted domain, theory and design insights from iteration one to three, from poster to software prototype, from user interface to feature appropriation and from general to specific design and analysis of affordances. The typical order from domain over theory to design insights can be ascribed to the fact that the domain and its stakeholders usually define the challenge, for which researchers and designers evolve a particular action possibility (affordance) that is informed by theory and incorporated in the design. This documentation via traceability chains also helped us establish evidence for validation across several design phases using different methods and design artefacts. For example, we gained evidence for the existence of “busy and flexible working time” and the different practices and affordances found therein. The concept was already established in the contextual inquiry, later validated through the box file exercise, and the division of collection and sensemaking in the paper and software prototype. This helps to build validation of the design into the process by means of establishing convergent evidence for a particular design decision. Traceability chains are related to conjecture maps suggested by Sandoval (2014), but with the focus on the domain knowledge, they also record evidence about the practice patterns found in the domain.
In addition, we analysed and synthesized these interrelated research insights and traceability chains alongside the DBR study to finally arrive at trustworthy key research insights consisting out of mutually corroborating domain, theory and design insights (Figure 10). We have presented two key research insights gained through the application of the Co-DAA method. These could also be understood as design principles (Kali, 2006), which could eventually be entered into a design principles database.
Limitations of appropriation-based design-based research
A possible limitation lies in the fact that we focussed on only one case of workplace learning, which limits generalizability to other contexts. An alternative would have been to validate the developed Co-DAA method across several cases in several different domains. We refrained from such an approach for two reasons. Firstly, conducting a DBR study in professional contexts requires a significant amount of trust building. While in academic settings, it might be feasible to recruit new cohorts of students year after year to design and validate a particular method, this is not feasible in limited and less-controlled professional learning settings. In such setting with smaller samples, where learning is not the main focus, it needs the building of significant trust to reach sufficient openness for conducting research in a collaborative way. For example, it requires trust to have professionals discussing challenges of informal learning (where the term “learning” is sometimes associated with a personal weakness). The second reason for choosing a longitudinal focus was the opportunity that comes from such a very contextualized approach. It allowed us to focus on some challenges that kept reappearing over time and that allowed us to address them with different methods and from different points of view. For example, the sensemaking affordances of categorization and grouping re-appeared throughout the iterations with the box file exercise, the corkboard (conceptual prototype) and the circles (paper and software prototype).
Besides generalizability, the longitudinal approach does lead to increased effort for the professionals. This introduces additional demands on top of their busy schedules. Instead of distributing the effort onto a bigger sample, however, professionals get the chance to see the effect of their feedback and to evaluate the re-designs and “revised” affordances increasing validity in this way. Even though we chose the long-term above the cross-sectional design approach (trust-building and validity reasons), our Co-DAA method has been leveraged in other long-term case studies of the Learning Layers European Integrated Project. For example, Pejoska-Laajola et al. (2017) and Virnes et al. (2017) leveraged the qualitative focus on usage (analysis of appropriation) in their research-based design process to co-design cognitive tools for the construction sector.
Conclusions and future work
The Co-DAA DBR method allowed us to set up hypotheses for domain, design and theory, gain traceable research insights and gather first evidence for the informal sensemaking process in the information rich and complex health-care domain. The professional’s appropriation of the cognitive tool B&P indicated successfully evolved practices. Further design iterations will now help to check the validity of the acquired key insights. This way, we continue monitoring the methods ability of embedding a design artefact in working practices and gaining trustworthy research insights at the same time.
Although our method is aiming at the workplace, its focus on the study of cognitive tools and generating traceable research insights makes it relevant for formal learning contexts as well. The application of e-portfolios in health-care education is one example of where we aim at demonstrating the benefits of the Co-DAA method in the co-design of cognitive tools in the university context.
Figures
Figure 1.
Flow diagram of appropriation process by Carroll (2004)
Sample of the co-design phase including abbreviation, role and participation in iterations per participant
| Abbreviation | Role | Iteration |
|---|---|---|
| GP1 | General practitioner | 1 |
| GP2 | General practitioner | 2 + 3 |
| PN1 | Practice nurse | 1 + 2 |
| PN2 | Practice nurse | 1 |
| PN3 | Practice nurse | 1 |
| PN4 | Practice nurse | 1 |
| HCA1 | Health-care assistant | 1 + 2 |
| HCA2 | Health-care assistant | 1 + 2 + 3 |
| PM1 | Practice manager | 1 |
| DSN1 | Diabetic specialist nurse | 2 + 3 |
| DSN2 | Diabetic specialist nurse | 2 + 3 |
Appendix 1. Two further exemplary traceability chains
In the following depiction of the traceability chains, all relevant domain, theory and design insights are formatted in bold font. These insights are also marked in Figure A1:
“sensemaking” traceability chain - bold and light grey
“re-evaluation” traceability chain - bold and dark grey
Traceability chain (paper prototype): Sensemaking → expected appropriation of the affordance “interactive sensemaking visualization” (specification by “unconstrained”) in the B&P paper prototype
Domain: Collecting notes with respect to a certain learning need in a box-file and understanding the diverse information at hand, requires professionals to relate their respective experiences to each other. Leveraging a corkboard or card sorting helps professionals gaining action-oriented knowledge and drawing a conclusion for practice.
Theory: HCI sensemaking theory by Russell et al. (1993) and Pirolli and Card (2005) explains the action-based process of semantically elaborating experiences to implement learning into practice (episodic to semantic). By manually relating information on a corkboard, for example, bottom up and top down processes generate or instantiate representations for “encoding retrieved [external] information to answer task-specific questions” (Russell et al., 1993).
Design: The need for active sensemaking combined with the desire of a corkboard styled approach led to the design idea of categorization by means of Venn diagrams. The respective paper-prototype, hence, allowed to manually cluster bits by labelled circles.
Appropriation: The paper-prototype is in line with the professionals’ practice of elaborating experiences to implement learning into practice. In particular, the unconstrained and action-based character of the sensemaking visualization was appreciated (expected appropriation).
Traceability chain (software prototype): Re-evaluation → unexpected appropriation of the affordance “evaluation” in terms of “RE-evaluation” in the B&P software prototype
Domain: Professionals collect two kinds of bits: bits that remind of past experiences such as notes on a conference (past focussed experiences) and bits that remind of upcoming experiences such as a to-read (future focussed experiences). Most bits are collected in short bursts and represent things professionals want to follow-up (haphazard collection). After remembering an experience, we observed a professional re-evaluating the corresponding bit: “[D]o I still need this? If I do need this, what am I going to do with it?” (HCA2). She examined the future focused experience in more detail than she was able during busy working time and assessed it again. Another participant puts it like that: “What you really need, is to be able to go back to something you’ve collected, go into it, read it and then […] make your own notes.” (DSN1).
Theory: Coming back to a collected bit, (re-)assessing the title and tags represents a cognitive reflection process which leads to an enrichment of the episodic memory trace with semantic concepts. Such an interaction of the semantic with the episodic ‘memory’ enables retrieval by semantic cues.
Design: Collected bits could be annotated by title and tags in Evernote. This annotation could be continued whenever the professionals wanted to re-evaluate the recorded experience as double clicking the bit in the Timeline View referred to the original bit in Evernote.
Appropriation: The professionals unexpectedly manipulated the bits’ titles and tags within Evernote during flexible working time and requested their manipulation in B&P, as well. This unexpected practice pointed to the need of a (re-)evaluation step: i.e. sometimes bits are collected under high time pressure complicating early reflection.
Appendix 2. Previous versions of traceability graph
The traceability graph has been created in each co-design iteration, successively. Therefore, each version is building upon the previous one. The final version is reflected in Figure 1. Subsequently, its predecessors are presented (Figure A2 and A3).
References
Ashley, C., Halcomb, E., Brown, A. and Peters, K. (2018), “Experiences of registered nurses transitioning from employment in acute care to primary health care – quantitative findings from a mixed-methods study”, Journal of Clinical Nursing, Vol. 27 Nos 1/2, pp. 355-362, doi: 10.1111/jocn.13930.
Bagnoli, A. (2009), “Beyond the standard interview: the use of graphic elicitation and arts-based methods”, Qualitative Research, Vol. 9 No. 5, pp. 547-570, doi: 10.1177/1468794109343625.
Barab, S. and Roth, W.-M. (2006), “Curriculum-based ecosystems: Supporting knowing from an ecological perspective”, Educational Researcher, Vol. 35 No. 5, pp. 3-13, doi: 10.3102/0013189X035005003.
Barab, S. and Squire, K. (2004), “Design-based research: putting a stake in the ground”, Journal of the Learning Sciences, Vol. 13 No. 1, pp. 1-14, doi: 10.1207/s15327809jls1301_1.
Billett, S. and Choy, S. (2013), “Learning through work: emerging perspectives and new challenges”, Journal of Workplace Learning, Vol. 25 No. 4, pp. 264-276, doi: 10.1108/13665621311316447.
Bødker, S. and Grønbæk, K. (1991), “Cooperative prototyping: users and designers in mutual activity”, International Journal of Man-Machine Studies, Vol. 34 No. 3, pp. 453-478, doi: 10.1016/0020-7373(91)90030-b.
Carroll, J. (2004), ‘“Completing design in use: closing the appropriation cycle”, in Proceedings of the European Conference of Information Systems, AIS Electronic Library (AISeL), Turku, Finland, p. 11.
Clemmensen, T., Kaptelinin, V. and Nardi, B. (2016), “Making HCI theory work: an analysis of the use of activity theory in HCI research”, Behaviour and Information Technology, Vol. 35 No. 8, pp. 608-627, doi: 10.1080/0144929x.2016.1175507.
DBRC (Design-Based Research Collective) (2003), “Design-based research: an emerging paradigm for educational inquiry”, Educational Researcher, Vol. 32 No. 1, pp. 5-8, doi: 10.3102/0013189X032001005.
DeSanctis, G. and Poole, M.S. (1994), “Capturing the complexity in advanced technology use: adaptive structuration theory”, Organization Science, Vol. 5 No. 2, pp. 121-147, doi: 10.1287/orsc.5.2.121.
DeWitt, D., Alias, N. and Siraj, S. (2015), “‘Collaborative learning: Interactive debates using padlet in a higher education institution”, Proceedings of the International Educational Technology Conference, Istanbul, Turkey, pp. 88-95.
Durall, E., Bauters, M., Hietala, I., Leinonen, T. and Kapros, E. (2020), “Co-creation and co-design in technology-enhanced learning: innovating science learning outside the classroom”, Interaction Design and Architecture(s), 42(Co-Creation in the Design, Development and Implementation of Technology-Enhanced Learning), pp. 202-226.
Edelson, D.C. (2002), “Design research: what we learn when we engage in design”, Journal of the Learning Sciences, Vol. 11 No. 1, pp. 105-121, doi: 10.1207/s15327809jls1101_4.
Engeström, Y. (2001), “Expansive learning at work: toward an activity theoretical reconceptualization”, Journal of Education and Work, Vol. 14 No. 1, pp. 133-156, doi: 10.1080/13639080123238.
Eraut, M. (2000), “Non-formal learning and tacit knowledge in professional work’, The ”, British Journal of Educational Psychology, Vol. 70 No. 1, pp. 113-136, doi: 10.1348/000709900158001.
Eraut, M. (2004), “Informal learning in the workplace”, Studies in Continuing Education, Vol. 26 No. 2, pp. 247-273, doi: 10.1080/158037042000225245.
Fessl, A., Wesiak, G., Rivera-Pelayo, V., Feyertag, S. and Pammer, V. (2017), “In-app reflection guidance: Lessons learned across four field trials at the workplace”, IEEE Transactions on Learning Technologies, Vol. 10 No. 4, pp. 488-501, doi: 10.1109/TLT.2017.2708097.
Fischer, G. (2013), “A conceptual framework for computer-supported collaborative learning at Work”, in Goggins, S. P. and Jahnke, I. (Eds) Computer-Supported Collaborative Learning at the Workplace, Springer, Boston, MA, pp. 23-42.
Fishman, B.J., Penuel, W.R., Allen, A.R., Cheng, B.H. and Sabelli, N.O.R.A. (2013), “Design-Based implementation research: an emerging model for transforming the relationship of research and practice”, Yearbook of the National Society for the Study of Education, Vol. 112 No. 2, pp. 136-156.
GMC (General Medical Council) (2012), “Continuing professional development: guidance for all doctors”, GMC, Manchester, available at: www.gmc-uk.org/Continuing_professional_development___guidance_for_all_doctors_0612.pdf_56438625.pdf (accessed 7 January 2016).
Gaß, O., Koppenhagen, N., Biegel, H., Maedche, A. and Müller, B. (2012), “Anatomy of knowledge bases used in design science research: a literature review”, Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), doi: 10.1007/978-3-642-29863-9_24.
Gaver, W. (1991), “Technology affordances”, Proceedings of the CHI Conference on Human Factors in Computing System, ACM Press, New Orleans, LA, New York, NY, pp. 79-84, doi: 10.1145/108844.108856.
Gibson, J.J. (2014), “The ecological approach to visual perception”, Behavioral Science, Psychology Press, Boston, doi: 10.4324/9781315740218.
Goggins, S.P. and Jahnke, I. (2010), “‘Computer supported collaborative learning at work: CSCL at work”, Proceedings of the 16th ACM International Conference on Supporting Group Work, GROUP’10, pp. 361-362, doi: 10.1145/1880071.1880151.
Gotz, D. (2007), ‘“The ScratchPad”, Proceedings of the 16th international conference on World Wide Web - WWW ‘07. ACM Press, New York, NY, pp. 1329-1330, doi: 10.1145/1242572.1242834.
Hevner, A.R., March, S.T., Park, J. and Ram, S. (2004), “Design science in information systems research”, MIS Quarterly, Vol. 28 No. 1, pp. 75-105, doi: 10.2307/25148625.
Hollan, J., Hutchins, E. and Kirsh, D. (2000), “Distributed cognition: toward a new foundation for human-computer interaction research”, ACM Transactions on Computer-Human Interaction (Tochi), Vol. 7 No. 2, pp. 174-196, doi: 10.1145/353485.353487.
Jarrahi, M.H., Philips, G., Sutherland, W., Sawyer, S. and Erickson, I. (2019), “Personalization of knowledge, personal knowledge ecology, and digital nomadism”, Journal of the Association for Information Science and Technology, Vol. 70 No. 4, pp. 313-324, doi: 10.1002/asi.24134.
Jarrahi, M.H. and Sawyer, S. (2015), “Theorizing on the take-up of social technologies, organizational policies and norms, and consultants’ knowledge-sharing practices”, Journal of the Association for Information Science and Technology, Vol. 66 No. 1, pp. 162-179, doi: 10.1002/asi.23161.
Joynes, V., Kerr, M. and Treasure-Jones, T. (2017), “Exploring informal workplace learning in primary healthcare for continuous professional development”, Education for Primary Care, Vol. 28 No. 4, pp. 216-222, doi: 10.1080/14739879.2017.1298405.
Kali, Y. (2006), “Collaborative knowledge building using the design principles database”, International Journal of Computer-Supported Collaborative Learning, Vol. 1 No. 2, pp. 187-201, doi: 10.1007/s11412-006-8993-x.
Kuutti, K. (1996). “Activity theory as a potential framework for human-computer interaction research”, in Nardi, B. A. (Ed.), Context and Consciousness: Activity Theory and Human-Computer Interaction, MIT Press, Massachusetts, pp. 17-44.
Kvan, T. (2000), “Collaborative design: what is it?”, Automation in Construction, Vol. 9 No. 4, pp. 409-415, doi: 10.1016/s0926-5805(99)00025-4.
Lave, J. and Wenger, E. (1991), “Situated learning: legitimate peripheral participation”, Pea, R. and Brown, J.S. (Eds), Learning in Doing, Cambridge University Press (Learning in doing), 10.2307/2804509.
Leinonen, T., Toikkanen, T. and Silfvast, K. (2008), “‘Software as hypothesis: research-based design methodology”, Proceedings of the Tenth Anniversary Conference on Participatory Design 2008, pp. 61-70.
Lewis, C. (1982), “Using the ‘thinking aloud’ method in cognitive interface design”, IBM Research Report, RC-9265, doi: 10.19173/irrodl.v17i3.2379.
Ley, T., Kump, B., Maas, A., Maiden, N., Albert, D., et al. (2009), “‘Evaluating the adaptation of a learning system before the prototype is ready: a paper-based lab Study’”, in Houben, G.J. (Eds) Proceedings of the User Modelling, Adaptation, and Personalization 17th International Conference, Springer, Trento, Italy: Heidelberg, pp. 331-336, doi: 10.1007/978-3-642-02247-0_32.
Ley, T., Cook, J., Dennerlein, S., Kravcik, M., Kunzmann, C., Pata, K., Purma, J., Sandars, J., Santos, P., Schmidt, A. and Al‐Smadi, M. (2014), “Scaling informal learning at the workplace: a model and four designs from a large-scale design-based research effort”, British Journal of Educational Technology, Vol. 45 No. 6, pp. 1036-1048, doi: 10.1111/bjet.12197.
Littlejohn, A., Milligan, C. and Margaryan, A. (2012), “Charting collective knowledge: supporting self-regulated learning in the workplace”, Journal of Workplace Learning, Vol. 24 No. 3, pp. 226-238, doi: 10.1108/13665621211209285.
Lonchamp, J. (2012), “An instrumental perspective on CSCL systems”, International Journal of Computer-Supported Collaborative Learning, Vol. 7 No. 2, pp. 211-237, doi: 10.1007/s11412-012-9141-4.
Maier, J.R.A. and Fadel, G.M. (2009), “Affordance based design: a relational theory for design”, Research in Engineering Design, Vol. 20 No. 1, pp. 13-27, doi: 10.1007/s00163-008-0060-3.
March, S.T. and Smith, G.F. (1995), “Design and natural science research on information technology”, Decision Support Systems, Vol. 15 No. 4, pp. 251-266, doi: 10.1016/0167-9236(94)00041-2.
Mayring, P. (2000), “Qualitative content analysis”, Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, Vol. 1 No. 2, doi: 10.3196/18642950085445190.
McGrenere, J. and Ho, W. (2000), “Affordances: Clarifying and evolving a concept”, Proceedings of the Graphics Interface 2000 Conference, Montréal, Québec, Canada, pp. 179-186, 10.20380/GI2000.24.
Mor, Y., Warburton, S. and Winters, N. (2012), “Participatory pattern workshops: a methodology for open learning design inquiry”, Research in Learning Technology, Vol. 20 No. sup1, doi: 10.3402/rlt.v20i0.19197.
Nonaka, I. (1991), “The knowledge creating company”, Harvard Business Review, Vol. 69 No. 6, pp. 96-104, doi: 10.1016/b978-0-7506-7009-8.50016-1.
Norman (1988), The Psychology of Everyday Things, Basic Books.
Paavola, S. and Hakkarainen, K. (2014), “‘Trialogical approach for knowledge Creation’”, in Tan, S.C., So, H.J. and Yeo, J. (Eds) Knowledge Creation in Education, Springer, Singapore, pp. 1-20, doi: 10.1007/978-981-287-047-6_4.
Pejoska-Laajola, J., Reponen, S., Virnes, M. and Leinonen, T. (2017), “Mobile augmented communication for remote collaboration in a physical work context”, Australasian Journal of Educational Technology, Vol. 33 No. 6, doi: 10.14742/ajet.3622.
Pirolli, P. and Card, S. (2005), “The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis”, Proceedings of International Conference on Intelligence Analysis. McLean, VA, pp. 2-4.
Rabardel, P. and Bourmaud, G. (2003), “From computer to instrument system: a developmental perspective”, Interacting with Computers, Vol. 15 No. 5, doi: 10.1016/S0953-5438(03)00058-4.
Renner, B., Kimmerle, J., Cavael, D., Ziegler, V., Reinmann, L. and Cress, U. (2014), “Web-based apps for reflection: a longitudinal study with hospital staff”, Journal of Medical Internet Research, Vol. 16 No. 3, doi: 10.2196/jmir.3040.
Robert, G., Cornwell, J., Locock, L., Purushotham, A., Sturmey, G. and Gager, M. (2015), “Patients and staff as codesigners of healthcare services”, British Medical Journal, Vol. 350 No. 1, doi: 10.1136/bmj.g7714.
Ruiz-Calleja, A., Prieto, L.P., Ley, T., Rodríguez-Triana, M.J. and Dennerlein, S. (2017), “Learning analytics for professional and workplace learning: a literature review”, Lecture Notes in Computer Science, pp. 164-178, doi: 10.1007/978-3-319-66610-5_13.
Russell, D.M., Stefik, M.J., Pirolli, P. and Card, S.K. (1993), “The cost structure of sensemaking”, Proceedings of the SIGCHI conference on Human factors in computing systems – CHI ‘93, ACM Press, New York, NY, pp. 269-276, doi: 10.1145/169059.169209.
Sandoval, W. (2014), “Conjecture mapping: an approach to systematic educational design research”, Journal of the Learning Sciences, Vol. 23 No. 1, pp. 18-36, doi: 10.1080/10508406.2013.778204.
Sandoval, W.A. and Reiser, B.J. (2004), “Explanation-driven inquiry: integrating conceptual and epistemic scaffolds for scientific inquiry”, Science Education, Vol. 88 No. 3, pp. 345-372, doi: 10.1002/sce.10130.
Seidel, S., Chandra Kruse, L., Székely, N., Gau, M. and Stieger, D. (2018), “Design principles for sensemaking support systems in environmental sustainability transformations”, European Journal of Information Systems, Vol. 27 No. 2, doi: 10.1057/s41303-017-0039-0.
Stahl, G. (2013), “Learning across levels”, International Journal of Computer-Supported Collaborative Learning, Vol. 8 No. 1, pp. 1-12, doi: 10.1007/s11412-013-9169-0.
Svihla, V. (2014), “Advances in design-based research”, Frontline Learning Research, Vol. 6 No. 2, pp. 35-45.
Thalmann, S., Borntrager, V., Treasure-Jones, T., Sandars, J., Maier, R., Widmann, K. and Kerr, M. (2013), “‘Designing scalable informal learning solutions with personas: a pilot study in the healthcare sector”, Proceedings of the 8th European Conference on Technology Enhanced Learning: Scaling up Learning for Sustained Impact, Springer International Publishing, Paphos, Cyprus, pp. 641-642, doi: 10.1007/978-3-642-40814-4_91.
Thorpe, M. and Gordon, J. (2012), “Online learning in the workplace: a hybrid model of participation in networked, professional learning”, Australasian Journal of Educational Technology, Vol. 28 No. 8, pp. 1267-1282, doi: 10.14742/ajet.763.
Treasure-Jones, T. (2017), “Healthcare teamwork in and between general practices”, EU-IP Learning Layers, available at: http://results.learning-layers.eu/impacts/c-08/ (accessed 2 August 2018).
Treasure-Jones, T., Sarigianni, C., Maier, R., Santos, P. and Dewey, R. (2019), “Scaffolded contributions, active meetings and scaled engagement: How technology shapes informal learning practices in healthcare SME networks”, Computers in Human Behavior, Vol. 95, pp. 1-13, doi: 10.1016/j.chb.2018.12.039.
van Arnhem, J.-P. (2013), “Unpacking evernote: Apps for Note-Taking and a repository for Note-Keeping”, The Charleston Advisor, Vol. 15 No. 1, pp. 55-57, doi: 10.5260/chara.15.1.55.
Van den Akker, J., et al. (2006), ‘“Introducing educational design Research’”, in Van den Akker, J. et al. (Eds) Educational Design Research, Taylor and Francis, New York, NY, pp. 3-7, doi: 10.1007/bf02504982.
Virnes, M., Thiele, J., Manhart, M. and Thalmann, S. (2017), ‘“Application scenarios of mobile learning in vocational training: a case study of ach so! In the construction sector”, EdMedia: World Conference on Educational Media and Technology. Association for the Advancement of Computing in Education (AACE), pp. 89-98.
Vygotsky, L.S. (1978), Mind in Society: The Development of Higher Psychological Processes, Harvard University Press, Cambridge.
Wang, F. and Hannafin, M.J. (2005), “Design-based research and technology-enhanced learning environments”, Educational Technology Research and Development, Vol. 53 No. 4, pp. 5-23, doi: 10.1007/bf02504682.
Wenger, E. (1998), “Communities of practice: Learning, meaning, and identity”, in Pea, R., Brown, J.S. and Hawkins, J., Learning in Doing, Cambridge University Press (Learning in doing), doi: 10.2277/0521663636.
Acknowledgements
The research has been co-funded by the European Union’s 7th Research Framework project Learning Layers (Grant agreement No. 318209; http://results.learning-layers.eu/), the European Union’s Horizon 2020 research and innovation programme project CEITER (Grant agreement No. 669074; http://ceiter.tlu.ee) and the Austrian COMET Program – Competence Centers for Excellent Technologies. The Know-Center is funded within the Austrian COMET Program under the auspices of the Austrian Federal Ministry of Digital and Economic Affairs, the Austrian Federal Ministry of Climate Action, Environment, Energy, Mobility, Innovation and Technology and by the State of Styria. All co-design interactions of the “Bits and Pieces” case study were subject to ethical application and approval through the University of Leeds, School of Medicine Research Ethics Committee (EDREC/12/009 and SoMREC/13/097) and the National Health Service Research Governance Process (B001_30_01_13_123029 and 001_07_08_14_161142).