Abstract
Purpose
Action design research (ADR) has become widely accepted as a prominent research method within information systems when managing design-oriented research projects. One purpose of the ADR method is to provide methodological guidance for the building of IT artefacts. However, several scholars have reported a lack of guidance of method support at the micro level. This article aims to complement the macro level of the ADR method by integrating prescriptive method support at the micro level.
Design/methodology/approach
A qualitative approach including direct content analysis. An empirical ADR project was analysed in order to identify method support that could be integrated into the ADR method.
Findings
Method support at the micro level was identified for all the stages of the ADR method. The method support consists of procedural support, guiding concepts, and various techniques for the documentation of project tasks stated in the ADR method.
Research limitations/implications
The contribution to theory consists of aspects concerning the integration of macro and micro levels: relationships between normative and prescriptive support, continuous focus shifts, and method completeness.
Practical implications
The contribution to practice consists of explicit suggestions for method support that could be integrated into the ADR method.
Originality/value
This study extends previously provided knowledge by offering empirical evidence concerning theoretical constructions consisting of explicit relationships between ADR tasks and integrated method support, and elaboration on the integration of macro and micro levels.
Keywords
Citation
Cronholm, S. and Göbel, H. (2022), "Action design research: integration of method support", International Journal of Managing Projects in Business, Vol. 15 No. 8, pp. 19-47. https://doi.org/10.1108/IJMPB-07-2021-0196
Publisher
:Emerald Publishing Limited
Copyright © 2022, Stefan Cronholm and Hannes Göbel
License
Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
1. Introduction
Design science research (DSR) has become established as a widely accepted research paradigm within the field of Information Systems (IS) (e.g. Gregor and Hevner, 2013; Vaishnavi and Kuechler, 2015; Baskerville, 2018) and in other fields such as data science (Mullarkey et al., 2019) and business administration (Dresch et al., 2015). One specific characteristic of DSR is the interest in socio-technical artefact development and artefact theorising (e.g. Hanseth and Lyytinen, 2016; Gregor and Hevner, 2013; Baskerville et al., 2018). Hanseth and Lyytinen (2016, p. 2) state that DSR is socio-technical “ […] because its design domain involves both technical and social elements and their relationships”. One purpose of DSR is to respond to the dual mission of making theoretical contributions and assisting in solving the problems of practitioners (e.g. Rosemann and Vessey, 2008; Sein et al., 2011).
The recognition of the DSR paradigm has created a need for useful DSR methods. According to the number of citations, one of the most popular DSR methods is action design research (ADR) (Sein et al., 2011). Sein et al. (2011, p. 40) state that “ADR is a research method for generating prescriptive design knowledge through building and evaluating ensemble IT artifacts in an organizational setting”. Furthermore, Sein et al. (2011, p. 53) state that the ADR method “[…] provides methodological guidance for IS researchers who study the design of ensemble artifacts”. Although several researchers report positive experiences from applying the ADR method (Gregor et al., 2014; Schuppan and Koehl, 2017; Cheng et al., 2018), there is room for improvements concerning guidance on the micro level. Keizer-Broers and de Reuver (2016a) assert that the ADR method leaves a lot of freedom to the researcher concerning which design methods to use. Collatto et al. (2017) state that the ADR method only stresses macro steps. Mullarkey and Hevner (2019) argue that there is a need to give better support to method users for how to structure the key decisions and activities required for a rigorous application of the ADR method. Sein and Rossi (2019, p. 21) respond to this statement by stating that “[…] we agree with some of their elaborations, such as unpacking the specific stages of ADR to make them more transparent and accessible”. In addition, Cronholm and Göbel (2019) report adequate support at the macro level but suggest the ADR method needs more detailed support for operationalisation in practice (i.e. the micro level).
Several studies assert that the combination of macro and micro levels is essential. Buckley et al. (2011) state there often exists a chasm between micro and macro and there is need to narrow this divide. Cronholm and Göbel (2019) state that method support on both the macro and micro levels is needed when translating the overall method strategy into operational actions. Several scholars have presented general arguments for combining macro and micro levels. Dopfer et al. (2004) emphasise the importance of focus on both macro and micro aspects during analysis. Moreover, they state that “The sum of micro is macro, and the decomposition of macro is micro” (p. 264). Uhrmacher et al. (2007, p. 871) state that specific " […] attributes and dynamics are described at micro level, whereas the macro level holds aggregated variables and functions that describe high level dynamics”. Furthermore, Bassin et al. (2013) state that when both macro and micro plans are coordinated, projects have the tools to meet quality goals, test budget targets, as well as schedule and time-to-market demands.
Based on these arguments, the problem addressed in this study reads: there is a lack of prescriptive ADR method support at the micro level. A balanced macro and micro-level support is essential because it can streamline methods with the purpose of being experienced as helpful. However, identifying method support at the micro level is often time consuming and usually requires a lot of effort from the project teams. Against this background, the aim of our paper is to complement the macro level of the ADR method by integrating prescriptive method support at the micro level. The objective of our paper is not to develop new method support. Instead, the purpose is to support efficient use of the ADR method by identifying existing forms of method support at the micro level, which could be integrated into the ADR method.
Our research question reads: What method support needs to be integrated in order to gain a better balance between macro and micro levels? This means that we are interested in adding prescriptive method support (how to do something) to the ADR method. In this study, prescriptive method support at the micro level is defined as procedural support, guiding concepts, or techniques for project documentation.
In order to respond to the research question, we have analysed an empirical ADR project. The ADR project was conducted within the field of IS. The objective of the ADR project was to develop socio-technical resources to support organisations seeking to improve services offered to customers. During the ADR project, method support at the micro level was integrated into the ADR method to support the objectives of the project. The ADR method regards the process of artefact development as a collaborative effort involving both practitioners and researchers (Sein et al., 2011). The practice in this study consists of ADR projects. This means that the contribution to practice addresses ADR projects involving both researchers and practitioners. The contribution to practice consists of concrete suggestions for method support at the micro level that could be integrated into the ADR method. We are, in particular, targeting novice users. The theoretical contribution consists of method knowledge concerning the need for balancing method support at the macro and micro levels in order to improve efficiency and usefulness in future ADR projects.
The ensuing section briefly presents the ADR method. After that, we discuss the concept of method followed by a description of the ADR project. Next, we will introduce the research method. Then, we provide a literature review concerning previous integrations of method support in ADR projects. After that, we present the analysis of the empirical ADR project. Next, we discuss of the findings in relation to the literature review. Then, we elaborate on the implications for practice and research. Finally, our conclusions will be drawn.
2. The ADR method
One purpose of the ADR method is to support research projects with the development of IT artefacts shaped by organisational contexts. Another purpose is to support the development of design principles. The ADR method draws on design research (DR) and action research (AR) (Sein et al., 2011). The underlying assumption is that and DR and AR will not suffice on their own. Sein et al. (2011, p. 39) state that current DR “ … are strong in their support of abstraction and invention, they consider organizational intervention to be secondary”. On the other hand, AR combines theory generation with researcher intervention to solve immediate organisational problems. Based on these characteristics, ADR has emerged as a cross-fertilisation between DR and AR.
The ADR method comprises four stages which are: Problem Formulation (identifying and conceptualising a research opportunity based on existing theories and technologies); Building, Intervention and Evaluation (realising the design of the artefact and articulating the design principles); Reflection and Learning (moves conceptually from building a solution for a particular instance to applying that learning to a broader class of problems); and Formalisation of Learning (the situated learning from an ADR project should be further developed into general solution concepts). Each stage involves principles and tasks (see Figure 1), which are further described in Section 7 (see also Sein et al. (2011) for a detailed description).
3. Method theory
In Section 1, we argued that the ADR method lacks method support on the micro level. The purpose of this section is to further strengthen our argument by comparing the ADR method to a model developed by Goldkuhl et al. (1997), which describes and explains fundamental method concepts. Goldkuhl et al. (1997) state that method theory involves several related method concepts (see Figure 2). One concept is perspective, which is defined as the conceptual and value bases of a method. A perspective involves values, principles and categories. Moreover, a perspective can be explicitly articulated in methods, or it can be inexplicit.
Another concept is framework, which is defined as a phase structure of the method. A framework provides possible method components to choose from. A framework can be regarded as an ordered phase structure of method components that can give information about why and what to do.
A method component is defined as a meaningful unit that links procedure, notation and concepts. Method components are often integrated with other method components but can be used separately and independently (Goldkuhl et al., 1997; Cronholm and Ågerfalk, 1999). Examples of method components are root cause analysis, data modelling, process modelling.
A procedure is defined as prescriptions for what actions to take and what questions to ask. A notation is defined as prescriptions for how answers to questions should be documented (e.g. diagrams, tables and text). Usually, the procedure and notation are tightly coupled to each other. The procedure involves some meta-concepts such as process, activity, information and object. Such general concepts are used when asking the questions, meaning they are parts of the prescribed procedure. They are also parts of the semantics of the notation. The concepts are the cement between procedure and notation; the overlapping parts of procedure and notation.
Co-operation forms are defined as the interaction of different people in research projects. This has to do with roles, responsibilities and division of work. It describes how people collaborate when performing method-guided work. Moreover, it describes who is asking and answering questions. Finally, collection forms describe how data will be gathered.
Based on these definitions and explanations of different method concepts, a comparison with the ADR method indicates that the resemblance is high. The ADR method explicitly denotes the socio-technical perspective as an overarching perspective. Moreover, the ADR method involves a framework that consists of an ordered phase structure, which involves four stages (problem formulation; building, intervention and evaluation; reflection and learning and formalisation of learning). Furthermore, the ADR method encompasses co-operation forms describing the collaboration between researchers and practitioners. This implies that the ADR method involves the perspectives of both practitioners and researchers.
The lower part of Figure 2 consists of method components. We interpret the concept of method component as similar to the concept of task which is used in the ADR method. The ADR method describes several tasks that should be conducted in different stages. These tasks and principles excellently describe what should be done and why something should be done. However, they do not provide explicit method support on the micro level that informs about how to do (procedure) and how to document something (notation).
One interpretation of the purpose of omitting support at the micro level is that the authors of the ADR method make it possible for ADR projects to choose method support on an individual basis. This freedom can be regarded as positive, but it can also be perceived as inefficient and constitutes a barrier for novice IS researchers or researchers unfamiliar with IS methods in general. In this study, method support at the micro level is defined as something that provides procedural support. This definition is similar to what Goldkuhl et al. (1997) define as method component.
4. The ADR project
This section aims to describe the ADR project analysed in order to identify method support integrated into the ADR method. The purpose of the ADR project was to (1) develop socio-technical resources (e.g. methods, models, algorithms and digital tools) that could help organisations to utilise data in order to develop their service offerings and (2) create design knowledge (design principles) for socio-technical resources supporting service development.
The ADR project followed the four stages of the ADR method over a period of three years. Moreover, it involved four researchers and nine organisations. The organisations consisted of service providers and their customers. The researchers consisted of two professors and two PhD students from the field of information systems. The line of business the organisations represented was the car industry, telecom, and IT. The roles of the participating practitioners were IT Quality Managers, Head of Architecture and Solutions, IT consultants, Manager of Consumer Services, Business Manager, CEO, IT Process Framework Manager and Manager Consumer Sales. There were frequent interactions between researchers and practitioners in the ADR project. Following the ADR method, the ADR project can also be characterised as iterative.
The ADR method provided excellent support at the macro level. However, the ADR project identified a need to integrate method support at the micro level for all the four stages of the ADR method when developing socio-technical resources. Identifying appropriate method support was experienced as time-consuming and required a lot of effort from the project team.
The ADR project comprised an exceptional opportunity to analyse the need for method support at the micro level. This extended research interest corresponds to the purpose of this paper. In this paper, the role of the ADR project was to serve as a case in order to generate knowledge concerning the integration of the macro and micro levels based on empirical evidence.
5. Research method
In order to identify prescriptive method support at the micro level, we have conducted a qualitative study. Qualitative research can be defined as the type of research that finds out about experiences and help us to understand what is important to people (Silverman, 2020). The reasons for conducting a qualitative study were that it supported: (1) the possibility to conduct an in-depth study (DiCicco-Bloom and Crabtree, 2006), (2) the analysis of complex situations (Silverman, 2020), and (3) the focus on qualitative questions such as “why”, “what” and “how” (Patton, 1990). The qualitative study included a single case as our analysis unit. Yin (2011) states that a case study is an empirical inquiry, which examines a contemporary phenomenon in its real-life context. Moreover, our qualitative study also included using the method qualitative content analysis (see Section 5.2).
The design of our research method was also based on the fact that the two of researchers acted in multiple roles: (1) participated in the ADR project in order to develop socio-technical resources that could help organisations to utilise data in order to develop their service offerings and (2) analysed the ADR project in order to create knowledge about method support on the micro level that could be integrated into the ADR method. The two researchers are also the authors of this paper and they participated in the ADR project from start to end. Moreover, the analysis of method support on the micro level commenced already at the beginning of the ADR project. This research design, which is not unusual in qualitative studies (Susman and Evered, 1978; Avison et al., 2001), meant that we analysed the ADR project simultaneously as we participated in the development of social-technical resources (see Figure 3).
The overarching research method involved three phases: (1) a literature review of existing attempts to integrate method support into the ADR method, (2) an analysis of an empirical ADR project where there was a need to integrate method support into the ADR method, and (3) a comparison of the literature review and the results from the analysis of the empirical ADR project. Each phase will be presented in detail below.
5.1 Phase 1 literature review
In order to identify existing knowledge concerning the integration of method support into the ADR method, we needed to apply a literature review strategy. We were inspired by the literature review method presented by Webster and Watson (2002). In the first step, “identifying the relevant literature”, we applied different keywords in order to find relevant articles. A search in the Scopus database using the simple search string “Action Design Research” resulted in more than 1 000 hits, which were not manageable to review. A review of leading journals included in the eight top IS journals using the same search string resulted in the opposite. We received less than 3o hits, and most of them were irrelevant with regard to the purpose of our study. The reviewed journals were European Journal of Information Systems (EJIS), Information Systems Journal (ISJ), Information Systems Research (ISR), Journal of Association of Information Systems (JAIS), Journal of Information Technology (JIT), Journal of Management Information Systems (JMIS), Journal of Strategic Information Systems (JSIS) and, Management Information Systems Quarterly (MISQ). Based on these unsatisfactory results, we decided to use backward reference searching (i.e. snowball sampling, e.g. Naderifar et al., 2017) by reviewing relevant papers cited in the identified articles in the leading IS Journals. In total, we analysed 48 papers. Out of these 48 papers, 14 were relevant to the purpose of this paper.
In the second step of the literature review, Webster and Watson (2002) recommend researchers to develop a concept-centric matrix. The purpose of the concept-centric matrix is to synthesise the literature. The concepts used to organise the matrix were the different types of method support identified in previously reported ADR projects.
5.2 Phase 2 analysis of an empirical ADR project
In order to identify method support on the micro level used in the ADR project, we decided to use qualitative content analysis (e.g. Hsieh and Shannon, 2005). One technique of qualitative content analysis is Direct Content Analysis (DCA). DCA is recommended when experiences of a phenomenon have been identified, but would benefit from further description. As shown in the literature review, some knowledge concerning the integration of method support can be further developed (see Section 6.1). We followed the DCA research process proposed by Seuring and Müller (2008), consisting of code selection, material collection and material evaluation.
Step 1 Code selection
The first step was to select analytical codes. According to Seuring and Müller (2008), the analytical codes are to be chosen before the analysis is carried out. We selected the ADR tasks as codes since they constitute the keys to what and how something should be carried out at the micro level. This meant that we had 18 codes in total (see Figure 1).
Step 2 Material collection
The purpose of the step “material collection” was to explore method support used on the micro level. We used the ADR tasks (the codes) as a lens to identify method support integrated into the ADR project. The base for identifying method support was project documentation involving design documentation, notes taken from workshops, meeting protocols, and videotapes. These materials were continuously produced during the three years the ADR project lasted.
First, we created a list of all method support used in the ADR project. Then, we created explicit relationships between the ADR tasks and the identified method support. This means that we constructed explicit relationships between what should be done (ADR tasks) and how something should be done (integrated method support in the ADR project). The creation of explicit relationships could mean that one ADR task was supported by zero, one or many method components. It also meant that the method support could be reusable in more than one ADR task.
Step 3 Material evaluation
In order to determine whether the integrated method support was successful or not, we evaluated the relationships between the ADR tasks and the integrated method support. Eisenhardt and Graebner (2007) state that theoretical constructions (the relationships between the ADR tasks and the integrated method support) should be evaluated by providing empirical evidence. In order to evaluate the relationships between the ADR tasks and the integrated method support, we searched in project documentation for empirical evidence concerning the value of the integrated method support (e.g. strengths and weaknesses). The predefined codes (the ADR tasks) were used as a lens to identify empirical statements. Furthermore, Eisenhardt and Graebner (2007) argue that a “[…] a separate table that summarises the evidence for each theoretical construct is a particularly effective way to present the case evidence”. We followed this advice when we developed and evaluated the theoretical constructions.
5.3 Phase 3 comparison of the literature review and the empirical ADR project
In order to identify how our study contributed with extended knowledge concerning method support at the micro level, we compared similarities and differences between the results of the analysis of the ADR project and the literature review. We used simple Venn diagrams (e.g. Chen and Boutros, 2011), which support set comparisons and visualises how two or more sets are overlapping. In our comparison, the first set consisted of method support identified in the ADR project, and the second set consisted of method support identified in the literature review (see Figure 4). The comparisons made it possible to identify (1) new method support used in the ADR project and not identified in the literature review (complement A), (2) method support used in both the ADR project and the literature review (intersection), and (3) method support found in the literature review but not used in the ADR project (complement B).
6. Phase 1 literature review
As mentioned in Section 5.1, this section follows the literature review method presented by Webster and Watson (2002). First, we present the identified literature related to the purpose of this paper. Second, we describe how we have structured the identified literature according to a concept-centric matrix. Finally, we summarise the literature review.
6.1 Identified relevant literature
The purpose of this section is to present existing knowledge concerning the integration of method support into the ADR method. Mullarkey and Hevner (2019) propose an elaborated process model for applying ADR by integrating concepts from the framework developed by Peffers et al. (2007). More specifically, Mullarkey and Hevner (2019, p. 6) suggest improvements that “better support users to structure the key decisions and activities necessary to rigorously apply ADR”. They have identified four distinct types of ADR cycles for diagnosis, design, implementation, and evolution of the growing artefact solution. The suggested process model also supports multiple entry points based on the current state of the problem environment and the goals of the ADR project.
Bub (2018) has developed a process model that has been integrated into the ADR method. One purpose of the process model is to ensure both practice-driven innovation as well as design science research. Bub (2018, p. 337) states that the proposed process model “ … presents a combination of Design Science Research processes with innovation processes that are characterised by stage-gate-orientation”. On the other hand, Sein et al. (2011) highlights the importance of concurrent building and evaluation and state that a critical characteristic of the ADR method is that “… evaluation is not a separate stage of the research process that follows building. In this, ADR differs from the stage-gate models proposed in prior work” (p. 43).
Haj-Bolouri et al. (2016) propose participation action design research (PADRE), which includes adopting principles and philosophy from participatory action research and participatory design. They argue that the ADR method can benefit from incorporating learning within and across all stages. Furthermore, the ADR method should include a learning nexus, which is a repository of knowledge that accumulates when the ADR stages are conducted. This means that the results from reflection and learning are documented iteratively into the learning nexus.
Keijzer-Broers and de Reuver (2016) have illustrated how agile methods could be integrated into the ADR method. The reason is that “… researchers often face severe constraints in terms of budget and time within the practical setting” (p. 68). Keijzer-Broers and de Reuver (2016) argue that agile methods will be efficient and stimulate a quick start of the design process when combined with UX design methods (Sy, 2007). Moreover, Keijzer-Broers et al. (2016) have used personas (Long, 2009), user stories (Cohn, 2004) and scenarios in order to evaluate the prototypes that are developed in the project.
Lüftenegger et al. (2017) have adopted the ADR method to guide close collaboration with the industry. The purpose of their study is to conceptualise a strategy based on service-dominant logic (SDL) (Vargo and Lusch, 2004) and to develop a tool that can support the translation of abstract SDL into actionable insights for practitioners. In order to fulfil this purpose, they have used a simplified version of the Delphi method. The Delphi method provides detailed principles for making design choices during the process that ensure a valid study (Okoli and Pawlowski, 2004). Furthermore, brainstorming sessions (e.g. Wilson, 2013) involving both researchers and practitioners were used in order to obtain consensus concerning the mapping of SDL concepts onto concepts familiar to the practitioners.
Schacht et al. (2015) have applied the ADR method to design a knowledge management system “ … that integrates the social and technical perspective by expressing and evaluating design principles according [to] the design science research approach” (p. 5). Method support used to complement the ADR method consisted of an exploratory interview study, a general inductive approach for analysing evaluation data (Thomas, 2006), and focus groups that facilitated discussions and enabled the gathering of participants' attitudes, opinions, and beliefs (Myers, 2009).
Venable et al. (2016, p. 77) state that “ … extant DSR literature provides insufficient guidance on evaluation to enable Design Science Researchers to effectively design and incorporate evaluation activities into a DSR project that can achieve DSR goals and objectives”. This general statement includes the ADR method. The purpose of their study is to propose a framework for the evaluation of DSR (FEDS). In particular, the purpose of the framework is to address the questions: “What strategies and methods should be used for evaluation in a particular DSR project?” and “How should such evaluations be designed and conducted as part of a DSR project?”
Ebel et al. (2016) have applied the ADR method in order to present a framework for developing tool support for the design and management of new business models. The literature review was conducted by following a multistep process (Zott et al., 2011) and by means of qualitative content analysis (Forman and Damschroder, 2008). In order to generate data, semi-structured expert interviews were conducted (DiCicco-Bloom and Crabtree, 2006). Moreover, they have used Intra-Class-Correlation (ICC) coefficients (Amabile, 1996) to check the inter-rater reliability of the analysed interviews. In order to evaluate the artefact's usability, the authors used the Questionnaire for User Interaction Satisfaction (QUIS) (Chin et al., 1988). Finally, the utility of the framework was confirmed by using exploratory focus groups (Hevner and Chatterjee, 2010).
Spagnoletti et al. (2015) have presented results from an ADR project aimed to generate and orchestrate personalised elderly care interventions from a geriatric unit. They have used the ADR method as the overarching methodological framework. Moreover, the authors state that “The ADR framework allows experimenting with localised methods for building an ensemble artefact, including situated interventions and the evaluation of outcomes through multiple iterations” (p. 132). Data were generated from direct observations and focus groups involving domain experts, researchers and potential users. A questionnaire was used to assess the elderly participants' degree of social integration.
Göbel and Cronholm (2016) present intermediate results from evaluating a digital service platform and nascent design principles enabling researchers and practitioners to leverage other instances of digital service platforms. In order to validate the platform, semi-structured interviews and group interviews were conducted (Patton, 1990). In order to justify theory generation, four grounding strategies were applied: value grounding (a reference to an addressed goal), conceptual grounding (talking about the world/defining categories), explanatory grounding (justification for statements), and empirical grounding (in terms of instantiation and evaluation) (Goldkuhl, 2004).
Giessmann and Legner (2016) present design principles for the development of business models concerning cloud platforms. The ADR method was used for generating prescriptive design knowledge through building and evaluating IT artefacts in an organisational setting. The business model canvas (Osterwalder and Pigneur, 2010) was used to analyse current business models and develop a business model for cloud platforms. Moreover, an ideation workshop was used to generate ideas for improving the business model.
Mettler (2018) state that professional social networks (PSN) is not widely developed in complex domains such as health care. They have used the ADR method in order to describe practical design propositions and possible tension along the contextualisation of PSN. The ADR method was complemented with focus groups, workshops and surveys.
Gregor et al. (2014) report from an ADR project that analysed the limited adoption of e-Government in Bangladesh. The method support integrated into the ADR method is related to the seven principles of the ADR method (see Figure 1). Unfortunately, the method support is mainly described in general terms and lacks references to scientific articles. One exception is the integration support for theorising and generalisation. The method support incorporated during the ADR stage Formalisation of Learning was a theorising framework suggested by Lee et al. (2011), which provided an overarching guide in support of generalisation.
Henriques and O'Neill (2021) supplies a systematic approach to integrating focus groups into the ADR method. One important insight is that rigorous and committed stakeholder engagement is a critical success factor in complex projects. Other insights are that the usage of focus groups: (1) provide an efficient way to study artefacts, (2) propose improvements in its design, and (3) acknowledge the utility of those artefacts in real field use.
6.2 Structuring the review
The purpose of this section is to structure the results of the literature review through the use of a concept-centric matrix (see Table 1). The concept-centric matrix shows that there is a variety in the method support integrated into the ADR method.
6.3 Summarising the review
To summarise, through the literature review several promising attempts to integrate method support into the ADR method have been identified. All these attempts have provided valuable insights into our study. We can conclude that several scholars agree on the necessity to integrate method support into the ADR method. However, the method support identified in the literature review is:
Not evaluated. (The integration of method support was not the primary objective of the articles reviewed, and therefore there was no discussion about the successfulness of the integration).
Fragmented.
Often described in general terms or in passing.
Not related to specific ADR tasks.
Moreover, the literature review revealed no suggestions concerning method support for the ADR stage Reflection and Learning and only little method support for the ADR stage Formalisation of Learning. We can conclude that the literature review strengthened our belief that there is a need for increased method knowledge concerning the balance of method support at the macro and micro levels.
7. Phase 2 analysis of the empirical ADR project
The purpose of Section 7.1 is to (1) present the tasks that guided the development of the digital tool and the design principles developed in the ADR project, and (2) describe the method support integrated into the ADR project. The purpose of Section 7.2 is to evaluate the integrated method support. Section 7.2 can also be regarded as a summary of the integrated method support.
7.1 Integration of method support in the ADR project
This section is organised according to the four ADR stages (see Figure 1). The focus is set on the tasks for each stage since they guided the progress of the ADR project. Below, we describe how the ADR project approached the tasks. Due to lack of space, we have (1) omitted descriptions of integration of method support that was also identified in the literature review, (2) excluded tasks when no integration was made in the ADR project, and (3) excluded descriptions of the most common and well-known method support such as interviews and workshops. This omission resulted in that not every task is described below.
7.1.1 ADR stage 1: problem formulation
7.1.1.1 Task: identify and conceptualise the research opportunity
The research opportunity should be identified at the intersection of technological and organisational domains (Sein et al., 2011). In order to identify a research opportunity concerning service assessment and service innovation, the ADR project conducted traditional semi-structured interviews and workshops with the participating organisations. This resulted in several research opportunities consisting of problems and needs concerning service assessment and service innovation, such as digital tools, structured processes for service assessment, and support for collaboration between service providers and their customers. The ADR project considered this strength since it illuminated different aspects of the problems identified, which contributed to a deeper understanding concerning the problems the organisations are facing.
To identify and visualise relationships in terms of cause and effect between the identified problems, the ADR project applied root-cause analysis (Wilson et al., 1993). The main problem was formulated as the lack of a digital tool supporting collaborative service assessment and service innovation. Based on the problem formulation, the ADR project decided to develop a digital tool that could support collaboration between service providers and their customers.
7.1.1.2 Task: formulate initial research questions
The ADR method does not provide much guidance on the formulation of initial research questions. In order to learn more about formulations of research questions concerning design-oriented projects, the ADR project consulted Gregor and Hevner (2013), who state that research questions in DSR studies are always descriptive and prescriptive. This usually means that the knowledge contribution is to inform about what to do and how to do something, whilst research questions formulated in other scientific fields often lack the prescriptive dimension.
Gregor and Hevner (2013, p. 343) argue that “Research questions typically center on how to increase some measure of operational utility vis-à-vis new or improved design artifacts”. The support for formulating the research question consists of two analytical enquiries: “What do we know already?” and “From what existing knowledge can we draw?” (ibid.).
In the ADR project, the research question was continuously revised based on emerging empirical observations and theoretical insights. To gradually refine the research question also meant that the researchers in the ADR project kept an open mind during the research process. Furthermore, the ADR project also investigated whether other similar artefacts that have been used to solve the same or similar research problems in the past were in existence. This means that existing appropriate descriptive and propositional knowledge informed us when formulating the research question.
7.1.1.3 Task: cast the problem as an instance of a class of problems
Sein et al. (2011, p. 40) state that “ … the action design researcher should generate knowledge that can be applied to the class of problems that the specific problem exemplifies”. Moreover, Sein et al. (2011) describe this task as a conceptual move from the specific-and-unique to generic-and-abstract.
In the description of a class of problems, the ADR project was inspired by UML class diagrams (Object Management Group, 2020). A class is defined as a blueprint that is used to create an object. An object is referred to as an instance of a class. Another purpose of a class diagram is to support graphical descriptions of the relationship between a class of problems and the instances (Berardi et al., 2005). The identified problems in the ADR project were regarded as problem instances. These instances formed a basis for the identification of a class of problems. The class diagram was elaborated on in workshops that involved both researchers and practitioners.
7.1.1.4 Task: identify contributing theoretical bases and prior technology advances
Sein et al. (2011, p. 41) state that “The action design researcher should inscribe theoretical elements in the ensemble artifact”. In order to identify relevant theories, the ADR project followed the literature review method suggested by Webster and Watson (2002). This method included the steps: identifying relevant literature, structuring the review, and theoretical development. In order to synthesise the literature, the ADR project created a concept-centric matrix. The relevant theories that had a crucial impact on the design of digital tool were service-dominant logic (Vargo and Lusch, 2004), collaboration theory (Mathiassen, 2002), and digital innovation (Nambisan et al., 2017).
In order to identify prior technological advances such as similar digital tools, the ADR project conducted an empirical multidimensional market analysis (Day, 1981). This included procedural support such as identification of strengths and weaknesses relative to the competition, and exploration of and comparison with existing technologies.
7.1.2 ADR stage 2: building, intervention and evaluation
7.1.2.1 Task: discover initial knowledge-creation target
Sein et al. (2011) state that the shaping of the artefact requires interaction between technological and contextual dimensions and that the interaction is manifested in knowledge-creation targets.
In order to decide the overall knowledge-creation target, the ADR project was inspired by the DSR contributions types suggested by Gregor and Hevner (2013). These types involve (1) situated implementation of artefact (level 1), which includes software products or implemented processes, (2) nascent design theory (level 2), which includes constructs, methods, models, design principles, technology and rules, and (3) well-developed design theory (level 3), which could be mid-range or grand theories. As mentioned in Section 4, the ADR project developed a digital tool concerning support for service assessment (level 1) and design principles that support the development of digital tools of this type (level 2). The ADR project also formulated specific goals such as: facilitate feasible and viable service assessment and service innovation, support an improved dialogue between service providers and customers, and embed a modern service innovation and value co-creation culture.
7.1.2.2 Task: execute building-intervention-evaluation (BIE) cycle(s)
Sein et al. (2011, p. 42) state that “The outcome of the BIE stage is the realised design of the artifact”. Furthermore, the BIE cycle involves formulating general design principles that can be applied outside the studied context. Moreover, Sein et al. (2011) state that IT artefacts are ensembles shaped by the organisational context.
In order to find support for identifying contextual factors, the ADR project used the context framework presented by Rosemann et al. (2008). The framework holds procedural support enabling users to derive relevant contextual information presented as an onion model. The purpose of the onion model was to identify, classify, understand and integrate relevant contextual factors. Examples of contextual factors found in the ADR project were service orientation, resources and service ecosystems, which were essential to consider during the design of the digital tool. The building of the digital tool followed agile development (Schwaber, 1997). The ADR project implemented several inspection points to ensure that the artefact was designed according to the goals.
7.1.2.3 Task: assess need for additional cycles, repeat
Sein et al. (2011, p. 43) state that the ADR method conceptualises the research process as" … interwoven activities of building the IT artifact, intervening in the organisation, and evaluating it concurrently” (p. 37). Moreover, the ADR method includes the naturalistic and formative evaluation of ensemble IT artefacts in a specific context whilst searching for new design knowledge (ibid.).
The ADR project was inspired by naturalistic evaluation described in the framework for evaluating design-oriented projects (FEDS) (Venable et al., 2016). This meant that the digital tool was empirically evaluated within the environments of the organisations', and feedback was collected from empirical use. One specific method support integrated into the ADR project was that of evaluation episodes. An evaluation episode is defined as “specific evaluation activities of specific evaluands using a specific evaluation method” (Venable et al., 2016, p. 81).
A typical evaluation episode lasted for 2 h. Each episode included 1–2 service providers, 1–2 customers and 1–2 researchers. The process of the evaluation episode consisted of the following steps: (a) the service provider individually assessed different aspects of the digital tool without the involvement of the customer, (b) the customer individually assessed various aspects of the digital tool without the involvement of the service provider, (c) the service provider and the customer collaboratively analysed the individual assessments, and (d) the service provider and the customer collaboratively suggested improvements. Steps (a) and (b) were conducted in parallel. This evaluation process meant that the emergent design of the digital tool was heavily based on intervention, including the collection of contextual requirements from the participating organisations.
The ADR project carried out three iterations of the BIE cycle, which means that approximately 25 evaluation episodes were conducted with nine organisations. Each iteration resulted in a large amount of feedback. In order to structure the qualitative feedback, the ADR project used Grounded Theory (Strauss and Corbin, 1998) to group the feedback into categories consisting of similar requirements that constituted a basis for a redesign.
7.1.3 ADR stage 3: reflection and learning
7.1.3.1 Task: reflect on the design and redesign during the project
7.1.3.1.1 Task: evaluate adherence to principles
The reason for presenting the two tasks, “Reflect on the design and redesign during the project” and “Evaluate adherence to principles” together is that, in parallel, the ADR project developed the digital tool and design principles concerning the development of digital tools supporting service assessment and service innovation. There is also a tight coupling between these tasks. This meant that identified method support was used to assist both tasks.
One purpose of developing artefacts is to support organisations to fulfil their goals. One purpose of design principles is to support the transfer of design knowledge from one situation to another (Chandra Kruse and Seidel, 2017). The ADR project identified a mutual dependency between the emergence of the digital tool and the design principles. In order to manage the mutual dependency, the ADR project was inspired by the concept of reciprocity which is part of principle 3 (see Figure 1). With respect to principle 3, reciprocity concerns the interplay between the IT artefact and the organisational context. In this case, we utilised the concept of reciprocity to analyse the interplay between the development of the IT artefact and the design principles. The concept of reciprocity emanates from social psychology and can be understood as a relation of mutual dependence, action, influence, and a mutual or cooperative interchange of favours between two parts (Gouldner, 1960; Ben-Ari and Enosh, 2013). The emergence of the digital tool and the design principles mutually informed each other during the BIE cycles in the following way:
The development of the digital tool was guided by the design principles that emerged during the BIE cycles. That is, the advances of the design principles were used to shape the digital tool.
The development of the design principles was guided by empirical feedback from the use of the digital tool. That is, the digital tool provided a platform for the evaluation of the design principles.
In the ADR project, these two dependencies formed a base for two overarching reflections:
Reflection on the design principles: Were the design principles sufficiently articulated to provide the necessary support for designing digital tools supporting service assessment and service innovation?
Reflection on the digital tool: Was the feedback on the digital tool sufficient to provide adequate guidance on the development of the design principles?
In order to find method support concerning reflection, the ADR project consulted Daudelin (1996), who suggests a structured reflection process consisting of four distinctive stages: (1) articulation of a problem (i.e. lack of digital support concerning service assessment and service innovation), (2) analysis of that problem (i.e. analysis of feedback concerning the usage of the digital tool in relation to problem formulation and goal formulation), (3) formulation and testing of a tentative theory to explain the problem (i.e. feedback from usage of the digital tool guided search for theoretical support), and (4) action (or deciding whether to act) (i.e. decision to redesign the digital tool according to new theoretical insights and empirical feedback).
7.1.4 ADR stage 4: formalisation of learning
7.1.4.1 Task: articulate outcomes as design principles
Sein et al. (2011) state that generalisation is challenging because of the highly situated nature of ADR outcomes that include organisational change along with the implementation of an IT artefact. However, the ADR method explicitly defines three levels of generalisation: the problem instance, the solution instance, and the derivation of design principles. In Section 7.1.1, we have described the process for generalisation of the problem instance. In the same section, we also presented the method support multidimensional market analysis. This analysis helped us to identify a solution class, which were called digital tools supporting service assessment and service innovation. In the ADR project, the task “articulate outcomes as design principles” was divided into two sub-tasks: generalisation of design principles and formulation of design principles.
7.1.4.2 Sub-task generalisation of design principles
In order to generalise design principles, the ADR project was inspired by the well-cited article written by Lee and Baskerville (2003), which make a distinction between what the researcher is generalising from and to. Based on this distinction, the authors suggest four types of generalisation: generalising from data to description (type EE), generalising from description to theory (type ET), generalising from theory to description (type TE), and generalising from concepts to theory (type TT). With regard to generalisation of design principles, the ADR method involves generalisation from description (empirical evidence identified in organisational contexts) and further to theory and generalising from concepts to theory (theoretical concepts identified in kernel theories were included as elements of general design principles).
In order to generalise the design principles, the ADR project utilised the fact that there were nine organisations participating. This meant that the emergence of the design principles was based on input from nine organisational contexts, which could be regarded as nine instances or cases. Although, these organisations shared an interest in developing a digital tool supporting service assessment and service innovation, their input varied concerning: problem formulations, goal descriptions, requirement specifications, and feedback concerning the design of the digital tool. Yin (1994) states that at high degree of variation will serve to support generalisation.
Moreover, the ADR project utilised the concept of analytical generalisation, which has received attention and approval from a prominent interpretive IS researcher (Lee and Baskerville, 2003). Yin (1994) argues that the goal of analytical generalisation is to expand theories beyond their current domain. Analytical generalisation involves a reasoned judgement about the extent to which the findings from one study can be used as a guide to what might occur in another situation (Kvale, 2007). Halkier (2011) claims that analytical generalisation involves generalisation on the basis of qualitative data, which correspond to the type ET in Lee and Baskerville (2003) and, consequently, also corresponds with the ADR method.
7.1.4.3 Sub-task: formulation of design principles
Sein et al. (2011) state that the design principles identified through the stage Reflection and Learning “ … are fully formulated and articulated during this stage of formalising learning” (p. 45). The ADR method does not provide any guidelines on the formulation of design principles.
The ADR project identified that several scholars criticise existing design principles for variances in how they are formulated (e.g. Chandra Kruse and Seidel, 2017). Consequently, the ADR project searched for support concerning formulation of design principles and identified several suggestions (meta-design principles) such as: Walls et al. (1992), Van den Akker (1999), Van Aken (2004), and Chandra Kruse et al. (2016). In order to support reusability, design principles within the same set should be uniformed in terms of structure, content, and level of abstraction (Cronholm and Göbel, 2018).
The ADR project decided to follow the generic guideline suggested by Van den Akker (1999). This guideline reads: “If you want to design intervention X [for the purpose/function Y in context Z], then you are best advised to give that intervention the characteristics A, B, and C [substantive emphasis], and to do that via procedures K, L, and M [procedural emphasis], because of arguments P, Q, and R.”. The reason for choosing this guideline is that it provides support for how to formulate and structure design principles.
7.2 Evaluation of integrated method support
The output from Section 7.1 consisted of descriptions of the integrated method support and its relationship to the ADR tasks, which can be regarded as theoretical constructions (Eisenhardt and Graebner, 2007). The purpose of this section is to evaluate the theoretical construction. Column three in Tables 2–5 consists of empirical evidence concerning the value of the integrated method support. As mentioned in Section 5, the empirical evidence was identified in the project documentation.
In the ADR project documentation, we have identified several notes and statements that show that the integrated method support in the ADR project was valuable. Furthermore, the evaluation confirms that the integrated method support improved the utility of the ADR tasks, which in turn, contributed to the fulfilment of project goals in the ADR project. Based on these observations, we claim that the theoretical constructs consisting of the ADR task and the integrated method support form a whole that responds to the questions: why to, what to, when to, and how to do something.
8. Phase 3 comparison of the ADR project and the literature review
The purpose of this section is to illuminate the knowledge contribution of our study by comparing the literature review (see Section 6) and the method support integrated into the ADR project (see Section 7).
As mentioned in Section 6.3, the 14 articles included in our literature review were not explicit about what ADR tasks the integrated methods are supposed to support. In these cases, we interpreted general descriptions of the situations in which the method support was used. Furthermore, explicit references to the used method support were omitted in some articles. In these cases, we have added references that serve as a pointer to the method support used. In interpretive approaches such as text-based analysis of literature reviews, the analyst makes various decisions about how to comprehend the data (Walsham, 1995). Risks concerning biased interpretation can be reduced by involving two or more researchers when searching for and analysing data. Therefore, two researchers authored this paper: (1) individually analysed the articles and used the predefined codes (tasks) to interpret the relationship between the task and the identified method support and (2) in a following step, the output from the individual analyses was jointly compared and reconciled. This meant that we had two comparable outputs; the output from the interpretation of the articles included in the literature review and the output from the analysis of the ADR project.
To illustrate the comparison between the method support integrated into the ADR project and method support identified in the literature review, we have used Venn diagrams (see Figures 5–14).
Based on the comparison, we have identified both similarities and differences. The similarities exist in the intersections of the Venn diagrams and represent method support identified in both the ADR project and the literature review. This support involves methods that are frequently being used in research projects. An analysis of the differences in the Venn diagrams reveals the following observations:
Additional method support. We identified additional method support in the ADR project that was not identified in the literature review.
Coverage of the ADR stages. We identified method support in the ADR project represented in all four ADR stages. The literature mainly identified method support for the first two ADR stages.
Inclusion of method support for artefact design and development of design principles. The ADR project involved method support for building and evaluating the IT artefact and developing design principles. Method support identified in the literature review is mainly oriented towards the building and evaluating of the IT artefact.
Based on the analysis of the literature review and the documentation of the ADR project, we can also state that there are differences concerning:
Clarity. The ADR project is explicit about which method support had been integrated into specific ADR tasks. The studies in the literature review sometimes mention what method support was used in passing. Probably, this is because the studies identified in the literature had other objectives than suggesting method support that could be integrated into the ADR method.
Provision of explicit references. The ADR project provides explicit references to the method support as integrated. The studies identified in the literature review sometimes omit explicit references to the method support.
9. Implications for practice and research
This section aims to discuss implications for practice (Section 9.1) and research (Section 9.2). As mentioned in Section 1, the practice in our study consists of ADR projects which usually involve both researchers and practitioners (Sein et al., 2011).
9.1 Implications for practice
Based on the evaluation of the empirical ADR project, we highlight three significant implications for ADR project managers: operationalisation, understanding, and quality. First, we claim that the pinpointed micro support contributes to project managers regarding the operationalisation of ADR projects. For example, in the ADR project, the efficiency related to tasks such as “assessing the need for additional cycles” and “generalization of design principles” significantly improved when the micro support had been identified, described and refined in the subsequent project iterations. Consequently, the resources (e.g. time and money) needed for the project managers to plan and implement operative tasks were reduced.
Second, we state that the combination of macro and micro support helps project managers to foster a better understanding of the ADR method for participating organisations and novice researchers. For instance, the associated references to micro support helped the ADR project managers to argue for the necessity of conducting ADR tasks that are not common in traditional business projects (e.g. generalisation of design principles). Moreover, the combination of macro and micro support helped the ADR project managers to describe the ADR method as a whole to novice project members efficiently.
Third, we argue that the combination of macro and micro support helps project managers to leverage a higher quality of the results of ADR projects. For example, in the ADR project, the design principles were considered structured, generalised and easy to communicate due to the added micro support. Another example concerns micro support related to evaluation. The added micro support enabled the ADR project to deliver validated socio-technical resources to support organisations.
9.2 Implications for research
Based on the evaluation of the theoretical constructions created in Section 7.2, we argue that it is essential to integrate method support at macro and micro levels. Our argumentation involves three statements.
The combined macro and micro perspective:
Identified relationships between normative and prescriptive method support. Gregor (2006) states that methods informing about what to do are normative, whilst methods informing how to do something are prescriptive. We have created explicit theoretical constructions that balance the need for both normative and explicit method support. As mentioned in Section 1, there was a need to unpack the specific stages of the ADR method to make them more transparent and accessible. In the ADR project, the translation of the macro level to the micro level was necessary in order to find support for operational tasks.
Supported a continuous shift of focus between the whole and its parts. Our analysis of the ADR project identified similarities with the hermeneutic perspective, which focuses on the relationships between the whole (the ADR method) and its parts (e.g. the integrated method support), and argues that the whole and the parts need to harmonise (Gadamer, 1975). This harmonisation requires a dialectic process that shifts between the whole and the parts and back to the whole (ibid.). In the ADR project, a constant focus shift generated an enhanced understanding of (1) the problem analysed, (2) the design of digital tool supporting service assessment and service innovation, (3) the development of the design principles, and d) the relationship between the development of the digital tool and design principles. A focus that is too one-sided on the macro level could mean that necessary details were overlooked, disregarded or omitted (e.g. what should be included in this function in the digital tool?). Vice versa, a too narrow-minded focus on the micro level could imply that the presence of overarching issues at the macro level was lost (e.g. is this functionality needed at all?).
Ensured the completeness of the ADR method in action. The existing macro level in the ADR method supported general and high-level issues often related to what to, when to, and why to do something. The added micro level in this study supported, in particular, issues related to how to do something. Strauss and Corbin (1998) call these generic questions ‘analytical’ and state that they should preferably be asked within empirical and qualitative research projects. The added method support on the micro level corresponds to the method component element presented by Goldkuhl et al. (1997), (see Figure 2). In this respect, the integrated method support on the micro level has contributed to the completeness of the ADR method in action.
In the ADR project, these three statements have supported knowledge acquisition, efficiency and usefulness of the ADR method in action. This is valid for both problem formulations and suggested solutions regarding the digital tool and developed design principles. In addition, they have also supported the management of the ADR project in terms of project planning, project realisation, and project evaluation.
10. Conclusion
This paper aims to complement the macro level of the ADR method by suggesting integrated method support at the micro level. We can conclude that the suggested method support on the micro level integrated into the ADR project contributed with added value to all the ADR stages. This conclusion is based on the fact that we have identified method support that contributed to the success of the ADR project in all the ADR stages. We can also conclude that the interplay between the macro level in the ADR method and the suggested micro level in the ADR project harmonised and contributed to the fulfilment of the project goals. Therefore, we regard the suggested method support as a supplement to the ADR method.
The contribution to practice consists of concrete suggestions for integrating method support into the ADR method (see Section 7) and a discussion about implications (see Section 9.1). In order to support practical use, we have created explicit links between recommended tasks in the ADR method and the supplementary method support on the micro level. In particular, this supplement targets novice users, and hopefully, it will be considered efficient and effective in future ADR projects. The theoretical contribution consists of method knowledge concerning the need for balanced method support between the macro and micro levels, strengthening the ADR method concerning knowledge acquisition, efficiency and usefulness. The contribution consists of three statements concerning a combined macro and micro perspective: (1) the identification of relationships between normative and prescriptive method support, (2) a continuous shift of focus between the whole and its parts, and (3) the completeness of the ADR method in action (see Section 9.2).
The contributions to theory and practice are based on a review of existing literature resulting in a compilation of fragmented available knowledge and analysis of an empirical ADR project that developed new insights. Consequently, the knowledge created in this study is theoretically informed and empirically justified. We can conclude that our study extends existing knowledge concerning integrated method support at the micro level. The findings from this project are based on an analysis of one ADR project. Findings valid to a single case context are not necessarily valid to other contexts. Because single case studies do not allow for statistical generalisation, analytical generalisation is common in qualitative studies (Lee and Baskerville, 2003; Wieringa, 2014). The purpose of analytical generalisation is to support the transfer of findings from one context to other contexts with similar characteristics (ibid.). In order to provide support for reuse of the findings, we have: (1) formulated abstractions of the findings on a generic level (see Section 9.2) and (2) provided transparent support for method integration on the specific level (see Section 7).
The fact that the ADR project was conducted within the field of IS, the primary scope of our findings is IS. However, we cannot foresee any barriers against implementing the suggested method support in future ADR projects carried out within other fields such as data science or business administration, or other research projects aiming at building artefacts with methods lacking micro levels. The method support can be regarded as a toolbox, and selections can be made in accordance with contextual project needs. To further validate the integrated method support, we recommend forthcoming ADR projects to implement and evaluate its usefulness.
Figures
Concept-centric matrix concerning the identified method support
| Method support/articles | Process model | Participatory action research | UX design method | Delphi method | Brainstorming | Dynamic capability framework | Evaluation framework | Qualitative content analysis | Semi-structured interviews | Direct observations | Focus groups | Business model canvas | Workshop | Survey | Theorising framework |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| X | |||||||||||||||
| Bub (2018) | X | ||||||||||||||
| Haj-Bolouri et al. (2016) | X | ||||||||||||||
| Keijzer-Broers and de Reuver (2016) | X | ||||||||||||||
| Lüftenegger et al. (2017) | X | X | |||||||||||||
| X | |||||||||||||||
| X | |||||||||||||||
| Venable et al. (2016, p. 77) | X | ||||||||||||||
| Ebel et al. (2016) | X | X | |||||||||||||
| Spagnoletti et al. (2015) | X | X | |||||||||||||
| Göbel and Cronholm (2016) | X | X | X | ||||||||||||
| Giessmann and Legner (2016) | X | X | |||||||||||||
| Mettler (2018) | X | X | X | ||||||||||||
| Gregor et al. (2014) | X | ||||||||||||||
| Henriques and O'Neill (2021) | X |
Theoretical constructions for the stage Problem Formulation
| Theoretical construction | Evaluation of theoretical construction | |
|---|---|---|
| Task in the ADR method | Integrated method support in the ADR project | Empirical evidence |
| Identify and conceptualise the research opportunity | Root-cause analysis (Wilson et al., 1993) | Enabled researchers and practitioners to: (1) jointly identify a generic problem valid for all organisations (2) visualise relationships between of organisation-specific problems |
| Formulate initial research questions | Formulation of research question (Gregor and Hevner, 2013) | Supported the ADR project to formulate and reformulate the research question with respect to emerging insights gained from empirical data and theoretical insights. The suggested questions “What do we know already?” and “From what existing knowledge can we draw?” were found useful |
| Cast the problem as an instance of a class of problems | UML class diagrams (Object Management Group, 2020) | Increased the understanding of the relationships between the instance and the class of problems by illustrating how attributes of the instance were passed on to the class |
| Identify contributing theoretical bases and prior technology advances | Literature review method (Webster and Watson, 2002) | Supported the ADR project to conduct an organised literature review to identify kernel theories concerning what is already known and what we need to know |
| Identify contributing theoretical bases and prior technology advances | Multidimensional market analysis (Day, 1981) | Supported the ADR project to systematically identify and evaluate technology advances (digital tools) that belonged to the same class of solutions |
Theoretical constructions for the stage Building, Intervention and Evaluation
| Theoretical construction | Evaluation of theoretical construction | |
|---|---|---|
| Task in the ADR method | Integrated method support in the ADR project | Empirical evidence |
| Discover initial knowledge-creation target | DSR contribution types (Gregor and Hevner, 2013) | Helped the ADR project to identify and decide about different levels of design knowledge that the ADR project should contribute to. These levels are: (1) a situated implementation of an artefact, (2) a nascent design theory, and (3) a well-developed design theory |
| Execute building-intervention-evaluation (BIE) cycle(s) | The context framework (Rosemann et al., 2008) | The onion model and procedural support included in the context framework enabled the ADR project to identify organisation specific factors and characteristics improving the design and evaluation of the digital tool |
| Assess need for additional cycles, repeat | Framework for Evaluation of Design Science Research (Venable et al., 2016) | The ADR project experiences a support for: (1) systematic evaluation approach involving planning of specific evaluation cycles, which included several evaluation episodes (2) encouraged the formulation of generic and specific evaluation properties, which were essential for the understanding of when the artefact had fulfilled the goals |
| Grounded Theory (Strauss and Corbin, 1998) | Supported the ADR project to categorise the feedback collected from evaluating the digital tool during the interventions made in practice. The generated categories were useful when assessing the need for additional ADR cycles | |
Theoretical constructions for the stage Reflection and Learning
| Theoretical construction | Evaluation of theoretical construction | |
|---|---|---|
| Task in the ADR method | Integrated method support in the ADR project | Empirical evidence |
| Reflect on the design and redesign during the project Evaluate adherence to principles | Guidance concerning reciprocity (Gouldner, 1960) | Reinforced the understanding of how the emergence of the digital tool and design principles mutually informed each other during the BIE cycles |
| The reflection process (Daudelin (1996) | Provided a structured approach that guided the ADR project to discuss anticipated and unanticipated consequences of the design of the digital tool. The process helped both practitioners and researchers to share experiences efficiently in order to evaluate adherence to design principles as well as the refinement of the digital tool | |
Theoretical constructions for the stage Formalisation of Learning
| Theoretical construction | Evaluation of theoretical construction | |
|---|---|---|
| Task in the ADR method | Integrated method support in the ADR project | Empirical evidence |
| Generalisation of design principles | Conceptualising Generalisability (Lee and Baskerville, 2003) Analytical generalisation (Kvale, 2007) | Supported the ADR project to decide about an appropriate generalisation type. The type selected was from description, the empirical evidence identified in organisational contexts, to theory Researchers and practitioners argued that the method support ensured the quality of the design principles formulated |
| Formulation of design principles | Guidelines for formulation of design principles (Van den Akker, 1999) | Pinpointed essential elements that should be involved when formulating design principles, and therefore ensured that no element was overlooked by the ADR project. The formulation of the design principles was considered easy to communicate and understand by both researchers and practitioners |
References
Amabile, T.M. (1996), Creativity in Context: Update to “The Social Psychology of Creativity”, Westview Press, Boulder, CO.
Avison, D., Baskerville, R. and Myers, M. (2001), “Controlling action research projects”, Information Technology and People, Vol. 14 No. 1, pp. 28-45.
Baskerville, R.L. (2018), “The emergence of design science research from decision theory”, Scandinavian Journal of Information Systems, Vol. 30 No. 2(2), p. 8.
Baskerville, R., Baiyere, A., Gregor, S., Hevner, A. and Rossi, M. (2018), “Design science research contributions: finding a balance between artifact and theory”, Journal of the Association for Information Systems, Vol. 19 No. 5, pp. 358-376.
Bassin, K.A., Huang, S., Kagan, S., Li, S.C., Li, Z.J., Liu, H.H. and Zhu, J. (2013), U.S. Patent No. 8,539,438, U.S. Patent and Trademark Office, Washington, DC.
Ben-Ari, A. and Enosh, G. (2013), “Power relations and reciprocity: dialectics of knowledge construction”, Qualitative Health Research, Vol. 23 No. 3, pp. 422-429.
Berardi, D., Calvanese, D. and De Giacomo, G. (2005), “Reasoning on UML class diagrams”, Artificial Intelligence, Vol. 168 Nos 1-2, pp. 70-118.
Bub, U. (2018), “Towards an integrated method for the engineering of digital innovation and design science research”, in European Conference on Advances in Databases and Information Systems, Springer, Cham, pp. 327-338.
Buckley, M.R., Hamdani, M.R., Klotz, A.C. and Valcea, S. (2011), “Into the great wide open: bridging the micro–macro divide in the organizational sciences”, in Building Methodological Bridges, Emerald Group Publishing.
Chandra Kruse, L. and Seidel, S. (2017), “Tensions in design principle formulation and reuse”, in Designing the Digital Transformation: DESRIST 2017 Research in Progress Proceedings of the 12th International Conference on Design Science Research in Information Systems and Technology, Karlsruhe, Germany, 30 May–1 June.
Chandra Kruse, L., Seidel, S. and Purao, S. (2016), in Parsson, J., Tuunanen, T., Venable, J., Donellan, B., Helfert, M. and Kenneally, J. (Eds), “Making use of design principles”, DESRIST2016: Proceedings of the 11th International Conference on Design Science Research in Information Technology and Systems, Springer, pp. 37-51.
Chen, H. and Boutros, P.C. (2011). “VennDiagram: a package for the generation of highly-customizable Venn and Euler diagrams”, Bioinformatics, Vol. 12 No. 1, pp. 1-7.
Cheng, X., Fu, S., Huang, J. and de Vreede, G.J., (2018). “Can process facilitation improve globally distributed collaboration? An action design research”, in Bui, T. (Ed.), HICSS51: Proceedings of the 51st Hawaii International Conference on System Sciences, p. 51.
Chin, J.P., Diehl, V.A. and Norman, K.L. (1988), “Development of an instrument measuring user satisfaction of the human-computer interface”, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp. 213-218.
Cohn, M. (2004), User Stories Applied: For Agile Software Development, Addison-Wesley Professional, Boston.
Collatto, D.C., Dresch, A., Lacerda, D.P. and Bentz, I.G. (2017), “Is action design research indeed necessary? Analysis and synergies between action research and design science research”, Systemic Practice and Action Research, Vol. 31 No. 3, pp. 239-267.
Cronholm, S. and Ågerfalk, P. (1999), in Käkölä, T. (Ed.), “On the concept of method in information systems development”, in Proceedings of the 22nd Information Systems Research in Scandinavia (IRIS 22), Keuruu, Finland, Aug 7–10.
Cronholm, S. and Göbel, H. (2018), “Guidelines supporting the formulation of design principles”, 29th Australasian Conference on Information Systems (ACIS), Sydney, December 3–5, 2018.
Cronholm, S. and Göbel, H. (2019), “Evaluation of action design research”, Scandinavian Journal of Information Systems, Vol. 31 No. 2, pp. 35-82.
Daudelin, M.W. (1996), “Learning from experience through reflection”, Organizational Dynamics, Vol. 24 No. 3, pp. 36-48.
Day, G.S. (1981), “Strategic market analysis and definition: an integrated approach”, Strategic Management Journal, Vol. 2 No. 3, pp. 281-299.
DiCicco-Bloom, B. and Crabtree, B.F. (2006), “The qualitative research interview”, Medical Education, Vol. 40, pp. 314-321.
Dopfer, K., Foster, J. and Potts, J. (2004), “Micro-meso-macro”, Journal of Evolutionary Economics, Vol. 14 No. 3, pp. 263-279.
Dresch, A., Lacerda, D.P. and Antunes, J.A.V. (2015), “Design science research”, Design Science Research, Springer, Cham, pp. 67-102.
Ebel, P., Bretschneider, U. and Leimeister, J.M. (2016), “Leveraging virtual business model innovation: a framework for designing business model development tools”, Information Systems Journal, Vol. 26 No. 5, pp. 519-550.
Eisenhardt, K.M. and Graebner, M.E. (2007), “Theory building from cases: opportunities and challenges”, Academy of Management Journal, Vol. 50 No. 1, pp. 25-32.
Forman, J. and Damschroder, L. (2008), “Qualitative content analysis”, in Baker, R. and Wayne, S. (Eds), Empirical Research for Bioethics: A Primer, Elsevier Publishing, Oxford, UK.
Gadamer, H.-G. (1975), Truth and Method (Revised by Joel Weinsheimer and Donald Marshall. (Original German Version Published in 1960)), Trans. Vol. 2nd Edn, Continuum, New York.
Giessmann, A. and Legner, C. (2016), “Designing business models for cloud platforms”, Information Systems Journal, Vol. 26 No. 5, pp. 551-579.
Göbel, H. and Cronholm, S. (2016), in Parsson, J., Tuunanen, T., Venable, J., Donellan, B., Helfert, M. and Kenneally, J. (Eds), “Nascent design principles enabling digital service platforms”, in DESRIST2016: Proceedings of the 11th International Conference on Design Science Research in Information Technology and Systems, Springer, pp. 52-67.
Goldkuhl, G. (2004), “Design theories in information systems - a need for multi-grounding”, Journal of Information Technology and Theory Application (JITTA), Vol. 6, pp. 59-72.
Goldkuhl, G., Lind, M. and Seigerroth, U. (1997), in Jayaratna, N., Fitzgerald, B., Wood-Harper, T. and Larrasquet, J.-M. (Eds), “Method integration as a learning process”, in Proceedings of the Fifth International Conference of the British Computer Society Information Systems Methodologies Specialist Group, Springer-Verlag, pp. 15-26.
Gouldner, A. (1960), “The norm of reciprocity: a preliminary statement”, American Sociological Review, Vol. 25, pp. 161-178, doi: 10.2307/2092623.
Gregor, S. (2006), “The nature of theory in information systems”, MIS Quarterly, pp. 611-642.
Gregor, S. and Hevner, A.R. (2013), “Positioning and presenting design science research for maximum impact”, MIS Quarterly, Vol. 27 No. 2, pp. 337-355.
Gregor, S., Imran, A. and Turner, T. (2014), “A ‘sweet spot' change strategy for a least developed country: leveraging e-Government in Bangladesh”, European Journal of Information Systems, Vol. 23 No. 6, pp. 655-671.
Haj-Bolouri, A., Bernhardsson, L. and Rossi, M. (2016), in Parsson, J., Tuunanen, T., Venable, J., Donellan, B., Helfert, M. and Kenneally, J. (Eds), “PADRE: a method for participatory action design research”, in DESRIST2016: Proceedings of the 11th International Conference on Design Science Research in Information Technology and Systems, Springer, pp. 19-36.
Halkier, B. (2011), “Methodological practicalities in analytical generalization”, Qualitative Inquiry, Vol. 17 No. 9, pp. 787-797.
Hanseth, O. and Lyytinen, K. (2016), “Design theory for dynamic complexity in information infrastructures: the case of building internet”, Enacting Research Methods in Information Systems, Palgrave Macmillan, Cham, pp. 104-142.
Henriques, T.A. and O’Neill, H. (2021), “Design science research with focus groups–a pragmatic meta-model”, International Journal of Managing Projects in Business, Vol. ahead-of-print No. ahead-of-print, doi: 10.1108/IJMPB-01-2020-0015.
Hevner, A. and Chatterjee, S. (2010), “The use of focus groups in design science research”, in Sharda, R. and Voß, S. (Eds), Design Research in Information Systems: Theory and Practice, Springer Science+Business Media, New York, Dordrecht, Heidelberg, London.
Hsieh, H.F. and Shannon, S.E. (2005), “Three approaches to qualitative content analysis”, Qualitative Health Research, Vol. 15 No. 9, pp. 1277-1288.
Keijzer-Broers, W.J. and de Reuver, M. (2016), “Applying agile design sprint methods in action design research: prototyping a health and wellbeing platform”, International Conference on Design Science Research in Information System and Technology, Springer, Cham, pp. 68-80.
Keijzer-Broers, W., Florez-Atehortua, L. and de Reuver, M., (2016). “Prototyping a health and wellbeing platform: an action design research approach”, in Bui, T.X. and Prague Jr, R.H. (Eds), HICSS49, Proceedings of the 49th Hawaii International Conference on System Sciences, pp. 3462-3471.
Kvale, S. (2007), “Validation and generalization of interview knowledge”, Doing Interviews, pp. 121-129.
Lee, A. and Baskerville, R.L. (2003), “Generalizing generalizability in information systems research”, Information Systems Research, Vol. 14 No. 3, pp. 221-243.
Lee, J., Pries-Heje, J. and Baskerville, R. (2011), “Theorizing in design science research”, in Jain, H., Sinh, A. and Vitharana, P. (Eds), Lecture Notes in Computer Science, Springer, Milwaukee, MI, Service-oriented perspectives in design science research (6th DESRIST), pp. 1-16.
Long, F. (2009), “Real or imaginary: the effectiveness of using personas in product design”, Proceedings of the Irish Ergonomics Society Annual Conference, pp. 1-10.
Lüftenegger, E., Comuzzi, M. and Grefen, P.W. (2017), “Designing a tool for service-dominant strategies using action design research”, Service Business, Vol. 11 No. 1, pp. 161-189.
Mathiassen, L. (2002), “Collaborative practice research”, Information Technology and People, Vol. 15 No. 4, pp. 321-345.
Mettler, T. (2018), “Contextualizing a professional social network for health care: experiences from an action design research study”, Information Systems Journal, Vol. 28 No. 4, pp. 684-707.
Mullarkey, M.T. and Hevner, A.R. (2019), “An elaborated action design research process model”, European Journal of Information Systems, Vol. 28 No. 1, pp. 6-20.
Mullarkey, M.T., Hevner, A.R., Grandon Gill, T. and Dutta, K. (2019), “Citizen data scientist: a design science research method for the conduct of data science projects”, in International Conference on Design Science Research in Information Systems and Technology, Springer, Cham, pp. 191-205.
Myers, M.D. (2009), Qualitative Research in Business & Management, Sage, Thousand Oaks, CA.
Naderifar, M., Goli, H. and Ghaljaie, F. (2017), “Snowball sampling: a purposeful method of sampling in qualitative research”, Strides in Development of Medical Education, Vol. 14 No. 3, pp. 1-6.
Nambisan, S., Lyytinen, K., Majchrzak, A. and Song, M. (2017), “Digital Innovation Management: reinventing innovation management research in a digital world”, MIS Quarterly, Vol. 41 No. 1, pp. 223-238.
Object Management Group (2020), “Unified modelling language”, available at: https://www.omg.org/spec/UML (accessed 5 July 2021).
Okoli, C. and Pawlowski, S.D. (2004), “The Delphi method as a research tool: an example, design considerations and applications”, Information and Management, Vol. 42 No. 1, pp. 15-29.
Osterwalder, A. and Pigneur, Y. (2010), Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers, 1st ed., John Wiley & Sons, Amster-dam, Netherlands.
Patton, M.Q. (1990), Qualitative Evaluation and Research Methods, SAGE Publications, California.
Peffers, K., Tuunanen, R., Ma, R. and Chatterjee, S. (2007), “A design science research methodology for information systems research”, Journal of Management Information Systems, Vol. 24 No. 3, pp. 45-77.
Rosemann, M. and Vessey, I. (2008), “Toward improving the relevance of information systems research to practice: the role of applicability checks”, MIS Quarterly, Vol. 21 No. 1, pp. 1-22.
Rosemann, M., Recker, J.C. and Flender, C. (2008), “Contextualisation of business processes”, International Journal of Business Process Integration and Management, Vol. 3 No. 1, pp. 47-60.
Schacht, S., Morana, S. and Maedche, A. (2015), “The evolution of design principles enabling knowledge reuse for projects: an action design research project”, JITTA: Journal of Information Technology Theory and Application, Vol. 16 No. 3, p. 5.
Schuppan, T. and Koehl, S. (2017), in Bui, T. (Ed.), “One stop government: stalled vision or a matter of design? - empirical findings from social services in Germany”, in HICSS50: Proceedings of the 50th Annual Hawaii International Conference on System Sciences.
Schwaber, K. (1997), “Scrum development process”, Business Object Design and Implementation, Springer, London, pp. 117-134.
Sein, M.K. and Rossi, M. (2019), “Elaborating ADR while drifting away from its essence: a commentary on Mullarkey and Hevner”, European Journal of Information Systems, Vol. 28 No. 1, pp. 21-25.
Sein, M.K., Henfridsson, O., Purao, S., Rossi, M. and Lindgren, R. (2011), “Action design research”, MIS Quarterly, Vol. 35 No. 1, pp. 37-56.
Seuring, S. and Müller, M. (2008), “From a literature review to a conceptual framework for sustainable supply chain management”, Journal of Cleaner Production, Vol. 16 No. 15, pp. 1699-1710.
Silverman, D. (2020), Qualitative Research, Sage Publications, California.
Spagnoletti, P., Resca, A. and Sæbø, Ø. (2015), “Design for social media engagement: insights from elderly care assistance”, The Journal of Strategic Information Systems, Vol. 24 No. 2, pp. 128-145.
Strauss, A. and Corbin, J. (1998), Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Sage Publications, California.
Susman, G. and Evered, R. (1978), “An assessment of the scientific merits of action research”, Administrative Science Quarterly, Vol. 23 No. 4, pp. 582-603.
Sy, D. (2007), “Adapting usability investigations for agile user-centered design”, Journal of Usability Studies, Vol. 2 No. 3, pp. 112-132.
Thomas, D.R. (2006), “A general inductive approach for analyzing qualitative evaluation data”, American Journal of Evaluation, Vol. 27 No. 2, pp. 237-246.
Uhrmacher, A.M., Ewald, R., John, M., Maus, C., Jeschke, M. and Biermann, S. (2007), “Combining micro and macro-modeling in DEVS for computational biology”, in Winter Simulation Conference, IEEE, pp. 871-880.
Vaishnavi, V.K. and Kuechler, W. (2015), Design Science Research Methods and Patterns: Innovating Information and Communication Technology, CRC Press.
Van Aken, J. (2004), “Management research based on the paradigm of the design sciences: the quest for field-tested and grounded technological rules”, Journal of Management Studies, Vol. 41 No. 2, pp. 219-246.
Van den Akker, J. (1999), “Principles and methods of development research”, Design Approaches and Tools in Education and Training, Springer Science and Business Media, pp. 1-14.
Vargo, S.L. and Lusch, R.F. (2004), “Evolving to a new dominant logic for marketing”, Journal of Marketing, Vol. 68 No. 1, pp. 1-17.
Venable, J., Pries-Heje, J. and Baskerville, R. (2016), “FEDS: a framework for evaluation in design science research”, European Journal of Information Systems, Vol. 25 No. 1, pp. 77-89.
Walls, J.G., Widmeyer, G.R. and El Sawy, O.A. (1992), “Building an information system design theory for vigilant EIS”, Information Systems Research, Vol. 3 No. 1, pp. 36-59.
Walsham, G. (1995), “Interpretive case studies in IS research: nature and method”, European Journal of Information Systems, Vol. 4 No. 2, pp. 74-81.
Webster, J. and Watson, R.T. (2002), “Analyzing the past to prepare for the future: writing a literature review”, MIS Quarterly, Vol. 26 No. 2, pp. xiii-xxiii.
Wieringa, R.J. (2014), Design Science Methodology for Information Systems and Software Engineering, Springer.
Wilson, C. (2013), Brainstorming and Beyond: A User-Centered Design Method, Newnes, Oxford.
Wilson, P.F., Dell, L.D. and Anderson, G.F. (1993), Root Cause Analysis: A Tool for Total Quality Management, ASQ Quality Press.
Yin, R. (1994), Case Study Research: Design and Methods, 2nd ed., Sage Publications, Thousand Oaks, CA.
Yin, R.K. (2011), Applications of Case Study Research, Sage.
Zott, C., Amit, R. and Massa, L. (2011), “The business model: recent developments and future research”, Journal of Management, Vol. 37, pp. 1019-1042.