Abstract
Purpose
A recent paper by Morris et al. (2013b) presented evidence that students can develop entrepreneurial competencies through international fieldwork. This paper explores whether the same results can be developed in a traditional classroom setting.
Design/methodology/approach
The study is a systematic replication of the Morris study with the addition of a matched pair, quasi-experimental design, with a self-replication. Data were collected on 13 self-reported competencies at the start of a semester from two groups using the Morris instrument. The treatment group was exposed to a curriculum designed to teach entrepreneurial competencies, and both groups were re-surveyed at the end of the semester. The process was then repeated with a different cohort, one year later, to replicate the initial study.
Findings
Five competencies saw significant increases in the first treatment group. However, only three of these competencies increased more in the treatment group than the control group. In the replication study, only one competency was significantly higher in the treatment group, and that competency was not one of the original three.
Practical implications
Educators and policymakers should select a curriculum that is valid and reliable. Entrepreneurship educators and policymaker should devote more time to evaluating the effectiveness of different pedagogical techniques for improving entrepreneurial competencies.
Originality/value
To the best of our knowledge, this is one of the first studies in entrepreneurship education to undertake a matched pair, quasi-experimental design with an in-study replication. The results indicate that serious inferential errors arise if simpler designs are used, even though such designs are the norm in entrepreneurship research.
Keywords
Citation
Glackin, C.E.W. and Phelan, S.E. (2020), "Improving entrepreneurial competencies in the classroom: an extension and in-study replication", New England Journal of Entrepreneurship, Vol. 23 No. 2, pp. 79-96. https://doi.org/10.1108/NEJE-04-2020-0005
Publisher
:Emerald Publishing Limited
Copyright © 2020, Caroline E.W. Glackin and Steven E. Phelan
License
Published in New England Journal of Entrepreneurship. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) license. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this license may be seen at http://creativecommons.org/licences/by/4.0/legalcode
1. Introduction
In 2013, Michael Morris and his colleagues reported in the Journal of Small Business Management that 40 students reported a significant increase in 9 of 13 entrepreneurial competencies following a six-week international consulting project with disadvantaged entrepreneurs in South Africa (Morris et al., 2013b). At the time, the paper was described as a pilot study, but the results have assumed greater importance given the rise of competency-based education in the interim.
A competency refers to the “knowledge, skills, attitudes, values, and behaviors that people need to successfully perform a particular task or activity” (Morris et al., 2013a, p. 45). Competence, the state of being proficient in a competency, has been associated with higher performance and productivity. There is also a widespread belief that competence can be developed over time with training and practice (Morris et al., 2013a).
Competency-based education is defined as “an outcome-based approach to education that incorporates modes of instructional delivery and assessment efforts designed to evaluate mastery of learning by students through their demonstration of the knowledge, attitudes, values, skills, and behaviors required for the degree sought” (Gervais, 2016, p. 99). In recent years, there has been a renewed interest in competence-based education, with educational institutions like Western Governor's University and Southern New Hampshire University making the demonstration of competence via project-based learning the centerpiece of their pedagogical strategy (Adams, 2019). Enrollments at both schools regularly exceed 100,000 students.
Presumably, competence-based entrepreneurship education (CBEE) requires the acquisition of knowledge, skills, attitudes, values and behaviors which result in “… venture birth, survival, and/or growth” (Bird, 1995, p. 51). However, Morris et al. (2013a) have argued “there is little consensus in regarding the relative importance of particular competencies in an entrepreneurial context” (p. 46). The finding by Morris et al. (2013b) that a planned intervention can have a measurable impact on entrepreneurial competencies in a short period of time is therefore of great importance for entrepreneurship educators and competence-based education more generally.
This paper extends the seminal work of Morris et al. (2013b) by asking whether perceived entrepreneurial competencies can be improved in a semester-long classroom setting. Few students can afford to spend six weeks abroad on an international consulting project to improve their entrepreneurial competencies, so the question is of intrinsic interest to educators serving lower socio-economic populations. While this study is not a direct replication of the initial paper, it is a form of systematic replication using the same instrument. Systematic replication “establishes the generality of a phenomenon over a wide range of situations, [including] varied intervention procedures, participant characteristics, implementers, and settings” (Tincani and Travers, 2019, p. 61).
1.1 The original study
The original study (Morris et al., 2013b) was conducted in two phases. First, it engaged 10 experienced entrepreneurs and 10 entrepreneurship faculty in a Delphi study to identify a set of entrepreneurial competencies. Morris et al. were able to move from an initial list of 265 competencies to a consensus list of 13 competencies specific to entrepreneurship, namely: opportunity recognition, opportunity assessment, risk management/mitigation, conveying a compelling vision, tenacity/perseverance, creative problem solving/imaginativeness, resource leveraging, guerilla skills, value creation, maintain focus yet adapt, resilience, self-efficacy and building and using networks.
Once the list of entrepreneurial competencies was established, the authors sought measurement instruments for each competency from existing tested instruments that involved self-reporting. The authors created a 131-item instrument by selecting 112 items from 11 prior research instruments and adding 19 items of their own for risk management/mitigation (9), conveying a compelling vision (8) and guerilla skills (2). The instrument was designed for self-reporting and the items from prior research all had acceptable reliability.
Finally, the authors applied this instrument to a convenience sample of students engaged in an intensive international consulting experience to assess whether entrepreneurial competencies could be improved through experiential entrepreneurship education. The authors did not create specific hypotheses for each competency; simply being curious about whether any of the self-reported competencies could be improved during the experience. The student sample consisted of 40 mixed undergraduate and graduate students within business and non-business programs with 25 students from the United States and 15 students from South Africa. The entrepreneurship education experience included three months of online instruction and six weeks of half-day classroom instruction plus field work in teams consulting with South African micro entrepreneurs for six weeks. Entrepreneurial competencies were not taught explicitly. Students completed the self-reporting using the Morris Instrument before and after the experience.
Ultimately, Morris et al. found that self-reported entrepreneurial competencies did increase after the entrepreneurship education. Using 78 of the original 131 items in the final analysis, they found improvements in all 13 competencies with the differences in mean scores between pre- and post-test scores significant for 9 competencies. According to the authors, “These results lend initial support to the hypothesis that competencies can be enhanced based on exposure to an entrepreneurship program, in this case one that was centered both on the classroom and intense experiential learning. When placed in a context that centers on experiencing and performing real tasks that support entrepreneurial outcomes, students appear to get better at demonstrating specific competencies” (Morris et al., 2013b, p. 361).
Based on the results, the authors recommend increased emphasis on experiential education in entrepreneurship, including pedagogies such as entrepreneur interviews, developing business models or plans, feasibility analyses, and/or “elevator pitches, business plan presentations, or construct mock entrepreneurial tasks or simulations” (Morris et al., 2013b, p. 363). They further state, “[W]e believe our research presents a foundation for guiding the development of entrepreneurship curricula. Courses can be tailored to developing specific competencies within students…” (Morris et al., 2013b, p. 364). While Morris et al. seem to claim generalizability, they caution that “We do not expect that all experiential learning tasks will lead to development of all competencies” (Morris et al., 2013b, p. 365).
1.2 The current study
The impetus for the present research was to explicitly foster an understanding of and growth in entrepreneurial competencies among early academic career students by emphasizing the 13 core competencies identified and measured by Morris et al. (2013b). The Morris study was the most comprehensive model available and, as indicated above, provided insights into the types of experiential education that should be offered. The results were nothing short of remarkable, showing improvements in competencies for the relatively small sample of 40 students across grades, disciplines and nationalities, without direct instruction in the entrepreneurial competencies. This led to an interest in determining how and whether such results could be replicated in a more common collegiate environment, namely an on-campus, full semester undergraduate course.
We sought to enhance the methods of the original study through a systematic/conceptual replication rather than a reproduction, recognizing that there is a continuum of replication options (Walker et al., 2018). We retained the same measurement instrument as Morris et al. and the pre-test and post-test design. However, we were concerned that the small sample size in the original study made it difficult to discern small true effects. As a result, we opted to use matched pairs in our design, which gave us approximately 10 times the statistical power of the original study (Hollenbeck and Wright, 2017). This is akin to using a larger telescope to search for faint objects in the night sky (Simonsohn, 2015).
In addition, our sample selection was deliberate and well-defined. By selecting a group of students enrolled in an initial college entrepreneurship course within the university core curriculum we were able to control for heterogeneity and create a larger sample than in the original study. With little previous exposure to entrepreneurship education, these students were poised to make the largest gains in self-reported competencies. The opportunity to add a control group of students with similar attributes also added validity to our results by ensuring that any gains were not just due to the passage of time or the general college experience but specific to the intervention. Similarly, we replicated the original design twelve months later to ensure that the findings could be reliability reproduced.
It is interesting that Morris et al. did not posit any specific hypotheses about which individual competencies were expected to increase because of the consulting experience. We chose to not do so in our study either. While the allure of running many significance tests and HARKing (Hypothesizing After Results are Known) is strong per Hollenbeck and Wright (2017), we resist the development of post-hoc generated hypotheses to provide affirmative results.
This study follows the prescription regarding replication which suggests that, “Replication needs to become a normal scientific practice…to improve the quality of scholarship, accumulate knowledge, develop midrange theories, distill lessons for practice, and identify robust size effects and boundary conditions” (Walker et al., 2018, p. 623). With a high-powered systematic replication like ours, with interventions specifically targeting the 13 competencies, we fully expected to find similar results to Morris et al. (2013b). Any failure to find a reliable association provides a check on the generalizability of the Morris et al. results.
It is important for the field to report whether a provocative finding is supported or unsupported in subsequent tests. Doing so avoids the “file drawer” effect of publication bias, confirmation bias and impacts of meta-analysis by reporting the results are they arose without HARKing. Walker et al. (2018, p. 623) make a valid point when they state, “Replication failures are not necessarily failures of good scientific practice; rather, they should be seen as the scientific process in action.”
2. Literature review
Developmental and experiential pedagogies have long been the tools of entrepreneurship educators in their efforts to impact entrepreneurial activities (Albornoz, 2011). Yet, there have been ongoing changes to the pedagogical questions of what to teach and how to teach it as educators continue to learn more about how students acquire, retain and apply knowledge and skills and, consequently, which pedagogies are most effective in higher education (Neck and Greene, 2011; Rubin et al., 2018). To answer these questions, the goals and objectives of entrepreneurship education must be considered. Since its infancy, one clear purpose of entrepreneurship education has been to teach students how to establish new ventures (Rasmussen and Sorheim, 2006). Even with this singular goal, inquiries continue regarding the best practices in terms of organization and content (Neck and Greene, 2011; Ronstadt, 1987; Rubin et al., 2018). Neck and Green (2011) report that gaining consensus about the essential and most valuable learning outcomes from entrepreneurship education continues to be elusive, although fostering an entrepreneurial mindset (McGrath and MacMillan, 2000) and developing entrepreneurial competencies have generally emerged as leading outcomes.
One way of understanding what works is to use evidence-based decision-making, and to develop and test assessments of mastery through scientific methods (including replication). Evidence-based decision-making means that decisions should be based on the latest and best evidence of what works. Evidence-based techniques began in medicine but have since been extended to management (Pfeffer and Sutton, 2006) and education (Davies, 1999; Slavin, 2002). Applying this concept to entrepreneurship education means sorting through research findings and identifying what constitutes good evidence. However, such information continues to be lacking in the field, and research has indicated a gap between the startup activities of nascent entrepreneurs and what is taught in entrepreneurship textbooks (Edelman et al., 2008). One of the few long-term studies in this area found that the number of entrepreneurship courses taken did not correlate with the operational performance of new ventures and that writing a business plan had no effect on success beyond initial fund raising (Lange et al., 2012).
Over time, the emerging approaches to entrepreneurial education have evolved into a distinction between a competency-based approach and a personality-based approach. Kyndt and Baert (2015, p. 14) have argued that entrepreneurial competencies as defined by Bird (1995) are “changeable, learnable and attainable through experience, training, or coaching” and highlight differences between the competency-based approach and a personality-based approach that presumes characteristics are comparatively immutable.
One of the appeals of the competency-based approach is that it supports the proposition that entrepreneurship can be taught (Dickson et al., 2008; Fayolle et al., 2005; Kuratko, 2005). It also allows entrepreneurship educators to argue that entrepreneurial competencies (such as innovativeness, proactiveness and risk-taking) are distinct from general business competencies and thus can be taught outside a business school context (Chandler and Jansen, 1992; Morris and Kaplan, 2014; Rubin et al., 2018). It also addresses the implicit long chain of causal reasoning that links competency training with entrepreneurial success as depicted in Figure 1 (Glackin et al., 2018). Demonstrating that entrepreneurial competencies can be clearly defined, learned and applied to affect entrepreneurial outcomes is therefore vital to the successful adaptation of the competency-based approach to entrepreneurship.
2.1 Competency frameworks
Although there has been some gradual convergence, there is still a lack of consensus on the definition of a canonical collection of entrepreneurial competencies, and some argue that this lack of consensus constrains progress in entrepreneurship education (Mitchelmore and Rowley, 2010). Prior frameworks have also been criticized by Morris et al. (2013b) due to their focus on too few competencies (often only four to six items) although research has continued to emphasize one or a few competencies rather than a comprehensive framework (Gonzalez-Lopez et al., 2018; Lee et al., 2018; Oosterbeek et al., 2010; Rezaei-Zadeh et al., 2017; Sánchez, 2011).
The past seven years has seen a concerted effort to create entrepreneurial competency frameworks such as the European Commission's EntreComp Framework Academics have also created frameworks and tested them to varying degrees (Gonzalez-Lopez et al., 2018; Lee et al., 2018; Oosterbeek et al., 2010; Rezaei-Zadeh et al., 2017; Sánchez, 2011).
The recognized value of entrepreneurship to national economic outcomes has led some governments to create comprehensive frameworks. Examples include reports from the UK government on enterprise education (Davies, 2002). The European Commission's EntreComp framework (Bacigalupo et al., 2016), and the U.S. Department of Labor's Entrepreneurship Competency Model (Consortium for Entrepreneurship Education, 2009). The EntreComp framework includes an extensive roster of entrepreneurial competencies and related learning outcomes (15 competencies and 442 learning outcomes) leading to certifications for secondary school students. Recently, the Evaluation of Entrepreneurship Programs in Higher Education Institutions and Centres (EEEPHEIC), supported by the European Commission and led by Ecorys, has begun to develop a toolkit and methodology for measuring the value of entrepreneurship education with an emphasis on impact (see http://epic.ecorys.com).
This paper employs the entrepreneurial competence framework created by Morris et al. (2013b). There are a few reasons for this selection. First, the Morris framework is derived from a Delphi survey of successful entrepreneurship educators and entrepreneurs providing it some face validity. Second, it is one of the most inclusive frameworks of competencies and intersects with the EntreComp framework in at least nine areas. Most significantly, it specifies a measurement tool for each competence and has published data for comparative purposes. Table 1 contains a list of the thirteen competencies, definitions and sample questions.
2.2 Measuring competence
Entrepreneurship researchers have used several approaches to measure entrepreneurial competencies. Self-reporting is a prevalent method, but less than ideal. For instance, Van de Mortel (2008) suggests that social desirability response bias is problematic in self-report research. Entrepreneurial competencies have overwhelmingly been measured using Likert scales in the past, with more recent studies transitioning to behavioral measures (Kyndt and Baert, 2015; Schelfhout et al., 2016). While a direct behavioral demonstration of a competence is probably the most desirable form of measurement, this must be weighed against the cost and time required to collect such data, including the development of activities and scoring systems that capture the relevant facets of the underlying competence. For these reasons, most competence frameworks use Likert scales to measure changes in competence albeit with the clear recognition that they will not correlate as well with the desired end behaviors nor serve as the best measure for a given competence (Ajzen, 2005). The current study also adopts this convention. Table 2 illustrates several studies and their methods. Of the fourteen studies in the table (not including our own), fully 100% rely on self-reporting of competency, but only 35% utilized a pre/post design, and only 42% utilized a control group. None of the previous studies attempted to replicate their initial results.
The results of these studies should be taken within the context of their methods and the rigor of the studies. While several studies have been conducted on the acquisition of entrepreneurial competencies, few are methodologically rigorous. Rauch and Hulsink (2015, p. 188) support the work of Fiet (2001) and Weaver et al. (2006) in stating, “In light of methodological weaknesses inherent in entrepreneurship education studies, some authors concluded that there is little evidence as to its effectiveness.” Rauch and Hulsink recommend the use of quasi-experimental designs with pre- and post-tests, timed according to the nature of outcomes to be studied.
Köhler et al. (2017) take this notion a step further by suggesting that studies, should be designed to exclude alternate explanations for significant effects, including the use of repeated measures and randomized or quasi-randomized control groups. Unfortunately, fewer than 50% of studies they reviewed in the Academy of Management Learning and Education journal followed this practice and virtually no studies utilized control groups. Similarly, Longva and Foss (2018) performed a systematic literature review (SLR) of 613 articles from 65 Association of Business Schools (ABS) journals and found only 17 utilized an experimental or quasi-experimental design. They lamented that a growth in rigor has not matched the growth in significant results. Only research by Gielnik et al. (2015) and Karlsson and Moberg (2013) rise to the Longva and Foss rigor standards with respect to this topic.
Even reaching further, Simmons et al. (2011) suggest a disclosure-based solution to presenting anything as significant with guidelines and requirements for authors and reviewers, including disclosure of all variables considered, data collection procedures and decisions, failed manipulations, and replication. Minimizing false positives using in-study replications has been recommended (Hsu et al., 2017; Uncles and Kwok, 2013). We deliberately incorporate these recommendations by using a quasi-experimental design with matched pair, pre- and posttests and in-study replication. The following sections provide details regarding our methodology and findings.
2.3 Competencies in the classroom
Multiple factors intersected to suggest that there are core entrepreneurial competencies and that they can be taught with proper pedagogy. The question of whether entrepreneurship writ-large can be taught has been studied repeatedly. The studies that have examined entrepreneurial competencies (Gonzalez-Lopez et al., 2018; Lee et al., 2018; Morris et al., 2013b; Rezaei-Zadeh et al., 2017; Solesvik, 2019) demonstrate improvements in competencies, although generally through self-reporting and without control groups and replication (Oosterbeek et al., 2010). Interestingly, Oosterbeek et al. (2010, p. 442) report regarding a leading entrepreneurship education for college students, the “Program does not have the intended effects: the effect on students' self-assessed entrepreneurial skills is insignificant and the effect on the intention to become an entrepreneur is even negative.” In addition, with the assessment requirements from regional, national and international accrediting bodies, there is pressure to demonstrate successful student learning outcomes. Based on these factors, we expected a course structured around building entrepreneurial competencies to result in large effects.
As a university core course under the “life skills” category at our institution, Entrepreneurial Thinking was designed to foster the development and growth of core entrepreneurial competencies in students across the university. The course was an attractive site to explore competency development as at the time the course was developed, entrepreneurial competencies described by Morris et al. (2013b) appeared to suit this course. Entrepreneurial Thinking was a twice weekly, 16-weeks semester course with sufficient time to cover each topic adequately, to incorporate numerous experiential exercises, and to engage students in understanding the concepts. Students were to identify entrepreneurial competencies in public figures and reflecting on past and future application in their own lives. It seemed logical to teach core competencies in the earliest entrepreneurship course and build on them in subsequent courses. An overview of entrepreneurship and the Morris core competencies kicked off the course, followed by a weekly focus on the individual competencies combining mini-lectures, experiential exercises, videos, guest speakers and reflective writing. By the end of the semester, we expected students graded as proficient in the course to be able to articulate and explain competencies, apply them to outside parties and in their own lives.
3. Method
3.1 Design
This paper uses an innovative design for research in this area beyond the use of the Morris instrument. First, a matched pairs design is adopted. Previous studies have compared the mean of the treatment group before and after an intervention (Uncles and Kwok, 2013). A matched pairs design, on the other hand, compares the mean change in competency for each individual. This design increases statistical power by an order of magnitude therefore making the design more sensitive to changes in small samples. This approach addresses a frequent criticism that replications lack statistical power.
Second, we utilize a control group of students from a similar class (Financial Literacy – FINC100). Several others have used control groups (Gonzalez-Lopez et al., 2018; Hosseini and Pouratashi, 2011; Oosterbeek et al., 2010; Rauch and Hulsink, 2015; Sánchez, 2011; Solesvik, 2019). However, only a few studies use both pre- and posttests and control groups as we do (Gonzalez-Lopez et al., 2018; Oosterbeek et al., 2010; Rauch and Hulsink, 2015; Sánchez, 2011). Arguably, any change in the dependent variable might have occurred without the planned intervention (for instance, due to the Hawthorne effect, self-selection, test familiarity, or simply the passage of time). An effect that is experienced by the treatment group but not the control group mitigates against such claims. In a true experiment, students would be randomly assigned to the treatment and control groups. We do the next best thing by using another 100-level business class in the life skills section of the general education curriculum. The data in Table 2 shows that the demographics of the two classes are very similar.
Third, we conduct an in-study replication. If taking Entrepreneurial Thinking really improves competencies and the results are robust, then we should have been able to replicate the results from semester to semester with the same significant relationships in the replications (assuming significant statistical power). This approach is recommended by Hsu et al. (2017) as a best practice for re-confirming the validity of a given relationship. Our replication occurred exactly one year from the first study and used the same instructor to deliver the course to ensure students received a common experience.
3.2 Sample
All students taking the freshman courses Entrepreneurial Thinking (ENTR100) and Financial Literacy (FINC100) at a mid-sized Historically Black University in the Southeastern United States were invited to participate in the survey. In the first cohort, completed in Fall 2016, a total of 89 students took the instrument with 57 students taking the instrument twice (30 ENTR, 27 FINC). In the second cohort, administered in Fall 2017, a total of 77 students took the instrument at least once, with 53 students taking the instrument twice (30 ENTR, 23 FINC). Eligible students who did not take the survey were simply absent on the day of administration rather than refusing to participate in the study.
Demographic information was collected from the 110 matched surveys that were completed across the two cohorts. This information is presented in Table 3. The entrepreneurship students were more likely to be male lower division students but did not differ markedly in age, number of previous entrepreneurship courses, or race.
3.3 Instrument
The study utilized the Morris Competency Instrument (MCI) introduced by Morris et al. (2013b). The instrument contains 104 items on a five-point Likert scale ranging from Strongly Agree to Strongly Disagree or Frequently to Never. Descriptive statistics for the MCI are presented in Table 3. Our results indicate low reliabilities for 6 of the 13 scales in the 104-item version whereas Morris reported scores above 0.6 on all scales. Attempts to replicate the factor structure of the subscales in Morris et al. (2013b) were unsuccessful.
3.4 Procedure
The pre-treatment instrument was administered to each class section during the second week of classes and the post-treatment surveys were completed during the final week of classes. All students who were present on the day of the assessment and over age 18 were invited to participate in the study.
During the semester, students in the Entrepreneurial Thinking (ENTR 100) class received specific training in each of the 13 Morris competencies through a combination of lectures, videos, role playing, guest speakers and classroom exercises. There were no prerequisites for this freshman-level core curriculum life skills class. Students were assigned readings and videos from popular media rather than having a textbook. Each week, they learned about one of the competencies with at least one experiential activity per week to reinforce the readings, videos and lectures. Guest speakers provided “live case studies” for the students. The program was developed by the primary author based upon recommendations in the Morris et al. article, best practices gleaned from the entrepreneurship education literature, exercises introduced through the United States Association for Small Business and Entrepreneurship (USASBE), and in consultation with other entrepreneurship faculty at the university. Students were not required to engage in a consulting experience unlike Morris et al. (2013b). However, they invested considerable time in learning about each competency, viewing videos of and identifying people who exhibited the competency, having guest speakers and engaging in experiential exercises related to each competency. They also selected “Superhero” entrepreneurs and apply their understanding of competencies by identifying and articulating examples with their chosen entrepreneurs. Finally, students reflected on how they exhibited entrepreneurial competencies in their personal lives and how they could incorporate them into their future lives.
4. Results/findings
In our initial analysis, the dependent variables in Table 4 were entered into a multiple regression model with the demographic variables in Table 3 as controls. Table 5 depicts the base model with the average competency score for all items as the dependent variable using the data from all four collection periods. Students who had taken more than one entrepreneurship course rated themselves higher on entrepreneurial competence, but none of the other parameters were significantly different from zero in the base model, and the overall model was not significant (mean = 3.54, n = 268, R2 = 0.045, F = 1.75, p > 0.05).
The central hypothesis in the study was that the post-instruction competencies would be higher than the pre-instruction competencies for the entrepreneurship class but not the finance class. The change in competence for each class and cohort is presented in Table 6. In the first cohort, competencies increased in five areas and declined in one area in the entrepreneurship class, with an overall increase in average competency from the start of the semester to the end. However, the finance class also saw an increase in two areas and a decline in two areas. Overall, the predicted interaction effect occurred in three of the 13 competencies.
In the replication study, we would expect to see the same interaction patterns if the treatment is reliably increasing the identified competencies. However, the desired interaction effect was present for only one competency that was not one of the initial three significant interactions. Instead, both classes saw an increase in average competency. Opportunity recognition and creative problem solving saw significant gains in both entrepreneurship samples, but these measures also increased in the second control group.
In the final analysis, we created a repeated measures multivariate model using Student ID as a random effect, and then entering control variables and independent variables with separate terms for cohort, class, time (pre/post) and the interaction between class and time. The results of this analysis are presented in Table 7. The model overall had a good fit due to controlling for within-subject variance and using the full sample size rather than matched pairs (mean = 3.54, n = 268, R2 = 0.88), but no interaction between class and time was detected. There was, however, a significant change in scores over time, with post-semester scores being significantly higher for both classes. Similar results occurred with each competency entered as the dependent variable. However, two competencies (risk mitigation and creative problem solving) showed an interaction effect in the expected direction with the larger sample size.
5. Discussion
This study was motivated by the desire to extend the work of Morris et al. (2013b) and to explore whether we could improve entrepreneurial competencies in a college classroom. The principal hypothesis in this study was that the 13 entrepreneurial competencies measured by the Morris Competency Instrument (MCI) could be systematically raised over the course of a semester through classroom instruction. The results from the current sample show only very limited support for this hypothesis. To start, half of the sub-scales were not reliable, including resource leveraging and risk mitigation, both significant factors in some analyses.
The results basically indicated that we were unable to reliably increase any of the 13 competencies in the classroom. One explanation for this result is that these competencies cannot be developed in the classroom. Another explanation is that the wrong pedagogical techniques were used. The solution to both issues is to keep experimenting with pedagogical techniques targeted at improving individual competencies. When positive results are found then these results should be archived and made available to teachers of entrepreneurship, thus promoting evidence-based entrepreneurship education (Frese et al., 2012). Over time, it might then be possible to build a full semester long course that draws on this evidence-based material.
The failure to replicate the significant interaction effects in the first cohort also alerted us to the possibility that we were not seeing true gains in reported competencies, but this result would not have been detected with a typical one-shot design. This lack of replication is indicative of other findings in the social sciences, collectively referred to as the “replication crisis” (Open Science Collaboration, 2015). Small sample sizes often have low statistical power making it difficult to discern small true effect, and a large number of significance tests can lead to the phenomenon of “HARKing,” Hypothesizing After Results are Known (Hollenbeck and Wright, 2017). Ionnaidis (2005) has stated that most published research findings are false because of these practices.
On a positive note, perceived competence in creative problem solving and opportunity recognition increased significantly in both entrepreneurship cohorts. This suggests that the exercises around creative problem solving and opportunity recognition might be capable of producing reliable increases in perceived competence. In the case of creative problem solving, the exercise used was “connecting the dots” where students listed their passions, hobbies, and frustrations on index cards. The cards were then grouped in three piles and students randomly drew a passion, hobby and frustration and attempted to brainstorm a business from the combination. Students then reflected on the best and worst ideas (Glackin, 2016). The opportunity recognition section of the course used a similar exercise called Milliken's “exercise in ridiculosity” (Shields, 2015). This exercise follows a similar approach to the connected dots exercise by placing cards in three piles labeled resources, skills and situations. Students are then required to use the business model canvas to create a business with the combination selected and reflect on the results. We encourage entrepreneurship educators to consider incorporating these exercises into their courses to verify their effectiveness in raising perceived competencies in the designated areas.
5.1 Limitations and future research
One limitation of the study is that it is a quasi-experiment not a true experiment. Subjects were not randomly assigned to the treatment and control groups. This means there might be some unobserved heterogeneity between the groups that accounts for the observed increase in creative problem solving and opportunity recognition despite our finding that the groups did not differ on several demographic variables.
Another possible explanation for the pattern of results is that each wave did not receive the same treatment. This may have led to subtle changes in the curriculum that were manifested as different survey results. Similarly, the curriculum was designed by a third party so may not have been faithfully implemented by either instructor even if perfectly designed to elicit the desired results. It was also discovered that a monthly reflective assignment requirement was dropped from the second cohort, which may have resulted in a different experience. Controlling for treatment effects over a semester is difficult but could be mitigated through extensive training of the instructors prior to taking the course to ensure a consistent application of the curriculum.
There were also issues with the reliability of the measurement instrument. An examination of the Cronbach’s alphas in Table 3 suggests a low level of reliability (under 0.70) for six of the 13 scales, whereas Morris et al. (2013b) reported alphas over 0.70 for all scales. This means the items in the scale are not measuring the same underlying construct in our sample and further work is needed to see if this is a product of the context of our study or a broader issue with the inventory. Discussions with the instructor after the class also suggested that the large gain in “guerilla skills” may be at least partially attributable to an increased comprehension of the word “guerilla” as much as any increase in perceived competency. This suggests that the inventory may have to be reworked for different comprehension levels. Future work should also focus on using a matched pair design to maximize statistical power and use in-study replications to minimize false positives (Hsu et al., 2017; Uncles and Kwok, 2013).
5.2 Conclusion
One reason for studying entrepreneurial competencies is to determine the critical knowledge, skills, and abilities for undergraduate entrepreneurship students to acquire during their degree programs. This represents a significant challenge for entrepreneurship educators. Students in undergraduate degree programs have approximately 300 contact hours in entrepreneurship courses (and perhaps 1,000 h in total if you include outside study, internships and work experience). Even counting other business-related courses, contact hours will rarely reach more than 1,000 h in a degree program. This leads to two observations. First, entrepreneurship students will probably not master the field during their undergraduate education (Ericsson et al., 1993). Second, entrepreneurship educators do not have time to waste teaching the wrong content or using ineffective pedagogies given the limited hours available. This highlights the importance of conducting quality research that assesses student learning across various entrepreneurship education pedagogies and tools and yield effective guidance for instructors to use their limited engagement time most effectively.
It is both valuable and important to continue to replicate our efforts with other groups, in other settings, and over time. The ability to raise perceived competencies is an important step in entrepreneurial education but perceived competency is not actual competence, and there is still a requirement to demonstrate that actual competence affects entrepreneurial intention, entrepreneurial founding and, ultimately, the performance of a new venture (Gonzalez-Lopez et al., 2018). Research relating the development of entrepreneurial competencies to intention, found and performance is a vital future step. This work forms an important first step in creating a body of evidence-based knowledge that can guide entrepreneurship education into the future.
Figures
Morris entrepreneurial competency frameworks
| Competency | Definition from Morris et al. (2013b) | Sample question |
|---|---|---|
| Opportunity recognition | The capacity to perceive changed conditions or overlooked possibilities in the environment that represent potential sources of profit or return to a venture | I am good at “connecting dots” |
| Creative problem solving/imaginativeness | The ability to relate previously unrelated objects or variables to produce novel and appropriate or useful outcomes | I think outside of the box |
| Conveying a compelling vision | The ability to conceive an image of a future organizational state and to articulate that image in a manner that empowers followers to enact it | I believe in a bold and daring view of the future |
| Opportunity assessment | The ability to evaluate the content structure of opportunities to accurately determine their relative attractiveness | I can distinguish between profitable opportunities and not so profitable opportunities |
| Self-efficacy | The ability to maintain a sense of self-confidence regarding one's ability to accomplish a particular task or attain a level of performance | I can shape whatever environment in which I find myself operating |
| Tenacity/perseverance | The ability to sustain goal-directed action and energy when confronting difficulties and obstacles that impede goal achievement | I have achieved a goal that took years of work |
| Resource leveraging | Skills at accessing resources one does not necessarily own or control to accomplish personal ends | The need for resources can be solved without any costs, for example by using resources that others control |
| Value creation | Capabilities of developing new products, services, and/or business models that generate revenues exceeding their costs and produce sufficient user benefits to bring about a fair return | I have a continuous flow of new business ideas that come through observing the world |
| Risk management/mitigation | “the taking of actions that reduce the probability of a risk occurring or reduce the potential impact if the risk were to occur” | My skills in recognizing and assessing risks are strong |
| Building and using networks | Social interaction skills that enable an individual to establish, develop and maintain sets of relationships with others who assist them in advancing their work or career | Gone to lunch with persons who can help you professionally |
| Maintain focus yet adapt | The ability to balance an emphasis on goal achievement and the strategic direction of the organization while addressing the need to identify and pursue actions to improve the fit between an organization and developments in the external environment | I find it easy to modify or change my ideas about how something should be done |
| Guerrilla skills | The capacity to take advantage of one's surroundings, employ unconventional, low-cost tactics not recognized by others, and do more with less | I could quickly identify three guerrilla ideas to help any start-up venture |
| Resilience | The ability to cope with stresses and disturbances such that one remains well, recovers, or even thrives in the face of adversity | I actively look for ways to replace the losses I encounter in life |
Examination of entrepreneurship education and competencies studies
| Study | Number of competencies | Population | Self-assessment? | Pretest and posttest? | Control group | Replication? |
|---|---|---|---|---|---|---|
| Akhmetshin et al. (2019) | 13 | Undergraduates | Yes | No | No | No |
| Baum and Locke (2004) | 6 | Entrepreneurs | Yes | No | No | No |
| Ferreras-Garcia et al. (2019) | 12 | Undergraduates | Yes | No | No | No |
| Gonzalez-Lopez et al. (2018) | 1 | Undergraduates | Yes | Yes | Yes | No |
| Hosseini and Pouratashi (2011) | 10 | Undergraduates | Yes | No | Yes | No |
| Kyndt and Baert (2015) | 12 | Aspiring entrepreneurs | Yes | No | No | No |
| Lee et al. (2018) | 3 | Undergraduates | Yes | No | No | No |
| Mojab et al. (2011) | 10 | IT students and entrepreneurs | Yes | No | No | No |
| Morris et al. (2013b) | 13 | Graduate students | Yes | Yes | No | No |
| Osterbeek et al. (2010) | 6 | Undergraduates | Yes | Yes | Yes | No |
| Rauch and Hulsink (2015) | 4 | Graduate students | Yes | Yes | Yes | No |
| Sánchez (2011) | 3 | Undergraduates | Yes | Yes | Yes | No |
| Schelfhout et al. (2016) | 11 | Undergraduates | Yes | No | No | No |
| Solesvik (2019) | 15 | Undergraduates | Yes | No | Yes | No |
| Current study (2019) | 13 | Undergraduates | Yes | Yes | Yes | Yes |
Study demographics
| Class | FINC | ENTR | ||
|---|---|---|---|---|
| Cohort | First | Second | First | Second |
| Number of responses | 27 | 23 | 30 | 30 |
| Average age | 22.7 | 22.6 | 22.6 | 20.1 |
| Previous entrepreneurship courses | 1.0 | 0.7 | 1.0 | 1.2 |
| Division | ||||
| Lower (<60 credit hours)% | 66.67 | 69.57 | 90.00 | 96.67 |
| Upper (>60 credit hours)% | 33.33 | 30.43 | 10.00 | 3.33 |
| Gender | ||||
| Male% | 29.63 | 26.09 | 56.67 | 53.33 |
| Female% | 70.37 | 73.91 | 43.33 | 46.67 |
| Race | ||||
| African-American or black% | 88.89 | 82.61 | 83.33 | 90.00 |
| White/Caucasian% | 11.11 | 8.70 | 10.00 | 3.33 |
| Asian% | 0.00 | 0.00 | 3.33 | 0.00 |
| Native Hawaiian or Pacific Islander% | 0.00 | 4.35 | 0.00 | 3.33 |
| Other% | 0.00 | 4.35 | 3.33 | 3.33 |
Descriptive statistics for the dependent variables (n = 268)
| Variable | Items (reversed) | Mean | SD | Item–total correlation | Cronbach’s alpha |
|---|---|---|---|---|---|
| Opportunity recognition | 6 | 3.84 | 0.53 | 0.62 | 0.75 |
| Opportunity assessment | 5 | 3.77 | 0.59 | 0.60 | 0.73 |
| Risk management/mitigation | 5 (2) | 3.66 | 0.56 | 0.38 | 0.49 |
| Conveying a compelling vision | 6 (2) | 3.95 | 0.55 | 0.65 | 0.54 |
| Tenacity/perseverance | 14 (4) | 3.90 | 0.51 | 0.58 | 0.81 |
| Creative problem-solving | 7 | 3.94 | 0.58 | 0.73 | 0.84 |
| Resource leveraging/bootstrapping | 8 (3) | 2.97 | 0.33 | 0.17 | 0.11 |
| Guerrilla skills | 2 | 3.17 | 0.77 | 0.56 | 0.72 |
| Value creation | 15 | 3.62 | 0.62 | 0.67 | 0.89 |
| Ability to maintain focus yet adapt | 6 (3) | 3.17 | 0.54 | 0.33 | 0.58 |
| Resilience | 9 (3) | 3.74 | 0.48 | 0.52 | 0.57 |
| Self-efficacy | 4 (3) | 3.57 | 0.57 | 0.35 | 0.45 |
| Building and using networks | 17 | 2.72 | 0.92 | 0.55 | 0.90 |
| All items | 104 | 3.54 | 0.31 | 1.00 | 0.92 |
Base model
| Variable | Estimate | Std error | t Ratio |
|---|---|---|---|
| Intercept | 3.62 | 0.06 | 59.04* |
| Gender [male] | 0.02 | 0.02 | 0.89 |
| Race [African-American or Black] | −0.00 | 0.06 | −0.02 |
| Race [White/Caucasian] | −0.10 | 0.08 | −1.28 |
| Race [Asian] | −0.05 | 0.18 | −0.28 |
| Race [native Hawaiian or Pacific Islander] | 0.18 | 0.13 | 1.39 |
| Division [lower] | −0.05 | 0.02 | −1.96 |
| Courses [multiple] | 0.06 | 0.03 | 2.36* |
Mean competencies by cohort and class
| First cohort (means) | ENTR (n = 28) | FINC (n = 27) | Interaction | ||||
|---|---|---|---|---|---|---|---|
| Pre | Post | Diff | Pre | Post | Diff | ||
| Opportunity recognition | 3.65 | 3.88 | 0.23* | 3.87 | 3.81 | −0.06 | Yes |
| Opportunity assessment | 3.74 | 3.90 | 0.16* | 3.72 | 3.60 | −0.12 | Yes |
| Risk management/mitigation | 3.58 | 3.74 | 0.16 | 3.76 | 3.59 | −0.17* | |
| Conveying a compelling vision | 3.88 | 4.00 | 0.12 | 3.84 | 3.86 | 0.02 | |
| Tenacity/perseverance | 3.95 | 3.92 | −0.03 | 3.74 | 3.67 | −0.07 | |
| Creative problem solving | 3.76 | 4.00 | 0.24* | 3.86 | 3.80 | −0.06 | Yes |
| Resource leveraging/bootstrapping | 2.94 | 2.97 | 0.03 | 2.97 | 3.06 | 0.09 | |
| Guerrilla skills | 2.99 | 3.57 | 0.58* | 2.81 | 3.26 | 0.45* | |
| Value creation | 3.49 | 3.76 | 0.27* | 3.49 | 3.65 | 0.16* | |
| Ability to maintain focus yet adapt | 3.24 | 3.11 | −0.13 | 3.12 | 3.13 | 0.01 | |
| Resilience | 3.65 | 3.65 | 0.00 | 3.71 | 3.79 | 0.08 | |
| Self-efficacy | 3.73 | 3.50 | −0.23* | 3.65 | 3.38 | −0.27* | |
| Building and using networks | 2.66 | 2.79 | 0.13 | 2.63 | 2.80 | 0.17 | |
| All items | 3.48 | 3.60 | 0.12* | 3.48 | 3.49 | 0.01 | Yes |
| Second cohort (means) | ENTR (n = 30) | FINC (n = 23) | Interaction | ||||
|---|---|---|---|---|---|---|---|
| Pre | Post | Diff | Pre | Post | Diff | ||
| Opportunity recognition | 3.78 | 4.01 | 0.23* | 3.80 | 3.95 | 0.15 | |
| Opportunity assessment | 3.79 | 3.87 | 0.08 | 3.67 | 3.86 | 0.19* | |
| Risk management/mitigation | 3.65 | 3.82 | 0.17* | 3.57 | 3.60 | 0.03 | |
| Conveying a compelling vision | 3.80 | 3.94 | 0.14 | 4.04 | 4.27 | 0.23* | |
| Tenacity/perseverance | 3.76 | 3.74 | −0.02 | 4.07 | 4.13 | 0.06 | |
| Creative problem solving | 3.93 | 4.08 | 0.15* | 3.92 | 4.07 | 0.15 | |
| Resource leveraging/bootstrapping | 2.84 | 3.08 | 0.24* | 3.03 | 2.93 | −0.10 | Yes |
| Guerrilla skills | 3.07 | 3.60 | 0.53* | 2.67 | 2.96 | 0.29* | |
| Value creation | 3.68 | 3.78 | 0.10 | 3.29 | 3.66 | 0.37* | |
| Ability to maintain focus yet adapt | 3.08 | 3.10 | 0.02 | 3.24 | 3.37 | 0.13 | |
| Resilience | 3.84 | 3.76 | −0.08 | 3.82 | 3.91 | 0.09 | |
| Self-efficacy | 3.65 | 3.62 | −0.03 | 3.58 | 3.73 | 0.15 | |
| Building and using networks | 2.76 | 2.72 | −0.04 | 2.50 | 2.51 | 0.01 | |
| All Items | 3.51 | 3.62 | 0.11* | 3.49 | 3.61 | 0.12* | |
Note(s): *p < 0.05, one-tailed t-test for dependent means
Repeated measures model for average competency
| Term | Estimate | Std error | t Ratio |
|---|---|---|---|
| Intercept | 12.46 | 7.81 | 1.60 |
| Class [FINC] | −0.01 | 0.02 | −0.31 |
| Time [pre] | −0.04 | 0.01 | −3.80* |
| Class [FINC] × Time [pre] | 0.01 | 0.01 | 1.01 |
| Division [lower] | −0.02 | 0.03 | −0.75 |
| Courses [multiple] | 0.02 | 0.02 | 0.87 |
| Cohort [first] | −0.03 | 0.02 | −1.10 |
| Gender [male] | 0.02 | 0.02 | 0.78 |
| Birth year | −0.00 | 0.00 | −1.14 |
Note(s): *p < 0.05
References
Adams, S. (2019), Meet the English Professor Creating the Billion-Dollar College of the Future, Forbes, available at: https://www.forbes.com/sites/susanadams/2019/03/28/meet-the-english-professor-creating-the-billion-dollar-college-of-the-future/#6144c2e0426b.
Ajzen, I. (2005), Attitudes, Personality, and Behavior, Open University Press, McGraw-Hill Education, New York.
Akhmetshin, E.M., Larionova, G.N., Lukiyanchina, E., Savitskaya, Y. and Aleynikova, O. (2019), “The influence of educational environment on the development of entrepreneurial skills and competencies in students”, Journal of Entrepreneurship Education, Vol. 22, pp. 1-13.
Albornoz, C.A. (2011), Exploring the Goals, Content, and Methods of Entrepreneurship Professors: A Multiple Case Study, FIU Electronic Theses and Dissertations, (Doctor of Education) Florida International University, available at: http://digitalcommons.fiu.edu/etd/542.
Bacigalupo, M., Kampylis, P., Punie, Y. and Van den Brande, G. (2016), Entrecomp: The Entrepreneurship Competence Framework, (EUR 27939 EN), available at: https://ec.europa.eu/jrc/en/publication/eur-scientific-and-technical-research-reports/entrecomp-entrepreneurship-competence-framework.
Baum, J.R. and Locke, E.A. (2004), “The relationship of entrepreneurial traits, skill, and motivation to subsequent venture growth”, Journal of Applied Psychology, Vol. 89 No. 4, pp. 587-598.
Bird, B. (1995), “Towards a theory of entrepreneurial competency”, Advances in Entrepreneurship, Firm Emergence and Growth, Vol. 2, pp. 51-72.
Chandler, G.N. and Jansen, E. (1992), “The founder's self-assessed competence and venture performance”, Journal of Business Venturing, Vol. 7 No. 3, pp. 223-236.
Consortium for Entrepreneurship Education (2009), Introducing the Dol Entrepreneurship Competency Model, available at: https://www.careeronestop.org/competencymodel/Info_Documents/CEEcomp-model.pdf.
Davies, P. (1999), “What is evidence‐based education?”, British Journal of Educational Studies, Vol. 47 No. 2, pp. 108-121.
Davies, H. (2002), A Review of Enterprise and the Economy in Education, Her Majesty's Stationery Office, Norwich.
Dickson, P.H., Solomon, G.T. and Weaver, K.M. (2008), “Entrepreneurial selection and success: does education matter?”, Journal of Small Business and Enterprise Development, Vol. 15 No. 2, pp. 239-258.
Edelman, L.F., Manolova, T.S. and Brush, C.G. (2008), “Entrepreneurship education: correspondence between practices of nascent entrepreneurs and textbook prescriptions for success”, The Academy of Management Learning and Education, Vol. 7 No. 1, pp. 56-70.
Ericsson, K.A., Krampe, R.T. and Tesch-Römer, C. (1993), “The role of deliberate practice in the acquisition of expert performance”, Psychological Review, Vol. 100 No. 3, p. 363.
Fayolle, A., Gailly, B. and Lassas-Clerc, N. (2005), “Capturing variations in attitudes and intentions: a longitudinal study to assess the pedagogical effectiveness of entrepreneurship teaching programs”, Paper presented at the International Conference for Small Business (ICSB) World Conference, District of Columbia, Washington.
Ferreras-Garcia, R., Hernandez-Lara, A.B. and Serradell-Lopez, E. (2019), “Entrepreneurial competencies in a higher education business plan course”, Education + Training, Vol. 61 Nos 7-8, pp. 850-869.
Fiet, J.O. (2001), “The theoretical side of teaching entrepreneurship”, Journal of Business Venturing, Vol. 16 No. 1, pp. 1-24.
Frese, M., Bausch, A., Schmidt, P., Rauch, A. and Kabst, R. (2012), “Evidence-based entrepreneurship: cumulative science, action principles, and bridging the gap between science and practice”, Foundations and Trends in Entrepreneurship, Vol. 8 No. 1, pp. 1-62.
Gervais, J. (2016), “The operational definition of competency-based education”, The Journal of Competency-Based Education, Vol. 1 No. 2, pp. 98-106, doi: 10.1002/cbe2.1011.
Gielnik, M.M., Frese, M., Kahara-Kawuki, A., Wasswa Katono, I., Kyejjusa, S., Ngoma, M., Munene, J., Namatovu-Dawa, R., Nansubuga, F., Orobia, L., Oyugi, J., Sejjaaka, S., Sserwanga, A., Walter, T., Bischoff, K.M. and Dlugosch, T.J. (2015), “Action and action-regulation in entrepreneurship: evaluating a student training for promoting entrepreneurship”, The Academy of Management Learning and Education, Vol. 14 No. 1, pp. 69-94.
Glackin, C. (2016), “Connecting the dots: making sense from nonsense”, Paper presented at the Annual Conference of the United States Association for Small Business and Entrepreneurhip (USASBE), January, San Diego, California.
Glackin, C., Phelan, S.E. and McCrowre, R. (2018), “Improving entrepreneurial competencies in the classroom”, Proceedings of the 2018 Annual Meeting of the United States Association for Small Business and Entrepreneurship, January, Los Angeles, CA.
Gonzalez-Lopez, M.J., Perez-Lopez, M.C. and Rodríguez-Ariza, L. (2018), “Clearing the hurdles in the entrepreneurial race: the role of resilience in entrepreneurship education”, The Academy of Management Learning and Education, Vol. 18 No. 3, pp. 457-483, doi: 10.5465/amle.2016.0377.
Hollenbeck, J.R. and Wright, P.M. (2017), “Harking, sharking, and tharking”, Journal of Management, Vol. 43 No. 1, pp. 5-18, doi: 10.1177/0149206316679487.
Hosseini, S.M. and Pouratashi, M. (2011), “Entrepreneurial competencies of agricultural students: the influence of entrepreneurship courses”, African Journal of Business Management, Vol. 5 No. 6, pp. 2159-2163.
Hsu, D.K., Simmons, S.A. and Wieland, A.M. (2017), “Designing entrepreneurship experiments”, Organizational Research Methods, Vol. 20 No. 3, p. 379.
Ioannidis, J.P. (2005), “Why most published research findings are false”, PLoS Medicine, Vol. 2 No. 8, p. e124.
Karlsson, T. and Moberg, K. (2013), “Improving perceived entrepreneurial abilities through education: exploratory testing of an entrepreneurial self efficacy scale in a pre-post setting”, International Journal of Management in Education, Vol. 11 No. 1, pp. 1-11.
Koehler, T., Landis, R.S. and Cortina, J.M. (2017), “From the editors: establishing methodological rigor in quantitative management and education research”, The Academy of Management Learning and Education, Vol. 16 No. 2, pp. 173-192.
Kuratko, D.F. (2005), “The emergence of entrepreneurship education: development, trends, and challenges”, Entrepreneurship: Theory and Practice, Vol. 29 No. 5, pp. 577-597.
Kyndt, E. and Baert, H. (2015), “Entrepreneurial competencies: assessment and predictive value for entrepreneurship”, Journal of Vocational Behavior, Vol. 90, pp. 13-25.
Lange, J., Marram, E. and Bygrave, W. (2012), “Human assets and entrepreneurial performance: a study of companies started by business school graduates”, Journal of Business and Entrepreneurship, Vol. 24 No. 1, p. 1.
Lee, Y., Kreiser, P.M. and Wrede, A.H. (2018), “University-based education and the formation of entrepreneurial capabilities”, Entrepreneurship Education and Pedagogy, Vol. 1 No. 4, pp. 304-329.
Longva, K.K. and Foss, L. (2018), “Measuring impact through experimental design in entrepreneurship education: a literature review and research agenda”, Industry and Higher Education, Vol. 32 No. 6, pp. 358-374.
McGrath, R.G. and MacMillan, I.C. (2000), The Entrepreneurial Mindset: Strategies for Continuously Creating Opportunity in an Age of Uncertainty, Harvard Business Press, Cambridge, Mass, Vol. 284.
Mitchelmore, S. and Rowley, J. (2010), “Entrepreneurial competencies: a literature review and development agenda”, International Journal of Entrepreneurial Behavior and Research, Vol. 16 No. 2, pp. 92-111.
Mojab, F., Zaefarian, R. and Azizi, A.H.D. (2011), “Applying a competency based approach for entrepreneurship education”, Procedia Social and Behavioral Sciences, Vol. 12, pp. 436-447.
Morris, M.H. and Kaplan, J. (2014), “Entrepreneurial (versus managerial) competencies as drivers of entrepreneurship education”, in Morris, M.H. and Matthews, C. (Eds), Annals of Entrepreneurship Education and Pedgagogy - 2014, Edward Elgar Publishing, Cheltenham, pp. 134-157.
Morris, M.H., Kuratko, D.F. and Cornwall, J.R. (2013a), Entrepreneurship Programs and the Modern University, Edward Elgar Publishing, Cheltenham.
Morris, M.H., Webb, J.W., Fu, J. and Singhal, S. (2013b), “A competency-based perspective on entrepreneurship education: conceptual and empirical insights”, Journal of Small Business Management, Vol. 51 No. 3, pp. 352-369, doi: 10.1111/jsbm.12023.
Neck, H. and Greene, P. (2011), “Entrepreneurship education: known worlds and new frontiers”, Journal of Small Business Management, Vol. 49 No. 1, pp. 55-70.
Oosterbeek, H., Van Praag, M. and Ijsselstein, A. (2010), “The impact of entrepreneurship education on entrepreneurship skills and motivation”, European Economic Review, Vol. 54 No. 3, pp. 442-454.
Open Science Collaboration (2015), “Estimating the reproducibility of psychological science”, Science, Vol. 349 No. 6251, doi: 10.1126/science.aac4716.
Pfeffer, J. and Sutton, R.I. (2006), “Evidence-based management”, Harvard Business Review, Vol. 84 No. 1, pp. 63-74.
Rasmussen, E. and Sorheim, R. (2006), “Action-based entrepreneurship education”, Technovation, Vol. 26, pp. 185-194.
Rauch, A. and Hulsink, W. (2015), “Putting entrepreneurship education where the intention to act lies: an investigation into the impact of entrepreneurship education on entrepreneurial behavior”, The Academy of Management Learning and Education, Vol. 14 No. 2, pp. 187-204.
Rezaei-Zadeh, M., Hogan, M., O'Reilly, J., Cunningham, J. and Murphy, E. (2017), “Core entrepreneurial competencies and their interdependencies: insights from a study of Irish and iranian entrepreneurs, university students and academics”, The International Entrepreneurship and Management Journal, Vol. 13 No. 1, pp. 35-73.
Ronstadt, R. (1987), “The educated entrepreneurs: a new era of entrepreneurial education is beginning”, American Journal of Small Business, Vol. 11 No. 4, pp. 37-53.
Rubin, Y., Lednev, M. and Mozhzhukhin, D. (2018), “Entrepreneurship education in action: a matrix of competencies for a bachelor's degree program”, in Matthew, C. and Liguori, E. (Eds), Annals of Entrepreneurship - 2018, Edward Elgar Publishing, Northampton, Mass, pp. 187-202.
Sánchez, J.C. (2011), “University training for entrepreneurial competencies: its impact on intention of venture creation”, International Entrepreneurship and Management Journal, Vol. 7 No. 2, pp. 239-254, doi: 10.1007/s11365-010-0156-x.
Schelfhout, W., Bruggeman, K. and De Maeyer, S. (2016), “Evaluation of entrepreneurial competence through scaled behavioral indicators: validation of an instrument”, Studies In Educational Evaluation, Vol. 51, pp. 29-41.
Shields, J.W. (2015), “Exercise in ridiculosity”, Paper presented at the Annual Conference of the United States Association for Small Business and Entrepreneurhip (USASBE), January, Tampa, Florida.
Simmons, J.P., Nelson, L.D. and Simonsohn, U. (2011), “False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant”, Psychological Science, Vol. 22 No. 11, pp. 1359-1366, doi: 10.1177/0956797611417632.
Simonsohn, U. (2015), “Small telescopes: detectability and the evaluation of replication results”, Psychological Science, Vol. 26 No. 5, pp. 559-569, doi: 10.1177/0956797614567341.
Slavin, R.E. (2002), “Evidence-based education policies: transforming educational practice and research”, Educational Researcher, Vol. 31 No. 7, pp. 15-21.
Solesvik, M.Z. (2019), “Entrepreneurial competencies and intentions: the role of higher education”, Forum Scientiae Oeconomia, Wydawnictwo Naukowe Akademii WSB, Vol. 7 No. 1, pp. 9-23.
Tincani, M. and Travers, J. (2019), “Replication research, publication bias, and applied behavior analysis”, Perspectives on Behavior Science, Vol. 42 No. 1, pp. 59-75, doi: 10.1007/s40614-019-00191-5.
Uncles, M.D. and Kwok, S. (2013), “Designing research with in-built differentiated replication”, Journal of Business Research, Vol. 66 No. 9, pp. 1398-1405.
Van de Mortel, T.F. (2008), “Faking it: social desirability response bias in self-report research”, Australian Journal of Advanced Nursing, Vol. 25 No. 4, p. 40.
Walker, R.M., Brewer, G.A., Lee, M.J., Petrovsky, N. and van Witteloostuijn, A. (2018), “Best practice recommendations for replicating experiments in public administration”, Journal of Public Administration Research and Theory, Vol. 29 No. 4, pp. 609-626, doi: 10.1093/jopart/muy047.
Weaver, M., Dickson, P. and Solomon, G. (2006), “Entrepreneurship and education: what is known and not known about the links between education and entrepreneurial activity”, The Small Business Economy: A Report to the President, United States Government Printing Office, Washington, DC, pp. 113-156.