An integrated methodology for supporting the development and the performance evaluation of academic spin-offs

2.1 Factors affecting development and performance of academic spin-offs

A significant number of papers on the performance measurement of ASOs are focused on identifying the factors that influence the performances themselves (Bigliardi et al., 2013; Hayter, 2013; Iacobucci and Micozzi, 2015; Helm et al., 2018; Poponi et al., 2017; Hossinger et al., 2019; Mathisen and Rasmussen, 2019).

Visintin and Pittino (2014), suggests that business performance may be affected by the composition of the entrepreneurial team (Rodríguez-Gulías et al., 2017). Evers et al. (2016) argue that the international relationships of involved researchers, typical of the academic world, strongly influence the internationalization of an ASO.

Other researchers argue that the calculation of spin-off performance depends also on the technological transfer systems implemented and the “incubators” available in the university of origin (Sternberg, 2014; Carrasco and Aceytuno, 2015; Furlan and Grandinetti, 2014; Minguillo and Thelwall, 2014; Vinig and Lips, 2015; Rodríguez-Gulías et al., 2016; Soetanto and Jack, 2016).

Poponi et al. (2017) identified nine factors associated with the performance: sustaining the start-up, the heterogeneity of the founding skills, access to finance, environmental richness, network capital, size, reliability, innovation and motivation.

Many studies discuss the barriers and the reasons that hinder the growth of spin-offs (Ayoub et al., 2016; Galati et al., 2016; Neves and Franco, 2018), taking into account that creating an ASO is relatively simple, more difficult is supporting the growth, especially if the economic and social context is not favorable.

Barbieri et al. (2018) compare the work of the researchers, in terms of the number of publications made and the number of patents filed, before and after the creation of the company, as well as of the possible change of relationship with other companies.

Another critical point is to decide the right time, for a spin-off, to leave the university. The premature exit can lead to the failure of the company (Müller, 2008).

Furlan and Grandinetti (2014) identify, in the early years of a start-up, two development phases: incubation and emergency phase. They focus on the factors, for each of the phases that influence the survival and early growth of start-ups.

2.2 Methodologies for evaluating the academic spin-offs and measuring their performance

Research-based spin-offs are a particular case of start-ups: they are firms strongly based on an innovative product, as it directly derives from research. ASOs are potentially the most efficient way to transfer new technological knowledge into business, i.e. into new products and services. This kind of start-ups has a great advantage, over patents or other transfer mechanisms that the tacit knowledge of the academics, which is otherwise so difficult to transfer, is indeed transferred – straight into a new company (Sternberg, 2014).

Many scholars analyzed methods for evaluating the performance of spin-offs, as well as the impact these have on the surrounding environment both in terms of economic and social development (Cesaroni and Piccaluga, 2015; Iacobucci and Micozzi, 2015; Brown, 2016; Del Giudice et al., 2017; Helm et al., 2018; Boh et al., 2016; Fini et al., 2017). Some authors studied the value creation measurement in knowledge-based organizations (Iazzolino and Laise, 2016).

Helm et al. (2018) compare different kinds of indicators (not only financial but also economic) to measure the performance of companies. They propose a theoretical framework for evaluating the performance from a different point of view, but it is still not clear if the spin-off classification varies according to the index used.

Cesaroni and Piccaluga (2015) identify the variables necessary to evaluate the efficiency of the knowledge transfer (KT) office. They measure it in terms of the success in commercializing the generated technology, and more in general, to concretize the so-called “third mission.” The amount of KT generated by Italian, European and US universities was compared.

Vinig and Lips (2015) present an innovative approach to measuring university technology transfer performance using meta-data analysis. In particular, they use the research products (patents, licensing deals and spin-offs) as meta-data to estimate the potential for technology transfer.

However, the analysis carried out shows that a methodology has not yet been developed specifically for the rating of ASOs.

2.3 The lean approaches for supporting startup development

Several statistics note that the startups have a high level of failure: 75 per cent following a Harvard Business School’s Shikhar Ghosh research (Blank, 2013).

One of the main reason arise from the difficulty to define and make stable their business models. Many scholars started to analyze that this condition is structural for a startup, then the typical tools supporting the first phase of innovative new venture firms are not suitable in this case. The business plan, for example, following a waterfall approach (based on a linear sequence: idea generation, search a technical solution, build a business model, implement this solution and collect the feedback), determines the typical situation shown in Figure 1, where the amount of resources allocated are maximum in the first phase, but customer feedback is evaluated only in the last phases, when a large amount of resources are now lost, especially in the case of hypotheses not validated by their target market.

A startup seeks its balance through the definition of its own business model. The balance must be:

• overall because the development areas of a spin-off are different and each is fundamental for the success of the business project; and

• dynamic because, in full compliance with the principles of lean thinking, the balance must be built and fed through tests and feedbacks that generate learning as a necessary tool to decide, which development path must be chosen.

According to the lean startup methodology (Ries, 2011) the learning process of the new company is based on some fundamental activities including:

• tests and feedback collection;

• feedback measurement and evaluation; and

• learning based on tests.

Figure 2 shows more in detail the phases related to the learning process.

Blank (2013) proposed the customer discovery process method, which outlines the ideal path of a startup in the phases represented in Figure 3.

It is clear that the initial phases of discovery and validation are essential for the business model research, rather than for its execution. This activity, moreover, is a typically managerial activity related to a waterfall approach, based on the implementation of strategic plans (business plans) based on already validated hypotheses that obviously a startup cannot have.

It is these difficult affirmation and development conditions for the startups that have favored the birth and development of the lean startup approach as a valid alternative to the waterfall approach. It can be noted the differences between the two approaches, with respect to the expected economic fundamentals of the business project as shown in Figure 4.

Practically, the lean approach makes possible to replace quickly not-validated hypotheses because of feedback collected from customers – in this case early adopters – already in the critical phases of the startup project (product-market FIT), in which the invested resources (not only financial) are still “low” (compared to the waterfall approach), and above all, manageable with a pivoting methodology.

According to the most recent and established approaches, it is necessary to conceptualize a business model to describe and visualize the startup logic to create, supply and capture value. The most recent innovations in this sense refer to the use of the “canvas” models, which help to “visualize” the ways a startup creates value, and above all, enable all the benefits of visual thinking, today an increasingly affirmed and effective methodology for the development of “innovations in organizations.” The most widespread and appreciated contributions in this sense are those proposed by Osterwalder and Pigneur (2010), and more recently, the one proposed by Maurya (2012) that can be considered as an upgrade of the first business model canvas in lean perspective (Figure 5).

These models have the worth of effectively breaking down, within a canvas, “problems to be solved or tasks to be performed” typical of a startup project. The canvas is then used as prototypes of business models, to be refined and consolidated through continuous cycles of experiments and tests.

3.1 The lean approach adopted for designing and defining the proposed methodology

The methodology here proposed has been developed within an Erasmus + research project, co-funded by the European Commission, called SOLA. The partners involved in this project are nine European and Latin-American universities and their TTOs.

The aim of the research was to design a methodology that could be used first of all by the TTOs and by the university incubators in their institutional activity of supporting ASOs.

Several scholars analyzing the startup’s development consider the lean approach very effective because it is founded on the continuous refining of the first proposed solutions taking into account the validation received by the early adopters.

Although the previously presented canvas models based on lean approach for startups are very effective and successful, in this research work it was considered appropriate to define a model that can answer more directly and comprehensively to the needs of university spin-offs, characterized by very specific problems on which the most recent models (especially the last “lean canvas”) do not place the necessary emphasis.

The most widespread model of lean canvas focuses particularly on three aspects of a startup risk/value: product, customer and market (Figure 6).

The model has to be adapted to the specificities of ASOs and to the contribution that the TTOs or incubators can offer.

There are two main questions:

Following a founding principle of the lean approach, (“get out of the building,” that is “go outside to learn”) the first choice has been to directly involve a “startupper” with particular sensitive to the lean startup methodologies, to have precious “in house” examples, practices and methods.

The second step has been the choice of a methodology, based on the idea to apply the statement: “to develop a lean project we must think lean”: a known canvas model has been applied to highlight the current and desired product-market FIT between the TTO (the incubator “Liaison Office” at University of Calabria in the real case) (supplier) and startups incubated (customers). The application of this model has a two-fold objective as follows:

1. to respond in a systematic and demonstrable way to the questions outlined in Figure 7, to approach the development of a specific “spin-off lean canvas”; and

2. to apply and to learn how to use a basic model to support startups on the market-fit concept.

Because of this approach, it was possible to clearly and operationally highlight the spin-offs needs and expectations, a fundamental starting point for the design of a new analysis framework specific for spinoffs.

Five workshops were held in this phase and the following sections describe the methodology elaborated by the actors involved in the research project.

3.2 The “lean acceleration canvas” framework

Following an analysis carried out jointly by the TTOs involved in the project, some critical features were found, additional to those of the generic startups, which the general lean model canvas underestimate:

• Governance and organization: what are the expected roles in the startup and how is the leadership distributed on the most important decisions?

• Networking and stakeholders: which stakeholders are needed to make the project sustainable? How can we meet the expectations of the stakeholders involved?

• Management skills: what are the currently available skills and which ones are missing and should be acquired?

• Motivation and commitment: how much have they decided to invest the founders, not only monetarily but also in terms of commitment?

• Scientific research and/or underlying technology: what are the research studies underlying the startup? Which intangibles are connected to the underlying research and/or technologies and how can they be valorized and defended?

• Project timing: how long will it be possible to obtain a first minimum viable product (MVP)?

Due to the nature of the ASOs and their characteristics, the proposed methodology is focused on a canvas model based on five fundamental risk areas, three of which specifically cover the criticalities of the spin-offs while the remaining two refer to the critical risk elements already considered by the lean canvas (Figure 8):

Each of the areas identified is necessary to achieve a dynamic balance in the development of the business as follows (Figure 9):

• problem-solution FIT: market risk;

• innovation FIT: technological risk;

• operations FIT: risk of implementation;

• stakeholders FIT: risk of governance; and

• economics FIT: economic and financial risk.

The model breaks down the spin-off evaluation into 5 macro-areas and 10 blocks. Each macro-area identifies a critical element for the risk and the value of the spin-off, while each block represents an operational problem that lends itself to specific solutions, facilitated by the use of certain methods of analysis and design thinking techniques.

3.3 The first part of the proposed methodology: supporting the development of academic spin-offs

Starting from the LAC framework just described, a series of issues to be investigated (Table I) were identified (qualitative part). These issues have to be evaluated through interviews held by a team of evaluators with the founders. The information obtained from the interviews are necessary to provide an overview of the company situation and to know its evolution over time. The interviews have to be conducted by at least three evaluators who, first individually, and then as an evaluation group, express a qualitative judgment on each of the aspects investigated, both by giving a score from one to six and by noting relevant aspects. Tables II and III are an example of the evaluation sheets held by the evaluators during the interview.

The weights of the various aspects considered change as the age class of the ASO being evaluated varies. The team of evaluators before establishes them jointly to start. The numerical evaluations of the individuals are appropriately normalized and summed to obtain an overall score and the overall judgments are given to the founders.

For example, the evaluators, within the SOLA project in which the methodology was developed and applied, were members of a TTO interested in monitoring the progress of incubated startups to support them in the growth process and the allocation of resources.

The average scores obtained by the spin-offs can be reported in diagrams showing more clearly where the weak areas are, as well as the evaluations that emerged during the meetings are very useful to the founders.

3.4 The second part of the proposed methodology: the performance evaluation of academic spin-offs

The second part of the methodology aims to define appropriate quantitative indicators that refer to the obtained performance in the various areas identified in the LAC framework.

This part aims to provide an evaluation of the sustainability and scalability of the business model pursued by the existing startups.

Coherently with the multidimensional approach before adopted, the key performance indicators (KPIs) and the metrics measured have been organized following a balanced scorecard approach based on the five dimensions identified in the LAC model (Table IV).

Tables V and VI are an example of the metrics considered for two of the five dimensions here proposed. Data have to be collected both through a survey and public data.

The weights of the various aspects considered change as the age class of the startup being evaluated varies. The resulting values are appropriately normalized and weighed to obtain the appropriate rankings by dimension, company and the distance from the reference threshold values.

3.5 The overall evaluation matrix

The application of the methodology ends with a joint analysis of the qualitative and quantitative aspects, and therefore, it is possible the ranking of companies.

In particular, the principle adopted was that the two different types of evaluation converge to the same final objective and mutually reinforce each other with their differentiated values and repercussions (Figure 10).

The qualitative first part of the proposed methodology refers to the potential of the business model the quantitative second part validates the results obtained by the considered startup.

Cross-referencing these two dimensions (qualitative-potential evaluation and quantitative-validation evaluation) we can define an overall evaluation matrix (Figure 11) that allows us to get important suggestions on how to support the growth and how to eventually correct the startup’s choices.