Evaluating the performance of open source software projects using data envelopment analysis

Abstract

Purpose

The purpose of this paper is to develop and test a model of the relative performance of open source software (OSS) projects.

Design/methodology/approach

This paper evaluates the relative performance of OSS projects by evaluating multiple project inputs and multiple project outputs by using a data envelopment analysis (DEA) model. The DEA model produces an efficiency score for each project based on project inputs and outputs. The method of producing an efficiency score is based on the convex envelopment technology structure. The efficiency measure quantifies a “distance” to an efficient frontier.

Findings

The DEA model produced an index of corresponding intensities linking an inefficient project to its benchmark efficient project(s). The inefficiency measures produced an ordering of inefficient projects. Eight projects were found to be “efficient” and used as benchmarking projects.

Research limitations/implications

This research is limited to only security‐based OSS projects. Future research on other areas of OSS projects is warranted.

Practical implications

The result of this research is a practical model that can be used by OSS project developers to evaluate the relative performance of their projects and make resource decisions.

Originality/value

This research extends the work of previous studies that have examined the relative performance of software development projects in a traditional development environment. As a result of this research, OSS projects can now be adequately benchmarked and evaluated according to project performance. An OSS project manger can effectively use these results to critically evaluate resources for their project and judge the relative efficiency of the resources.

Citation

Wray, B. and Mathieu, R. (2008), "Evaluating the performance of open source software projects using data envelopment analysis", Information Management & Computer Security, Vol. 16 No. 5, pp. 449-462. https://doi.org/10.1108/09685220810920530