Evaluative design of e-government projects

A community development perspective

The Authors

Mike Grimsley, Faculty of Arts, Computing, Engineering and Science, Sheffield Hallam University, Sheffield, UK

Anthony Meehan, Department of Computing, The Open University, Milton Keynes, UK

Anna Tan, The Office of The Deputy Prime Minister (Directgov Home and Community Franchise Team), London, UK

Abstract

Purpose – The purpose of this paper is to develop a framework supporting the design and evaluation of e-government projects, especially those involving voluntary and community organisations.

Design/methodology/approach – The research adapts a socio-economic model of community sustainability, reinterpreting it in the context of e-government. It documents the evolution of a conceptual framework for evaluative design through study of a project in London, UK. An interpretive approach was adopted, within which research was guided by the structured-case method.

Findings – The research provides an evaluative framework for e-government projects featuring four forms of community capital: infrastructural, environmental, human and social. An ex post evaluation illustrates how the framework identifies design and management issues that are not considered by conventional evaluative frameworks.

Research limitations/implications – The main sources of data reflect project management perspectives and information from monitoring the evolution of activities undertaken by participant organisations. There has been limited direct engagement with the latter and the next phase of research will apply the framework from their perspectives to identify factors promoting and inhibiting ongoing engagement with the system.

Practical implications – The framework provides an analytic tool for designers and managers of e-government systems, especially those which feature online community building as a strategic outcome. All project stakeholders can use the framework to structure engagement with system design and management.

Originality/value – The distinctive contribution is to reinterpret e-government from a community development perspective. It offers a means of identifying project shortcomings ignored by methods taking a narrower approach to e-government information systems development.

Article Type:

Research paper

Keyword(s):

Government; Communication technologies; Information systems; Community development; United Kingdom.

Journal:

Transforming Government: People, Process and Policy

Volume:

1

Number:

2

Year:

2007

pp:

174-193

Copyright ©

Emerald Group Publishing Limited

ISSN:

1750-6166

Introduction

The UK Government, in common with many others, is promoting electronically mediated approaches to democratic participation and public service provision. A crucial component of the government's strategy is the development and deployment of information and communications technologies (ICT) to support the new service-delivery structures (Cabinet Office 2005). The aim is partly to realise efficiencies, but there are also prominent socio-political aims such as facilitating participation in community decision making, and promoting community well-being and sustainability. Achievement of both social and economic goals depends upon widespread adoption e-government technologies.

A particular concern is that those who most need government services are least likely to use e-government systems, either because they cannot access them (Pekonen and Pulkkinen, 2002) or because they do not trust them (Duffy et al., 2003). Historically, a large number of voluntary and community organisations (the so-called Third Sector) have come into being to offer citizens, especially those most in need, advice, support and advocacy in relation to government and public services, fulfilling the important role of trusted intermediary. In the context of the rapid rollout of e-government services demanded by UK Government targets, there is a widely perceived risk that along with individual citizens, the community organisations upon which they depend may also experience digital exclusion Loader et al. (2000). Increasingly, the UK Government casts the Third Sector as key intermediaries in the relationship between potentially excluded citizen and government services (UK Government e-Envoy, 2003).

This strategic positioning of voluntary and community organisations brings additional diversity (and arguably a changed distribution of power) to the nature and range of stakeholder interests to be addressed within many e-government initiatives. However, the goals of national government, local government, public service providers (both profit-driven and not-for-profit), voluntary and community organisations and individual members of the public are rarely congruent. In so far as potential synergies exist, they do so in a web of relationships, each with an associated “history” (Vangen and Huxman, 2003). For long-term success, many e-government projects will require information systems that facilitate the construction and mediation of complex and delicate partnerships involving Third Sector organisations.

The development of such systems will require appropriate techniques of evaluation, capable of supporting all stages of the development life-cycle. Traditional approaches to evaluation, and especially those used in practice, tend to focus on organisational performance and related economic measures (Serafeimidis and Smithson, 2003; Klecun and Cornford, 2005), often in a context in which a single investing organisation has a final determining viewpoint. Modern approaches to evaluation advocate the importance of more holistic evaluation and acknowledge multiple stakeholder perspectives (Walsham, 1993, 1999; Farbey et al., 1999), but there is limited evidence that such approaches are used in practice (Serafeimidis and Smithson, 2003; Klecun and Cornford, 2005). It remains the case that evaluation is “underdeveloped” (Irani and Love, 2001), and perhaps especially so in relation to e-government (Damodaran et al., 2005).

In this paper, we articulate the development of a conceptual framework (CF) which is intended to support the evaluative design of e-government projects, and especially projects in which there is a perceived need to facilitate the contribution of Third Sector organisations. The framework has evolved from engagement with, and reflection upon, a significant e-government project-CASweb (www.casweb.org). CASweb is a strategic e-government initiative in central London, UK, which aims to supporting a large number of voluntary and community organisations in developing an internet presence and in developing new ways of organising to support their members and clients in relation to e-government services. Using an interpretive approach, we view CASweb and similar e-government initiatives as community development projects. This leads us to adapt and develop a socio-economic model of community sustainability developed at the World Bank (Grootaert, 1998) to provide a framework for evaluative design of e-government projects. At the heart of the framework are four forms of capital: infrastructural, environmental, human and social, and the dependencies between them.

Using a research process described in Section 3, the CF at the centre of this paper has evolved through iterative engagement with the case study project. The structure of this paper is intended to reflect this and so relevant literature is considered as the paper unfolds rather than being located in a single section. The paper is organised as follows. In Section 2, we review information systems evaluation in relation to e-government. Section 3 describes our methodology. Section 4 introduces the case study project, CASweb. Section 5 motivates our initial CF and documents its evolution on the basis of the engagement with the case study project. Section 6 uses the framework and the case study to draw out some generic issues and lessons. Section 7 makes some concluding remarks.

Evaluation and e-government

In all information systems contexts evaluation has a role to play, helping to assure alignment of outcomes with an evolving perception of strategic need. However, as Willcocks and Lester (1999) observe, evaluation is one of the most neglected areas of IS design and development. The most frequently cited obstacles to evaluation are: problems identifying and quantifying benefits, unfamiliarity with evaluation techniques, difficulty interpreting results, and lack of time, data, information, or interest (Ballantine et al., 1999, p. 142). Not only are there well recognised problems with objective evaluation metrics, but there is also little experience (at least outside of the academic world) of using methods, which seek to interpret and/or quantify the subjective, such as feelings, attitudes and perceptions (Powell, 1999, p. 159), or “the intangible” (Farbey et al., 1999, p. 184).

In response to the difficulties of evaluation considered above, many academics have argued for interpretive techniques which gain a deep understanding of the otherwise undocumented evaluations of all stakeholders during system design, development and deployment, accommodating subjective assessment and complementing objective techniques (Walsham, 1999; Hirchheim and Smithson, 1999; Jones Hughes, 2001).

The interpretive approach is demanding of time and commitment from stakeholders, so is not necessarily justified in all contexts. But, in a comprehensive survey of evaluative approaches and IS project contexts, Farbey et al. (1999) consider qualitative, interpretive, case study approaches as the method of choice for radical projects with fuzzy objectives; a category within which many e-government initiatives fall.

The nature of many e-government initiatives is such that the familiar problems and obstacles to evaluation, as outlined above, are posed in a particularly sharp fashion. Intangible objectives such as community sustainability, democratic participation, social inclusion, government legitimacy, citizen satisfaction and trust are poorly understood and difficult to manage and measure. And in a democratic context e-government objectives such as efficiency (cost cutting to reduce overall public spending) and effectiveness (using efficiencies to release public resources for improved performance or to fund additional projects and priorities) are likely to be contested by a diverse and political stakeholder community. Whilst managing conflicting viewpoints is a familiar problem in IS design, in the context of e-government power is distributed much more equitably than is often the case in stakeholder groups. In the introduction, we described how a strategic reliance on voluntary and community organisations had emerged, which puts them in a particularly influential position in relation to many e-government projects. This distribution of legitimacy and influence makes the problem of designing e-government systems for long-term success particularly complex.

Given the difficulties considered above, it is unsurprising to find little evidence of stakeholder involvement in evaluative design of e-government projects. In a recent “survey of surveys” looking for evidence of stakeholder-centric approaches within e-government Damodaran et al. (2005) summarise the situation as one in which “there appears to be far more emphasis on technological aspects of delivery than on engaging citizens in identifying real needs and participating in decision-making regarding perceived priorities and methods of service delivery” and that “understanding of user requirements are therefore insufficient to inform service design and delivery”. This situation exists despite a report to the ODPM on the implementation of local e-government in the UK in which researchers observed that “most importantly, for e-government to achieve its goals, active participation of individuals and business as customers, interlocutors, clients and citizens needs to be configured” (ODPM, 2003).

Recently, recognition of the need to foster evaluation in e-government has led the UK's Engineering and Physical Sciences Research Council (EPSRC) to fund a Network for e-Government Integration and Systems Evaluation (e-GISE) with the goal of embedding evaluation within government, the public sector, and their partner organisations (http://gow.epsrc.ac.uk/). This paper is a contribution to that objective.

Developing an evaluative framework: elements of method

Because the term evaluation continues to be used in a number of different ways in the literature, it is appropriate that we make clear that in this paper our use of the term is rooted in the notion of a stakeholder-centric process which makes explicit both the desired and the current outcomes, and which supports the managed alignment of these throughout the project life-cycle. (See Farbey et al. (1999a) and Walsham (1999) for accounts of the development of evaluation design.)

The role of an evaluative design framework is to support the evaluator and stakeholders in structuring their mutual engagement at various stages in the design life-cycle. A framework may have a number of elements or devices, such as lists, diagrams, keyword collections, scenarios, etc. It should be interpretable by all parties concerned, though not necessarily consensually.

Walsham (1999, p. 374-7) identifies the main elements in interpretive evaluative design for information systems:

A distinctive feature of the interpretive approach is that the emphasis is upon shared learning, rather than formulation of some definitive Olympian judgement, which would only risk engendering distrust or opposition in one or more stakeholder groups. Structuring interpretive evaluation is a difficult task and one criticism of the approach is that the evaluator(s) and stakeholders are offered little support for conceptualising purpose, content and context. The CF developed within this paper is intended to provide such support, and especially for e-government projects in which Third Sector organisations play a strategic role.

Our chosen method for developing the CF is structured-case (Carroll and Swatman, 2000). Structured-case provides researchers with a processual model with three components:

  1. An evolving CF representing the current state of a researcher's (evaluator's) aims, theoretical foundations and understandings. The researcher begins with an initial CF based upon prior knowledge and experience, iteratively revising it until the enquiry terminates.
  2. A research cycle organises data collection, analysis, interpretation and synthesis.
  3. Literature-based scrutiny is used to compare and contrast the (evolving) outcomes of the enquiry with extant literature.

The term “case” is used to define the object of study; it may be a person, a group, a project, an organisation, a process, an IS, &c. A structured-case case study is distinguished from other forms of case study by the use of the three elements of the above framework to structure the enquiry. The goal of structured-case is to produce new or revised knowledge, and sometimes theory, that describes relationships between (abstract) concepts but is demonstrably rooted in observation. Structured-case has been used to develop understanding of ways in which local authorities undertake ex ante evaluation of e-Government projects (Irani et al., 2005).

The combination of interpretive enquiry with structured-case provides the basis for engaging with radical, complex and “fuzzy” e-Government IS projects.

Data capture

The analysis arrived at in this paper emerged through a cycle of data collection and interpretation as described above when outlining the structured-case framework. Data relating to CASweb was acquired over a period of a year and in a variety of ways, including:

At each stage the data from the case project were used to ground the emerging CF.

Case project: CASweb

For reasons of space, we can only provide an overview of the CASweb project at this point. For a fuller account, see http://mcs.open.ac.uk/am4469/CASweb/CASweb.pdf; the project may be found at: www.casweb.org/.

Socio-political context

The CASweb project represents a strategic response to the risk of digital exclusion in the provision of e-government services across central London, UK. The central goal of the project is to attenuate this risk by helping voluntary and community advice services develop an online presence and independent ICT capability that would enable them to avoid the potential “digital divide” and to continue serving their respective communities and clients as e-government is rolled-out. The two strategic aims of the project are to enable community agencies to maintain or improve their clients' access to public and community services, and to facilitate interagency working so that clients' needs can be viewed and supported more holistically.

Stakeholders and stakeholder relations

The principal stakeholder groups are:

System development

CASweb seeks to realise the first of it strategic aims by adopting the role of portal. In keeping with portals in other domains, it provides users with public directory services in relation to participating organisations. To provide a basic level of interactivity, CASweb allows organisations to initiate surveys and polls so they can consult their respective communities on topical issues, and obtain feedback on policies and practices.

To address its second objective, tools promoting inter-agency awareness, and facilitating sharing and cooperation are provided. CASweb supports connectivity between its member organisations and national and local e-government services via Directgov (the national e-government portal) or any local authority web site that meets interoperability standards.

In order to minimise costs associated with purchasing/licensing proprietary software (an issue for all stakeholders, especially the smallest) CASweb integrates a variety of browser-based public domain open-source components.

System deployment

The five boroughs of the LCP are major public sector organisations each employing large-scale and complex ICT systems. In contrast, even the most developed of the community organisations had fairly rudimentary networked systems running standard “office” software with related bespoke applications. The smallest agencies operate at a level comparable with a personal domestic user with internet access.

The provision of relevant software and technology for the project reflected the scale of this diversity. Once developed, the system was hosted by the Greater London Authority (a pan-London local government authority) in a way that maintains the availability of CASweb services across the capital, whilst ensuring the availability and integrity of each organisations web space and of shared systems.

To support organisations acquiring or upgrading technology in order to participate in CASweb, the local authorities introduced a scheme which allowed them to purchase recycled hardware.

Evolution of the 4-capitals framework

Our initial conceptual framework (CF1) springs from our interpretation of e-government initiatives which posit a strategic role for the Third Sector as community development projects. Accordingly, in seeking a conceptual point of departure, we take a prominent model for evaluating community development and sustainability; specifically, we take the 4-capitals model developed at the World Bank (Grootaert, 1998), a model that had its origins in the literature on growth and environmental economics (Ekins et al., 1992; Perlman et al., 2003). In much the same way as we have taken the model as a point of departure in the context of evaluation of e-government information systems, Hancock (2001) had given an interpretation of the CF from a health perspective and Green et al. (2001, 2005) had developed the model and incorporated the four capitals as key drivers of community regeneration in disadvantaged neighbourhoods.

The four-capitals of the original model are:

  1. Manufactured or fixed capital. Representing the buildings, plant and machinery used in the production of goods and services, along with roads and railways.
  2. Human capital. The sum of the skills and knowledge in a community.
  3. Environmental or natural capital. Includes natural resources, green space and blue space and may be extended to community amenities, especially those supporting social interaction. (Examples are shops, cafes, public transport, and school gates.)
  4. Social capital. Considered as different from other forms of capital in that it is said to “inhere[s] in the structure of relations between and among persons” (Coleman, 1990) and may be considered as the level of productive investment in social relations (Warren, 2001). Social capital has been described as making possible the achievement of community ends that would not be attainable in its absence (Coleman, 1990) and as the “glue” holding all other forms of capital together (Grootaert, 1998).

The presence of social capital in the model made it particularly attractive as a point of departure. In Putnam et al.'s (1993) seminal work social capital and the role it is perceived to play in societies is characterised by high levels of political engagement and economic performance, thus providing an underpinning rationale for a strategic emphasis on promoting the role of community organisations in e-government strategy. However, Putnam's emphasis on horizontal voluntary relations with high-levels of face-to-face contact means that it does not sit entirely comfortably in this context.

Clearly, these forms of capital are not immediately interpretable in the context of e-government IS initiatives.

Interpreting 4-capitals in the e-government domain

The first refinement of the framework (CF2) was driven by the need to arrive at a meaningful interpretation of the 4-capitals of the World Bank model in the context of e-government IS. Recognising that the notion of capital is not unproblematic, we begin by summarising our understanding of the term. In the context of evaluation, we focus on capital as a source of “value”. Thereafter, we document the (re)interpretation of each capital form in turn.

We take capital to be an entity that embodies or, more crudely, “stores” value, which may be realised through human use or application. The source of value is human work; the term valorisation expresses the process of adding value in this way. The value of capital is realised through its use, which itself involves human agency. Traditionally, the processes of realising value and of (re)valorisation are distinguished; the former is associated with depreciation of value with use and the latter is seen as the inverse of this, arising from creation, maintenance, repair or enhancement. These processes apply to both physical capital, such as plant or machinery, and abstract capital, such as social relationships. In fact, we suggest that it is sometimes difficult to disentangle within a single human action that which realises value in capital from that which revalorises the same capital; we give two examples. First, in the act of applying a previously acquired human skill, that same skill may itself be reinforced (practice makes perfect), and even extended, e.g. if minor variations and novelties of context are encountered and absorbed. Second, in the context of social relations, active engagement in a relationship may simultaneously realise the benefits of the relationship and regenerate its value. This is perhaps best illustrated in trust-based relations where mutually sustained trustworthy conduct induces renewed, or even enhanced levels of, trust.

The above account of capital reveals two distinct dimensions of capital: the first dimension relates to whether capital has a physical or an abstract expression; the second relates to whether capital may be revalorised in its use. The binary nature of each of these dimensions engenders four distinct expressions of capital:

  1. physical expression which is not revalorised with use;
  2. physical expression which is revalorised with use;
  3. abstract expression which is not revalorised with use; and
  4. abstract expression which is revalorised with use.

Interpreting capital – infrastructural capital

It is relatively straightforward to identify the elements of manufactured capital in an e-government project. These are the artefacts which are either brought into being, or purchased, or integrated into the technological infrastructure that facilitates the expression of the goals of the system. Thus, such capital comprises principally the computing and communications hardware and systems-level software deployed along with the documentation that supports it. Considering the nature of these artefacts we adopt the term “infrastructural capital” in preference to “manufactured capital” as we feel this better reflects the role of this form of capital in this domain.

Environmental capital

In reinterpreting environmental capital, we begin with the notion of community environment as “any amenity which facilitates social interaction” such as public parks, shops, the school gate, public transport, place of work. The internet and other digital communications technologies have significantly extended the potential scope for social interaction, facilitating social relations within and between people and organisations. Thus, in the realm of e-government, it seems natural to interpret the internet and related communications modalities (e-mail, video and webcams, CCTV, conferencing of many sorts, blogs, SMS/txt, &c.) as expressions of environmental capital which complement existing community amenities.

Human capital

Considered in the context of ICT-mediated community development, there is a distinction to be made between two distinct knowledge and skill sets. The first comprises the technical skills needed to develop, configure and maintain the ICT infrastructure (traditionally, systems management and systems administration). The second comprises the user skills needed by citizens, staff, volunteers, etc. to make effective use of the available computing and communication environment. This set of skills extends significantly beyond the technical and includes communication skills, relationship management skills, and capacities for consultation, negotiation, collective decision-making, and organisational governance, all of which are mediated by ICT. These latter skills are especially important in the functioning of independent and voluntary community organisations. In an e-government context involving co-production of services by voluntary and community organisations, it is important to recognise that these organisations are much more constrained to work with the skills they posses as that they cannot simply recruit or retrain volunteers to meet the demands of a new system.

Social capital

There is a considerable literature on forms of social capital and its measurement and it continues to be an issue of debate, even controversy. The UK Office of National Statistics (ONS) has attempted to harmonise a number of approaches measures (Harper and Kelly, 2003). The ONS summarise these types of social capital: bonding, bridging, and linking (Putnam, 2000). We adapt the definitions to reflect our domain of as follows:

The concepts of bonding and bridging relate naturally to a distinction between intra-organisational and inter-organisational relations. Linking social capital captures the power differentials that occur in a context featuring governmental and managerial relations.

In considering Social Capital we distinguish two components of value. The first relates to the very existence of the relation; the second to the extent to which the relationship is characterised by high or low levels of trust. The value of trust is the value of the work that does not need to be done to research, formalise and monitor the experience of a relationship – low-trust relations are expensive in terms of opportunity-costs. Some people avoid contact with public and community services that they do not trust (even essential health services) unless it is absolutely essential (Duffy et al., 2003). Thus, e-government initiatives must not only establish (new) electronically-mediated communicative relations, but they also must do so in a way that engenders trust.

The validity of considering the four capitals we have described above as distinct forms of capital is supported by the fact that there is a one-to-one correspondence between them and the four expressions of capital identified earlier (Figure 1).

Relationships between forms of capital

Our most recent evolution of the CF3 focuses on ways in which the 4-capitals are related to each other in our chosen domain. The need for this evolution of CF2 arose from a recognition that a focus on each form of capital in itself does not support a deep conceptual engagement with evaluative design. The 4-capitals are useful as elements of a taxonomic structure that supports an accounting approach to evaluation. However, we find that important, often less prominent, design issues are better accessed by considering the interdependencies of the 4-capitals in a pairwise fashion. For example, in examining the relationship between human capital and any of the other three capitals, evaluative design decisions in relation to skills needed and skills possessed are raised; the existence of any disparity poses questions as to how the gap may be addressed. This may be through revision of the system (infrastructural, environmental or relational specifications), or through the provision of training, or in recruitment of additional human resource, or some combination of these.

The four forms of capital give rise to six binary relations between them (Figure 2), each encompassing a set of evaluative design issues arises:

  1. Architectural design addresses the well rehearsed issues, embedded in the software development life-cycle, of marrying ICT infrastructure to satisfy functional design requirements.
  2. Connectivity concerns the configuration of infrastructural components to ensure that the desired community relationships are supported by the technological infrastructure.
  3. Communicative mediation (of relationships) identifies the modes of cooperative communication that are to be supported (examples are conferencing, document sharing, or secure case working) and the related issues of privacy and security needed to promote and sustain trust.
  4. System management seeks to marry the system technology with the skills and competences of people involved in its configuration, management and administration.
  5. Relational conduct seeks to ensure that the people and organizations involved are equipped to conduct community relationships; for example, do they have skills in electronic communication, participative decision-making, management, or ethical governance.
  6. Affordance design (the domain of human-computer interaction – HCI) concerns the design of the communicative interface(s) to support the selected modes of cooperative communication in ways that reflect both the attainable technological skills and competencies of the ordinary user.

The combination of the 4-capitals and the six relations between them constitute a framework which can be used by an evaluative designer when discussing, exploring or negotiating with stakeholders. The precise state of development of the model and each of its elements is governed by the unfolding of the project life-cycle and the intensity of the stakeholder engagement.

Identifying generic issues and lessons

In this section, we illustrate how the elements of the 4-capitals framework provide evaluative structure by considering CASweb as an example of an ICT-based community development project. In the ex post picture presented (Table I) it is clear that where the project managers and developers had the time and resources to actively address elements of the framework (mainly infrastructural, environmental and technical human capital) then the outcome has been highly positive; on the other hand, where elements of the model were not addressed, for whatever reason, then success is less obvious.

On the basis of the above we were able to draw out generic issues which may subsequently be considered as part of the 4-capitals framework (Table II).

In addition to these generic issues we can identify some lessons which seem worthy of re-emphasising as, in our view, they presented significant challenges to the long-term success of the project.

Firstly, CASweb, like many e-government initiatives, was funded as a capital project. However, community development demonstrably needs sustained effort, which entails ongoing revenue-based support. If this is not adequate, the potential of the investment is unlikely to be realised.

Second, all organisations can benefit from support and development to appraise how their core-processes and ways of working might be need to be revised in the light of the potential offered by an online connection to clients and intra-and inter-organisation tools for collaborative work. Third Sector organisations may prefer independent advice in achieving this.

Third, a number of the stakeholder parties involved, including authorities of the LCP, relied heavily on “local champions” to play a key part in community building through CASweb. But a strategy which is reliant upon local champions is unlikely to be sustainable; organisations, be they large local authorities or the most modest voluntary organisations, need to plan for succession of individuals, e.g. using organisational knowledge management techniques.

Finally, it all four forms of capital need to be developed in parallel, but, as we have suggested, in relation to human and social capital, continuous rather than “one off” revalorative activity is needed – a case of “use it or lose it?”

We conclude this section by considering how designers of e-government systems should approach the evaluative design problem in terms of the 4-capitals framework we have defined. Our conclusion flows from identifying constraining dependencies between capital forms in relation to strategic goals. The result is shown in Figure 3, in which we suggest that the starting point should be an assessment of the social relations that must be supported. Thereafter, consideration of the environmental capital and human capital can proceed in parallel, ensuring that there needs to be an alignment of these before the infrastructure capital is specified. Of course, design over this framework will be an iterative process as the project life-cycle unfolds, but we suggest that this general direction of travel should be maintained.

Concluding remarks

This paper has given an account of an attempt to use interpretive enquiry to develop a framework for evaluative design of e-government projects that seek to contribute to community development by involving Third Sector organisations in co-production of e-government services. The enquiry was grounded in a case project which sought to bridge a potential digital divide between e-government service providers and community advice organisations and to promote inter-agency approaches to tackling clients' problems holistically.

The framework that emerged is a reinterpretation and extension of a socio-economic model of sustainable development to suit the context of e-government information systems. The framework supports evaluators and stakeholders in identifying and developing four forms of capital (4-capitals) needed for sustainable project development: infrastructural capital, human capital, environmental capital and social capital. Consideration of the interplay between these forms of capital gives rise to and structures an agenda of design decisions to be explored.

In considering the extent to which it achieves that goal, the 4-capitals model can itself be evaluated in relation to the suggested shortcomings of traditional IS evaluation introduced at the beginning of this paper. Farbey et al. (1999, p. 184) pointed to the need for an approach that helped identify many of the “intangibles” that contribute to the successful attainment of system goals. Ballantine et al. (1999, p. 142) signalled the need for instruments to support evaluators in addressing the needs of stakeholders during system design, development and deployment. Willcocks and Lester (1999, p. 84) highlighted the need for an approach that enables “timely decisions”. We consider each of these in turn.

Firstly, the 4-capitals model of CF3 makes concrete and brings to prominence a number of the important intangibles contributing to e-government IS design: the social (and political) relations which must be established and supported, the diversity of human and organisational skills needed, the communicative environment which mediates relations, and the technology required to facilitate all of these. Secondly, the 4-capitals and the interdependencies between them are readily interpretable and stakeholders can usually relate strongly to at least some of the elements or the issues arising in respect of their interdependencies. The architecture of Figure 2 allows stakeholders to see where they and others contribute to the project whole. Finally, the elements of the framework persist throughout the life-cycle of the project even as their description (documentation) evolves as the project unfolds. This means there is always some a meaningful “interpretation” of the elements to be arrived at, whether at project conception or project conclusion. Thus, at each stage of the life-cycle, they provide a foundation for stakeholder evaluation of past development and future need. In this way, the framework supports timely review and forward planning, diminishing the likelihood of the need for expensive post hoc revision.

ImageTaxonomic foundation for four expressions of capital exemplified by each of the capitals in the 4-capitals model
Figure 1Taxonomic foundation for four expressions of capital exemplified by each of the capitals in the 4-capitals model

ImageRelations between 4-capitals
Figure 2Relations between 4-capitals

ImageOrder of development in the 4-capitals framework
Figure 3Order of development in the 4-capitals framework

Image evaluative record of the CASweb project illustrating application of the 4-capitals framework
Table I Ex post evaluative record of the CASweb project illustrating application of the 4-capitals framework

ImageSome generic issues andlessons relating to evaluative designdecisions raised by the 4-capitals framework
Table IISome generic issues andlessons relating to evaluative designdecisions raised by the 4-capitals framework

References

Ballantine, J.A., Galliers, R.D., Stray, S.J. (1999), "Information systems/technology evaluation practices: evidence from UK organisations", in Willcocks, L.P., Lester, S. (Eds),Beyond the IT Productivity Paradox, Wiley, Chichester, pp.123-49.

[Manual request] [Infotrieve]

Carroll, J.M., Swatman, P.A. (2000), "Structured-case: a methodological framework for building theory in information systems research", European Journal of Information Systems, Vol. 9 pp.235-42.

[Manual request] [Infotrieve]

Coleman, J. (1990), Foundations of Social Theory, Harvard University Press, Cambridge, MA, .

[Manual request] [Infotrieve]

Damodaran, L., Nicholls, J., Henney, A., Land, F., Farbey, B. (2005), "The contribution of sociotechnical systems thinking to the effective adoption of e-government and the enhancement of democracy", The Electronic Journal of e-Government, available at: www.ejeg.com/, Vol. 3 No.1, pp.1-12.

[Manual request] [Infotrieve]

Duffy, B., Browning, P., Skinner, G. (2003), Trust in Public Institutions: A Report for the Audit Commission, MORI, London, .

[Manual request] [Infotrieve]

Ekins, P., Hillman, M., Hutchinson, R. (1992), Wealth Beyond Measure: An Atlas of New Economics, 3rd ed., Gaia, London, .

[Manual request] [Infotrieve]

Farbey, B., Land, F., Targett, D. (1999a), "Moving IS evaluation forward: learning themes and research issues", Journal of Strategic Information Systems, Vol. 8 No.2, pp.189-207.

[Manual request] [Infotrieve]

Farbey, B., Land, F., Targett, D. (1999b), "Evaluating investments in IT: findings and a framework", in Willcocks, L.P., Lester, S. (Eds),Beyond the IT Productivity Paradox, Wiley, Chichester, pp.183-215.

[Manual request] [Infotrieve]

Green, G., Grimsley, M., Stafford, B. (2001), Capital Accounting for Neighbourhood Sustainability, CRESR, Sheffield Hallam University, Sheffield, .

[Manual request] [Infotrieve]

Green, G., Grimsley, M., Stafford, B. (2005), The Dynamics of Neighbourhood Sustainability, Joseph Rowntree Foundation, York Publishing Services, York, .

[Manual request] [Infotrieve]

Grootaert, G. (1998), "Social capital: the missing link", World Bank, Washington, DC, SCI Working Paper No. 3, .

[Manual request] [Infotrieve]

Hancock, T. (2001), "People, partnerships and human progress: building community capital", Health Promotion International, Vol. 16 No.3, pp.275-80.

[Manual request] [Infotrieve]

Harper, R., Kelly, M. (2003), Measuring Social Capital in the United Kingdom, ONS, London, available at: www.statistics.gov.uk/socialcapital, .

[Manual request] [Infotrieve]

Hirchheim, R., Smithson, S. (1999), "Evaluation of information systems: a critical assessment", in Willcocks, L.P., Lester, S. (Eds),Beyond the IT Productivity Paradox, Wiley, Chichester, pp.381-409.

[Manual request] [Infotrieve]

Irani, Z., Love, P.E.D. (2001), "Information systems evaluation: past present and future", European Journal of Information Systems, Vol. 10 No.4, pp.183-8.

[Manual request] [Infotrieve]

Irani, Z., Love, P.E.D., Elliman, T., Jones, S., Themistocleous, M. (2005), "Evaluating e-government: learning from the experiences of two UK local authorities", Information Systems Journal, Vol. 15 pp.61-82.

[Manual request] [Infotrieve]

Jones, S., Hughes, J. (2001), "Understanding IS evaluation as a complex social process: a case study of a UK local authority", European Journal of Information Systems, Vol. 10 pp.189-203.

[Manual request] [Infotrieve]

Kelly, G.A. (1955), The Psychology of Personal Constructs, Norton, New York, NY, .

[Manual request] [Infotrieve]

Klecun, E., Cornford, T. (2005), "A critical approach to evaluation", European Journal of Information Systems, Vol. 14 No.3, pp.229-43.

[Manual request] [Infotrieve]

Loader, B.D., Hague, B.N., Eagle, D. (2000), in Guerstein, M. (Eds),Community Informatics: Enabling Communities with Information and Communication Technologies, Idea Group Publishing, Hershey, PA, .

[Manual request] [Infotrieve]

ODPM (2003), Local E-government: Process Evaluation of the Implementation of Electronic Local Government in England, Office of the Deputy Prime Minister, London, .

[Manual request] [Infotrieve]

Pekonen, O., Pulkkinen, L. (2002), Social Capital and the Development of Information and Communication Technology: Report for the Committee for the Future of the Parliament of Finland, University of Jyväskylä, Jyväskylä, 26 April, .

[Manual request] [Infotrieve]

Perlman, R., Ma, Y., McGilvray, J., Common, M. (2003), Natural Resources and Environmental Economics, 3rd ed., Pearson, Harlow, .

[Manual request] [Infotrieve]

Powell, P.L. (1999), "Evaluation of information technology investments: business as usual", in Willcocks, L.P., Lester, S. (Eds),Beyond the IT Productivity Paradox, Wiley, Chichester, pp.151-82.

[Manual request] [Infotrieve]

Putnam, R. (2000), Bowling Alone – The Collapse and Revival of American Community, Simon & Schuster, New York, NY, .

[Manual request] [Infotrieve]

Putnam, R., Leonari, R., Nanetti, R. (1993), Making Democracy Work: Civic Traditions in Modern Italy, Princeton University Press, Princeton, NJ, .

[Manual request] [Infotrieve]

Rugg, G., McGeorge, P. (1999), "The concept sorting techniques", The Encyclopedia of Library and Information Science, Marcel Dekker, Inc., New York, NY, supplement 28, Vol. Vol. 65 pp.43-71.

[Manual request] [Infotrieve]

Serafeimidis, V., Smithson, S. (2003), "Information systems evaluation as an organisational institution – experience from a case study", Information Systems Journal, Vol. 13 No.3, pp.251-74.

[Manual request] [Infotrieve]

UK Government e-Envoy (2003), UK Online Annual Report, Office of the e-Envoy, London, .

[Manual request] [Infotrieve]

Vangen, S., Huxman, C. (2003), "Nurturing collaborative relations", Journal of Applied Behavioural Science, Vol. 39 No.1, pp.5-31.

[Manual request] [Infotrieve]

Walsham, G. (1993), Interpreting Information Systems in Organisations, Wiley, Chichester, .

[Manual request] [Infotrieve]

Walsham, G. (1999), "Interpretive evaluation design for information systems", in Willcocks, L.P., Lester, S. (Eds),Beyond the IT Productivity Paradox, Wiley, Chichester, pp.183-215.

[Manual request] [Infotrieve]

Warren, M.E. (2001), "Social capital and corruption", in Castiglione, D.S. (Eds),Social Capital: Interdisciplinary Perspectives, paper presented at EURESCO Conference on Social Capital, Exeter, .

[Manual request] [Infotrieve]

Willcocks, L.P., Lester, S. (1999), "In search of information technology productivity: assessment issues", in Willcocks, L.P., Lester, S. (Eds),Beyond the IT Productivity Paradox, Wiley, Chichester, pp.69-97.

[Manual request] [Infotrieve]

Further Reading

Gilbertson, J., Green, G., Grimsley, M., Manning, J. (2005), "The Dynamic of Social Capital", Health and Economy, CRESR, Sheffield Hallam University, Sheffield, .

[Manual request] [Infotrieve]

Irani, Z., Love, P.E.D. (2002), "Developing a frame of reference for ex-ante IT/IS investment evaluation", European Journal of Information Systems, Vol. 11 No.1, pp.74-82.

[Manual request] [Infotrieve]

Loader, B.D., Keeble, L. (2004), Challenging the Digital Divide?, Joseph Rowntree Foundation, York, .

[Manual request] [Infotrieve]

Love, P.E.D., Irani, Z. (2003), "A project management quality cost information system for the construction industry", Information & Management, Vol. 40 No.7, pp.649-61.

[Manual request] [Infotrieve]

ODPM (2004), Breaking the Cycle' Social Exclusion Unit, Office of the Deputy Prime Minister, London, available at: wwwsocialexclusiongovuk/downloaddocasp?id = 262, .

[Manual request] [Infotrieve]

ODPM (2005), "Inclusion through innovation: tackling social exclusion through new technologies", Social Exclusion Unit Final Report, Office of the Deputy Prime Minister, London, available at: www.socialexclusionunit.gov.uk/page.asp?id = 583 for report and supplementary annexes (accessed November), .

[Manual request] [Infotrieve]

About the authors

Mike Grimsley is a Principal Lecturer in Statistics at Sheffield Hallam University. Mike gained a BSc(Econ) and MSc in Statistics from LSE (1969), a PGCE from Goldsmiths College (1970). He was formerly a Lecturer in statistics and research statistician at the Institute of Child Health, Un of London. He teaches multivariate and survey methods and has supervised many doctoral students. Particular research interests include: modelling pathways between deprivation, social capital, crime and fear of crime, health and well-being; the relevance of health and housing to social and economic regeneration; evaluation of policy interventions; the concept and measurement of community trust. E-mail: m.f.grimsley @ shu.ac.uk

Anthony Meehan is a member of the Centre for Research in Computing (CRC) and the Department of Computing at the The Open University, UK. His current research aims to identify principles and frameworks which support e-government systems development. Anthony also supervises research in computational neuropsychology. His current teaching interests are Information Security Management and accreditation of professional and vocational learning. Anthony Meehan is the corresponding author and can be contacted at: a.s.meehan@open.ac.uk

Anna Tan leads the ODPM's “Directgov” Home and Community Franchise Team responsible for e-government initiatives supporting community organisations. Prior to joining the ODPM, she worked as a project leader with the London Borough of Camden and the London Central Partnership, including leadership of CASweb. Her early career involved work in both the private and voluntary sectors. E-mail: anna.tan@odpm.gsi.gov.uk