The use of economic evaluations in NHS decision-making: a review and empirical investigation

Citation

(2008), "The use of economic evaluations in NHS decision-making: a review and empirical investigation", Clinical Governance: An International Journal, Vol. 13 No. 3. https://doi.org/10.1108/cgij.2008.24813cae.003

The use of economic evaluations in NHS decision-making: a review and empirical investigation

Article Type: Health technology assessment From: Clinical Governance: An International Journal, Volume 13, Issue 3

I. Williams, S. McIver, D. Moore and S. Bryan

Objectives

This report is concerned with the use of research evidence relating to economic analyses in healthcare decision-making. The research described in this report addresses two principal questions:

RQ1. To what extent, and in what ways, is health economic information used in health policy decision making in the UK?;

RQ2. What factors are associated with the utilisation (or non-utilisation) of such research findings?

Methods

Systematic review

Major electronic databases were searched up to 2004 and a systematic review of the literature was undertaken. This considered existing reviews on the use of economic evaluations in policy decision making, health and non-health literature on the use of economic analyses in policy making and studies that have identified actual or perceived barriers to the use of economic evaluations.

Empirical research methods

The research team adopted a predominantly qualitative approach involving primarily the use of case study methods. This included documentary analysis, meeting observation and semi-structured interviewing. Five case studies were conducted in total, including committees from four local and one national organisation. The national case study was the Technology Appraisal Committee of the NICE. Case studies were augmented with a documentary analysis of new technology request forms used by local decision-making committees and workshop discussions with members of local decision-making committees.

Results

Systematic review results

Overall, the systematic review exposed the difficulties of attempting systematically to search for evidence when considering topics such as this. Despite these difficulties, the review established the following:

• •

  There are very few previous systematic reviews of the evidence in this area.

• •

  A number of previous studies in healthcare have looked at the use of economic evaluations in decision making. Although these undoubtedly contribute to our knowledge on this topic, there are some concerns about the methodological approach adopted in these studies.

• •

  There is a continuing need for research that addresses the range of policy decision-making levels and which takes an in-depth, qualitative approach to addressing the research question.

Empirical research results: the local level

There are a range of local formulary decision-making committees in existence. These vary in terms of: the geographical and organisational scope of responsibility; level of resource and capacity available to them; their perceived role and functions; and the types of information they request and use. Our main research finding at the local level in the NHS is that it is an exception for economic evaluation to inform technology coverage decisions.

Our data suggest that local decision-making focuses primarily on evidence of clinical benefit and cost implications. Information on implementation is also frequently requested. Cost-effectiveness information is not routinely requested by the majority of committees and was rarely accessed by the committees included as case studies. Outcomes of deliberation rarely, if ever, included disinvestments in current practices.

Case study committees appeared to operate without any direct control over resource allocation, although some committee members were clearly concerned to control spending. This added to the impression that the principal aim was to manage the introduction of technologies into the health economy (via the formulary) rather than making technology coverage decisions based on principles of efficiency and/or opportunity cost. Committee members acted as advocates of sectional, organisational or departmental interests, and demonstrated a limited capacity to access and interpret economic evaluations.

Attitudes and practices of decision makers are shaped by the institutional constraints in which they operate. A number of features of the decision-making environment appeared to militate against emphasis on cost-effectiveness analysis. These were:

• •

  a lack of clarity as to the objectives of the committees and their relationship to broader structures and processes;

• •

  an explicitly political decision-making process that involved the satisfying of interests; and

• •

  the absence of a defined budget held and allocated by the committees.

These factors, combined with constraints on the capacity to generate, access and interpret information, led to a minor role for cost-effectiveness analysis in the decision-making process.

Empirical research results: the national case study

At the national policy level, our main research finding is that economic analysis is highly integrated into the decision-making process of NICE’s technology appraisal programme. This is evidenced by the remit of NICE (to consider cost-effectiveness), the nature of the assessment reports commissioned specifically for NICE and the committee composition. In addition, data drawn from observation and interviews with Appraisal Committee members added considerable support to this overall impression. Attitudes to economic evaluation were found to vary from one committee member to another, and other factors dilute the influence of the health economics analysis available to the committee. There was strong evidence of an ordinal approach to consideration of clinical effectiveness and cost-effectiveness information. Some interviewees considered the key role of the cost-effectiveness analysis to be provision of a framework for the decision-making process. The NICE committee deliberations that we observed saw significant disagreement among committee members and these mainly revolved around the economic evaluation.

Interviewees indicated that the NICE committee did make use of some form of cost-effectiveness threshold but expressed concerns around both its basis (especially where the threshold in use currently might have come from) and its use in decision making. Overall, interviewees praised the processes employed by NICE and indicated, in general terms, that the appraisal process worked very well. However, frustrations with the appraisal process were expressed in terms of the scope of the policy question sometimes being addressed. The suggestion was made that an opportunity to clarify and identify clearly the relevant policy question should be a more formal part of the appraisal process.

Interviewees generally felt that the committee included a sufficient number of professional health economists on each branch. There was less agreement concerning levels of expertise in health economics amongst the broader committee. A number of interviewees indicated that they were concerned not only by their own personal lack of understanding of the economic analyses but also the level of understanding by others on the committee. If the economic analysis is to be used effectively to provide the framework for the discussion, then there is clearly a requirement that a minimum level of understanding of the analyses exists amongst committee members.

A particular issue brought up by many interviewees was the great benefit for a decision-making body such as NICE of a single measure of benefit such as the quality-adjusted life-year, in allowing comparison of very many disparate health interventions and in providing a benchmark for later decisions. Particularly in the context of model-based analyses, the importance of ensuring that committee members understand the limitations of the analysis was highlighted.

Conclusions and recommendations for further research

• •

  Research into healthcare organisational forms that can explore the alternative structures, processes and mechanisms by which technology coverage decisions can and should be made.

• •

  The further development of “resource centres” that can provide information relating to high-quality independent published analyses and are able to support decision-makers with some local re-analysis and interpretation of findings.

• •

  The development of improved methods of economic analysis that take account of the concerns raised by practitioners and users of such analyses in this research.

• •

  The design of more accessible forms of presentation of economic analyses.

• •

  Further assessment of the feasibility and value to be derived from a formal process of discussion and deliberation concerning the objectives that we seek from investments in healthcare.

Further reading

Williams, I., McIver, S., Moore, D. and Bryan, S., (2008), “The use of economic evaluations in NHS decision-making: a review and empirical investigation”, Health Technol Assess, Vol. 12 No. 7.

© 2008 Crown Copyright

About the authors

I. Williams is based at the Health Economics Facility and the Health Services Management Centre, University of Birmingham, Birmingham, UK. S. McIver is based at the Health Services Management Centre, University of Birmingham, Birmingham, UK. D. Moore is based in the Department of Public Health and Epidemiology, University of Birmingham, UK. S. Bryan is based at the Health Economics Facility, University of Birmingham, Birmingham, UK. S. Bryan is the corresponding author.