BOOK REVIEW

Citation

Calvert, P. (2001), "BOOK REVIEW", Library Hi Tech News, Vol. 18 No. 4. https://doi.org/10.1108/lhtn.2001.23918dae.005

BOOK REVIEW

BOOK REVIEW

Statistics and Performance Measures for Public Library Networked Servicesby John Carlo Bertot, Charles R. McClure and Joe RyanAmerican Library Association,Chicago, IL2001103 pp.US$38.00ISBN 0838907962 (soft)

Steadily over the last decade, the topic of performance measurement has been increasingly critical for good library management. Stakeholders expect greater accountability than was once the case, so managers must be able to deliver proof that they are delivering an effective service. It is not enough to run a good library service, you've got to be able to provide concrete evidence that it is good. Library associations, including IFLA, have responded by publishing recommended sets of performance measures, though in the context of this review it is necessary to point out that almost all of them only describe the activities of a traditional print-based library. Scholars have been active in developing new measures and the renowned Northumbria conferences allow them to share new knowledge (Bertot and McClure both spoke at the 1999 Northumbria Conference.) Somewhere along the way, electronic service provision emerged as a major part of library activity, and unfortunately most of the existing performance measures didn't cope at all well with virtual services. New research projects were started in the USA and in Europe with the intent of developing measures appropriate for a new environment in which electronic and networked functionality in the library was just as important as print, videos, and CD-ROMs. Statistics and Performance Measures for Public Library Networked Services is at once a report on a US project and a how-to manual intended to guide public library managers through the process of measuring and assessing services provided across online networks.

The authors provide five reasons why a library (with the term being used generically) should measure the use of its services and resources. It assists staff making resource allocation decisions, it provides data for peer comparisons with similar libraries, it aids with the evaluation of existing services, it helps present the nature of the library to the public, and it can improve the political climate for funding. It will be apparent to many public library managers that, while existing measures perform some or all of these tasks adequately for conventional services, they don't do them so well, or even at all, for newer services based on networked access to information. The goals of this manual are to: "establish a core of network national statistics and performance measures that describe public library use of the Internet and Web-based services and resources" (p. 2); aid librarians with the actual measurement of those services; use the resulting statistics to describe the library to various audiences; and to incorporate existing statistics into the broader picture. It is also interesting to consider what the manual is not. Quite sensibly the authors have not attempted to supplant existing measures. They do not claim to have written the last word on the subject and they expect change in networked library services to become so rapid that few measures last for long. Indeed, they make a statement that leaves one uneasy when they say, "Statistics and measures provided in this manual may have a limited shelf-life of three to five years at most" (p. 3). It also becomes apparent that the collection of precise data for some aspects of networked service is almost impossible; hence the authors make the point that estimates are only estimates, but they are better than no data at all. Naturally this will require a mind-set shift amongst managers used to the precision (or assumed precision) of statistics such as circulation and door counts.

Chapter 2 contains the "Recommended Statistics", and for each there is a definition, an indication of who should collect it, a suggested survey period, procedures for collection, any special considerations, and finally, the all-important analysis and use. The recommended statistics include: number of public access Internet workstations, number of virtual reference transactions, number of full-text titles available by subscription, number of virtual visits to networked library resources, and user information technology instruction. Each one of these statistics is part of a larger group covering the same dimension of service provision. Although the statistics seem clear enough, it becomes apparent that even such simple data are not easy to collect. The extent to which libraries will be dependent upon database vendors for information about the number of titles available, and the number of sessions (log-ins) with online databases came as a surprise. This is a new development in library measurement and only time will tell if cooperation between vendor and library will work.

The authors see the recommended statistics as raw data that still require some manipulation before they are useful to managers. Hence in chapter 3 they describe ways to transform the raw data into "composite and performance measures". The method they suggest is based on the ARL approach of calculating everything as a ratio, for example, "percentage of virtual reference transactions to total reference transactions" (p. 29). Each description of a composite measure contains a definition, method of calculation, and suggestions for analysis and use. Many of the proposed measures combine data from networked services with traditional data; and the authors acknowledge that they need to overcome a possible credibility gap, if people do not believe that the measures are comparing oranges with oranges and not lemons.

This manual goes beyond many similar sets of performance measures in the proposals for use of the resulting figures. It seems that the authors are determined not to be seen as living in an ivory tower making suggestions that practising librarians will ignore, and so they have worked with several library services and as a result propose actions that can follow the calculation of the performance measures. As an example "user information technology instruction as percentage of total reference activity" is a concern to many reference library managers struggling with the problem of limited resources and huge demand for IT instruction, so the proposed use of the measure allows for comparisons over time, and with peer libraries, and perhaps even with national data, presumably so that "best practice" will in time emerge.

The following two chapters are concerned with data collection, and give detailed advice on conducting focus groups, interviews, and surveys with customers, then on how to instruct staff to collect data accurately and consistently. Naturally the authors foresee problems with inconsistent interpretation of collection instructions and have tried hard to be specific in what they propose.

The theme of chapter 6 is the issues that require consideration at a local level before the manager selects appropriate measures. The authors acknowledge "the increasingly likely situation that the measures here proposed are not sufficient for a library's needs" (p. 62). There are a number of highly pertinent questions asked in this chapter, such as "How can you reduce the measure's potential error if you can't get exact data?", "Should you measure capacity, use, impact, or outcome?", "Are you measuring something just because you can?" and "How can the measure be used to enforce compliance, reward or sanction?" and each one is given a thorough and honest answer.

It would be easy to criticise this manual if one forgot that the authors are trying to make sense of a complex and rapidly changing field. Most of the measures count inputs or simple outputs, much as in the tradition of other library statistics. They are useful for resource allocation decisions and these are the sorts of measures that funding agencies have become accustomed to receiving from libraries. What the suggested measures don't do well is describe the customer's experience of networked services. Admittedly, measuring service quality is itself a very new topic for investigation, and Bertot spoke at the ARL's Symposium on this subject in October 2000. In an Appendix, the authors have suggested more measures that they say need further consideration before they are put forward as concrete proposals. Measures such as "Average Web page loading time", and the average search time of different databases, will tell us more about customer service than most of the measures in chapter 3.

This is a US publication designed, for the most part, for US public libraries. There are other projects developing performance measures for the electronic library environment, such as Equinox in Europe http://equinox.dcu.ie/ that should have received more credit (though Equinox hasn't suggested many measures significantly different from the US team's proposals ­ perhaps the use of cost data is the only major variation).

There is probably no need to introduce the authors. John Carlo Bertot has been investigating performance measures for electronic services for many years, Chuck McClure is already a legend in the profession, and Joe Ryan is a consultant specialising in networked services. Even though they have put many warning notes into the manual, there is enough here for public library managers to use straight off the bat. In time, the measures proposed could become the basis of a national set of performance measures. It is to be hoped that more can be added to describe the customer's experience.

Philip CalvertVictoria University of Wellington, New Zealand