Assessing Reference and User Services in a Digital Age

Citation

Duckett, B. (2009), "Assessing Reference and User Services in a Digital Age", Library Review, Vol. 58 No. 4, pp. 326-327. https://doi.org/10.1108/00242530910952909

This collection of twelve essays is designed to address the lack of reported research on, and the lack of standards for, evaluating electronic reference services in libraries. The title of the book could be more specific, namely, Assessing the Provision and Use of Electronic Reference Services in Libraries in the USA. The essays are grouped into three sections: Library Case Studies and Research Results; Standards and Methods for Evaluating Virtual Reference; and Assessing Library Instruction in an Online Environment.

No overall critique or observation is provided and no instruction for front line staff. Like other titles from the Haworth stable, this is largely a collection of accounts of how individual libraries have tackled the issues in question. It is a record of efforts to plan, implement and manage e‐reference services, and to evaluate them. The book provides a number of case studies which will stimulate and inspire LIS managers.

There is advice on how public libraries can build a case for a chat service and what data should be collected; there is discussion on efforts to create an evaluation “toolkit” for use by a multi‐type library consortium providing a virtual reference project; experience of a university library entering a state‐wide e‐reference consortium and the benefits of such a cooperative arrangements; and a model for evaluating electronic reference service that can be used in either public or academic library settings. An interesting development in e‐reference is the evaluation of transcripts of the entire reference transaction and the analyses of chat transcripts feature in many of the studies reported here. One is a comparison of question categories for in‐person and online reference transactions; while another reports using the transcripts for instruction purposes. Chat transcripts also help in developing a set of quality measures for chat reference. These can also help in identifying problem areas, both subject and staff performance. Other evaluative strategies discussed are feedback questionnaires on sources used and the experiences in response to trial questions asked by students.

“Reference services” are defined to include reference (i.e. answering questions) as well as instructional activities provided online, and one such online instructional module based on user needs concluded that “students want fast, easy information – they think they are good searchers, and they confuse speed and convenience with accuracy or ease of use.” The implications for public service and web designs are discussed. There is also an analysis of the amount and type of instruction provided in online reference encounters compared to in‐person reference. Does the medium used affect the nature of the instruction? (I was sorry not to see more discussion of the use of printed resources vis à vis electronic ones.)

The editor admits that there are no standards or consensus on effectiveness, but the contributors were asked to include any “instruments” or surveys they used. These provide a useful starting point for further debate. All the essays carry abstracts and references, and the book has a composite index. The texts are well presented with clear headings.

A finding from a study of on‐line use indicates the nature of the problems we face: “In sum, student confusion was likely caused by a combination of factors, including:

• •

  A lack of understanding of different types of knowledge and publishing patterns (e.g. why a book might be a better choice than an article, or vice versa).

• •

  A lack of understanding of information formats and corresponding search tools.

• •

  A lack of knowledge of how databases are structured and searched effectively.

These barriers led to the inability to evaluate the effectiveness of a search and modify search strategies as a result.” (p. 198)

As e‐reference comes of age such findings are a challenge and this useful collection of essays signpost a number of directions to take. It is timely and useful.