lm10.1108/lmLibrary Management0143-5124Emerald Group Publishing Limited10.1108/lm.2002.23.8_9.449.2e-reviewReviewcat-LISCLibrary & information sciencecat-LLMLibrarianship/library managementcat-HILBHR in librariescat-LPMLibrary promotioncat-LSTRLibrary strategyBook ReviewEditorial Peer Review: Its Strengths and WeaknessesMaurice B. LineInformation and Library Consultant, Harrogate, UK01122002238/94494502002Ann C. Weller. Editorial Peer Review: Its Strengths and Weaknesses. Medford, NJ: Information Today for ASIST 2001. xv + 342 pp., ISBN: ISBN 1 57387 100 1 US$35.60 (ASIST members) US$44.50 others© MCB UP Limited2002Publishing, Quality controlpeer-reviewednoacademic-contentyesrightslinkincluded

I was especially interested to read this book. I edit a journal, of which only a proportion of articles are refereed; I have done a great deal of refereeing for other journals; I am on the editorial boards of seven journals (mostly, it appears, in an ornamental capacity); and I have written far too many articles, of which only one was rejected (it was accepted by another journal, and subsequently appeared in a volume of the year’s best LIS articles).

The author has certainly done her work thoroughly. Her search for articles was exceptionally difficult, as relevant material was widely scattered and often poorly indexed; she thinks it likely that some material has been missed, but it cannot be much. One limitation was self‐imposed: she restricted the search to material in the English language. This is very understandable, interesting though it would have been to know of differences between different languages, especially in non‐scientific subjects. Peer review is incidentally relatively recent; it was very rare in anything like its present form before the Second World War, and has become steadily more common since then. References to the subject traced by Weller grew from under five in 1950 to over 80 in 1990.

Each chapter shows the same meticulous approach, covering the literature on peer review in almost obsessional detail. After an introduction to the peer review process, “The rejected manuscript” deals mainly with what happens to rejected papers (most of them get published elsewhere) and how authors react to rejection. Rejection rates have risen slightly over the years – possibly because of more pressure to publish, but Weller does not permit herself any speculation. So close to the literature does the author stick that there is no discussion whatever of the startlingly larger rejection rate for non‐scientific journals. Various reasons could be suggested for this interesting phenomenon, among them the smaller size of such journals, a greater emphasis on good writing, and greater difficulty in producing “hard” results.

Next comes “Editors and editorial boards: who they are and what they do”. In some cases editorial boards form the refereeing panel. The matter of “ornamental” members is hardly mentioned. Rather more interesting is the chapter on “The authorship problem”, which turns out to be simply the allocation of responsibility between multiple authors (multiple authorship is something else that has grown).

The following three chapters are all concerned with reviewers – their role, the extent of agreement among them, and potential biases. They seem to be generally conscientious and altruistic, though there are rewards such as increased knowledge of a field and knowing more about the opposition (I would add the mild satisfaction of sitting in judgement on others). Agreement is not high, even in scientific fields where there would not appear to be much cause for dispute. As for biases, these can be reduced somewhat by blind refereeing (where the authors are not known to the referees); this practice and that of anonymous reporting (where the referees are not known to the authors) have both increased. As an editor, I can be fairly sure I get the result I want by using referees with known characteristics and attitudes, notably their harshness or kindness. As a referee, I am aware that the same paper can read differently according to whether I am fresh or tired, short of time or with time on my hands. In the first example, I try to achieve a balance, and in the second I try to allow for such factors. The point is that I may not succeed, and the same must apply to other editors and referees – even supposing that all of them try. There is no way of eliminating bias.

The chapter on “Peer review and statistical review” evidently deals with a matter close to the author’s heart; she is clearly very much at home with statistics herself. In medicine in particular, many submissions are deeply flawed in their statistical methods, analysis and presentation; there are startling differences between acceptance of articles before and after statistical review, which has become much more widely accepted but is not yet universal.

Most chapters conclude with comments on limitations of the studies reviewed and recommendations (e.g. for best practice), and then with some conclusions. These generally do not summarise the chapters; perhaps this was considered impossible in view of the body of disparate data. As Weller points out, many studies have serious inadequacies, and the recommendations she makes for improvement are valuable.

In all the mass of information she reviews there is much that has not been written about. Studies have tended to be restricted to one journal, or one small field, and authors have made no attempt to produce comparable data. As a result, we do not know, for example, what percentage of articles in different subjects is refereed and what is not. And there is not one example of a study of peer review in the humanities.

Electronic publishing could change matters somewhat. Several such journals already have peer review, mainly because it is a main way of achieving parity of recognition with paper journals. Papers can be exposed to public view as pre‐prints before being refereed, perhaps also in public view, or subjected to an initial review before publication and then to post‐publication reviews. These are among the models mentioned in “Peer review in an electronic environment”. Watch this space.

At the end of all this, we are not very much clearer about the virtues or otherwise of peer review as far as the journal itself is concerned. It is inevitably a highly imperfect process, but it does result in the improvement of many articles, it weeds out much chaff, and where statistical data are present it is essential. However, most editors could do most of this for most articles. What peer review does do is to add respectability to a journal, to the extent that few serious research (as opposed to news) journals now dare be without it, even though it adds to costs and delays. Weller states firmly on the last page of her final admirable chapter of “Conclusions” that “Some type of review by someone other than the author is essential” – but essential for what types of article? For a scientific study, certainly; but for an “ideas” article in the social sciences or humanities, or for a purely descriptive paper?

The work is well written, though there are some lapses (e.g. “different to”, “dissimilar from”) and quite a few misprints. There is a good index (but who would look for, e.g. the American Journal of Diseases of Children under Diseases?)

The book is, within its self‐imposed limitations, excellent. Anyone wanting information on what has been written about peer review will find virtually everything they could want here, and more. But in the end it does not amount to much more than a comprehensive literature review, superb though it is as such; there are many unanswered questions, and time and again I found myself longing for some discussion and speculation.