Citation
Overstreet, R.E. and Hazen, B.T. (2016), "Editorial: tips for conducting high-quality reviews", The International Journal of Logistics Management, Vol. 27 No. 2. https://doi.org/10.1108/IJLM-04-2016-0109
Publisher
:Emerald Group Publishing Limited
Editorial: tips for conducting high-quality reviews
Article Type: Editorial Regular issue From: The International Journal of Logistics Management, Volume 27, Issue 2.
Introduction
Any discussion of the peer review process must begin with a sincere “thank you” to the many volunteer International Journal of Logistics Management (IJLM) reviewers. In our discipline, all journal stakeholders rely on reviewers to be the guardians of the logistics and supply chain management body of knowledge, ensuring that each and every manuscript accepted for publication makes a genuine contribution. We recognize that a well done review is a gracious contribution of the reviewer's time and expertise. It is her or his thoughtful reflection on the research that manifests in a cogent, insightful review that informs the editor, encourages the researcher, and improves the quality of the research – whether the manuscript is ultimately accepted for publication or not.
Completing a peer review of an academic manuscript is challenging, as it should be. Each manuscript may represent years of work by one or more authors who anxiously await the editorial team's decision (Carter and Ellram, 2010). One can take pride in knowing that the editorial team has made a deliberate decision to request your specific expertise in determining the publishability of the submitted manuscript. In an effort to make the mechanics of the review process somewhat less challenging, this editorial provides some general thoughts on performing the review, to include discussion of the specific areas of the IJLM peer review and some technical points regarding the review process.
Sections of the structured review
In accomplishing an IJLM review, the editorial team asks that each reviewer address six major areas in the peer review of a submitted manuscript, and provide additional feedback in the form of comments to authors and the editors. Below, each area is discussed in greater detail to provide insight into what kinds of comments can help the editorial team make a publication decision, as well as what kind of feedback may be helpful to the author(s):
1. Originality: does the paper contain new and significant information adequate to justify publication? Ensuring that an article makes a significant, new contribution is arguably the most important criterion when assessing whether or not a manuscript should be published. That said, often times we see that a piece of research indeed can make a contribution, yet the contribution is not brought out well in the article. This is where the mentorship role of a reviewer comes into play. Many authors, especially those who are more junior in the field, can use assistance in teasing out the significant contributions of the study. Even more senior scholars can sometimes overlook various contributions that the research may be able to make. Not to say that the reviewer needs to do the work for the author, but perhaps the reviewer can offer some advice as to what the real contributions of the article might be and how to better bring them to surface.For example, some (especially junior scholars) feel that if results are insignificant or opposite of the hypothesized direction, then the research has failed. Indeed, many authors may go to great lengths to explain the hypotheses that were supported while making only scant reference to any that were not supported. This may leave the most original and compelling parts of the manuscript up to the reader to decipher. While this is only one example, it provides some indication as to where seasoned reviewers can mentor as part of the review process.
Examples of an original contribution to our discipline may include (but surely are not limited to):
Bringing in theories and phenomenon from other domains. That is, our research area has ample opportunity to include significant thoughts and theories from disciplines such as economics, organizational behavior, psychology, or sociology.
Applying, when justified by the problem, new techniques that go beyond the commonly used research designs and methodologies in our discipline.
Developing new theories and ways of thinking that are specific to logistics and supply chain management.
Showing the evolution of theory and practice in consideration of innovation, new technology, new cultural norms, emerging geo-political landscapes, and other phenomena.
2. Relationship to literature: does the paper demonstrate an adequate understanding of the relevant literature in the field and cite an appropriate range of literature sources? Is any significant work ignored? Depending upon the rationality of your selection by the editorial team, you may or may not be the expert in the specific topic area. If you are, it may be readily apparent to you, whether or not the authors understand the literature and have employed it correctly. Please comment on the adequacy of the review, and include ideas from other articles (or even related topics)that should be included in a revision. If you are not an expert in the topic area, your feedback regarding the literature review is still valuable. Please determine if the literature presented builds an adequate foundation for motivating the topic, facilitates the use of theory, identifies areas where research has been done, and uncovers areas needing research (Webster and Watson, 2002) from the perspective of someone unfamiliar with the literature. If, as a non-expert in the topic, you understand the background and motivation for the paper, then this is one indication of well-done review.Regardless of the level of expertise on the topic area, below are some indicators of an effective literature review that any reviewer can use to gauge adequacy:
it is up to date;
it is well integrated into the text, rather than a simple list of who said what;
it is not biased toward a few select publications at the exclusion of other possibly relevant journals;
it shows a clear understanding of how the topic is being addressed in IJLM and related journals;
it identifies important gaps in our understanding of the phenomena being studied;
it identifies both the research and practical implications of filling the research gap;
it is systematic in nature, and represents a comprehensive understanding of the extant literature; and
it gives the reader confidence that the authors are well-versed in the topic area.
3. Methodology: is the paper's argument built on an appropriate base of theory, concepts or other ideas? Has the research or equivalent intellectual work on which the paper is based been well designed? Are the methods employed appropriate? It is not enough that the manuscript address a new and interesting problem. For it to be worthy of publication it must make a compelling case using extant literature and theory (Van de Ven, 1989). In fact, a multi-theoretical lens may be particularly interesting by more thoroughly explaining the phenomena under investigation (Dai et al., 2014). Reviewers should ensure that the methodology is a natural outflow of the problem, literature, and theory. That is, the problem should drive the method, not vice versa.
After determining if the appropriate method has been chosen, it is then time to determine if it has been used correctly. Although it was intended as a guide for authors, the article by Fawcett et al. (2014) provides authors and reviewers alike with key methodological items for which to consider. In the article, the authors provide specific tips for conceptual, qualitative, and survey research. Each method used in research evokes different seminal and updated references and manuals guiding its proper execution. We ask that reviewers who are experts in the method employed in the research ensure that authors executed the method in accordance with the most recent and relevant literature. For reviewers who are not experts in the method, we still ask that you comment on your reaction to the method employed, and whether or not it seems to make sense at face value. Again, being able to explain the method and analysis to a “non-expert” is one sign of a well-written research paper.
4. Results: are results presented clearly and analyzed appropriately? Do the conclusions adequately tie together the other elements of the paper? Whether you are an expert in the particular method or not, the results should be explained in an easily understandable format such that anyone can decipher them. For a qualitative design, authors should provide enough information that any competent researcher could replicate the study (how the data were collected, analyzed, and interpreted). Alternatively, for a quantitative design, does the author provide evidence of common measures of merit (power, sample size, etc.), reliability (Cronbach's , composite reliability), multiple forms of validity (face, construct, convergent, and divergent) (Carter and Ellram, 2010), and other commonly accepted criteria? The bottom line is to determine whether or not the reviewer believes the results and, if so, if those results serve to answer the hypotheses or research questions appropriately. Finally, the results section should simply report the results, where interpretation of results and discussion takes place in later sections.
5. Implications for research, practice and/or society: does the paper identify clearly any implications for research, practice and/or society? Does the paper bridge the gap between theory and practice? How can the research be used in practice (economic and commercial impact), in teaching, to influence public policy, and/or in research (contributing to the body of knowledge)? What is the impact upon society (influencing public attitudes, affecting quality of life)? Are these implications consistent with the findings and conclusions of the paper? The section of the paper regarding implications is where the investment made by the author in finding a significant problem, telling a compelling story using extant literature, and in the application of theory has the potential to make a significant difference in the value of the manuscript. The implications should follow logically from the results of the analysis, drawing dividends from the meticulous investment in literature and theory up front. With logistics and supply chain management being an applied discipline that is relatively new when compared to other disciplines, there is ample opportunity to generate noteworthy implications for theory, practice, and society. If sufficient implications are not made or conveyed, mentor the author through straightforward critique providing suggestions to consider.
6. Quality of communication: does the paper clearly express its case, measured against the technical language of the fields and the expected knowledge of the journal's readership? Has attention been paid to the clarity of expression and readability, such as sentence structure, jargon use, acronyms, etc.? After addressing the previous elements of the review, make an evaluation of the overall quality of communication. Please keep in mind that this is an international journal. While the journal is published in English, authors are working all over the world to make contributions to our discipline. The editorial team does not expect you to identify and recommend corrective action for every error in the document; however, it would be helpful to identify types of errors that need to be addressed throughout the manuscript. It may be appropriate to tactfully recommend that the author take advantage of professional editing services for non-native (or even for native) English-speaking authors.
Following the six specific questions, each reviewer is asked to provide additional feedback to the author. Please take this opportunity to provide more constructive feedback, and outline any other areas for improvement that are not necessarily captured in the abovementioned sections. In the comments to authors, we ask that you be most cautious, as this summarizes your direct assessment of the article. It should be straightforward without being offensive, listing the manuscript's strengths and weaknesses with specific, actionable comments (Lee, 1995). This is your opportunity to mentor new authors and/or possibly sharpen the skills of more seasoned authors. Sometimes it is appropriate to be brief, whereas sometimes it is important to provide a lengthily discussion of what needs to be corrected and why. This judgment is certainly left to the reviewer.
Finally, it is time to make an overall recommendation for the article of accept, minor revision, major revision, or reject. This should follow logically from the review and the comments to the author(s). The editor takes these recommendations seriously, and must also weigh the quality of the review in making determinations between disparate reviews. This is where it is important to take the time to write an unvarnished summary about the article as it is written, and if appropriate, the potential contributions (and likelihood of success) of a revised version. These comments are confidential to the editor and should accurately reflect the amount of time and consideration that you have invested in the manuscript.
Technical notes for reviewers
Now that we have discussed some reviewing best practices, there are some technical points about reviewing that should be mentioned:
1. First, when you are invited to review, please respond to the email immediately with your intentions. If you do not have time to make the commitment or are otherwise unavailable, that is understandable. Academics have busy and sometimes chaotic work schedules, and the editorial team understands that not all potential reviewers are available at all times. In addition, simply ignoring the request leads to increased ambiguity in the review process. In both cases (being late to respond or ignoring the request), the editorial team is unaware whether or not the invited reviewer will be available and, in turn, hesitates to invite additional reviewers. This can significantly affect review cycle times. Second, along the same line, we have heard from reviewers that they do not want to immediately accept an invitation so as to “pad their time” for the review. This is highly undesirable and leads to the same ambiguity as described above. If you would like to accept the request but know that you might not be able to meet the deadline, please be upfront with the editor that invited you to review. It is better to know that a review will be completed late than it is to not know whether or not an invited reviewer will agree. Third, next, if you are not committed to providing a substantial review, please decline the invitation. Receiving reviews where simple yes/no answers are provided, or brief comments are given is not useful to the editorial team or the author. Other unacceptable practices include poor tone and a lack of actionable feedback. Such reviews will often not be included in the decision letter, and reflect poorly on the reviewer. To this end, if you are no longer interested in reviewing for IJLM, you can delete your profile, or ask the editorial team to do so for you. Everyone makes transitions in their careers and there is a time where every academic will retire or move on to another career or calling. If it is your time to bow out, then there are no hard feelings regarding your decision, and we thank you for the contributions you made during your time as a reviewer and author.
2. Some reviewers may question why they are asked to review a certain manuscript. Having read the abstract in the review invitation, it might not seem that you are qualified to review the selected manuscript. However, please know that your expertise was carefully considered when making the decision to choose you. For instance, you might not have specific knowledge of the topic, but perhaps you were called upon for methodological advice, or perhaps vice versa. If you do not understand why you might have been chosen, it would be much appreciated if you were to simply reply back to the editor for clarification before simply dismissing the review opportunity.
3. Sometimes errors are made or it is not documented that you have just completed (or are working on) a review. There are many reasons for this, and most are functions of the manuscript management system. However, the editorial team makes mistakes as well. We will typically not ask for more than one review every six months, unless otherwise noted in your profile preferences. If you feel that you have received a review too soon or otherwise feel overtasked, please just reply to the invitation and ask to not be considered for reviewing this time around.
4. Please keep your profile up-to-date in Scholar One (the Manuscript Central submission website). Updating your email will allow you to receive all messages, review requests, and author notifications. If your expertise has changed, please reflect this as well. Also, you can update your availability to note when you might be out of town for an extended period or might otherwise be unavailable to review.
Concluding remarks
In conclusion, a good reviewer should:
be willing to invest the time and energy into a comprehensive review of the manuscript;
be objective;
be constructive when making comments to the editor and the author;
provide clear feedback to the editor that contributes to a timely decision;
provide the author with encouraging and actionable feedback; and
provide a clear expectation for the author by numbering and ordering comments such that each should be addressed in the reply to reviewer.
The intended takeaway of this editorial is that every scholar has the potential to make our discipline (and the Journal) better, both as an author and as a reviewer. Please take the time to invest in the work of others the way you hope others invest in your work. We would like to close by thanking everyone in our community for their hard work and support of one another's research.
Robert E. Overstreet and Benjamin T. Hazen
References
Carter, C.R. and Ellram, L.M. (2010), “Crafting high-quality reviews: guidelines, examples and feedback”, Journal of Supply Chain Management, Vol. 46 No. 3, pp. 3-5
Dai, J., Montabon, F.L. and Cantor, D.E. (2014), “Linking rival and stakeholder pressure to green supply management: mediating role of top management support”, Transportation Research Part E: Logistics and Transportation Review, Vol. 74, pp. 124-138
Fawcett, S.E., Waller, M.A., Miller, J.W., Schwieterman, M.A., Hazen, B.T. and Overstreet, R.E. (2014), “A trail guide to publishing success: tips on writing influential conceptual, qualitative, and survey research”, Journal of Business Logistics, Vol. 35 No. 1, pp. 1-16
Lee, A.S. (1995), “Reviewing a manuscript for publication”, Journal of Operations Management, Vol. 13 No. 1, pp. 87-92
Van de Ven, A. (1989), “Nothing is quite so practical as a good theory”, Academy of Management Review, Vol. 14 No. 4, pp. 486-489
Webster, J. and Watson, R.T. (2002), “Analyzing the past to prepare for the future: writing a literature review”, Management Information Systems Quarterly, Vol. 26 No. 2, pp. xiii-xxiii
Further reading
Kassirer, J.P. and Campion, E.W. (1994), “Peer review: crude and understudied, but indispensable”, Journal of the American Medical Association, Vol. 272 No. 2, pp. 96-97