Why manuscripts submitted to an international peer reviewed journal in education are rejected

Why manuscripts submitted to an international peer reviewed journal in education are rejected

Article Type: Commentary From: International Journal of Mentoring and Coaching in Education, Volume 3, Issue 2.

Introduction

Like myself, many readers of the International Journal of Mentoring and Coaching in Education (IJMCE) will have endured the heart-sinking moment of discovering that a manuscript we have submitted to a peer reviewed journal is rejected. For some of us, one of the most prominent emotions associated with submitting a manuscript, thereby putting ourselves at the mercy of journal editors and reviewers, and awaiting the outcome of their deliberations, is that of hope. Without revealing too much about my own insecurities, for me it is not so much hope that the paper will be accepted, but hope that it won’t be rejected. For some, it may be fear that their papers will be rejected. And I am aware that at least some academics avoid submitting papers to refereed journals for this reason. If our papers are rejected, it can be difficult to avoid the feeling that we as individuals are somehow being rejected by the academic community to which we (seek to) belong, peripherally or otherwise. It can affect our occupational identity and well-being, and provide a test of our resilience.

Whilst I cannot, hand on heart, claim that making a decision to reject a paper as Editor is anywhere near as painful as the experience of having one's own paper rejected, it gives me no pleasure whatsoever to have to inform numerous authors of work submitted to IJMCE that their papers cannot be accepted for publication. Sadly, at present I have to reject more manuscripts than I am able to accept for publication. In an effort to help prospective authors – at least those who take the trouble to read past issues of journals to which they are submitting their papers – to avoid the same fate, I decided that I should analyse the content of the "decision letters" and (where appropriate) associated reviews, and publish the findings. Before elaborating on how I went about this, I will provide some additional contextual information in the form of a brief account of how the IJMCE review and editorial process currently operates.

The IJMCE review and editing process

The review and editing process is managed using the ScholarOne web-based system, which supports the entire process from submission of manuscripts through to production and publication. When a paper is first submitted, it comes to the Editor-in-Chief (EiC) to undertake initial checks as to whether it is potentially suitable for publication in the journal (i.e. relevant to the aims and scope of the journal) and presented in an appropriate form to undergo peer review (e.g. it is anonymised for blind review, and includes the required structured abstract). At this stage, as EiC I will make one of three possible decisions:

* reject the manuscript, notably if it is judged not to be relevant for publication in IJMCE;

* "unsubmit" the manuscript, if it is potentially suitable for publication in IJMCE but not presented in an appropriate form to undergo peer review, and advise the corresponding author of the reason for this; or

* assign the manuscript to a "Co-Editor" (one of whom is myself!) to oversee the review and (if it successfully negotiates peer review) editing process for that particular paper.

Once a manuscript is assigned to a Co-Editor, they will invite appropriate reviewers (based on their research interests and expertise) from our wide international review panel. Each manuscript is reviewed by three independent reviewers from at least two different countries. Reviewers are asked to complete a review template (see Figure 1). When the Co-Editor has received all three reviews, s/he will examine the various recommendations and comments (which can sometimes be quite divergent), revisit the manuscript as appropriate, and make and communicate a decision to the corresponding author – potentially one of the four listed in Figure 1 but almost always one of the following:

1. accept subject to the appropriate completion of minor revisions;

2. accept subject to the appropriate completion of major revisions; or

3. reject.

Figure 1 IJMCE review template

The decision letter to the author (sent by e-mail) is accompanied by the three independent reviews. On occasion, these are edited slightly to reduce the occurrence of typographical errors or where it is felt that some of the language used by reviewers could be perceived to be over-critical and/or likely to cause offence. Where the manuscript is accepted subject to the satisfactory completion of (minor or major) revisions, authors are always invited to comment if they feel that some of the requested revisions are inappropriate, and to explain why this is the case. Where a "major revisions" decision is given, it is stressed to the author that this does not guarantee eventual publication, and that if the revised manuscript is judged to still require major revision, it will be rejected at that stage.

In the majority of cases where authors are given a "major" or "minor revisions" decision, they normally choose to revise and resubmit the manuscript[1], at which point it will return to the same Co-Editor, who will then invite the original reviewers to review the revised paper. Assuming that the revised manuscript is now judged to require only minor revisions (otherwise, as stated above, it will be rejected), the process outlined above is repeated until the authors are judged to have satisfactorily responded to the reviewer comments and suggestions: it will normally take between two and four rounds of peer review to get to this stage. At that point the Co-Editor then reads the paper in detail. If s/he feels that any substantive issues remain, the author(s) will be asked to undertake further minor revision. Once the Co-Editor is satisfied that the paper is ready (or almost ready) for publication, s/he will then invite the journal's Associate Editor to examine the manuscript. The Associate Editor pays particular attention to potential typographical and grammatical errors, and to issues of clarity of meaning, but may also notify the Co-Editor if she feels that more substantive issues may have been missed during the earlier stages of the peer review and editing process. The Co-Editor will then invite the author to undertake "final revisions" prior to giving a final "accept" (accepted for publication "as it stands") decision.

Analysing "reject" decisions

In order to systematically identify why manuscripts submitted to IJMCE were being rejected, I decided to analyse the first 25 "reject" decision letters and accompanying reviews (where applicable) that I returned to corresponding authors in my capacity as EiC. The letters were sent between April 2013 and February 2014 inclusive. Ten related to manuscripts which were rejected without recourse to peer review, and 15 to manuscripts which were rejected following peer review (12 following a single round of review, two following two rounds of review and one after four rounds of review).

All documents were imported into MAXQDA (qualitative data analysis) software and subsequently analysed using an "open coding" approach (Strauss and Corbin, 1990). Hence, a coding tree was gradually developed and refined, with new categories or "nodes" created, as data were coded. The main outcomes of this process are outlined below.

Findings

Before I detail what my analyses revealed as the reasons given for not accepting manuscripts for publication, it is important to state that rejected manuscripts are rarely considered to be without merit: on the contrary, they are often judged to have a number of strengths. The following excerpts from the decision letters sent to authors of various rejected manuscripts identify a range of different perceived strengths of the papers:

This study is timely and worthwhile.

This paper is very well-written and pleasurable to read.

The paper demonstrates an adequate awareness of the literature which is well-contextualised.

The theoretical foundations of the paper seem solid.

[…] it offers a new perspective on mentorship, which may justify publication if the new knowledge can be delineated.

Manuscripts were thus rejected on the grounds that their perceived strengths were outweighed by their perceived limitations.

Turning now to the substantive reasons for rejecting papers, my analyses revealed that there were both similarities and differences between those associated with manuscripts that were rejected immediately by the Editor ("pre-peer review") and those associated with manuscripts which were judged suitable to undergo peer review but nonetheless rejected after one or more rounds of review. I consider the two categories of manuscript/decision in turn.

Reasons for rejection pre-peer review

The factors found to be associated with the rejection of manuscripts without recourse to the peer review process are listed below, in order of frequency of coded segments across the ten papers. It is important to recognise that some of the considerations overlap (e.g. the second factor – "insufficient substance" – was often related to two or more of the more specific third, fourth, fifth, sixth and seventh factors), while some (e.g. failing to follow Instructions to Authors) were not sufficient in themselves to lead to a reject decision but were nonetheless contributory factors. On the other hand, some considerations were sufficient in themselves to cause a reject decision, most notably lack of relevance to the aims of journal and "insufficient substance", each of which are illustrated below with quotations from some of the decision letters:

(1)lack of relevance to aims of journal (9 coded segments):

[…] there is insufficient focus on mentoring and/or coaching in the paper

[…] there are only 3 references to mentoring in the entire paper (2 in the literature review on p.3, one in the Methodology on p.4)

[…] there is insufficient conceptual or empirical focus on mentoring (or coaching) in the manuscript for this particular journal, and it is felt that the manuscript would be more suitable for a higher education journal.

[…] it is not felt that it is suitable for publication in IJMCE because there is insufficient focus on mentoring or coaching. In fact, I couldn't see any explicit reference to mentoring or coaching throughout the paper.

[…] while your manuscript looks very interesting and potentially publishable, it is not suitable for publication in the International Journal of Mentoring and Coaching IN EDUCATION […]

(2)insufficient substance (4 coded segments):

[…] it seems to simply touch the surface on all the points presented.

[…] it is not felt that the manuscript is appropriate (e.g. sufficiently developed as an academic paper) to be published in the International Journal of Mentoring and Coaching in Education.

(3)lack of methodological rigour or transparency (4 coded segments);

(4)lack of engagement with papers already published in journal (3 coded segments);

(5)insufficiently grounded in relevant literature (2 coded segments);

(6)insufficient discussion of findings (2 coded segments);

(7)insufficient theorisation (2 coded segments);

(8)instructions to authors not followed (2 coded segments);

(9)lack of clarity regarding the aims of the study and/or argument of paper (1 coded segment);

(10)doesn't make original contribution to knowledge (1 coded segment); and (11) fails to address international audience (1 coded segment).

Reasons for rejection post-peer review

All 11 factors associated with rejecting manuscripts which did not make it as far as the peer review process were also associated with the rejection of manuscripts following peer review. However, there were a number of substantive differences between the two categories of rejection. First, a larger number of factors (16 in total) were identified as contributing to manuscripts being rejected post-peer review. This is probably due to the fact that three separate reviewers (and to a certain extent the Co-Editor overseeing the review process) were each digging a little more deeply into the content of the manuscripts than occurred at the earlier stage, when as EiC I was mainly concerned with establishing whether a manuscript was potentially suitable for publication in the journal and could therefore be sent out for peer review. The additional factors identified included insufficient discussion of the context and/or phenomenon under investigation and a perceived lack of clarity and/or criticality regarding key concepts. Another factor which was necessarily only identified at the post-peer review stage and was qualitatively different from all other factors contributing to reject decisions is insufficient improvement following previous reviews. The eight coded segments attached to this node related to three manuscripts that had initially been accepted subject to the satisfactory completion of revisions, where it was subsequently judged that the revised manuscript was not sufficiently improved to be worthy of publication.

The second substantive difference between the two categories of manuscript reject decisions is that the most prominent factor explaining the rejection of manuscripts pre-peer review – lack of relevance to the aims of the journal – was only given as a contributory factor to the rejection of one manuscript (by a single reviewer) post peer review. The most frequently coded factor associated with the rejection of manuscripts post-peer review was a perceived lack of methodological rigour or transparency, which attracted 48 out of a total of 218 coded segments for manuscripts rejected post peer review (i.e. 22 per cent of all coding), followed by the judgement that manuscripts were insufficiently grounded in relevant literature and/or did not include a sufficiently critical account of literature cited (36 coded segments; 17 per cent of all coding). All 16 factors which were found to contribute to post-peer review rejection are listed below, in order of frequency of coding and with illustrative quotations:

(1) lack of methodological rigour or transparency – including perceived flaws in or fitness for purpose of research design, lack of evidence to support findings claims and failure to acknowledge limitations of research (48 coded segments):

The problem with the study's methodology is that the mentees seem to have been left out. It would seem that, certainly with respect to research question number 2 ("how beneficial is peer mentoring?") and to a lesser extent with the other questions, the perspective of the student being mentored is crucial.

[…] the small sample sizes and lack of detail really limits was can be gleaned from this paper.

The methodology was poorly described

[…] There was inadequate data about the quantitative sample

[…] how it was chosen and how mixed the sample was across the colleges involved in the study. There was no information about a second questionnaire until results were reported in the qualitative data section.

(2) insufficiently grounded in relevant literature and/or lack of criticality regarding literature cited (36 coded segments):

[…] the main area missing is more comprehensive links to relevant, recent literature in the field.

Where potentially important points from the literature are being used they are not always critically explored as to relevance to the enquiry nor problematised.

(3) quality of communication (23 coded segments):

[…]this paper is weak in its logical connection from one paragraph to another

[…]The paper, as reviewed, is poorly written and would benefit greatly from some solid proofreading and a re-write. There are numerous typos on every page as well as problematical grammatical and syntax errors throughout, the author (s) may wish to reflect upon the overall structure of the paper, and consider adopting a strategy that presents the material in a logical and systematic manner.

(4) conclusions and/or implications not sufficiently developed and/or consistent with findings presented (20 coded segments):

Potentially there is a real benefit in this paper, however as it stands the conclusions drawn are not clearly substantiated.

Implications are minimal and focus only on how one might examine one's language in the context of a mentoring situation. No implications for teacher education are addressed.

The conclusions arising have a tenuous link and insufficient link with the way data appears to have been collected, analysed and the results actually given.

(5) doesn't make original contribution to knowledge, or insufficiently clear what this contribution is (17 coded segments):

[…] this manuscript fails to offer any new significant insights into coaching.

The author needs to make his/her contributions to scholarship and to practice clearer.

(6) insufficient discussion of context and/or phenomenon under investigation (16 coded segments):

The context of the research was limited. I wanted to know more about

[…] teacher education program within which the research took place.

What is missing is a qualitative description of what actually took place in the mentoring processes.

(7) lack of clarity and/or criticality regarding key concepts (13 coded segments):

[…] "mentoring" itself is taken for granted really – no definition given.

The other main criticism is that the paper is full of education jargon and many of the terms and phrases are problematic and/or contested and so need to be treated far more carefully.

(8) lack of clarity regarding rationale for and/or argument of paper and/or aims of study (9 coded segments):

All in all I don't understand the rationale for the paper. It's a think piece that doesn't seem to have been thought through very carefully.

[…] at the end of the paper I was still not completely clear if your focus was on mentoring beginning teachers or established teachers or both.

(9) insufficient substance (9 coded segments):

Because of the lack of exploration of focused literature the enquiry questions are rather simplistic and too general. They are not focused enough on detailed exploration of specific aspects of for example the mentoring relationship which might be unique to this specific programme in question.

The whole argument is rather flimsy.

(10) insufficient improvement following previous review(s) (8 coded segments):

[…] the reviewers do feel that the paper had been improved from the initial submission, but despite this do not feel that it is ready for publication or that further revisions would result in an acceptable paper.

Unfortunately, much as I hoped I would be able to support publication of your article I am unable to recommend it as you have not undertaken a major revision the editor asked for

[…] your manuscript has not been sufficiently improved to merit publication […].

(11) instructions to authors not followed and other unhelpful and/or careless presentation (6 coded segments):

In relation to blind review, generally the year and title of the work as well as the author's name should be omitted. Otherwise, an internet search could easily identify you

Many references cited in the paper are not listed in the reference list so it is difficult to determine the actual sources. (12) insufficient theorisation (3 coded segments):The discussion is not sufficiently informed by relevant theoretical frameworks.

There is no mention of theory or other foundation inquiry system. This is the major weakness of the article.

(13) fails to address international audience (2 coded segments):

It appears to have been written for a local, US, audience rather than an international one, acronyms and details of setting are not always fully explained.

(14) over-generalisation and insufficient precision (2 coded segments):

The discussion of literature on evaluating mentoring begins to bring together literature from outside the educational sphere on this issue with educational mentoring literature

[…] However, it is not made clear that these literatures are distinct. So, for example, you mention the International Mentoring Association, but without explanation that this is a general rather than educational specific association.

(15) lack of engagement with IJMCE readership/papers already published in the journal (2 coded segments):

[…] the paper does not sufficiently engage with existing issues and debates associated with papers previously published in the journal.

(16) lack of relevance to aims of journal (1 coded segment):

I think this work would be more fitting for publication in an action research oriented journal.

Conclusion

Since I would normally expect an author of a research paper to acknowledge some of the limitations of the work being discussed, I feel it is important to do the same here. The limitations of the analyses I have undertaken caution against the conclusion that I have presented an exhaustive account of the reasons why some manuscripts submitted to IJMCE are, have been, or will in the future be rejected. For example, the study only covered a specified (11 month) period during which I was EiC, so cannot tell us anything about papers rejected outside of this period; while I am solely responsible for the analyses another researcher would doubtless have interpreted some of the data differently and produced different "findings". Despite this, I hope the account I have provided will be of interest to readers and helpful to prospective authors in preparing manuscripts for submission to IJMCE.

It is also important to highlight that reviewers' and editors' reasons for rejecting papers – and certainly the reasons that are documented in the peer review paper-trail – will be influenced by the review template in use. Hence, should additional questions be introduced into the review template (e.g. a more explicit emphasis on whether the research reported had been conducted in an ethical manner), or others dropped, we would be likely to see subtle changes in the reasons given for rejection. This, and the fact that my analyses relate to a single peer reviewed journal (where other journals use different templates), means that we cannot confidently assume that the reasons identified for rejecting manuscripts submitted to IJMCE will have wider applicability. Nonetheless, to the extent that some of the identified reasons for rejection may also apply elsewhere, this Commentary might potentially inform the development of manuscripts for submission to other peer reviewed journals in education and perhaps even further afield. With this in mind, following the completion of my analyses, I sought to identify other sources of information relating to why papers are rejected. The most pertinent was an Editorial published in the journal Educational Action Research (2012), which identified the following reasons for the rejection of papers from that journal:

* We seem to get a large number of articles that are not about action research at all. They may be about practice, or even written by a practitioner, but they do not fit any of the traditions of action research which hold action and research together […]

* We expect articles that come to us not only to locate the work as action research, but to identify how they understand that and how it is embodied or enacted in the work about which they are writing […]

* Sometimes we are sent articles that are purely descriptive. They do not advance the thinking and practice of the community that the journal serves.

* Sometimes we are sent articles which present work on a topic as if no one else has ever written about it […] [I]t appears as if the writer has no sense of a community of other action researchers building knowledge, understanding and practice. There seems to be no sense of accumulation of knowledge, no commitment to furthering debates, and no acknowledgement of what has gone before […]

Finally, authors do not always write in a way that acknowledges the broad international readership of Educational Action Research. Sometimes articles are sent that make reference to local contexts or use technical terms without explaining them to an international readership (Educational Action Research, 2012, pp. 481-482; emphasis added).

The specifics differ, as one would expect, yet the general points raised here are all similar to ones identified in my own analyses. In conclusion, contributors to IJMCE and perhaps other journals too might use the reasons identified for rejection in this Commentary as a checklist to try to ensure that they avoid the same pitfalls with respect to their own manuscripts. One of the ways of avoiding some such pitfalls was highlighted by the Editors of Educational Action Research (2012, p. 482), who urged potential authors, especially those new to that particular community, "to read some past issues to get a sense of the interests, debates and gaps that there inevitably are in the field".

Andrew J. Hobson

Education Research Centre, University of Brighton, Brighton, UK

Note

Authors who for whatever reason feel unhappy with the outcome of the review process, are unable or unwilling to carry out the requested revisions, or unwilling to explain why they consider requested revisions to be inappropriate, are free to withdraw the manuscript. This has only occurred on one occasion during my first 12 months as Editor.

Acknowledgements

The author thanks Pat Ashby, Angi Malderez, Mark Boylan and Andy Townsend.

References

Educational Action Research (2012), "Editorial", Educational Action Research, Vol. 20 No. 4, pp. 481-482

Strauss, A. and Corbin, J.M. (1990), Basics of Qualitative Research: Grounded Theory Procedures and Techniques, Sage, Newbury Park, CA

Further reading

Murray, R. (2005), Writing for Academic Journals, Open University Press, Maidenhead

Thomson, P. and Kamler, B. (2012), Writing for Peer Reviewed Journals: Strategies for Getting Published, Routledge, London and New York, NY