Why we need a taxonomy of publishing impact

Citation

Gorman, G.E. (2014), "Why we need a taxonomy of publishing impact", Online Information Review, Vol. 38 No. 4. https://doi.org/10.1108/OIR-05-2014-0110

Why we need a taxonomy of publishing impact

Article Type: Editorial From: Online Information Review, Volume 38, Issue 4.

Working hard for a crowning achievement like being published in a high-impact journal is fine. The key is to keep some perspective. Realise that publishing in a second-tier or open-source journal is something to be proud of and realise that you can always publish in the future, from industry or otherwise (Hankel, 2014).

Sound advice indeed from Isaiah Hankel, but advice that so few seem to heed. We might say that the arcane academic game of Slippery-Ladder-of-Success, the embedded tradition of high Impact Factor (IF) journals and the Byzantine evolution of university evaluation criteria are pretty powerful barriers to taking an iconoclastic view of journal “quality” and the publishing game as we define it today.

Look at how we induct new and emerging researchers into the mysterious world of academic journal publishing: experienced academics/editors like myself offer workshops on “getting published in quality journals”, publishers churn out standard guidelines on how to be a successful author (in their journals), books and articles (hundreds of them) are replete with advice on breaking into publication …. In nearly all of these activities our concern is helping researchers get through the peer review process and have a paper accepted in a “high impact” journal, without regard for the quality or value of that paper. As Rob Kitchin (2014) reflects, “if a piece got published it is assumed it must have some inherent value”.

We need to be taking a much cleverer approach: educate those around us on how to write with impact in mind, and seek to provide them with a deeper, less mechanistic, understanding of the actual variables that determine journal quality. In helping us focus our minds more rigorously on this issue, Kitchin (2014) asks a simple yet profound question: “[…] what is required to produce work that editors are delighted to accept and readers find so useful that they want to cite in their own work?”

This is an important question for two reasons:

A meaningful answer will assist researchers in the present system to produce papers that we as editors genuinely want to publish, that our readers want to read and that just might make a difference by widening our understanding of whatever phenomenon is being addressed.

Part of the discussion around this question might give us pause for thought as we begin to critique more thoroughly new alternatives to traditional means of publishing " and here I am thinking of Open Access in particular (popular for many practical reasons, but not necessarily well understood as a key component in the new publishing landscape). We need to consciously control these emerging alternatives so that we can avoid inventing something as flawed as the present system.

Enter Rob Kitchin (2014) and his proposed taxonomy of publishing impact. Using the example of a paper on airport security, he argues that papers generally fall into one of four categories or levels, with the paper’s impact rising as one moves across the four levels. In Figure 1 academic quality, wider readership and impact run from left to right.

The full discussion of this taxonomy is available at: www.nuim.ie/progcity/2014/04/writing-for-impact-how-to-craft-papers-that-will-be-cited/. Here we can only summarise the salient points as a way of encouraging deeper discussion and debate, and to foreshadow a Call for Papers in this journal on the theme of Publishing Futures in the Online Environment.

Kitchin describes a Level 1 paper as “basically empiricist in nature”, making little use of theory, essentially descriptive and adding little to our knowledge of how airport security (or any other topic) works. “Generally, such papers appear in trade magazines or national level journals and are rarely cited”.

A Level 2 paper employs established theory to help make sense of a phenomenon. Here one draws on established theory to help cast new light on a phenomenon. “I am not, however, providing new theoretical or methodological tools but drawing on established ones”. This piece again does not extend our boundaries of knowledge very much and “might be accepted in a low-ranking international journal”.

At Level 3 we begin to see more groundbreaking work. A Level 3 paper “extends/reworks established theory” to help us make sense of a phenomenon. This paper would rework an existing theory “to provide new theoretical tools that others can apply to their own case material”. Because such a paper has wider utility, it will be accepted in “high-ranking international journals”.

A Level 4 paper studies a phenomenon “to rethink a meta-concept or proposes a radically reworked or new theory”. Such new thinking tends to be well cited because it can be applied in many contexts beyond that of the original paper. “Writing a Level 4 piece requires a huge amount of skill, knowledge and insight, which is why so few academics work and publish at this level. Such pieces will be accepted into the very top ranked journals”.

This taxonomy reflects my experience as an editor of five journals over several decades. But do not take my word for it: how often have you cited a Level 1 or Level 2 paper in your own work? In Kitchin’s opinion:

[…] the majority of papers being published in international journals are Level 2/borderline 3 with some minor extension/reworking that has limited utility beyond making sense of a specific phenomena, or Level 3/borderline 2 with narrow, timid or uninspiring extension/reworking that constrains the paper’s broader appeal.

Many fewer Level 3 papers with utility beyond the paper’s focus are published, and Level 4 papers “that really push the boundaries of thought and praxis are relatively rare”.

To test the validity of his views, Kitchin reflects on his experience as an Editor (three journals). He finds that “the relationship between level and citation generally holds " those papers that push boundaries and provide new thinking tend to be better cited”. Again, look at your own work and your own reading: how many really exceptional papers have you read in your career? And haven’t they invariably opened your mind to unexpected, new ways of thinking, and changed your own views of whatever the topic?

On so many levels Kitchin’s insights resonate with our academic experience and untested intuition; we should think carefully on this taxonomy as we plot our path into the publishing future.

G.E. Gorman

References

Hankel, I. (2014), “Five things successful PhD students refuse to do”, The Guardian, Higher Education Network, 2 May, available at: www.theguardian.com/higher-education-network/blog/2014/may/02/five-things-successful-phd-students-refuse-to-do?CMP=new_1194 (accessed 13 May 2014)

Kitchin, R. (2014), “Writing for impact: how to craft papers that will be cited”, The Programmable City, 23 April, available at: www.nuim.ie/progcity/2014/04/writing-for-impact-how-to-craft-papers-that-will-be-cited/ (accessed 27 April 2014)