Contamination by citations: references to predatory journals in the peer-reviewed marketing literature

Abstract

Purpose

Predatory publishing is a growing and global issue infecting all scientific domains. Predatory publishers create counterfeit, not (properly) peer-reviewed journals to exploit the open access (OA) model in which the author pays. The plethora of predatory marketing journals along with the sophisticated deceptive practices of their publishers may create total confusion. One of the many highly likely risks of that bewilderment is when peer-reviewed, prestigious marketing journals cite these pseudo-marketing journals. This phenomenon is called citation contamination. This study aims to investigate the extent of citation contamination in the peer-reviewed marketing literature.

Design/methodology/approach

Using Google Scholar as a citation gathering tool, this study investigates references to four predatory marketing journals in 68 peer-reviewed marketing journals listed in the 2018 version of the Academic Journal Guide by the Chartered Association of Business Schools (CABSs).

Findings

Results indicate that 59 of the 68 CABS-ranked peer-reviewed marketing journals were, up to late January 2021, contaminated by at least one of the four sampled predatory journals. Together, these four pseudo-journals received (at least) 605 citations. Findings from nonparametric statistical procedures show that citation contamination occurred irrespective of the age of a journal or its 2019 Journal Impact Factor (JIF). They also point out that citation contamination happened independently from the fact that a journal is recognized by Clarivate Analytics or not.

Research limitations/implications

This study investigated citations to only four predatory marketing journals in only 68 CABS-listed peer-reviewed marketing journals.

Practical implications

These findings should sound an alarm to the entire marketing community (including academics and practitioners). To counteract citation contamination, recommendations are provided for researchers, practitioners, journal editors and academic and professional associations.

Originality/value

This study is the first to offer a systematic assessment of references to predatory journals in the peer-reviewed marketing literature.

Citation

Moussa, S. (2021), "Contamination by citations: references to predatory journals in the peer-reviewed marketing literature", South Asian Journal of Marketing, Vol. 2 No. 1, pp. 5-27. https://doi.org/10.1108/SAJM-02-2021-0021

1. Introduction

1.1 Peer-reviewed vs predatory journals in marketing

Peer-reviewed journals are of paramount importance both to marketing theory and practice. As Baumgartner and Pieters (2003, p. 213) indicate, they are the “primary medium to communicate scholarly knowledge in marketing”. In their article on marketing academics' perception of the peer-review process, Bailey et al. (2012, p. 263) wrote, “peer review is central to the creation and dissemination of new knowledge in the marketing discipline and is the screening device used to identify research that is worthy of publication”. Discussing the importance of the review process in marketing, Lehmann and Winer (2017, p. 589) stated that a “strong, effective review process helps ensure that papers that are accepted do not have fatal flaws”.

Peer-reviewed journals—like the Journal of Retailing, which is often alluded to as the earliest marketing journal (having been incepted in 1925), the Journal of Marketing (1936), Journal of Marketing Research (1964) and Journal of Consumer Research (1973)—have played a key role in shaping marketing (both practice and academia) as it is known today (Baumgartner and Pieters, 2003; Lehmann, 2005; Wilkie and Moore, 2003). Several other marketing journals were launched during the 1980s (e.g. Marketing Science in 1982), the 1990s (e.g. Journal of Consumer Psychology in 1992) and the 2000s (e.g. Marketing Theory in 2001). As Hult et al. (2009, p. 1) state, “[e]ach journal fills a role in a marketplace where ‘publish or perish’ in journals is increasingly becoming a norm”. Today, the “market” of marketing journals is extremely broad, with Cabells' Journalytics (the subscription-based service by Cabells' Scholarly Analytics) listing 965 publication venues that are associated with marketing in their aims and scope (Moussa and Linacre, 2020).

Those who have been monitoring the market of marketing journals for the last few years have almost certainly noticed that a new type of journals has tapped into it: predatory journals (Moussa, 2021a, b; Moussa and Linacre, 2020). Predatory journals are “entities that prioritize self-interest at the expense of scholarship and are characterized by false or misleading information, deviation from best editorial/publication practices, lack of transparency, and/or use of aggressive and indiscriminate solicitation practices” (Grudniewicz et al., 2019, p. 211). They “are money-making stations characterized by charging publication fees and an absence of ‘true’ peer review” (Memon, 2018a, p. 146). To put it briefly, predatory journals are open access (OA) journals that publish not (or poorly) peer-reviewed articles for fees (commonly known as article processing charges or APCs for short). According to Moussa and Linacre (2020), Cabells' Predatory Reports included, up to November 2020, some 140 predatory journals related to marketing. Combining data from Cabells' Journalytics and Predatory Reports, the size of the market of marketing journals is 1,105 publication outlets, and 12.699% of them are predatory.

Predatory journals are a threat to the credibility and the integrity of any scientific domain. “By faking or neglecting peer review, they pollute the scholarly record with fringe or junk science” (Beall, 2016a, p. 326). Predatory publishing imposes two significant threats on marketing: The first threat posed by this phenomenon is that not (really) peer-reviewed manuscripts that are available without a subscription wall to anyone and anywhere become a source of “marketing knowledge” for managers and policymakers. The second threat is that not (truly) peer-reviewed articles published in OA become a source of “marketing knowledge” for new studies, hypotheses, frameworks, models, meta-analyzes and findings that can be used to attack the validity of both previous and future marketing research.

Despite the relatively large number of predatory marketing journals (i.e. 140 journals) and the gravity of the damages that they may cause to marketing knowledge, predatory publishing has received scant—read insignificant— interest within marketing. To the best of the author's knowledge, predatory publishing was evoked in only one article that appeared in a marketing journal (i.e. Linacre et al., 2019).

1.2 Citation contamination: a definition

According to Smith (1981, p. 83), a citation is “the acknowledgment that one document receives from another”. In general, a citation implies a relationship between a part or the whole of the cited document and a part or the whole of the citing document (Smith, 1981). Leydesdorff (1999, p. 8) proposes a similar definition according to which citations “are references to another textual element, from the perspective of the citing article”. To have citations, there must be a citing-cited pair.

A citation, Zinkhan (2004, p. 370) said, “is an indicator that a published article is not being ‘lost’ or ‘overlooked’ and that it is having some impact on the field and the work of future authors”. In their influential article on marketing journals structural influence, Baumgartner and Pieters (2003, p. 125) indicate that “[i]f a journal publishes an article that is cited by articles in other journals, it contributes to the exchange of ideas in a field of inquiry and is thus considered influential”. This statement is sound on one condition: the cited-citing pairs are composed of peer-reviewed journals. When the cited journal is a predatory journal and the citing journal is a peer-reviewed one, we must no longer speak about impact or influence but rather about citation contamination.

Citation contamination is a term coined by Rick Anderson, a university librarian at Brigham Young University in Utah (in the United States [US]). According to Anderson (2019), citation contamination occurs when “articles published in predatory journals are […] cited in the legitimate scientific literature”.

Citation contamination is a phenomenon that has been lately garnering great attention in science, technology and medicine (see, e.g. Akça and Akbulut, 2021; Anderson, 2019; Björk et al., 2020; Oermann et al., 2019; Severin and Low, 2019). Citation contamination is a serious science integrity issue that should warrant the attention of researchers in every scientific domain or discipline. To the best of the author's knowledge, only Moussa (2021a) has dealt with citation contamination in the marketing discipline.

Partially entitled “citation contagion”, the recent study by Moussa (2021a) is perhaps the first to provide solid evidence that some predatory marketing journals are being (extensively) cited, with some of them receiving thousands of citations from various sources, a few peer-reviewed marketing journals included (see Moussa, 2021a, p. 500). Though it provides hard proof that some marketing journals are contaminated by citations to predatory journals, the study by Moussa (2021a) remains descriptive in nature and mainly focused on the predatory marketing journals, not the peer-reviewed marketing journals. Furthermore, it uses Google Scholar as the citation source. However, the scholarly search engine by Google covers “all output regardless of whether or not it is peer-reviewed” (Halevi et al., 2017, p. 825). Stated differently, the study by Moussa (2021a) does not offer a systematic assessment of citation contamination in a large sample of peer-reviewed marketing journals.

1.3 Aims and research questions

The study at hand has two main aims: The first aim is to assess the contamination level that predatory publishing has caused to high-quality, peer-reviewed marketing journals. The second aim is to propose and test a set of hypotheses that may explain why some peer-reviewed marketing journals are more contaminated by predatory journals than others.

Within this study, predatory publishing is represented by four predatory marketing journals. As for the peer-reviewed marketing literature, it is represented by the 68 marketing journals included in the 2018 version of the Academic Journal Guide by the Chartered Association of Business Schools (CABSs).

To put it more clearly, this study tries to provide answers to the following research questions (RQs):

RQ1.

Which of the 68 CABS-listed marketing journals have (not) been so far contaminated by these four predatory journals?

RQ2.

Which are the most (vs least) contaminated CABS-listed marketing journals?

RQ3.

What are the features of the most (vs least) contaminated journals?

RQ4.

Which marketing subfield its journals are more contaminated than others?

RQ5.

Why some journals are less (vs more) contaminated than others?

It must be mentioned at the forefront that the author's intent is not to “name and shame” those journals that cite predatory journals. The author believes that before proposing solutions that prevent and ultimately eradicate the phenomenon of citation contamination, we (marketing academics and practitioners) must first diagnose it and investigate it thoroughly.

By answering the five aforesaid RQs, this study, along with its findings, might be extremely helpful for assessing the level and widespread of citation contamination in the peer-reviewed marketing literature. The author of this paper is not aware of any published marketing study that has undertaken such an endeavor. Hence, this study could hold several new and original insights for marketing theory and practice.

The rest of this paper is organized as follows. The next section offers a brief background on predatory publishing and the features of a predatory journal. The hypotheses are developed in section 2.3. The citation gathering procedure and data collection are described in section 3. Results are presented in section 4. A discussion of the findings is provided in section 5. Recommendations are listed in section 6. Limitations, as well as future research directions, are enumerated in section 7. The section 8 and final section is the conclusion.

2. Literature review

2.1 Predatory publishing

In 2010, Jeffrey Beall, an academic librarian at the University of Colorado in Denver (US), posted on his blog a list of “potential, possible, or probable predatory” journals and publishers. “Predatory publishing” is a term he coined to describe entities that

[P]ublish counterfeit journals to exploit the open-access model in which the author pays. These predatory publishers are dishonest and lack transparency. They aim to dupe researchers, especially those inexperienced in scholarly communication (Beall, 2012a, p. 179).

Beall's list has been controversial and was fervently castigated by the predatory publishers Beall debunked and other badly behaving scientists masquerading as advocates of the OA movement. In February 2013, a Canadian OA publisher, which publishes 42 journals including a marketing journal (hereinafter named Journal A), sent a letter to Beall stating that the inclusion of its company on his list amounted to defamation and libel. The letter also stated that if Beall did not remove the company from his list, it would subject him to “civil action” (Flaherty, 2013). Other legal threats were sent to Beall by several OA publishers, including the most questionable and the most profitable one (i.e. OMICS International) (Linacre et al., 2019; Yadav, 2018). After being placed under heavy pressure from his employer (i.e. the University of Colorado) and receiving several legal threats, Beall decided to remove his list from his blog on 15 January 2017 (Strielkowski, 2017).

The same year (in June 2017), Cabells' Scholarly Analytics seized the occasion and filled the vacuum left by the shutdown of Beall's list and began its fee-walled “Journal Blacklist” (which is currently known as Cabells' Predatory Reports) (Linacre et al., 2019). New free-to-access, active and up-to-date lists of predatory publishers were also created. These include the Dolos list (https://www.professeur-alexandre-georges.info/dolos-list), the Kscien list (http://kscien.org/predatory.php?id=1) and the Stop Predatory Journals list (https://predatoryjournals.com/publishers/), to name but a few.

With the ever-increasing number of predatory publishers and journals, the Committee on Publication Ethics (COPE) drafted, in November 2019, a discussion document simply entitled “Predatory Publishing” (COPE, 2019). In that document, the COPE defined predatory publishing as “the systematic for-profit publication of purportedly scholarly content (in journals and articles, monographs, books, or conference proceedings) in a deceptive or fraudulent way and without any regard for quality assurance” (COPE, 2019, p. 3).

2.2 The main features of a predatory journal

According to Manca et al. (2016, p. 161), we should “consider the predatory publishing phenomenon as an infectious disease”. Like an infectious disease characterized by specific and detectable clinical symptoms, predatory journals have many telltale signs. For instance, Beall (2012b) originally used an inventory of 25 predatory practices (classified into five groups) and some less fraudulent practices as the basis for his now-defunct list. Dadkhah and Bianciardi (2016, p. 2) have proposed a metric, called the “Predatory Rate”, to assess the “degree of dangerousness” of a predatory journal, which is based on 14 criteria. The COPE (2019, pp. 6–7) has issued a list of 16 warning signs of predatory journals.

The most distinctive feature of predatory journals is the fact that “they accept all submitted papers and pretend to have a peer review process” (Memon, 2019, p. 3). The COPE indicates that “the main hallmark of predatory publishing is simply that there is no or minimal quality control over the scholarly material in the publications (akin to practices of ‘vanity presses')” (COPE, 2019, p. 5). Predatory publishers, the COPE states, “are either silent about peer review or make false claims that the journal is peer-reviewed” (COPE, 2019, p. 5). Predatory journals are fee-gouging entities lacking “an appropriate peer-review process” (Cukier et al., 2020, p. 2) and “promising a short or ultra-short review cycle ranging from a few days to a few weeks” (Memon, 2018b, p. 1619). For instance, a predatory marketing journal (hereafter named Journal B)—published by one subsidiary of the (in)famous predatory publisher OMICS International—claims an acceptance rate of 30%. An examination of one of Journal B's latest issues (i.e. issue four of volume 24) reveals that it includes five articles in which the same researcher is the author or the coauthor. In total, four other articles (co)authored by that same scholar appear in a subsequent and more recent issue of Journal B (i.e. issue one of volume 25). Such a finding casts serious doubts not solely on the veracity of the advertised acceptance rate but also on the very existence of a peer-reviewing process for that journal. For readers' information, Journal B promises prospective authors a “21 days rapid review process with international peer-review standards”. Ironically, Journal B is listed in the Australian Business Deans Council's (ABDC) Journal Quality List for 2019 as a “B” journal (i.e. a “well-regarded” journal).

Journal C is a marketing journal published by an OA publisher that claims to be a US-based business research institute. Published volumes of Journal C abound with articles written by the same author(s). Journal C claims nonetheless to have a 60 to 90-day peer review cycle. The content of one particular volume of Journal C (i.e. volume 19) suggests the contrary. It contains three articles written by the same group of authors. If the peer review process for Journal C lasts as it is claimed 60–90 days and assuming that these three articles were submitted in quick succession (say with one-month laps), many months should have passed before the peer review and publication processes were complete, making it unlikely that they will overlap in the same journal volume.

Predatory journals are deceptive. Some of them falsely claim to be indexed in the Directory of Open Access Journals (DOAJ)—which is a community-curated online directory whose stated mission is to index and provide access to high-quality, OA, peer-reviewed journals (https://doaj.org/)— and have DOAJ's logo on their websites (Moussa, 2021a). The DOAJ provides, of course, a list of these deceptive journals (see https://docs.google.com/spreadsheets/d/1Y_Sza4rPDkf-NNX9kwiErGrKeNTM75md9B63A_gVpaQ/edit#gid=0). On that list, one can find a supposedly “British” marketing journal (henceforth labeled Journal D).

A predatory publisher may “appoint” an eminent scholar, without his/her knowledge or permission, as the editor-in-chief of one (or more) of its journals (Eriksson and Helgesson, 2017). Back to the case of Journal B, that journal's ongoing editor-in-chief is simultaneously the editor-in-chief of an accounting and finance journal and the coeditor-in-chief of a strategic management journal (the three of which are published by the same predatory publisher). Hence, the editor-in-chief of Journal B is also the (co)editor-in-chief of two other journals with different subjects including accounting, finance and strategic management. The author of this paper has tried to contact that multi-editor-in-chief (on 3 January 2021) but in vain.

Predatory publishers may also use fake nationalities and fake addresses for their “offices” while being located thousands of miles away in another continent (Moussa and Linacre, 2020). The publisher of Journal D publishes 82 journals, Journal D included. This is a quite remarkable feature since that publisher lists a small terraced house in Gillingham (the United Kingdom [UK]) as its main “headquarters”. This publisher claims to have its headquarters in the UK, yet the Whois website (https://www.whois.com/whois)—which provides information regarding the registrar of an Internet domain name, its date of registration and expiration, its name server and the registrant contact—indicates that the registrant contact of the Internet domain name used by that publisher is located in the US. Similarly, the publisher of Journal C claims to be based in the US. The Whois website shows, however, that the registrant contact of the Internet domain name used by that publisher is in Toronto, Canada.

The COPE stresses that a “further possible sign of a predatory publisher is if the owner or publisher is also the editor-in-chief” (COPE, 2019, p. 5). The founder/executive director of the allegedly US-based OA publisher is also the editor-in-chief of Journal C.

Predatory publishers send unsolicited or bulk spam emails inviting applications for editor/reviewer positions and/or manuscript submissions (i.e. call for papers) (Memon, 2018b). On two occasions (the first on 2 November 2011 and the second on 27 March 2013), the then editor-in-chief of Journal A has sent the author of this paper an invitation both to submit a manuscript and to become a member of that journal's review board. On 26 June 2016, the editorial assistant of Journal B has sent the author of this paper an email inviting him to submit his work before 26 July 2016 “so that it can be processed for publication in [that] journal's upcoming issue”. None of these three emails mentioned that there is an APC that has to be paid. The verbatim of these three spam emails is available upon request.

According to the COPE, predatory journals and publishers have “odd organizational mailing addresses such as post office (PO) Boxes” (COPE, 2019, p. 5; boldface in original). For instance, the publisher of Journal C uses three different addresses: the first one is termed “mailing address,” and it is a mail-drop address (i.e. a PO Box) in Ponte Vedra Beach, Florida (US); the second address, used for those wishing to mail bank checks made payable to that publisher, is a PO Box in Jacksonville, Florida; the third address, which is called a “business address”, corresponds to a privately owned, 2,950ft² black-tile-roofed residence in Ponte Vedra, Florida. Likewise, the publisher of Journal A seems to share its “offices” (Suite# 301) with at least two other Toronto-based OA publishers and two banking institutions. For the sake of information, these five entities have the same director/officer, a given W.Z.

As Jalalian and Mahboubi (2014) point out, almost anyone who has even minimal knowledge of how to design a website can claim to be a publisher and create a fleet of OA journals using open-source Content Management Systems (like Open Journal Systems by Public Knowledge Project).

2.3 Citation contamination as a journal quality issue: hypothesis development

For Nicholson (2017, p. 55), “citing articles from predatory journals in legitimate journals ‘muddies the water' of future science”. High-quality, “respectable journals should not cite articles from predatory journals” (Beall, 2016a, p. 326). Citation contamination could be thought of as a quality issue for the citing peer-reviewed journals. A high-quality journal is normally less prone to be infected by citations to predatory journals than a moderate or low-quality journal.

The “quality” of scholarly journals is a concept that has been the subject of a deep divide in marketing. Some believe that it is a psychological and socially constructed concept (Fuat Firat, 2010). Hence, they defend perception and opinion-based rankings of marketing journals (see, e.g. Hult et al., 2009, p. 1). Others adhere to the idea that citation-based rankings are more objective and reliable (e.g. Baumgartner and Pieters, 2003). The premise being that high-quality journals tend to be more cited than medium or low-quality journals. In this study, the quality of a marketing journal is viewed as a multidimensional construct. To measure it, one must take into account citation-based metrics, opinion-based rankings and other characteristics of a journal.

Most marketing scholars (over) trust and use citation-based metrics, like Clarivate Analytics' Journal Impact Factor (JIF), as proxies to journals' quality. As Dowling (2014, p. 283) stresses, many are those who believe that “the impact factor is a proxy measure of the journal's academic status”. For some other marketing scholars, like Stewart (2007), “[c]itation counts of various types, including the JIF […] are at best imperfect measures of influence, and such measures are subject to active manipulation by some journals” (Stewart, 2007, p. 4). Assuming that the JIF is a valid measure of the “quality” of a scholarly journal, it is expected that journals with high JIFs are less contaminated by citations to predatory journals than journals with low JIFs.

H1.

The higher the JIF of a journal is, the lower it cites predatory journals.

Quality lists of marketing journals, such as the CABS' Guide, are based on the opinion of a panel of experts. As the CABS (2018, p. 4) states, the Guide “reflects the outcomes of consultation carried out by the Scientific Committee of subject experts with expert peers and scholarly associations as to the relative standing of journals in each subject area. As a consequence, there is no mechanistic metrics-based formula that will capture the published ratings”. The editors of the Guide assert that it “is intended to give both emerging and established scholars greater clarity as to which journals to aim for, and where the best work in their field tends to be clustered” (CABS, 2018, p. 4). In the CABS' Guide, marketing journals are classified into five ordered categories (i.e. 1, 2, 3, 4 and 4*) with “1” intended for a “recognized journal” and “4*” for a “world elite journal”. It is highly unlikely that “world elite” journals cite predatory journals. It could be then hypothesized that

H2.

The higher the CABS rating of a journal is, the lower it cites predatory journals (if any).

Some authors have also proposed the age of a journal as an indicator of its quality (Baumgartner and Pieters, 2003; Touzani and Moussa, 2010; Moussa, 2019). Established journals have a long tradition of publishing research outputs. They have painstaking peer-review processes that have been operating for a while. Additionally, they have their reputations at stake. So, it is expected that a long-established journal will be (not or) less contaminated by citations to predatory journals than a younger journal.

H3.

The greater the age of a journal is, the lower it cites predatory journals.

The methodology adopted for testing these three hypotheses is described in the next section.

3. Methodology

3.1 The cited predatory marketing journals

In total, four predatory journals were selected for analysis. They were named Journal A to Journal D. The author is not naming these journals publicly to avoid exposing both himself and his institution to possible litigation. The names will be nonetheless provided upon request.

Examining citation contamination caused by all extent predatory marketing journals (i.e. 140 journals) is way beyond the scope of a single study. Only four predatory marketing journals were selected for analysis because they comply with the following selection criteria:

1. First, the predatory journal must be a mainstream (i.e. a nonspecialized) marketing journal. Predatory journals that are specialized in a particular marketing subfield were not selected for analysis. The four retained predatory journals are mainstream publication venues that have the word “marketing” in the title;

2. Second, the predatory journal has to be an author-pay OA (i.e. also termed a gold OA) publication venue. The four selected predatory journals are author-pay OA journals. The APC for Journal A is of 300US$. The APCs for journals B and D depend on the length of the manuscript. For Journal C, eventual authors have to pay a submission charge for their papers to be “peer-reviewed”. The submission charge depends on the number of pages of the manuscript. If the paper is accepted, the “lucky” author(s) has (have) to pay a publication charge that also depends on the length of the article;

3. Third, the predatory journal must be an active (i.e. still publishing) OA journal. Extinct predatory journals were not selected for analysis. The four selected predatory journals have published volumes in 2020. Journal B has even published issue number one of its 2021 volume;

4. Fourth, the predatory journal should be in existence for at least six years. Newly created predatory journals were not retained for analysis since they tend to receive very few citations (Moussa, 2021a). The four selected predatory journals were launched before 2014;

5. Fifth, there must be compelling evidence from various sources regarding the predatory nature of each journal. The subsection entitled the main features of a predatory journal provides such evidence. Besides, none of these four journals is currently indexed in the DOAJ (though Journal D claims so). None of the publishers of these four journals is presently a member of the Open Access Scholarly Publishers Association (OASPA). None of these four journals is to date covered by Elsevier's Scopus. Scopus' coverage of Journal B was discontinued (i.e. suspended) in 2016 for “publication concerns”. Also, the publishers of these four journals are included in the Dolos list, the Kscien list and the Stop Predatory Journals list. All of these four journals are currently included or under review for inclusion in Cabells' Predatory Reports (S. Linacre, personal communication, 9 February 2021).

3.2 The citing peer-reviewed marketing journals

The peer-reviewed marketing literature is herein represented by the marketing journals included in the 2018 version of CABS' Guide. The Guide was retained for three principal reasons: (1) It is one of the most widely used and internationally recognized journal quality lists (Feng et al., 2020); (2) it is way less elitist than the Financial Times' list of Top 50 management and business journals (FT50), which includes only six marketing journals (exclusively US-based journals), and (3) the Guide is less fatally flawed than the 2019 version of the ABDC's Journal Quality List. The latter contains not only predatory journals (e.g. Journal B) but also marketing journals that ceased to exist (e.g. The Canadian Journal of Marketing Research).

The Guide has 22 fields with marketing as the 12th field. Under the marketing field, the CABS' Guide proposes a ranking of 70 publication outlets including Advances in Consumer Research and Advances in International Marketing. These two publication venues are not peer-reviewed journals but rather a conference proceeding and a book series, respectively. Since this study focuses on peer-reviewed journals, both of these publication outlets were discarded from further analysis.

3.3 Citations gathering procedure

The four predatory journals (Journal A to Journal D) are indexed neither in Elsevier's Scopus nor in Clarivate Analytics' Social Sciences Citation Index (SSCI). The only citation source that covers predatory journals is Google Scholar (Moussa, 2021a). Google's scholarly search engine also covers the entirety of the 68 CABS-listed, peer-reviewed marketing journals (Moussa, 2019). It should be stressed that Google Scholar was used as a citation gathering tool, not as a citation source. The sources of citations are and should be the 68 CABS-listed peer-reviewed journals.

The citations gathering procedure could be described as follows: Using the “Advanced search” features of Google Scholar, the author inserted the title of each of the four predatory journals in the “find articles with the exact phrase” field and the title of each of the 68 peer-reviewed journals in the “Return articles published in” field. During the Google Scholar query, the title of each of the 68 peer-reviewed journals was put between quotation marks (“ ”) to avoid false hits (i.e. an article in a title-alike journal). The results of all automatic search queries have then undergone a careful visual inspection to detect false hits. The process was repeated 4 × 68 times. The citation gathering lasted three days from 26 to 28 January 2021.

3.4 Variables

The focal variable in this study is citation contamination. It corresponds to the number of articles published in the 68 CABS-indexed marketing journals that cited (at least) on one occasion one of the four sampled predatory marketing journals.

The JIFs for 2019 (published mid-2020) were collected using Clarivate Analytics' Journal Citation Reports (accessed via an institutional subscription on 28 January 2021). Only 42 of the 68 CABS-listed journals are indexed in the SSCI and have JIFs. In late 2015, Clarivate Analytics created the Emerging Sources Citation Index (ESCI). Of the 26 journals not indexed in the SSCI, ten are included in the ESCI. The remaining 16 journals are not recognized by Clarivate Analytics. So, an additional categorical variable (called Clarivate Analytics recognition) was created with three categories, namely SSCI-indexed, ESCI-indexed and nonrecognized by Clarivate Analytics.

Information about the age of a given journal was collected by visiting each journal's website. During these visits, the author has noticed that two journals have changed their names. The first journal is the Journal of Marketing Channels (Journal of Inter-Organizational Relationships as of 2021) and the second one is the International Journal of Nonprofit and Voluntary Sector Marketing (Journal of Philanthropy and Marketing since 2021). After a cautious check, it turned out that these title changes were made on the first of January 2021. As such, these changes in the titles have no deteriorating effects on this study's results for the period under scrutiny (i.e. before 2020).

To further investigate the phenomenon of citation contamination and respond to RQ4, a variable termed “subfield” was created. Consistent with and inspired by Baumgatner and Pieters (2003) and Lehmann (2005), the 68 peer-reviewed marketing journals were classified into 12 categories entitled: mainstream (ten journals), consumer behavior (ten journals), quantitative marketing research (seven journals), nonprofit and social marketing (seven journals), advertising and communication (six journals), services marketing (five journals), retailing (four journals), industrial/business-to-business marketing (four journals), Internet and interactive marketing (four journals), international marketing (three journals) and branding and reputation (three journals). The remaining five journals were put under the “miscellaneous” category.

Information about the current publisher of each journal and the number of its issues per volume (or annum) were collected while visiting these journals' websites. Of the 68 CABS-listed peer-reviewed marketing journals, 17 are published by Taylor & Francis, 16 by Emerald, nine by Sage, nine by Springer Nature, six by Wiley and five by Elsevier. The remaining six journals are published by other lesser-known publishers. The journal with the lowest number of issues per volume is the Review of Marketing Science (with one issue per volume). The journal that publishes the highest number of issues per annum is the Journal of Marketing Management (with 18 issues per volume).

3.5 Statistical procedures

As mentioned, citation contamination is a count variable. The distribution of counts is discrete, not continuous, and is limited to nonnegative, integer values. Distributions of (informetric) count data are unlikely to follow a normal distribution (Ajiferuke and Famoye, 2015). The nature of the focal variable of this study, the categorical/ordinal nature of most investigated variables (e.g. Clarivate Analytics recognition), a large number of ties for some others (e.g. Journal age) and the relatively small number of CABS-listed journals (N = 68) together impose the use of nonparametric statistical procedures.

To test hypotheses H1 and H3, this study uses Kendall's tau-b (τ_b), the nonparametric rank-order correlation coefficient. Kendall's tau-b corrects for the presence of ties and also has a range between −1 and 1. A τ_b “of at least 0.7 represents a very strong relationship; 0.4–0.699, a strong relationship; 0.3–0.399, a moderate relationship, 0.2–0.299, a weak relationship; and 0–0.199 implies that the variables are likely to be unrelated” (Ferrucci et al., 2020, p. 3).

To test H2 and provide answers to some of the four RQs (i.e. RQ4 and RQ5), Kruskal–Wallis H tests (also called one-way ANOVA on ranks) were performed followed by Mann–Whitney U tests. Kruskal–Wallis H test is a rank-based nonparametric test widely used to determine if there are statistically significant differences between two or more groups of an independent variable on a continuous or ordinal dependent variable (see, e.g. Arora et al., 2018). The Mann–Whitney U test is commonly used to compare differences between two independent groups when the dependent variable is either ordinal or continuous, but not normally distributed (see, e.g. Daou et al., 2018). All statistical analyzes were performed using IBM's SPSS (version 18).

4. Results

4.1 Descriptive statistics for the cited predatory journals

The four predatory marketing journals received together, and at least, 605 citations from the 68 CABS-listed peer-reviewed marketing journals (see Table 1).

The most cited predatory journal is Journal A with 259 received citations. As Table 2 indicates, 52 of the 68 peer-reviewed journals listed in the CABS' Guide have cited it at least on one occasion. Journal A received 3.81 citations in mean (Mdn = 2). The second most cited predatory journal is Journal B, which gathered 236 citations with a mean number of citations of 3.47 (Mdn = 2). As Table 2 indicates, 53 of the 68 peer-reviewed journals listed in the Guide cited it at least once. Hence, though Journal A received more citations Journal B has contaminated more journals. Journal C and Journal D received 93 and 17 citations, respectively. They have so far contaminated 39 and 12 peer-reviewed journals, respectively.

These four predatory journals received most of their citations from 1-, 2- and 3-rated peer-reviewed journals (see Figure 1).

4.2 Which CABS-listed journals are contaminated and which are not?

In total, 59 of the 68 (i.e. 86.765%) peer-reviewed journals listed in the CABS' Guide have at least cited on one occasion one of the four predatory journals (see Table 2). The list of contaminated journals includes “World elite” (i.e. 4*-rated) journals (e.g. Journal of Marketing Research and Journal of the Academy of Marketing Science), “Top” (i.e. 4-ranked) journals (e.g. Journal of Retailing) and “Highly-regarded” (3-graded) journals (e.g. European Journal of Marketing).

The list of the peer-reviewed journals that were, up to late January 2021, not contaminated includes only nine journals (i.e. 13.235%). Among that list, there are three “World elite” journals (namely Journal of Marketing, Journal of Consumer Research and Marketing Science), two “Highly regarded” journals (Journal of Public Policy and Marketing and Quantitative Marketing and Economics) and four “Recognized” journals (e.g. Australasian Marketing Journal).

4.3 Which are the most vs least contaminated CABS-listed journals?

The three most severely contaminated peer-reviewed journals by citations to the four predatory journals under scrutiny are (in decreasing level of citation contamination) as follows: International Journal of Bank Marketing (with 40 citations), International Journal of Retail and Distribution Management (with 36 citations) and Journal of Strategic Marketing (with 32 citations).

Some of the barely contaminated journals are Journal of International Marketing, Marketing Theory and Social Marketing Quarterly, with only one or two citations to a predatory journal each (see Table 3).

4.4 Results from hypothesis testing and statistical analyses

Table 4 indicates that H1 and H3 were not supported by the data. Table 5 displays the results from a Kruskal–Wallis H test indicating that there is a statistically significant difference in citation contamination between the five CABS categories (which supports H2).

Kendall's tau-b correlation between the 2019 JIFs and citation contamination was not statistically significantly different from zero (τ_b = −0.098; p > 0.10, n = 42). Stated differently, there is no evidence that a high (or low) JIF is negatively (or positively) associated with citation contamination.

No statistically significant monotone relationship between journal age and citation contamination was found (τ_b = −0.003; p > 0.10, N = 68). To put it more plainly, being a long-established (vs a young) journal does not necessarily mean that it is immune (vs inclined) to citation contamination.

Exploring the relationship between the various variables, Kendall's tau-b correlation between the number of issues per journal volume and citation contamination got a statistically significant, weak and positive value of τ_b = 0.290 (p < 0.01, N = 68). This finding indicates that there is a weak, monotonous and positive relationship between the number of issues per volume for a journal and its level of citation contamination (see Table 4).

A Kruskal–Wallis H test with citation contamination as the dependent variable and CABS rating as the independent categorical-ordered variable yielded a statistically significant χ² value of 13.251 (p < 0.05, N = 68, df = 4). CABS categories 1, 2 and 3 obtained the highest mean rank values with 40.290, 38.420 and 28.250, respectively (see Table 5). The lowest mean rank value was 13.080 for “World elite” journals. Using Mann–Whitney U tests for pairwise differences of mean rank values, four-star journals were statistically and significantly less contaminated by citations to predatory journals than “Well-regarded” (U = 11.000, p < 0.01) or “Recognized” journals (U = 24.500, p < 0.01).

A second Kruskal–Wallis test with citation contamination as the dependent variable and Clarivate Analytics' recognition as ordered categorical variable yielded a nonstatistically significant χ² value of 0.426 (p > 0.10, N = 68, df = 2). Consistent with the findings for H1 (i.e. no statistically significant relationship between the JIFs for 2019 and citation contamination), this test shows that citation contamination occurs regardless of whether the journal is SSCI-indexed, ESCI-indexed or not recognized by Clarivate Analytics.

A third Kruskal–Wallis test with citation contamination counts as the dependent variable and the subfield as a categorical variable yielded a statistically significant χ² value of 21.378 (p < 0.05, N = 68, df = 11). The three highest mean rank values were 57.300 for services marketing, 53.000 for branding and reputation and 42.880 for industrial/business-to-business marketing (see Table 6). The three smallest mean rank values were 13.000 for quantitative marketing research, 24.710 for nonprofit and social marketing, and 29.000 for retailing. Using Mann–Whitney U tests for pairwise comparisons of mean rank values, services marketing journals were statistically and significantly more contaminated by citations to predatory journals than consumer behavior (U = 8.000, p < 0.05) or advertising and communication journals (U = 2.000, p < 0.05). Quantitative marketing research journals are statistically and significantly suffering less from citation contamination than mainstream (U = 14.500, p < 0.05) or Internet and interactive marketing journals (U = 2.500, p < 0.05).

5. Discussion

Writing in 2015, Beall (2015, p. 473) said that predatory publishers and journals “carry out a fake or incomplete peer review that allows low-quality, un-vetted research to become a part of the scholarly record”.

Investigating references to four predatory marketing journals in the 68 peer-reviewed marketing journals listed in the CABS' Guide, the author has found that 59 of them (i.e. 86.765%) were already infected. Only nine journals have so far escaped citation contamination. The findings also point out that citation contamination occurred irrespective of the JIF of a journal, its age or whether it is or not recognized by Clarivate Analytics.

The main conclusion to draw from the statistical analyzes and hypothesis testing is that citation contamination is not essentially a matter of “quality” (as hypothesized). It is the result of something else: unawareness. Most marketing scholars (whether acting as authors, reviewers or editors) are probably unaware of the predatory nature of these four journals. One compelling fact that corroborates the soundness of this explanation lies in the findings: There is a positive correlation between the number of issues per volume for a journal and its level of citation contamination (τ_b = 0.290, p < 0.01, N = 68). A 3-rated journal that publishes 12 issues per volume (e.g. European Journal of Marketing) is severely contaminated, while a 2-rated journal that publishes only four issues per annum (e.g. Electronic Markets) is barely contaminated. An SSCI-indexed journal that publishes 18 issues (e.g. Journal of Marketing Management) is severely contaminated, while an ESCI-indexed journal that publishes only four (e.g. Australasian Marketing Journal) is not contaminated. A journal with a JIF of 2.800 that publishes seven issues per volume (i.e. International Journal of Bank Marketing) is way more contaminated than a journal with a JIF of 0.543 that publishes four issues per annum (i.e. Journal of Business-to-Business Marketing). It is not a matter of a CABS' rating, a JIF value, or inclusion or not in a given Clarivate Analytics index. The lack of awareness about the predatory nature of these four journals combined with the high number of published issues of the citing journal has led to a higher level of citation contamination. Peer-reviewed journals that publish four to six issues are either not or barely contaminated (see Figure 2). Someone may ask why four-star marketing journals are (statistically and significantly) less contaminated than “well regarded” or “recognized” journals? The response to that question is as follows: the six “World elite” marketing journals publish four (e.g. Journal of Consumer Psychology) to six issues (e.g. Journal of Marketing) per volume.

The fifth research question (i.e. RQ5) asked why some journals are less (vs more) contaminated than others. The results indicate that quantitative marketing research journals are the least contaminated among those falling under the 12 marketing subfields. The first explanation behind such findings resides in the fact that most quantitative marketing research journals publish a small number of issues per volume. For instance, the Journal of Marketing Research (as well as Marketing Science) publishes six issues per annum. As for the International Journal of Research in Marketing, Marketing Letters, and Quantitative Marketing and Economics, they publish four issues per volume. The Review of Marketing Science publishes a single issue per annum. The second explanation is that articles published in these quantitatively oriented journals tend to cite articles published in quantitatively oriented journals. As Lehmann et al. (2011, p. 159) indicate, these journals use “too complex statistical methods” and “too technical writing styles”. Lehmann et al. (2011, p. 155) further added that one “consequence of increased complexity is that marketing scholars are partitioning themselves into small groups, in which each group is defined by the particular kind of complexity that group favors”. To put it in a more critical tone, quantitative marketing research journals are currently dominated by modelers (Fuat Firat, 2010). Modelers tend to cite modeling journals. The sampled four predatory marketing journals are not modeling journals and do not publish complex and technical manuscripts. Hence, they are not cited by these modeling journals.

The findings show that quantitative marketing research journals are statistically and significantly less suffering from citation contamination than mainstream journals (U = 14.500, p < 0.05). A mainstream, four-star journal that is not a modeling journal, like the Journal of the Academy of Marketing Science, falls under the slightly contaminated category. Other mainstream, nonmodeling marketing journals (and depending on their number of issues per volume) are either moderately (e.g. Journal of Marketing Theory and Practice with four issues) or severely contaminated (e.g. European Journal of Marketing with 12 issues) by citations to the sampled four predatory journals.

As the findings suggest, services marketing journals were the most contaminated publication venues by citations to the four predatory journals under scrutiny (with a mean rank value of 57.300). This might be explained by the fact that services marketing is a very popular subfield. Following Vargo and Lusch's service-dominant logic for marketing (Vargo and Lusch, 2004), services marketing has become a trendy subfield that attracts and falls within the expertise of most marketing researchers (Moussa and Touzani, 2010; Wilden et al., 2017). So, it is highly likely that many of the articles published in the four predatory marketing journals delve into topics related to services marketing (see Table 7).

6. Recommendations

The findings reported in this study should sound an alarm to the entire marketing community (including academics and practitioners). Predatory journals will continue to contaminate marketing knowledge unless all stakeholders take specific and coordinated actions against them. Counteracting citation contamination that results from predatory publishing is an obligation that needs to be collectively embraced by the entire marketing community (including researchers, practitioners, journal editors and academic and professional associations).

6.1 For researchers

According to Severin and Low (2019, p. 1124), “[i]f researchers cite articles in scientifically questionable journals, their reputation could risk as much damage as if they appear as authors in these publications”. For Beall (2016b, p. 1512), “if a scientific paper cites earlier articles published in predatory journals, it may itself be considered questionable”. In medicine, a discipline severely contaminated by predatory journals (Cukier et al., 2020), the International Committee of Medical Journals Editors (2019, p. 17) recommends that “[a]uthors should avoid citing articles from predatory or pseudo-journals”.

Marketing researchers that base their research activities on poor-quality or even fabricated findings and cite these in the reference lists of their publications are the first and main contributors to the contamination of marketing knowledge. Respectable authors should not cite predatory journals. One recommendation for authors is always to read each paper, check its content and source before citing it (read, check and cite). A citation is an acknowledgment and homage that stays forever in the annals of marketing knowledge. Citations should be taken seriously. One additional piece of advice is that not every journal covered by Google Scholar is a peer-reviewed one. Google Scholar is not an index like Elsevier's Scopus or Clarivate Analytics' SSCI but rather a search engine (Halevi et al., 2017).

6.2 For practitioners

Articles published in predatory journals contain theoretical as well as managerial implications and recommendations. Imagine that a practitioner (i.e. a marketer, manager or a decision-maker) stumbles upon an article that appeared in an OA journal that has a fleeting or nonexistential peer-review process. Imagine now that the same practitioner has made one or more of his/her decisions based on recommendations found in an unvetted/false/erroneous research published in a questionable journal. Predatory journals can harmfully influence policies, decision-making and marketing practices. Practitioners have to be extremely cautious about the content of such journals.

6.3 For journal editors

The COPE advises that reviewers, editors and journals “[d]iscourage citation of articles published in fake journals” (COPE, 2019, p. 12).

Editors of marketing journals should institute procedures to reduce the risk of citation contamination. They can start by alerting their associate editors (if any) and review board members to the need to check reference lists of the submitted manuscripts carefully. Several free as well as fee-walled resources are available to assist them in that duty. These include the DOAJ, the OASPA and up-to-date lists of predatory publishers and journals.

New journal submission guidelines should be created. They must ask prospective authors to assess the cited literature thoroughly. For instance, the Journal of Threatened Taxa (JoTT, https://threatenedtaxa.org/index.php/JoTT/about) has issued the following guidelines regarding the citation of work from predatory journals:

1. “JoTT discourages citation of work published in journals from any of the known predatory publishers or stand-alone predatory/deceptive journals”.

2. “If the author must cite the work published in a predatory journal, as it is important in the study, they should cite it in the text as ‘published in a predatory journal’ (since JoTT does not trust that such work was published following scientific procedure of peer review). For instance, ‘……was suggested by Author et al. (published in a predatory journal) or …was suggested (Author et al., published in a predatory journal)’”.

Predatory journals are tarnishing the gold OA model. High-quality, peer-reviewed gold OA marketing journals have to differentiate themselves from predatory journals. This could be done in a multitude of ways (e.g. open peer review or post-publication peer review). The most effective way is to adhere to the OA publishing industry standards as outlined by the COPE, OASPA and DOAJ. A high-quality, peer-reviewed gold OA marketing journal is Innovative Marketing (which is published by the Ukrainian publisher Business Perspectives). Business Perspectives is currently a member of the COPE as well as a member of the OASPA. Innovative Marketing along with the 11 gold OA journals published by Business Perspectives have been lately awarded the DOAJ seal, which is a mark of certification for OA journals for the achievement of a high level of openness, adhering to best practices and having high publishing standards.

6.4 For editors of journal quality lists

Journal B is currently listed in the 2019 version of ABDC's list as a “B” marketing journal. This fact raises a number of queries: Are the four members of the panel for the marketing field of research (FoR code 1505) unaware of the predatory features of Journal B? Why have they upgraded it from a “C” (i.e. a “Recognized journal”) to a “B” (i.e. a “Well regarded”) journal? How many marketing scholars were misled by ABDC's Journal “Quality” List?

Editors of journal quality lists have to be extremely cautious. Before including and ranking a journal, they must first make sure that it is a peer-reviewed journal and not a journal that fakes the peer-reviewing process.

6.5 For academic and professional associations

According to Wilkie and Moore (2003), academic and professional associations like the American Marketing Association (AMA) and the Association for Consumer Research have played key roles in the development of marketing knowledge. In total, two other influential marketing associations are the European Marketing Academy (eMac) and the UK-based Academy of Marketing (AM). Marketing knowledge is currently facing the serious threat of predatory publishing. Those at the AM are fully aware of that as there is currently a predatory journal that is brandjacking the identity of their flagship journal, Journal ofMarketing Management (see https://www.jmmnews.com/journal-of-marketing-management-think-check-submit/). Another predatory journal whose name is confusingly similar to eMac's International Journal of Research in Marketing was launched in 2019 (see Moussa, 2021a, p. 490). There is even a predatory journal that claims to be “the” American Journal of Marketing Research (not AMA's Journal of Marketing Research). These associations (and their flagship journals) are today under the menace of predatory publishing. These, and other, associations have to combat this phenomenon. They must inform and educate their membership not solely about the risks of publishing in predatory journals but also about the threat they pose to the integrity of marketing knowledge. They can create special interest groups (similar to AMA's SIGs) dedicated to publication ethics and responsible authorship. They can organize workshops and symposia on the perils of predatory journals.

A decade ago, Lutz (2011) wrote an article entitled “Marketing scholarship 2.0”. In that article, Lutz envisioned an “optimal and inevitable” system in which all marketing journals are online, peer-reviewed and OA. Lutz (2011, p. 230) recommended that “journals at the AMA should adopt the OA green path” (which is also called self-archiving). To combat predatory publishing and citation contamination, the author advises taking Lutz's recommendation a bit further by adopting the diamond OA (also referred to as the platinum OA) path. Diamond OA means permanent and free access to published scientific work for readers with no APCs for the authors. According to the DOAJ, 11,505 of the 15,825 OA journals indexed in it (as of 7 February 2021) are diamond OA journals.

Diamond OA marketing journals do exist. The first example is Market-Tržište (the official journal of the Marketing Department at the Faculty of Economics and Business – Zagreb and the Croatian Marketing Association). The Spanish Journal of Marketing – ESIC is currently published in diamond OA by the Emerald Publishing Group in association with the Spanish Association of Academic and Professional Marketing and the ESIC Business and Marketing School. Another diamond OA marketing journal is the South Asian Journal of Marketing, which was newly launched by the Sri Lanka Institute of Marketing in collaboration with the Emerald Publishing Group. The AMA, the eMac and the AM are advised to follow these examples. The author believes that a continuous “deluge” of high-quality, peer-reviewed, OA marketing literature is the best, long-term solution to wash away the dark and depressing episodes of predatory journals and citations to them from the records of marketing knowledge.

7. Limitations and further research directions

While this study provides several new insights, it has some limitations. However, these limitations point to questions that could be the subject of future work. First, this study investigated citations to only four active predatory marketing journals. As Moussa and Linacre (2020) indicate, there are currently 140 journals listed in Cabells' Predatory Reports. Future research may investigate citations to a larger number of predatory publication venues. Second, the peer-reviewed marketing literature was represented by only the 68 CABS-ranked journals. Future studies may investigate citation contamination for other than these 68 journals (e.g. journals included in the lists provided by the India University Grants Commission – Consortium for Academic and Research Ethics) provided that they are covered by Google Scholar. Third, this study did not investigate hijacked (also called clone or duplicate) marketing journals. As previously mentioned (in Subsection 6.5.), there is one journal that is currently using the same title as the UK-based Journal of Marketing Management (see Moussa, 2021b). Future studies may investigate the case of this and other clone journals. Fourth, this study investigated direct citation contamination (i.e. when a peer-reviewed journal cites a predatory one). Contamination could also be indirect, and it occurs when an article published in a peer-reviewed journal cites the peer-reviewed journal article that first cited the predatory journal. In this context, we should speak about first-generation citation contamination, second-generation citation contamination, etc.

8. Conclusion

As Linacre et al. (2019), Moussa and Linarce (2020) and Moussa (2021a) indicate, the scholarly publishing scenery has changed a lot these last years. Marketing scholars and practitioners are living in an environment full of fake research, fake journals and fake publishers. Marketing journals that were created during the last century are now in a market where legitimate and illegitimate publication outlets coexist. Everyone must exercise vigilance and/or vigilantism. The OA movement has brought with it threats (i.e. predatory publishing) but also possibilities (i.e. the diamond OA path).

Figures

Figure 1

Citation contamination by the CABS' ranks

Figure 2

Relationship between citation contamination and the number of published issues per volume

Table 1

Descriptive statistics for the cited predatory marketing journals

	Journal A	Journal B	Journal C	Journal D
Mean	3.810	3.440	1.370	0.250
Median	2.000	2.000	1.000	0.000
Minimum–maximum	0–21	0–14	0–9	0–3
Sum	259	236	93	17

Table 2

Citation contamination

Peer-reviewed journals	Citation contamination count	Predatory journals
		A	B	C	D
Journal of Marketing	0	0	0	0	0
Journal of Consumer Research	0	0	0	0	0
Marketing Science	0	0	0	0	0
Journal of Public Policy and Marketing	0	0	0	0	0
Quantitative Marketing and Economics	0	0	0	0	0
Australasian Marketing Journal	0	0	0	0	0
International Journal of Nonprofit and Voluntary Sector Marketing (Journal of Philanthropy and Marketing as of 2021)	0	0	0	0	0
International Review of Retail, Distribution and Consumer Research	0	0	0	0	0
Review of Marketing Science	0	0	0	0	0
Journal of Marketing Research	1	0	0	1	0
Journal of Retailing	1	1	0	0	0
Journal of International Marketing	1	1	0	0	0
Marketing Letters	1	0	0	0	1
Consumption, Markets and Culture	1	0	1	0	0
Social Marketing Quarterly	1	1	0	0	0
Journal of Consumer Psychology	2	0	1	1	0
International Journal of Research in Marketing	2	1	1	0	0
Marketing Theory	2	0	2	0	0
Psychology and Marketing	2	0	2	0	0
Academy of Marketing Science Review	2	2	0	0	0
Electronic Markets	2	1	1	0	0
Journal of Consumer Affairs	2	1	1	0	0
Journal of Communication Management	2	0	2	0	0
Journal of Interactive Marketing	3	2	1	0	0
Journal of Consumer Behavior	3	1	2	0	0
Health Marketing Quarterly	3	2	1	0	0
Journal of Business-to-Business Marketing	4	2	1	1	0
Corporate Communications	4	2	1	1	0
Journal of the Academy of Marketing Science	5	2	1	1	1
International Journal of Consumer Studies	5	3	2	0	0
International Journal of Market Research	5	3	1	1	0
Journal of Macromarketing	5	2	2	1	0
Journal of Retailing and Consumer Services	5	2	2	0	1
Corporate Reputation Review	5	3	0	2	0
Journal of Social Marketing	6	3	2	1	0
Journal of Personal Selling and Sales Management	7	1	4	2	0
Qualitative Market Research	7	0	4	3	0
Journal of Marketing channels (Journal of Inter-Organizational Relationships as of 2021)	7	3	3	1	0
Journal of Advertising Research	8	3	2	3	0
Journal of Marketing Theory and Practice	8	2	5	1	0
Journal of Fashion Marketing and Management	8	3	4	1	0
Journal of Nonprofit and Public Sector Marketing	8	2	4	1	1
International Review on Public and Nonprofit Marketing	9	5	2	0	2
Journal of Global Marketing	9	6	2	0	1
Journal of Advertising	10	2	3	5	0
Journal of Services Marketing	11	3	6	2	0
International Journal of Internet Marketing and Advertising	11	4	4	2	1
Industrial Marketing Management	12	4	5	0	3
International Journal of Advertising	12	4	4	4	0
International Marketing Review	13	5	8	0	0
Journal of Marketing for Higher Education	13	7	5	1	0
Journal of Marketing Communications	14	3	8	3	0
Journal of Customer Behavior	15	9	5	1	0
European Journal of Marketing	16	4	7	5	0
Journal of Marketing Management	16	5	9	2	0
Journal of Financial Services Marketing	16	10	4	2	0
Services Marketing Quarterly	16	4	10	2	0
Journal of Business and Industrial Marketing	17	3	10	4	0
Journal of Research in Interactive Marketing	17	8	7	2	0
Journal of Consumer Marketing	18	8	7	2	1
Journal of Relationship Marketing	19	10	5	4	0
Journal of Brand Management	24	11	8	4	1
Young Consumers	24	12	9	3	0
Marketing Intelligence and Planning	28	15	10	3	0
Journal of Product and Brand Management	29	13	14	2	0
Journal of Strategic Marketing	32	14	12	5	1
International Journal of Retail and Distribution Management	36	21	12	3	0
International Journal of Bank Marketing	40	20	8	9	3

Table 3

Citation contamination categories

Category	Number of journals per category	%
Not contaminated	9	13.235
Barely contaminated (1–2 citations)	14	20.589
Slightly contaminated (3–5.5 citations)	11	16.176
Moderately contaminated (6–13.75 citations)	17	25.000
Severely contaminated (more than 13.75 citations)	17	25.000
Total	N = 68	100

Table 4

Kendall's tau-b correlations and descriptive statistics

	Citation contamination	Journal Impact Factors 2019	Journal age	Number of issues per volume
Citation contamination	1.000
Journal Impact Factors 2019	−0.098	1.000
Journal age	−0.003	0.239*	1.000
Number of issues per volume	0.290**	0.016	0.277**	1.000
Descriptive statistics
Mean	8.900	2.963	34.120	5. 430
Median	5.500	2.487	30.500	4.000
Minimum–maximum	0–40	0.543–7.959	10–96	1–18
Skewness	1.437	1.122	1.670	2.459
Kurtosis	1.814	0.987	4.549	8.649
Quartiles (1st Q; 3rd Q)	2; 13.75	1.779; 3.759	25;39	4; 6

Note(s): *p < 0.05, **p < 0.01

Table 5

Results from Kruskal–Wallis H and Mann–Whitney U for Hypothesis 2 testing

CABS category	Number of journals per category	Mean rank	Test
Kruskal–Wallis H test
4*	6	13.080	χ2(4) = 13.251 (p = 0.018)
4	2	16.000
3	12	28.250
2	20	38.420
1	28	40.290
Mann–Whitney U tests (only statistically significant differences are displayed)
4*–1	6–28	7.580–19.630	U = 24.500, p = 0.007
4*–2	6–20	5.330–15.950	U = 11.000, p = 0.003

Table 6

Kruskal–Wallis H and Mann–Whitney U tests for citation contamination by marketing subfields

Subfield	Number of journals per subfield	Mean rank	Test
Kruskal–Wallis H test
Mainstream	10	36.200	χ2(11) = 21.378 (p = 0.030)
Consumer behaviour	10	30.800
Quantitative marketing research	7	13.000
Nonprofit and social marketing	7	24.710
Advertising and communication	6	38.830
Services marketing	5	57.300
Retailing	4	29.000
Industrial/business-to-business marketing	4	42.880
Internet and interactive marketing	4	37.380
International marketing	3	35.500
Branding and reputation	3	53.000
Miscellaneous	5	38.000
Mann–Whitney U tests (only some statistically significant differences are displayed)
Consumer behavior – Services marketing	10–5	6.300–11.400	U = 8.000, p = 0.036
Advertising and communication – Services marketing	6–5	3.830–8.600	U = 2.000, p = 0.017
Mainstream – Quantitative marketing research	10–7	11.050–6.070	U = 14.500, p = 0.042
Quantitative marketing research – Internet and interactive marketing	7–4	4.360–8.880	U = 2.500, p = 0.028

Table 7

Summary of hypotheses testing

Hypothesis	Verbatim	Hypothesis testing	Interpretation
H1	The higher the JIF of a journal is, the lower it cites predatory journals	Not supported	Citation contamination occurred irrespective of the 2019 JIF value of the citing journal
H2	The higher the CABS rating of a journal is, the lower it cites predatory journals (if any)	Partially supported	Only 4*-rated journals were significantly less contaminated than 2 or 1-rated journals
H3	The greater the age of a journal is, the lower it cites predatory journals	Not supported	Citation contamination occurred irrespective of the age of the citing journal