Wikipedia:Fools Gold
Essay on editing Wikipedia
From Wikipedia, the free encyclopedia
Since I first joined Wikipedia, I've participated in a lot of WP:AfDs and WP:AfC reviews where an article's sourcing relies on academic journals nobody has heard of. The references section looks long. Well yes... The links do resolve. The publications have professional-sounding names and claim peer review. But.... geez, a troubling number of them are, to put it plainly, junk. Boy..! I am serious... I say it as someone who has now clicked on enough sketchy journal websites to have developed a Pavlovian twitch whenever I see the words "International," "Advanced," and "Multidisciplinary" in the same sentence.
 | This is an essay. It contains the advice or opinions of one or more Wikipedia contributors. This page is not an encyclopedia article or a Wikipedia policy, as it has not been reviewed by the community. |
 | This page in a nutshell: Not everything with an ISSN and a claim of peer review is a reliable source. Predatory journals exist to collect fees, not to advance knowledge. If you were sent here from a link in a WP:AfD, WP:AfC, or similar discussion, you are being asked to verify that the journals cited actually meet the criteria at WP:RS and WP:SCHOLARSHIP. If the journal would publish your grocery list for $300, it is not a reliable source. |
This is not a niche problem. It shows up with particular regularity in articles about academics, universities, organizations, and public figures from countries with large and fast-growing research sectors, India being a prominent example. This is not because Indian scholarship is poor! I am an Indian academic myself... Well, India produces enormous volumes of excellent, research published in internationally recognized venues. It is because the same conditions that produce a thriving research ecosystem, a large population of working academics under institutional pressure to publish, also create a massive market for publishers whose entire business model is collecting article processing charges from authors who need a line on their CV, with little or no genuine editorial oversight[1]. Think of it as the academic equivalent of a diploma mill, except instead of selling you a degree, they sell you the appearance of having done research.
Well, the result is a class of publication that mimics the form of scholarship without the substance. It is the academic equivalent of a film set: the building looks real from the front, but if you walk around the back, there is nothing there but plywood and staples. When these publications get cited on Wikipedia as evidence of notability or as reliable sources, we have a real problem. The article looks well-sourced. But the foundation is sand, and somebody just turned on the sprinklers lol!
Why this matters for Wikipedia
The basis of the sourcing policies on Wikipedia, especially the rule WP:RS, is the reasonable assumption that academic journals are more reliable than, say, an arbitrary blog posting or press release, because they have an editorial and peer review process. The assumption is true as long as the journal actually follows its own process. The assumption is completely false if the journal will publish anything that is sent to them, as long as the payment is made[2].
A paper published in such a journal is not evidence that the paper was reviewed by experts and found to be accurate. It is evidence that the authors are willing to pay to have the paper typeset. There is a big difference between the two. Your local copy shop will typeset papers for money too. We do not consider Staples a peer-reviewed source.
In the case where you are presented with a source from a journal you are unfamiliar with, the proper course of action is not to assume that the source must be valid because "it's a journal." The proper course of action is to ask yourself, "Would this journal reject a bad paper?" If the answer is "Probably not," or "They'll publish a paper saying the moon is made of cheese if the check clears," then you have your answer. ;)
How to spot them
Sorry if this is disappointing but sadly, there is no single test. Predatory publishing is a spectrum, not a binary. Some operations are blatantly fraudulent[3]. Others occupy a grey zone of sloppy but not outright dishonest practice. But there are patterns, and once you learn to see them, you cannot unsee them. It is like learning to spot bad CGI in movies. Ruins the experience, but someone has to do it.
The editorial board is suspect
Check the editorial board, if one is listed.
- Are they real people? Do they have affiliations that check out? A quick internet search will usually answer this question. If the "Editor-in-Chief" does not exist outside of this journal's website, that is what we call "a problem."
- If you were to contact one of these individuals, would they even know they were on the board? Would they have given permission? Some predatory journals feature academics without their knowledge or permission. Imagine being informed that you are on the editorial board of the "Global Journal of Innovative Multidisciplinary Advanced Everything" because someone found your name on some conference website.
- Are they all from the same country? Are they all affiliated with the same university? A legitimate international journal will have an editorial board that is also international. A journal that refers to itself as "International" but whose editorial board works in the same building is using the word "international" in a manner that would bring tears to the eyes of any dictionary.
- Are the editorial board members also writers for other predatory journals? This is circular reasoning, but it is also useful. If the editorial board is writing exclusively for journals just like this one, they are not a community of scholars. They are a support group.
It is not indexed where it matters
Indexing is one of the more reliable signs, though certainly not infallible.
- Is the journal indexed in PubMed, Scopus, or Web of Science? These databases have inclusion criteria, and while they are certainly not perfect, they filter out at least half of the predatory journals. They are like the bouncers at the door of a nightclub. They don't stop everyone with a fake ID, but they stop most of them.
- Be very cautious about journals that say they are indexed in Google Scholar. Google Scholar indexes just about everything. Being indexed in Google Scholar is utterly irrelevant to journal quality. It is like saying, "My restaurant is listed on Google Maps." So is the dumpster behind my restaurant.
- Being on WordPress or another website host with a website that is accessible on Google is also irrelevant. My aunt has a recipe website hosted on WordPress. Her website is accessible on Google. We don't cite it at WP:RSN. Her Smoked pork is excellent, however.
- Look at the Directory of Open Access Journals listing. Being listed is good. Not being listed is neither here nor there.
The journal publishes everything
Look through the journal's archives. Read some of the articles. I know this sounds like a lot of work, and it is, but bear with me here. It's an investment in not looking like a complete idiot at AfD.
- Does it appear to have some thematic organization, or do they publish articles on dentistry, electrical engineering, and 14th century Marathi poetry all in the same issue? A journal with no apparent scope is a journal with no apparent standards.
- Are the articles well-written? Do they follow standard academic conventions for their discipline?
- Does it appear to have been heavily edited, or do published articles have obvious errors, things like "INSERT REFERENCE HERE," or formatting issues that would have been caught by any reviewer with a pulse?
- Does it publish an implausible number of articles for its apparent scope and resources?
A journal that publishes 500 articles a month across 40 different disciplines is not a peer-reviewed journal. It's a printing press. A very busy, very undiscriminating printing press.
Check it against known tools and lists
There are a number of resources dedicated specifically to this problem. Use them. The fact that this problem is large enough to have an entire database dedicated to it should tell you something.
- Beall's List[4] and its various clones and successors was the original attempt at cataloging predatory publishers. While it is no longer maintained by its creator, various archived and updated versions of it float around. It's like the original "wall of shame."
- Cabells' Predatory Reports[5] is a subscription-based service with its own curated list based on set criteria.
- The Directory of Open Access Journals[6] maintains an inclusion list with its own set of criteria.
- Think. Check. Submit.[7] is a tool developed by a coalition of publishers and scholarly societies. The name says it all, and it's also a sad commentary on how authors often skip the first two steps of this process.
However please note that not all list are credible. For instance there is a cite called predatoryjournals.com that lists legit journals and extort money from these journals in return of having their name removed[8].
The Indian UGC
India has a University Grants Commission (UGC), which has for many years maintained a list of approved journals for career advancement in academics. Being published in a journal listed by the UGC has, in many academic institutions in India, been a requirement for hiring, promotion, and granting tenure. This has resulted in a huge demand for slots in these journals. And when there is demand, supply follows, and a new industry has evolved to meet this demand with the enthusiasm of a street vendor who has just spotted a crowd of hungry tourists. Some of these journals are legitimate, and many are not. The UGC has from time to time revised and refined its list, and the introduction of the UGC-CARE (Consortium for Academic and Research Ethics) list was a move to try and enforce stricter criteria, but the problem remains far from solved, and a large number of publications that were floating around during the less-stringent criteria continue to be cited on Wikipedia as if they were legitimate academic publications. In the past noting the quality of UGC listed journals India had to cull a lot of the approved journals[9]. But heyyy... despite the mass culling, a sting operation by researchers of Northeast India still found just this, that most of UGC listed journals were utter nonsense[10].
So, what this means for Wikipedia editors is quite simple: if you come across a source from a journal you've never heard of, don't assume that just because it exists, has an ISSN, and was once listed in one or more UGC lists, it's therefore reliable. Use the same checks that you would use with any unknown journal. It's not bias against Indian scholarship; it's just applying the same standard of scrutiny that we use to all unknown sources, adjusted for the known rate of predatory publishing in that region. We'd say the same thing about any region where there's a known problem with predatory journals, and we're saying it here because this is where that problem most commonly shows up in AfD discussions I've participated in.
The best Indian research will be found in well-known, internationally recognized journals, or Indian journals with long histories, real editorial standards, and real indexing in major databases. These are good sources. Examples of good Indian Journal are Indian Journal of Physics, Pramana (journal), Proceedings of the National Academy of Sciences, India Section A, etc.
If in doubt the Predatory journal list by Council of Scientific and Industrial Research, Pune is quite helpful[11].
What to do in AfD and AfC
If you are reviewing an article and the sources are journals you've never heard of:
- Choose the two or three sources that the article's proponents think are strongest. (See WP:THREE if you like this approach. It's good advice, and it will save you from reading the International Journal of Innovative Multidisciplinary Advanced Synergistic Research on your Saturday night.)
- Check those sources using the criteria above.
- If those journals do poorly, say so in your discussion section. Be specific. "This source is from [journal name], which is published by [publisher]. The journal is not indexed in Scopus or Web of Science, claims a 72-hour peer review turnaround, and publishes papers on topics from dentistry to electrical engineering in the same issue. I do not consider this a reliable source under WP:RS." This is persuasive. "This is a predatory journal" without evidence is not. Show your work. We're, after all, an encyclopedia.
- Don't be smug about it. Nobody likes having their sources told they're bad. Be factual, be clear, and don't make it personal. See WP:NPA
If you are defending an article and someone expresses concerns about the reliability of your sources being predatory:
- Take the concern seriously. Do not dismiss it as cultural bias or elitism. The person expressing the concern is probably correct, and getting defensive will not make the source more reliable.
- Find better sources. If the topic is really important enough to have an article, it is probably covered in some reliable source. Find those reliable sources and use them. Your article will be better for it.
- If you are really sure that the journal is reliable, make the case. Provide the journal's indexing. Provide the journal's rejection rate, if they will disclose it. Provide proof that the journal's editorial board consists of credentialed researchers at real institutions who know they are on the board. Evidence is more convincing than assertion.
- Limit yourself to your best two or three arguments. The same rationale that supports WP:THREE supports this. If you cannot convince with two or three good arguments, no number of additional arguments will. They will just cause people to glaze over, and glazed eyes do not vote to keep.
A note on good faith
Most editors who mention these journals do so in an honest manner, but they are most likely the authors of the article, or the authors' colleagues, or simply people who stumbled across the article and didn't know how to go about determining the legitimacy of the journal in which the article appeared. Assume good faith. Explain the situation to them, and direct them to this essay or other resources on the subject of predatory publishing. Our goal here is to improve the quality of sourcing, not to catch people in the act of misusing those sources. We are all here to create an encyclopedia, not to score points in an argument. (Well, most of us, anyway. I'm looking at you, too.)
However, do not let good faith excuse bad sourcing. An article based on unreliable sources is an article based on nothing, no matter how pure the heart of the person assembling those sources may be. Wikipedia's readers are better than that, and they expect us to get the job done, no matter what. The very least we can do is make an effort to check the sources.
But if, after all of the above, you are still uncertain about the legitimacy of a particular journal, then simply ask yourself one simple question: "Would I be embarrassed if someone found out I got this information from here?" If the answer is "yes," or "maybe," then go ahead and find a better source. Trust me, they are out there, but they are just a little harder to find, since they do not send you spam messages promising you publication within 72 hours.