Wikipedia:Wikipedia Signpost/2026-03-10/Op-ed

This article needs to clarify whether the problem observed is an effect of AI (slop added to our article because the AI model learned nonsense from somewhere else) or whether AI is spreading a WP mistake (just as any search-engine would, but only more-so because AI is so credulous of what it ingests). Or maybe citogenesis of unknown root origin? I assumed it was the first meaning, given the general WP distrust of LLM and the note to AI-Cleanup folks suggesting the uncovering of a newly-identified pile of slop that had been lurking. But PMID 9 was added to List of Disney television films in 2020. DMacks (talk) 04:55, 10 March 2026 (UTC)

(edit conflict) Came here to say the same thing.

This article is an example why we need better quality control for Signpost submissions.

ChatGPT thinks that proper Wikipedia articles contain PMID links (PubMed identifiers).

- what's the evidence for this assertion, and in particular for the assumption that all the bogus citations listed in the table were due to use of ChatGPT?

I spot-checked Polygnotus' example for PMID 47, i.e. the paper "The influence of pH on the interaction of inhibitors with triosephosphate isomerase and determination of the pKa of the active-site carboxyl group" being cited in Air pollution in Delhi. That bogus citation was added in 2019, i.e. years before ChatGPT's launch, which casts serious doubts on the entire premise of this Signpost article. Considering the title previously chosen by the author , it's hard to escape the impression that these conclusions were more motivated by a diffuse anti-AI animus than by a careful analysis of the facts.

Regards, HaeB (talk) 05:10, 10 March 2026 (UTC)

@HaeB: We had serious problems with automated accounts in 2019. I made Wikipedia:Sock farming at that time because I did not know what to call the inhuman looking edits that I was seeing. Perhaps this is not AI but the example you gave, Special:Contributions/Mr.Ludicrous, does not seem like a conscious human is behind it. This looks like automation to me. I am not sure that it is worthwhile to make a distinction between AI and non-human automated editing routines. Bluerasberry (talk) 18:17, 10 March 2026 (UTC)

Given that they used Claude for part of this, I don't think that the motivation here was an "anti-AI animus." Although it definitely wasn't a careful analysis of the facts either. Gnomingstuff (talk) 07:52, 12 March 2026 (UTC)

I offer a compromise solution. Block anyone who adds nonsense sources and can't explain themselves. Also block anyone who is inserting LLM content. Thebiguglyalien (talk) 05:13, 10 March 2026 (UTC)

Yep. This article, and the data, are valuable to me because it confirms my long-held suspicion that not every article deleted under WP:G15 for having completely nonsense sources was LLM generated - but does it matter? Source fraud is an issue, and one which we aren't too good at combatting. I'll take an imperfect solution that works. GreenLipstickLesbian💌🧸 06:30, 10 March 2026 (UTC)

Yup. Crap is crap, and crap must be treated as such regardless of what information-flow and work-flow led to that result. The fact that our material is recognizeable as it flows out to other platforms is a good indication of how reliable and valuable those other platforms consider our content. DMacks (talk) 07:09, 10 March 2026 (UTC)

@Thebiguglyalien: I agree, and moreover, we should block anyone who is inserting ONN content. ONNs constantly make mistakes, and there's never been any proof that they're capable of genuine understanding. jp×g🗯️ 08:14, 10 March 2026 (UTC)

I think I see where this is going. Pray tell, what does ONN stand for? Thebiguglyalien (talk) 15:18, 10 March 2026 (UTC)

Either Onion News Network or LLM I think. ✶Quxyz✶ (talk) 18:13, 11 March 2026 (UTC)

An ONN is a very unreliable type of neural network, which rather than actual thoughts contain nothing more than electricity, and which studies have shown are inherently prone to bias, and use tons of resources to even do simple tasks. But despite this, some ONN bros have been proposing letting them write content on Wikipedia. A lot of the comments in support of this are themselves written by ONNs, which should really tell you a lot. jp×g🗯️ 22:57, 14 March 2026 (UTC)

 Courtesy link: Oscillatory neural network ✶Quxyz✶ (talk) 02:03, 15 March 2026 (UTC)

Might as well be Onion News Network Large Language Model lol.

Is this what the discussions of having a pan-language modular Wikipedia are about? ✶Quxyz✶ (talk) 02:09, 15 March 2026 (UTC)

@Thebiguglyalien: JPxG is attempting to be clever, a trait that does not come naturally for them. It's "organic neural network", i.e., the human brain. --JBL (talk) 17:28, 15 March 2026 (UTC)

That was my guess. I was then going to respond that if he prefers an LLM encyclopedia to an ONN encyclopedia, I wish him luck when he leaves Wikipedia. Thebiguglyalien (talk) 18:01, 15 March 2026 (UTC)

I prefer Wikipedia to Conservapedia, which is the typical place you see comments to the effect that "you're either with us or you're with the terrorists". jp×g🗯️ 05:55, 5 April 2026 (UTC)

I mean, none of the claims are incorrect: it's a bunch of melodramatic humbug that can be said about anything. jp×g🗯️ 06:06, 5 April 2026 (UTC)

I agree, the premise of this article (that AI is responsible for these nonsense reference) seems flawed, I spot-checked the List of Disney television films reference, and that was added in 2020 (by a user who has edited wikipedia as recently as this month), so clearly the issue here isn't necessarily (only) LLMs, however the method described seems like it could be a useful way to spot potential problematic references and flag them for review by a human. I also think that a tool that allows mass undos of such errors/vandalism or at least mass flag suspected diffs would be useful, I recently encountered a subtle vandal from 2010 who was identified as a disruptive editor at the time, and who appears to have only made disruptive edits, but I would say at least half of the stuff they added was still up, and I ended up spending a fair bit of time tracking it down and removing it some 15 years after it was added. Having a way to at the very least flag edits that follow a certain pattern (e.g. adding PMID links to articles with no plausible connection to the subject, or edits by a user identified as a disruptive editor) would help getting this stuff off the encyclopedia earlier (making article maintenance easier and reducing WP:WIKIDEBT), as well as keeping this stuff out of anything downstream from us. Giulio 12:33, 13 March 2026 (UTC)

A very quick browse through the list showed 18 occurrences in Vivian Hunt. What's going on? -- Michael Bednarek (talk) 13:47, 10 March 2026 (UTC)

Now there's an article in need of some tender loving TNT. The PMID refs come from this edit. DMacks (talk) 14:27, 10 March 2026 (UTC)

My 2 cents: AI detection would definitely be good. Even if someone uses it properly, in accordance with all P&G and without producing slop, they'd be indistinguishable from high-quality, human edits. So AI detection would just be AI slop detection, which is sorely needed. --Firestar464 (talk) 16:15, 10 March 2026 (UTC)

I think that it's plausible that AI is responsible for this but this remains a hypothesis, especially since some of the edits apparently date to 2019. Maybe the analysis of editors who added these sources can throw some light on the root cause? Alaexis_¿question? 17:57, 10 March 2026 (UTC)

This was a long-standing VisualEditor bug. See this 2019 ANI thread in which I, too, was fooled into thinking something more nefarious was going on. See also filter 979 (hist · log), which has been tracking this problem since April 2019 (the problem goes well back before that). That said, AFAIK the bug has been fixed. If people are still inserting these references, perhaps AI is part of the problem. Or maybe it's just copy-pasting. Suffusion of Yellow (talk) 20:34, 10 March 2026 (UTC)

This comment should be the first one! I suppose it suggests that OP should repeat the experiment with some higher pubmed ID numbers? //Replayful (talk | contribs) 16:02, 11 March 2026 (UTC)

Yeap. Came here to say this. The last paragraph of Wikipedia:Signs of AI writing#DOIs that lead to unrelated articles states...

Note: From 2018 to 2023, a UX issue in VisualEditor led many editors to accidentally insert references to PubMed articles with low ID numbers (PMIDs), resulting in obviously irrelevant citations like an article about rat livers (PMID 9) being cited in List of Disney television films in 2020. These may resemble AI hallucinations (and should be fixed regardless), but they generally are not.

Someone should probably check the timestamps of the revisions that added these PMID citations. If they are from 2024–2026, then it's ChatGPT and friends. If it's 2018–2023, then it's probably this Visual Editor bug. –Novem Linguae (talk) 02:53, 14 March 2026 (UTC)

The first three examples were still around in 2022, before chatbots were widespread, so they were almost definitely not AI's fault. Sometimes they were around well before 2022; for instance, the Disney rat liver reference was added in 2020. I don't mean to be rude here, but literally the first thing you should do before calling something AI is to make sure it was actually added in the past few years. Gnomingstuff (talk) 03:06, 12 March 2026 (UTC)

For what it's worth, I have been tracking the occurrence of tokens in article text prior to 2023 compared to article text that is AI-generated; I don't include on that page the bottom half of the list, i.e., the tokens more common in human-written articles, because there is an enormous amount of noise and dozens of confounding factors due to template/formatting changes, subject matter sampling, etc. But pmid and related tokens consistently show up at the very bottom of the list, i.e., much more common in article versions written before AI chatbots. Again, there are dozens of confounding variables here but if "ChatGPT thinks that proper Wikipedia articles contain PMID links (PubMed identifiers)" then you'd expect them to not be largely absent from articles most likely generated by ChatGPT. Gnomingstuff (talk) 03:24, 12 March 2026 (UTC)

The trouble is that there are a lot of errors, coming from a lot of sources. For example, for some reason we deprecated {{Cite PMID}}, so footnote 109 in Maternal death is just a big red error message. All the best: Rich Farmbrough 20:09, 17 March 2026 (UTC).