Wikipedia talk:Signs of AI writing

Don't think this section is worth keeping. Unlikely to have been accurate even in the past or with old edits. We don't need a cruft section of "old advice". -- Colin°^Talk 09:08, 7 April 2026 (UTC)

I think this section misses the point by reducing "non-encyclopaedic writing style" down to specific examples that are claimed as AI signs. Most of the "AI vocabulary" words are signs of writing where the author is trying to prove a point, advance an opinion, or convince the reader. It is a common writing style but not encyclopaedic writing. We already cover this in the first two sections on undue emphasis on significance, legacy, trends, notability, media coverage, etc. If you are doing that sort of writing, you will use these so-called AI vocab words. A detailed breakdown of which words were overused by which variants of ChatGPT probably belongs on a nerdy blog about ChatGPT, not a Wikipedia advice page.

The vocab section cites "How much are LLMs changing the language of academic papers after ChatGPT? A multi-database and full text analysis". I think this is typical of the sources here. Someone has gone looking for change in vocab by searching for words they were already prejudiced in associating with AI ("we tracked terms commonly associated with LLM-associated texts.") Not surprisingly they found them. But things identifiable on a statistical level don't necessarily make a good sign. a The statistical comment about is/are phrases can be a consequence of changing matter-of-fact descriptions (which can be rather dry) into convincing arguments for/about something. Wikipedia prose can be very choppy and lacking flow. While we don't want it to be "convincing arguments", tighter or more elegant prose in itself isn't a bad thing. The examples describe text revision, not new text, so aren't actually signs of AI-generated content. That editor may be using AI to copyedit existing text but that's a separate issue.

The "Negative parallelisms" section verbosely describes an argumentative style. Which is not an appropriate style for a Wikipedia article. One of the lengthy examples is also an example of AI text in a deletion discussion. I think there should be a quite separate advice page for signs someone is using AI on talk pages. Because the style on a talk page is quite different to the style on an article.

The "Rule of three" and "Elegant variation" sections don't belong here. Why should WP:RO3 point here? A clue that both are not AI issues is that we have Wikipedia articles on them (Rule of three (writing) and Elegant variation) and a Wikipedia essay from 2018 on one (Wikipedia:The problem with elegant variation). I've got a RO3 example in my second sentence at the start of this section.

There's a 200-word example of a Soviet artist article text. Is that needed to demonstrate what elegant variation looks like? The supposed "repetition-penalty code" in AI is merely an attempt to enforce a copyedit practice that humans impose too. We don't like reading tediously repetitive text, and get irritated when people reuse the same word choice again and again.

I'm not seeing anything in the "language and grammar" section that isn't better handled by the earlier sections that describe superficially decent prose that is written for the wrong audience and has an inappropriate non-encyclopaedic style. We humans can detect such style without going into details. Would you write an essay on how to spot opinionated writing or argumentative prose by trying to teach others which words such writing uses, or whether they form lists of three rather than two or four? Or would you assume that many adult humans are capable of spotting opinionated writing or argumentative prose themselves? -- Colin°^Talk 11:40, 7 April 2026 (UTC)

I think there should be a quite separate advice page for signs someone is using AI on talk pages. Because the style on a talk page is quite different to the style on an article.

I agree. There is a page about AI signs in unblock requests, but that one is significantly narrower in scope, and the signs listed there can appear (and have appeared) in other kinds of messages too, alongside nonexistent shortcuts. – MrPersonHumanGuy (talk) 12:18, 7 April 2026 (UTC)

WP:AITALKSIGNS would be a good shortcut for this hypothetical page. – MrPersonHumanGuy (talk) 21:19, 9 April 2026 (UTC)

The point is that these things show up in AI-generated text far more often, as in a 4000% increase. It is probably the single most useful and applicable way to detect AI-generated Wikipedia writing and/or copyediting (which is also prohibited now per WP:NEWLLM, so I don't understand your point at all there).

The other thing to note is that that these signs are time-bound. AI-generated text from 2024 is empirically and observably different from AI-generated text from 2026. So even if you assume that science is bullshit and none of this is real and people just suddenly decided to start writing like that en masse in 2023 for no reason at all, then they would also have to have stopped writing like that en masse in 2025 for no reason at all, given that most of the indicators of GPT-4-generated text basically disappeared at that time. "Delve" disappeared a while ago, "underscore" has basically disappeared. (I've seen people hypothesize that this isn't because of AI but because people actively changed their writing to stop "sounding like AI," but I really do not buy that, because I highly doubt most people care that much, certainly not enough to create such a dramatic effect.)

I agree that there is some redundancy between the vocab sections and the first two sections, but I'm not aware of any study that makes the explicit connection that LLMs overuse those words specifically in order to do those things, so we don't have a source that we can condense that with. The word "pivotal," for instance, usually shows up in the phrase "plays a pivotal role." "Pivotal" beginning a sentence is much more indicative of human writing. I don't have a source for that besides going through literally every single example on Wikipedia of "Pivotal to" or "Pivotal in" beginning a sentence and noting that virtually all of them were added before 2023. Gnomingstuff (talk) 20:01, 7 April 2026 (UTC)

I think you've completely missed the point. -- Colin°^Talk 08:37, 9 April 2026 (UTC)

No, I think you are being deliberately obtuse just like everyone else who makes this argument, despite the large amount of research showing that they are wrong. If you actually read the studies, which clearly you have not, you would know that the specific indicators were not a common writing style and in some cases were almost unheard of in the corpus human writing until 2023. The whole point of this page is to assist people in identifying AI-generated text; these are the most consistent and thus the most useful identifiers. Gnomingstuff (talk) 06:33, 10 April 2026 (UTC)

To be fair, I wasn't sure if the Yankilevsky text was a good example of elegant variation in AI-generated text either, but it seemed like the best I could find to demonstrate it. If anyone else can find (or has found) a better example, put it at WP:AIELEVAR. – MrPersonHumanGuy (talk) 23:15, 12 April 2026 (UTC)

To my knowledge, AI images are not very prevalent on Wikipedia (unless they are explicitly for the purpose of being AI-generated, like on hallucination (artificial intelligence), but I think these are still a risk, especially because they can be used to create deliberate hoaxes. ~2026-25364-97 (talk) 16:01, 25 April 2026 (UTC)

Perhaps this is something Wikimedia Commons might be interested in? There are definitely a lot of AI-generated images there and not all of them are clearly tagged as such. Gnomingstuff (talk) 07:30, 1 May 2026 (UTC)

Where could I create such a page there? ~2026-16755-69 (talk) 16:25, 7 May 2026 (UTC)

Not sure if this should go with the above topic, so I apologize if this is the wrong place. However, I've noticed that Gemini in particular really loves the phrase "smoking gun". I haven't observed this with other chatbots like Grok, DeepSeek, ChatGPT, or Claude. Not just that phrase, but I feel like a lot of words and phrases common to AI may be underreported since they aren't used superfluously by ChatGPT, which (probably) makes up the majority of AI-generated writing submitted to Wikipedia. Also of note is that while ChatGPT mostly fixed the em dash issue, Gemini still has some problems with it while Grok still displays the incredibly superfluous use of it that old versions of ChatGPT had. ~2026-25364-97 (talk) 16:07, 25 April 2026 (UTC)

Interesting -- I believe you but haven't seen it around Wikipedia, for what it's worth. It might be a result of LLMs associating "formal/encyclopedic/neutral tone" with not using metaphorical phrases like that. (The GPT gremlin/goblin thing also doesn't seem to have made the jump here, possibly for the same reason.)

As far as words/phrases common to other LLMs, it's definitely a partial list, mostly because we're limited to research we can cite and we almost never know what LLM models people on Wikipedia are using. Most of the papers I've seen that compare LLMs use open-source stuff like Llama; this paper compares across LLMs we might actually see, but it doesn't compare them to human text and is a bit hard to extract practical use cases from. (and is also partly AI-based, at least the language analysis parts) Gnomingstuff (talk) 07:29, 1 May 2026 (UTC)

I've mostly seen the phrase "smoking gun" in situations where you give Gemini a handful of clues and it will usually say that one is the "smoking gun". While its use of this phrase is technically appropriate, it uses it more than humans ever would. Although, it does align with most AI words and phrases being mangled metaphors. I don't think you'd see this in an encyclopedic context, but I wouldn't rule it out as AI is terrible at knowing what is appropriate in a specific context.

The main thing from it appearing is, in my opinion, Gemini's tendency to give short responses. I don't think Gemini has a hard limit on token output, since I've seen that it's possible to get it to generate absurdly large amounts of text just by telling it to repeat a word over and over again. If you give it a word limit, it will usually completely ignore it, and if it doesn't, it will repeat the same word a few hundred or thousand times at the end to meet the limit. I haven't seen this problem with ChatGPT or Grok, as both seem to be able to generate large amounts of text without issue. Since Gemini gives very short responses, it isn't used much for AI-generated Wikipedia articles, even if it is a popular chatbot.

The difference between the "gremlin" and "goblin" thing with ChatGPT and the "smoking gun" problem with Gemini probably also relates to how "gremlin" and "goblin" were both terms used to describe users, which would not make sense in a Wikipedia article (AI is quite bad with context and appropriate talk, but I don't think it's this bad). Also, calling users "gremlins" and "goblins" is very unusual and probably drove some of OpenAI's customers away. Even if the prevalence wasn't very high, the words being prevalent at all probably led to customer complaints. In all, the issue wasn't that the words "gremlin" and "goblin" were overused, but that they were used at all. By contrast, calling a piece of evidence a "smoking gun" is a well-known figure of speech, and is not nearly as unusual or offensive as calling users "gremlins" and "goblins". As well, problem solving might be just close enough to an encyclopedic tone for a chatbot to confuse them. ~2026-16755-69 (talk) 16:32, 7 May 2026 (UTC)

The "smoking gun" thing might also be a tone issue, as I remember seeing someone who coded LLMs that noticed that telling LLMs to act like a detective could make them better at solving problems, and it would make significantly more sense for a detective to use the phrase "smoking gun" than the average person. Not sure if Gemini does this, especially since this detective thing was in 2023 or 2024. ~2026-16755-69 (talk) 16:36, 7 May 2026 (UTC)

I don't think this specific example is a very good indicator of LLM use:

In the United States, university-based incubators and accelerators have expanded alongside these centers; an official Library of Congress review found that 31.5% of SBA [Small Business Administration] Growth Accelerator Fund Competition winners from 2014–2016 were university-based programs.

Many humans can and do use WP:INTEXT attribution for uncontroversial facts even though it's discouraged. Even when I was in grade school, teachers sometimes encouraged attributing articles like this. Accessedgrant (Epicgenius mobile alt) (talk) 02:54, 4 May 2026 (UTC)

I also don't get how the 31.5% statistic can be considered "uncontroversial" enough to where it would be wrong to provide attribution for it. It's probably not as uncontroversial as the sky being blue or the Earth not being flat. – MrPersonHumanGuy (talk) 17:58, 4 May 2026 (UTC)

I agree. I propose removing this - in this specific example it's just as likely that a human actually would have added in-text attribution. Accessedgrant (Epicgenius mobile alt) (talk) 02:29, 5 May 2026 (UTC)