User:Gnomingstuff/AI tells (species articles)

From Wikipedia, the free encyclopedia

This is a list of common tells for AI-generated species articles, a supplement to our larger article about AI tells on Wikipedia. These tend to follow an extremely formulaic pattern even by LLM standards, which is immediately obvious after looking at many of them but may be harder to identify in isolation.

While this text may appear superficially "OK," it is not guaranteed to be based on real sources, true to what the sources actually say, or sourced at all. It is also heavy on speculation, opinions, puffery, which are not only against our policies -- verifiability, original research, synthesis, our Manual of Style, etc. -- but hollow and meaningless considering that every single AI article says the exact same things.

This list is not comprehensive and describes only one way AI text may show up.

It is unclear what training material the LLM reproduced, as this very rigid pattern is not especially common in non-AI species articles. Parts of it may be based on the text of IUCN Red List entries.

If you are not a Wikipedia editor

This article structure is specific to Wikipedia, about the rigid, idiosyncratic way that LLM output tends to shows up when asked to generate a "Wikipedia-style article" on a species. It is not intended as a statement about any other form of writing. You are allowed to use curly quotes, em-dashes, title case, and correct grammar. The fact that LLMs disproportionately use them on Wikipedia, in ways that did not really appear prior to generative AI, has nothing to do with you; it is not a statement about your academic or content writing legitimacy, and your career will not be ruined by AI accusations.

If you complain that we are policing your style anyway, you are revealing that you didn't read the text carefully -- which, ironically, does reflect poorly upon your analytical skills.

Examples

This is a chronological listing of (probable) AI edits to/versions of species articles, made over the course of ~3 years by unrelated editors (though some editors are prolific and appear multiple times). By viewing these edits as a group instead of in isolation, the pattern will quickly emerge -- particularly in later articles.

More information Extended content ...
Close

Confirmed/disclosed examples

These are rare instances of the LLM and/or version actually being disclosed. Some are longer and less formulaic than the above list.

General characteristics

All AI tells about verbiage, formatting, etc. may apply here.

Editor info

  • Sometimes come from Newcomer Tasks, specifically the biology category
  • Can be dozens/hundreds of people each making near-identically structured one-off edits or one person making a string of edits all about the same kind of species
  • Edits may be one large edit or piecemeal additions
  • Edits are probably not in bad faith -- they tend to be good faith attempts to improve articles, regardless of whether they have actually succeeded at that -- and probably not sockpuppets.

Formatting

  • All Title Case Headers with relatively short body text are very common here. The headers may sometimes be formatted as bold text, or headers with boldface text (very uncommon in non-AI wikitext).

Puffery

  • LLMs, as usual, will endlessly glaze whatever it is they are generating text about.
  • Sample phrases: the diverse and complex ecosystem, making it a notable presence (LLMs love to say things are "notable," probably mimicking editors signaling at WP:GNG), known for its rich marine biodiversity, these pristine environments provide the ideal setting, attractive appearance and interesting behavior, not only highlights the artistic ability of the species but also underscores its ecological importance, a fascinating species that exemplifies the diversity...

Assertions of significance

  • As with LLM text, such articles may not merely describe a species but spitball about how important they are to the world or contribute to a lineage; not everything in life plays a "crucial role" in stuff.
  • Sample phrases: integral part of the marine food web, contributing to the ecological balance, This ecological dynamic emphasizes the plant's importance as a hub for biodiversity, underscores its ecological role, contributing to the overall dynamics and biodiversity, contributes significantly to biodiversity, These insights situate [species] within a lineage, aligns with the geological context, reflecting the challenges of studying, highlights the rich biodiversity of these regions, contributing to our understanding of its evolutionary relationships.
  • This may even happen for even the most mundane stuff, e.g., formally described in scientific literature, contributing to the taxonomy and biodiversity records..., intricately classified based on its morphological and genetic traits, unique etymology highlights the species’ distinct traits, the existence of these synonyms reflects the historical challenges

Speculation

  • If a LLM using retrieval-augmented generation or web search does not turn up species -- which is likely for species articles that are stubs -- it will output a sentence to that effect. These tend to appear in the same places in articles, and are not usually woven into verbose, complex sentences unless the text has been edited.
  • Known phrases: while specific details are tq, relatively obscure, detailed descriptions of its morphology, limited in available literature, though its specific interactions within ecosystems are not well-studied, not well documented in public databases, not widely accessible, While little is known, not readily available, remain to be comprehensively documented, detailed records are sparse, not widely published, detailed, standalone morphological description... is not widely accessible outside of its original taxonomic publication, specific details... may require further research, extensively available in common databases, not detailed in readily available information, limited in accessible scientific literature...
  • Some may just mention the search process outright: without access to [the] original description, exact details remain uncertain, not detailed in the search results, not included in the brief reference, specific pages... were not accessible in the search, (if available from updated databases or studies, specific species names could be mentioned), limited in the provided sources
  • Then, the LLM may speculate on the answer based on what "similar" species are like, violating WP:OR and WP:SYNTH. If so the sentence will start with "despite," "though," etc.
  • Known phrases: it is likely that, like other members of its family, However, its genus..., as with many marine species, observed in related species, can be inferred by its, that would likely apply, Additional field research could help clarify, presumed to follow, current understanding... suggests the following, allows for general inferences, presumed to be similar...
  • Note: Even the claim that a species is "not documented" is itself speculation. Just because the AI didn't find sources does not mean they do not exist.

Specific sections

Note: None of these sections are signs in isolation. It's just that with AI articles, they almost always show up in these near-exact ways.

Key Features

  • LLMs disproportionately include "Key ____" headers in general, which are not unheard of from human editors on articles about technology, software, etc., but are not otherwise common for species articles.

Taxonomy header

  • When multi-header articles are generated this will almost always be here
  • If the species appears to be named after a person, the LLM may say so in a way that sounds like it's speculating
  • May contain pointless statements about taxonomy in general in LLM's endless fixation on ascribing meaning to everything, e.g., This taxonomic classification provides a way of categorizing and identifying...

Description header

  • Note before complaining: This is particularly common in non-AI articles too! If it shows up here it is often as a parent header for the below.

Morphology and Identification header

(Habitat and) Distribution header

  • May go off on tangents describing the habitat and not the species

Diet header

  • Less common

Predators header

  • Less common

Ecology header

Toxicity/Venom/Culturing/etc

  • Not common
  • Generally serves as a warning to potential readers, likely the LLM trying to cover its ass
  • Typical phrases: handling live specimens is discouraged, It's important to note that cases of toxicity..., Here are some important factors to consider when culturing...

Conservation header

  • Optional: Sentence about how little is known, the species is not common, etc
  • Conspicuous mention of conservation status -- even if the status is unknown and no serious efforts exist. May or may not be sourced.
  • As in the Speculation and Assertions of Significance sections, speculation about what future research is "needed," unattributed to a source, and often in a call-to-action style. Any claim that something is "needed" is an opinion -- nothing in the world is objectively a need.
  • Example phrases may include further studies are needed to provide a comprehensive description, Further research on this species could provide valuable insights, crucial for its continued survival, Continued research and monitoring are necessary to fully understand, underscoring the need for further paleontological research, Conservation efforts are essential to mitigate, studies are important for the conservation of biodiversity, If more places around the world could adopt policies like this, we could see drastic change, Future research... could focus on its ecological role, remain crucial for ensuring the long-term survival.
  • Note: Versions of above concluding sentences are also common in non-AI articles -- probably the ones LLMs are trained on. In particular, "further research is needed" is not, by itself, an AI tell.

Significance header

  • May be rolled into the above headers

Anti-tells

  • Articles that deviate significantly from this structure, or are generally long and dense, are unlikely to be primarily AI-generated. They may contain new AI-generated rewrites or content, or they may have been substantially built upon by human editors. For obscure species, this is unlikely; these articles were often previously substubs for years. Note: Edits prior to mid-2024 seem to follow this template less rigidly, though they still display AI tells.
  • In non-AI articles, "Further research is needed"-type statements are often placed in quotation marks and attributed to sources. AI articles do not seem to ever do this.

Non-examples

These edits unambiguously did not involve generative AI, being done prior to 2023. While they contain some of the indicators above, they do not adhere fully to this structure, in ways that will be clear upon comparison.

Related Articles

Wikiwand AI