Wikipedia:Village pump (WMF)

Taking a step back: could AI suggestions be beneficial?

As pointed out above, the current development stage is way too early for a broad rollout. This should have been better clarified, both to reassure the community about the experiment, and to provide clearer development goals.

However, taking a step back, a discussion can still be held regarding the potential of this whole endeavor. Would the community, in theory, agree to an AI model surfacing suggestions to newcomers in such a way, assuming the current pitfalls could be smoothed out in development? More concretely, are these expectations realistic, and is it worth investing further resources in this project?

These are questions I don't, personally, hold the answers to. However, we should be discussing them together, alongside members of the Editing team involved in its development (courtesy ping to @Quiddity (WMF)), if we want them to work in sync with community sentiment, and avoid investing resources in dead ends. Chaotic Enby (in solidarity · talk · contribs) 23:57, 15 August 2026 (UTC)

Would the community, in theory, agree to an AI model surfacing suggestions to newcomers. I think a better way to explore this tool would be to make it available to established editors first. People who have the experience and policy knowledge to be able to properly evaluate the suggestions. Maybe the people will say "The suggestions were all spot-on and incredibly valuable". Maybe they will say "Nothing this thing suggested made any sense at all, it's total garbage". More likely, somewhere in between. But let's do the experiment rather than pre-judging it. RoySmith (talk) 00:08, 16 August 2026 (UTC)

I think a better way to explore this tool would be to make it available to established editors first. While it might not have been clear at first, this is, in fact, exactly what the experiment is planning to do. The tool is still in development, and we can't say, in advance, how it will end up in terms of quality. For now, I'm just trying to figure out the proportion of editors who either will find it a non-starter in principle (regardless of the suggestion quality) or are opposed to investing further resources in its development for any other reason. Chaotic Enby (in solidarity · talk · contribs) 00:20, 16 August 2026 (UTC)

Mark me down as "opposed to investing further resources in its development for any other reason". Wikipedia has a large amount of technical debt that would be easy for a WMF developer to fix, but instead they are doing this?

As one example, Arbcom is a very important function, and many people who participate get stressed over whether they are under the word count limit. But the tool that puts a banner at the top of your comment fails to accurately count your words! Worse, you can't invoke it while composing -- you have to post and hope that you didn't go over. And if an arb replies to you inline, that increases your count! This is the sort of thing that a WMF developer could fix in an afternoon. It's important, but making an accurate arbcom word counter will never hit the top of any survey of things many editors want to see fixed.

Another example: what happens if when I sign this comment I accidentally hit the "~" key three times instead of four? How about 5, 6 or 7? This is a typo that happens again and again. How hard would it be for a WMF developer to make it so that you get a "are you sure" message before accepting a signature that is almost always a typo?

There are hundreds and hundreds of these easy to fix things that depend on old scripts written by volunteers and all too often no longer maintained.

I think the WMF should spend a significant amount of developer effort -- 75% or 80% -- fixing these small, non-sexy quality of life issues and only then devote the other 20% - 25% to fun things like AI suggestions. --Guy Macon (talk) 01:02, 16 August 2026 (UTC)

Or, if you don't like the above 4-tilde signature, --Guy Macon (talk) (3 tildes),  --01:13, 16 August 2026 (UTC) (5 tildes), or --01:13, 16 August 2026 (UTC)Guy Macon (talk) (7 tildes).

Guy, on that last one, you don't need to add a signature at all anymore. Discussiontools will do it for you. I haven't added one to the end of this post, for example. In solidarity, asilvering (talk) 01:04, 16 August 2026 (UTC)

Will I (Guy Macon) get an error message for this unsigned post or do I have to make a preferences change to get the autosign goodness? Show preview says it will post the unsigned comment with no error message.

Discussiontools is a nice tool, but it is no substitute for software baked into Wikipedia that checks for common errors and throws up an "Are you sure?" message. I could give you a hundred examples of places where the WMF is depending on unpaid volunteers to maintain basic functions that keep the Encyclopedia running smoothly while focusing on the exciting new stuff --01:30, 16 August 2026 (UTC) Guy Macon (talk)

Guy would need to be using DiscussionTools for that, but unfortunately they are using classic full page source editing where none of those helpful things will occur. DLynch (talk) 13:13, 16 August 2026 (UTC)

FYI I made Module:Word count, I can refine this further if you think it would be useful. It does not 100% solve the issue you mention Czarking0 (talk) 15:57, 16 August 2026 (UTC)

Although I agree with a lot of this sentiment, I think an organizational strategy which places say 20% of the engineering resources on long term tech rather than present issues is reasonable. As long as the development of this feature counts under that I do not see the problem. Czarking0 (talk) 16:00, 16 August 2026 (UTC)

+1, that seems to be what is happening here (see the comment below about 'avoiding sunk costs' and getting feedback early and often from established editors). I can see individual categories of suggestion being useful to experienced editors, focus on making something that works for them before considering anything that might be visible to newcomers. (They already have Special:Homepage) – SJ + 23:14, 18 August 2026 (UTC)

I agree with Fram above in that this feels like a way (though a much more subtle way than Simple Summaries) for the WMF to influence editorial decisions which, with the exceptions of legal reasons, they just should not be a part of. ♠JCW555 (talk)♠ 01:55, 16 August 2026 (UTC)

There isn't really any plans for the WMF to get involved in editorial descision (and I say that as somebody who has through m:PTAC reviewed the Annual Plan). The plan for Edit Suggestions is purely meant as a assistive tool to help editors and kinda comes from the idea of being able to surface gadget/userscript suggestions to everyone without having to have coding knowledge. Sohom (talk) 02:57, 16 August 2026 (UTC)

But the fact that the AI is making these suggestions at all in the first place is the WMF having a subtle hand on editorial decisions in my mind. The examples Gnomingstuff lists above, like the FreedomWorks example, is an example of the AI making an editorial judgement that it should not be doing. Some of the others are more subtle like the DeAndrea G. Benjamin example, but they're still editorial decisions that the WMF shouldn't be engaging in period. ♠JCW555 (talk)♠ 03:23, 16 August 2026 (UTC)

They are using generic models without fine tuning for these suggestions, so out of all the parties that could be said to have made editorial decisions in this scenario I would say OpenAI and Google would rank above the WMF, and I really wouldn't consider either of those companies to have made any editorial decisions... the problem IMO is potentially encouraging uh... not really making any editorial decisions, and vibing through things without considering the context of, e.g. the contents of the sources as we are supposed to. Alpha3031 (t • c) 08:28, 16 August 2026 (UTC)

It is not worth creating AI prose suggestions for newcomers (especially if it takes a year for each model!). Putting aside quality questions, editors here are expected to be competent and able to contribute in English. While there are different ways to be confident, we expect the ability to read and write in English, and thus to some extent to have the tools to be able to copyedit themselves. We also expect editors to be able to read and comprehend our guidelines and policies (pre-emptive clarification, read, not memorise). The best way for us to be able to understand competence in these areas, and thus to be able to assess and offer advice if needed, is to see their edits. Seeing instead a whole slew of new editors making the same llm-prompted changes is harmful to the community in being able to understand and accommodate new editors, and harmful to the new editors in giving them a misleading picture of how things work, and more harmful when the llm doesn't understand our policies and practices (as this one does not). Apologies to Sohom, but "There isn't really any plans for the WMF to get involved in editorial descision" just isn't true if this sort of system is being set up. This sort of suggestion task is the WMF making editorial decisions, even if they're making it through an llm. There are many reasons time would be better spent anywhere else. (I distinguish "prose suggestions" from say the add-a-link task, as that at least teaches a technical competence that we do not expect new editors to have. Such teaching does seem useful, although as mentioned above it would be nice if it was developed further.) CMD (talk) 03:45, 16 August 2026 (UTC)

Seeing instead a whole slew of new editors making the same llm-prompted changes is harmful to the community in being able to understand and accommodate new editors I just want to emphasize this -- the English Wikipedia community is not good at welcoming newbies who make mistakes. That is a problem.

The English Wikipedia community is actively hostile to editors making mistakes with large language models.

If a newbie puts a poor LLM-based reword into an article, an experienced editor is almost certainly going to WP:BITE them off. And a newbie, operating in a system they're unfamiliar with, with a human-sounding voice telling them "This copyedit is right", is never going to have the knowledge and is almost certainly not going to have the confidence to challenge the AI when it presents them with a bad suggestion. That is going to set them up for failure when a grumpy human editor, burnt out from dealing with LLM edits, callously reverts them.

I think using machine learning to help with encyclopedia maintenance is a wonderful thing! And I think large language models are really cool -- but they require a high degree of skill to use correctly in the manner that looks like it's being explored here. And, again, the community is so burnt out from dealing with poor quality LLM content that any editor who uses this tool and (inevitably) makes a mistake will be attacked by the community. Does the team working on this understand that, @MMiller (WMF)? GreenLipstickLesbian💌🧸 05:10, 16 August 2026 (UTC)

Seconding that this could be helpful for all sorts of maintenance but should be aimed at experienced editors while working out kinks.

I would personally like to see rubrics for highlighting potential vandalism and LLM edits. And I want much less text in my sidebar and zero suggested text: just a few words indicating the kind of issue to look for / the kind of style guidelines to check.

And I'd like to see explicit self-evals of each rubric for suitability to the task and for false positive/negative rates, which could also be compiled and developed by community maintainers. – SJ + 23:52, 18 August 2026 (UTC)

@GreenLipstickLesbian -- I think that the most important thing that would prevent against the situation you're describing is that the suggestions wouldn't propose text for the newbie to accept/reject. It would just point out the spot in the article that needs attention, e.g. "Does this sentence need to be rewritten to be easier to read?" -- it would not give them a re-written sentence to add. I know that in that situation, the AI may be wrong about whether the sentence needs to be rewritten, and the sentence may be perfectly fine -- but the newbie might be like, "Hmmm, well I guess I'll reword it?" and they may make a pointless edit, or may make the sentence worse.

Suggestion tasks like "add a link" and "add an image" actually do give newbies specific edits to accept or reject, and we see them generally apply good judgment and be constructive. Yes, many of them mess up and get reverted, but that might have happened to them anyway if they were left to their own devices. So these tasks cause a bunch of things to happen at once, and the question is whether it all adds up to a net positive or net negative, you know?

What do you think? MMiller (WMF) (talk) 21:13, 19 August 2026 (UTC)

@MMiller (WMF) Thanks for the response! Yes, I think these sound like reasonable precautions. I do really want to emphasize the point about making it clear to experienced editors what the newbies actually are seeing is very important.

Having had the beta experimental version of suggested edits enabled for a few days, I definitely see the vision. And I see how it could be very useful. (My response to many of the "consider adding a citation" suggestions has, admittedly, been a)"lol no, I'm removing the unsourced text for other PAG issues", b)"this is cited, it just needs an inline citation", c) "... yes, i see how citogenesis is made", or, d)"... we need an easily accessible essay on 'how to source a statement' that we can link to the newbies here" because wow, i'm having trouble". ) But I also see the biting that happens when newbies do make pointless edits/less than ideal edits ( has some which took more than a few years to rectify), and so I am worried about adding any "they're just thoughtlessly adding machine suggestions to articles"-esque ammunition, even it's it's not strictly true. GreenLipstickLesbian💌🧸 21:33, 19 August 2026 (UTC)

This is why I suggested some basic FAQ type popups upon first edit, including an agreement to not use LLMs, to prevent true good faith mistakes and remove plausible deniability for the rest. ChompyTheGogoat (talk) 17:37, 26 August 2026 (UTC)

RE: especially if it takes a year for each model! to be fair to the teams involved the stated idea behind this specific experiment seems to be wanting to try something that won't take a year per task on what the editing and ML teams think are relatively low risk tasks... I'm just not sure that the community and the teams involved have a sufficiently compatible idea of what tasks are low risk. I think it would be possible to develop something acceptable to the community, but unless very sure about it, it may be best to get a vibe check from the community before thinking something is "low risk", and treating things as "high risk" otherwise. Alpha3031 (t • c) 08:36, 16 August 2026 (UTC)

Would the community, in theory, agree to an AI model surfacing suggestions to newcomers in such a way – I won't, and if that ever happens I'm gone. Models are bias black boxes, and even if suggestions were 100% accurate this would still be an issue. fifteen thousand two hundred twenty four (talk) 04:37, 16 August 2026 (UTC)

If AI had far more quality control, I wouldn’t be opposed to this. Except it doesn’t yet. LLMs hallucinate, and those hallucinations are not something you want informing newcomers who are near-clueless as to how Wikipedia works, let alone people learning English who won’t be able to tell when an AI-based edit suggestion system instructs them to insert a grammatical error or something similar, like the above pyusician —> musician.

This could probably work in theory. But it would assume an AI with a reasonable knowledge of the given article subject, some common sense, and a degree of fluency in English (by which I mean not doing things like pyusician/musician), and, given the above examples provided by Gnomingstuff, this is definitively not that.

I oppose AI-based features being integrated into Wikipedia software, and will continue to do so unless a day comes when LLMs equal or surpass the common sense of a human. Perhaps in a few years LLMs will be reliable enough for this to work smoothly and without issue. With the current state of AI, I don’t think we’re there yet. Cheers, 𝔰𝔥𝔞𝔡𝔢𝔰𝔱𝔞𝔯 ^{(𝔱𝔞𝔩𝔨)} (any/all) In solidarity. 04:59, 16 August 2026 (UTC)

The AI could work 100% of the time and I'd still oppose it because these edit suggestions are being directed by an AI that's controlled by the WMF, which outside of legal purposes, should never have any editorial control on Wikipedia in the first place. ♠JCW555 (talk)♠ 05:17, 16 August 2026 (UTC)

That’s a fair point; I hadn’t thought of that. (Yet another reason why this is a terrible idea.) Cheers, 𝔰𝔥𝔞𝔡𝔢𝔰𝔱𝔞𝔯 ^{(𝔱𝔞𝔩𝔨)} (any/all) In solidarity. 05:39, 16 August 2026 (UTC)

The solution for this is making prompts publicly available and allowing each community to tweak them. Alaexis_¿question? 06:36, 16 August 2026 (UTC)

LLMs hallucinate, and those hallucinations are not something you want informing newcomers who are near-clueless as to how Wikipedia works, let alone people learning English who won’t be able to tell when an AI-based edit suggestion system instructs them to insert a grammatical error or something similar, like the above pyusician —> musician. - that's a rather odd example to get hung up on, given that automated spell checkers and their admittedly sometimes absurd correction suggestions have been around for decades (long before the term "hallucinations" came to be used in this context). Many or even most professional writers - journalists, book authors, scholars - use them routinely, and have learned to live with the occasional fail of this type (pyusician —> musician).

Indeed, in stark contrast to your logic here, WP:SPELLCHECK has long described them as potentially useful for Wikipedia editors, too, as long as they don't blindly rely on such tools:

Spellchecking software and online tools can be helpful when copyediting Wikipedia articles. [...]

• No spellchecker is completely accurate. You must check the output of any tool you use. [...]
• You are responsible for all spelling and grammar changes you make, even if the changes are suggested by an error-checking tool, such as Grammarly or ChatGPT.

Maybe it is time to conceive of Wikipedia editors a bit more as adults who are generally capable enough of using such tools even though their suggestions are not 100% error-free (very few things are).

That said, I do generally agree with your point that AI needs quality control, and folks should definitely ask if WMF has done enough here yet (I'm not sure it has). It's just that - as the spell checker example shows - it's not realistic to demand 100.000% accuracy, or to point to isolated failure cases without assessing how frequent they are. (To be fair, User:Gnomingstuff did already make an informal heuristical effort at the latter with regard to their list above: it took me only about ~1-2 hours to find the above issues, and that was only a spot check, i.e. there is informal evidence that these are not very rare at this point. But ultimately I'd be interested in more concrete assessments of how likely editors using this tool will be to encounter each of those failure cases in practice.)

Regards, HaeB (talk) 06:54, 16 August 2026 (UTC)

Most adults have a lot more experience with spelling than they do with Wikipedia's guidelines. LittlePuppers (talk) 06:58, 16 August 2026 (UTC)

Sure, but that doesn't mean that we keep the edit button away from them.

See Wikipedia:Competence is required, which is perhaps a better known version of this principle that we generally expect editors to be competent adults (metaphorically, with apologies to all the very smart and capable teenage editors among us) that do not require special restrictions and safeguards to protect them against their own mistakes.

Regards, HaeB (talk) 07:30, 16 August 2026 (UTC)

The thing I’m concerned about is everyone knows that spellcheck is obviously not infallible, as the Internet often likes to humorously point out. But to a newcomer, anything that comes from ‘Wikipedia’ seems naturally correct and reliable.

Certainly not everyone would fall into this trap, since, as you point out, it isn’t like all newcomers are to be treated as children who don’t know what they’re doing. But it’s likely that some would; perhaps enough that the consequences of Wikipedia’s reputation for factual accuracy is something we should take into account on this. Given all of that, I don’t think we can draw a one-to-one comparison between run-of-the-mill spellcheck and an edit suggestion feature built into Wikipedia itself.

That’s why I argue that we should hold ourselves to a higher standard; after I’ve given your points some thought, however, I think you are correct that my own request for equal [to] or surpass[ing] human-level reliability is asking a bit much of a literal artificial intelligence. Simultaneously, I think spellcheck-level absurdity is far too low a bar for this feature.

Cheers, 𝔰𝔥𝔞𝔡𝔢𝔰𝔱𝔞𝔯 ^{(𝔱𝔞𝔩𝔨)} (any/all) In solidarity. 09:11, 16 August 2026 (UTC)

Sounds like something that could be addressed with a user level disclaimer about LLMs before the feature is enabled on their account. Czarking0 (talk) 16:02, 16 August 2026 (UTC)

That would definitely fix that issue (I’m surprised I didn’t think of that, to be honest). Cheers, 𝔰𝔥𝔞𝔡𝔢𝔰𝔱𝔞𝔯 ^{(𝔱𝔞𝔩𝔨)} (any/all) In solidarity. 00:11, 17 August 2026 (UTC)

I agree with @RoySmith that some suggestions are more suitable for experienced editors. I believe that checking citations is a good use case with an LLM flagging potentially problematic citations and editors verifying them manually (we have a proof-of-concept that I've been maintaining but as long as it's a userscript it won't make a dent in the sourcing problem). This is a task that is not too complex but requires familiarity with WP:RS and some experience. Alaexis_¿question? 06:35, 16 August 2026 (UTC)

Based on the suggestions here, my stance is basically the same as what WP:LLM already says: typos and punctuation suggestions seem unnecessary but mostly fine (except when they're not, see the "physician" example), anything beyond that is unlikely to be fine, and suggestions related to NPOV are light years away from fine. 100% accuracy isn't possible but, realistically speaking, people are going to rubber-stamp whatever suggestions they receive.

As far as the spot-check goes, the timeframe is not really all that scientific since I actually saw this last night (not even while doing AI stuff, I was doing Commons new image patrolling and saw one of the screenshots from it), slept on it, and did the full writeup the next day. I also have seen tens of thousands of AI edit suggestions, as well as thousands of snippets of parsed Simple Summary text, so I already knew the places where this was likely to run into issues. This is also why I think the "experienced editor"/"new editor" dichotomy is not helpful, in general. Experienced editors are not necessarily experienced in copyediting and/or the specific problems that crop up with AI, and newcomers are not fragile baby birds who have never edited a piece of writing in their lives. Gnomingstuff (talk) 07:10, 16 August 2026 (UTC)

Automated suggestions could be helpful in very limited circumstances. I can imagine an experienced editor who has chosen to opt in to them assessing each proposal carefully, skilfully rejecting those with the sorts of problem listed above, and improving Wikipedia by acting only on the good ideas. I'm sceptical that AI is the best way to produce such input but I'm willing to be proven wrong. Methods such as this are totally unsuitable for new editors, many of whom will blindly obey The System, make bad edits in good faith, and get reprimanded or blocked. There is also the unoriginal but valid argument that editors would prefer the WMF to devote its resources to other activities, even if we don't always agree exactly on which ones. We're seeing a lot of these "Here's a new toy no one asked for that we've secretly blown this year's development budget on" announcements. I do honestly try to hope for the best, but I'm afraid my initial reaction has become "Oh no, what have they broken now?" Certes (talk) 09:57, 16 August 2026 (UTC)

No, LLM suggestions would be the opposite of useful. They would only reduce the amount of thinking an editor does, leading inexperienced editors to potentially make an edit because the LLM said so. People trust tool output, so if wikipedias own tooling gives them a suggestion, however ridiculous, they might easily do it.

This should be stopped at all costs. TietoTeekkari (talk) 19:48, 20 September 2026 (UTC)

Wouldn't it make more sense to focus on basic copyediting suggestions? I just went to a random article, Mircea II of Wallachia, and requested some copyedits from chatgpt.

Most of these seem like pretty decent suggestions, don't dip into npov/tone danger zones, and are simple enough for new editors to review and handle. ScottishFinnishRadish (talk) 00:14, 20 August 2026 (UTC)

Also, that might be the first time my first random article click wasn't about a sports ball player. ScottishFinnishRadish (talk) 00:15, 20 August 2026 (UTC)

the nice thing about having an insite spell/grammar checker would be less people would use Grammarly (we could tell people using Grammarly to use the insite one instead, similar to how PasteCheck works). I've found asking a chatbot for corrections gives decent results, but obv you can't defer to them (esp. on content) Kowal2701 (talk, contribs) 19:59, 20 August 2026 (UTC)

Why is it a problem that people use Grammarly? RoySmith (talk) 20:02, 20 August 2026 (UTC)

it goes beyond its scope as a spell/grammar checker and rewrites content, and a lot of people using it don't know it's LLM-powered. It's the worst because people will put time and effort into writing, then put it through Grammarly which turns it into slop. We see it at AINB a fair bit Kowal2701 (talk, contribs) 20:10, 20 August 2026 (UTC)

Context from Editing Team

Hi y'all – I'm Peter Pelberg, product manager of the Editing Team. Together with the Machine Learning Team, we've been working on the experimental model-generated edit suggestions. You have raised a range of valid concerns/questions that warrant responses. You can expect those when I'm back online in earnest next week.

In the meantime, I'd like to clarify some aspects of the work and the thinking that's informing it.

First and most importantly: there are no plans right now to deploy LLM-generated edit suggestions as default-on to anyone. The closest thing to a deployment that we've talked about is a potential A/B experiment that would not move forward until we, volunteers and staff, have deemed these suggestions reliable and promising. This commitment is in Phabricator by way of the experiment (T431377) needing T428311 to happen first. For context, T428311 states, "Learn whether experienced volunteers at en.wiki think the experimental suggestions are sufficiently reliable to be shown to newcomers so that we can decide: Will we invest the effort to scale this initial set of LLM-generated MoS suggestions across languages via T431376?"

Now, with regard to what this initial set of 30,000 LLM-generated suggestions is and is not…

1. Who has access to these experimental suggestions? In this initial, experimental phase, these suggestions will only be available to people who A) enable the Suggestion Mode beta feature and B) install this user script. Note: We will soon introduce a setting within Special:Preferences so people interested in trying out experimental suggestions don't need to install a user script to access them.
2. What do these experimental suggestions do? This initial batch contains experimental suggestions for three types of improvements/issues (listed below). Each suggestion will highlight the span of text it is relevant to, offer a generic description of the issue, and crucially, ask the experienced volunteers encountering it to indicate whether you think the suggestion itself is valid/useful or not. And if not, offer an explanation as to why. Said another way: These suggestions do not offer fixes or ask experienced editors to make any.

   

   

3. What are "experimental" suggestions? "Experimental" suggestions are a set of – as I think @Sohom Datta put it well – pre-alpha suggestions. The purpose of them is to evaluate their reliability. Currently, there are a total of 6 experimental suggestions available at en.wiki to people who opt-in to seeing them. 2 of these 6 suggestions are powered by machine learning models; the other 4 are based on deterministic heuristics. You can see the full list here: Special:EditChecks#Experimental checks.
4. What types of suggestions is this LLM generating? This initial batch contains suggestions for simplifying language, rewriting language in a neutral point of view, and adjusting place-based names to follow Wikipedia's geographical guidelines. We are by no means committed to this set of suggestions or to using LLMs in general. We chose this initial set because we found them to be reasonably accurate and assumed they would be relatively low-risk.

Last thing for now: we are aware of the sunk cost fallacy. Thank you for raising it, @Kowal2701. In fact, as I hope the above demonstrates, the whole point of developing experimental suggestions, and making them available in production to experienced volunteers who have explicitly opted into them, is as @Chaotic Enby described: to assess whether they're reliable. Ones that aren't, we will abandon. The ones that are, we'll work together to figure out how and to whom to make them available.PPelberg (WMF) (talk) 05:48, 16 August 2026 (UTC)

as […] “1.” and “3.” demonstrate. This is perhaps not the most important point, but it is bugging me: were the numbers meant to all be “1.”? Comment struck due to this being fixed; apologies. Cheers, 𝔰𝔥𝔞𝔡𝔢𝔰𝔱𝔞𝔯 ^{(𝔱𝔞𝔩𝔨)} (any/all) In solidarity. 06:08, 16 August 2026 (UTC)

Thanks for responding now and letting us know you don't have time to fully engage immediately, hopefully you will have time to go through all the comments above later in the week as you say. As the process of contingent experiments has been brought up again, perhaps it would help to answer the experimental question plainly, because it can be very easily answered without a complicated multi-month experiment. The suggestions are not reliable, and definitely not reliable enough to show to newcomers (not that this is the only metric that could be considered). To look further, re We chose this initial set because we found them to be reasonably accurate and assumed they would be relatively low-risk, the linked section lacks an assessment of accuracy or risk. Part of the issue here is that a simple glance is all that is needed to identify issues with many the suggestions, so it feels like even that is not being done before these are farmed off to volunteers. There is a mismatch in understanding somewhere. CMD (talk) 06:18, 16 August 2026 (UTC)

Part of the issue here is that a simple glance is all that is needed to identify issues with many the suggestions, so it feels like even that is not being done before these are farmed off to volunteers.

Honestly can't put it any better than that. I do appreciate the response and understand it is the weekend. Gnomingstuff (talk) 07:18, 16 August 2026 (UTC)

Tbh this is a systemic issue to do with WMF governance rather than a reflection on anyone here, where developers and idea labs seem to typically be distant from the projects and communities, and everyone at both ends is left straining to compensate for this. Phab being largely open to the public goes a long way, but I wonder if there could be something like a WP:VPI on meta for developing/screening ideas (obv staff's medium is meetings and private convos etc. and idrk what currently happens, but something like "John and I were talking yesterday about this, X is a potential solution" would be good to get some feedback at the earliest stage). I've previously suggested that staff should be encouraged to do a little bit of editing each week at any wiki they choose (and reduce hours/week a bit for this while keeping salaries the same), but it risks the WMF's status as a 501(c)(3) organization (I was told) Kowal2701 (talk, contribs) 07:57, 16 August 2026 (UTC)

I don't really know that it has anything to do with governance in this case. The issue would be the same regardless of whether the development process was public or semi-public or completely closed off; it's a classic, perennial issue of workplaces. The issue being -- and I am really trying to be as polite as possible here -- that QA is either not being done, or not being done correctly.

From what I understand, the suggestions were assessed for quality via AI, and then someone seems to have reviewed 15 suggestions manually. Unless there's more internal discussion (there probably is, this stuff is nearly impossible to follow), that seems to be the whole QA process.

Meanwhile, I actually have worked in QA. For a task like this, we would be given a spreadsheet like the ones here and would review every single line individually. We caught a lot of errors this way, the final product was better as a result, and it only took a day or two of work at most. I do not feel this is an unreasonable amount of work to be expected from a professional organization. Gnomingstuff (talk) 17:52, 16 August 2026 (UTC)

A stage of basic competitive fault-finding, annotating a large sample by hand to flag and classify problems, goes a long way. Your immediate reflex to do this as the first response above demonstrated this nicely.

If an originating team works in public to catch and patch issues in that way, before others run into them, with high standards for accuracy, that would set discussions like this off on a better foot. – SJ + 00:23, 19 August 2026 (UTC)

Thank you. Sorry, I didn't know who was in the team working on it, I trust all these people. Kowal2701 (talk, contribs) 06:51, 16 August 2026 (UTC)

I oppose any A/B tests of this feature. It is antithetical to what Wikipedia should be, and is not something the Wmf should attempt. Suggestions sent by some central, biased, singleminded entity diminishes the wide variety of input, style, viewpoint, ... that makes the richness of the community. Wikipedia ia a beacon of human diversity, an LLM is the opposite of this. Fram (talk) 06:58, 16 August 2026 (UTC)

In contrast to such hyperbolical rhetorical flourishes, the community has already been widely using AI suggestions sent by some central, biased, singleminded entity controlled by WMF for over a decade, in form of ORES (or now the "revert risk" models). These AI suggestions (on whether to revert another editor's changes as vandalism) are made available to any editor at Special:RecentChanges even though they might be seen as more consequential on average than those of the new tool under development here, and have a fairly high error (false positive) rate. I have personally implemented many thousands (probably tens of thousands) of those AI suggestions over the years, and survived skipping many more mistaken suggestions.

I don't want to entirely dismiss concerns about control by the Foundation though. In case of these vandalism detection AI models, the more recent work on them seems to have been dominated by decisions and aims of the WMF Research department that may not entirely align with the community here (for example, they seem to have foregone possibly substantial quality improvements for English Wikipedia in favor of language equity, and implemented their own conceptions of "fairness" with regard to IP editors).

The main employee behind the success and broad community acceptance of the original ORES left WMF years ago, and his parting recommendations for "Community-centered Evaluation of AI Models on Wikipedia" do by and large not seem to have been taken up by WMF.

Regards, HaeB (talk) 08:36, 16 August 2026 (UTC)

As I said earlier, keeping the prompts (and the pipeline in general) publicly available and enabling each community to tweak them would go a long way in assuaging these concerns. Some competence is required to maintain these tools, but there is no reason not to make prompts and benchmarks transparent. Alaexis_¿question? 10:50, 16 August 2026 (UTC)

The system prompts used for the model appear to be published on gitlab here according to the mw:VisualEditor/Suggestion Mode/Model-generated editing suggestions#Research findings. Not sure which size Gemma gemma4:latest is, maybe the E4B? I would be interested to know if other models were evaluated, for example Nemotron 3 Super is a similarly sized model to gpt-oss:120b, but a bit newer. Mistral is dense so might be a bit slow to run. In any case, there are many newer models in the same weight class, surely it wouldn't be that hard to generate say ~1000 from each of them to see if anything is clearly better, given the only thing that has been modified is the system prompt? Alpha3031 (t • c) 12:22, 16 August 2026 (UTC)

Thanks. Going off of that, it appears that no reference information is given, which is not surprising looking at the NPOV suggestions, and that no actual Wikipedia policies/guidelines are included in the prompt. LittlePuppers (talk) 15:24, 16 August 2026 (UTC)

A successful model would likely go beyond a prompt alone and be equipped, at the very least, with retrieval-augmented generation capabilities in order to access the text of the policies/guidelines themselves, and, in the case of NPOV, ground its answer in available reliable sources. I don't think that would be enough for NPOV specifically (there is still too much of a black-box effect in the model's weights, that won't be fully offset by prompting or sourcing), but this is to say that the current approach is far from optimal. Chaotic Enby (in solidarity · talk · contribs) 15:46, 16 August 2026 (UTC)

The particular challenge is, I think, that there seems to be a trend to use increasingly sophisticated models to push increasingly challenging tasks increasingly to newer editors. LittlePuppers (talk) 15:33, 16 August 2026 (UTC)

Regarding point 4., community feedback makes it pretty clear that anything regarding NPOV or tone will not be seen as low-risk, and might be interpreted as an attempt by the WMF to influence editorial decisions. I believe it would help the prospects of the experiment to commit to follow the emerging consensus, which is clearest on this specific aspect.

On a broader level, I will reiterate my suggestions on Phabricator on working with the community:

[...] the scope and timeline of any planned community rollout, and the importance of working with the editor community and respecting a future consensus on whether to deploy it, should be made as clear as possible to avoid a repeat of Simple Summaries.

Chaotic Enby (in solidarity · talk · contribs) 12:31, 16 August 2026 (UTC)

I'm with the rest of the group in saying that the NPOV suggestions are high risk and difficult to get right. I'm excited about trying to figure out if a good suggestion model can be developed for simplifying language. There is a large body of research showing that Wikipedia's science content is often way and way too complicated and linguistic complexity is an important part of that. We've had discussions on Wikipedia about using heuristics (sentence lenght, number of syllables per word), which is contentious. Perhaps an AI model can do this better than a heuristic, as it can distinguish between a long sentence with an easy sentence structure and an impenetrable long sentence. In solidarity, —Femke (talk) 🐦 15:27, 16 August 2026 (UTC)

@Femke I have tried both AI models and conventional readability tests like Flesch–Kincaid and this is an area in which current AI tech cannot and should not be used. And my conclusion was that conventional readability tests also suck.

NPOV is another area where AIs cannot and should not be used.

We can use AI to find typos, but fixing them actually requires an experienced Wikipedian. Polygnotus (talk) 15:33, 16 August 2026 (UTC)

I too have plenty of experience using AI for this purpose and find that they can be used successfully in identifying overly difficult text and decently enough for suggesting alternatives if one pays attention to subtle changes in meaning. As these edit checks are only about identifying overly complicated text and because we have an abundance of low-hanging fruit in this area, I'm confident that a tool can be developed. The big question is if within the limits of a certain budget, it can become good enough, and to what extent it attracts the right editors to fix it. In solidarity, —Femke (talk) 🐦 15:40, 16 August 2026 (UTC)

One of my common complaints at FAC (especially for scientific articles) is choppy writing style (sort of the opposite problem from overly difficult text). I often advise authors that they should try using some of the LLM tools to make suggestions for how their writing could be improved. I don't know how well this advice is received. I also don't know how much it is used in the intended way of offering suggestions vs just copy-pasting the output into their article; that would be an abuse of the tool, but people abuse tools all the time and the fact that they do so doesn't mean the tool is bad. RoySmith (talk) 15:43, 16 August 2026 (UTC)

find that they can be used successfully in identifying overly difficult text and decently enough for suggesting alternatives if one pays attention to subtle changes in meaning. Subtle changes in meaning convert text supported by reliable source(s) to text not supported by reliable source(s).

I'm confident that a tool can be developed Yeah, developing a tool is the easy part. The hard part is ensuring it is a net positive.

The big question is if within the limits of a certain budget The WMF is not the right party to create such a tool because they are unfamiliar with the challenges Wikipedia editors face and because they have a tendency to waste a lot of time and effort on creating shiny new tools while neglecting decades of tech debt.

The good news is that volunteer devs can do it, but they will run into the problem that it won't work as I explained above. Polygnotus (talk) 15:46, 16 August 2026 (UTC)

Some teams are unfamiliar with the editing challenges. Other teams have a track record of listening, or have editors in their team. The editing team is an example of the latter. For instance, when people at enwiki and dewiki asked for better VE performance recently, the team managed to fix this tech debt really rapidly.

I'm very willing to help test this part of the tool. I have experience with editors doing this well and less well. In solidarity, —Femke (talk) 🐦 15:58, 16 August 2026 (UTC)

@Femke You can try WP:SCRIPTREQ, the people there are a lot more agile than the WMF is.

Please ping me if you have something I may be able to help test it. Polygnotus (talk) 16:02, 16 August 2026 (UTC)

Tbh I still think we should look at incorporating SEWP articles here as 'simple summaries', but that's a different discussion Kowal2701 (talk, contribs) 15:47, 16 August 2026 (UTC)

I wouldn't assume they'd want us to. Simplewiki is a tiny wiki with just a few active people. If we incorporate their content their vandalism rates will go through the roof. Polygnotus (talk) 15:49, 16 August 2026 (UTC)

They'd also get more good-faith editors though! What I'm thinking is that SEWP still exists as a separate wiki, we just have an opt-in feature for displaying one of their articles behind a button. IIRC @Ferien was sort of open to the general idea (may be wrong) Kowal2701 (talk, contribs) 15:53, 16 August 2026 (UTC)

Adding a CTA, if we get consent from the Simplewiki regulars, may be a good idea. Polygnotus (talk) 16:04, 16 August 2026 (UTC)

Yeah, I am personally quite open to the idea, though I am not too sure what our community as a whole would make of it at the minute. --Ferien (talk) 21:04, 17 August 2026 (UTC)

What kind of typos is AI fit to resolve that WP:AWB is not? Czarking0 (talk) 20:13, 16 August 2026 (UTC)

One thing I'm having in mind are typos that depend on the context to make sense of them (or to whether there is a typo to begin with). Chaotic Enby (in solidarity · talk · contribs) 20:19, 16 August 2026 (UTC)

FWIW, the fixes in Special:Diff/1369578064 were all suggested by Claude. I imagine most of them would have been caught by other tools, but I was impressed by the flagging of Freilberg. I don't remember exactly what it said, but the gist was that it spotted that I had "Peter Freiberg" in one place and "Peter Freilberg" in another. It figured out that these were probably referring to the same person and while it didn't know which was wrong, it assumed one of them was, and left it up to me to figure out which. RoySmith (talk) 20:24, 16 August 2026 (UTC)

And I note the firsts of those fixes is changing a direct quote in a way that is inconsistent with the source. Sure, you can probably justify that (MOS:QUOTE allows minor typographic things to be silently corrected) but I don't think that's something an AI should be recommending in any way. * Pppery * _{(alt) in solidarity} 20:49, 16 August 2026 (UTC)

So, you're saying the fix is correct, and if a human had suggested it you would agree with it, but since an AI suggested it there's a problem? RoySmith (talk) 20:56, 16 August 2026 (UTC)

I'm saying it might be correct (not that it is correct), but determining whether it is is a judgement call I don't want an AI to make. * Pppery * _{(alt) in solidarity} 21:22, 16 August 2026 (UTC)

The AI didn't make the judgment call. It just brought this to my attention and I made the judgement call. That's why my name is on the diff. I'm really not seeing the issue here. I made an error, a tool alerted me to it, and I fixed it. How is this a problem? RoySmith (talk) 21:36, 16 August 2026 (UTC)

The funny thing about using LLMs here is that they start working against each other. The default behavior of AI trying to "rewrite articles into formal encyclopedic tone" is to undo any text simplification (and to do so poorly, for instance replace "is" with "serves as", "uses" with "utilizes", etc). The "text simplification" category here, however, seems to be basically a catch-all. "Simplification" suggestions here range from stuff like Correct the misspelling of the player's first name from "Russel" to the proper "Russell" (which isn't even correct!) to "break up these sentences."

There's also the problem that telling someone to Rewrite the sentence for clarity and smoother flow is not actionable to the majority of people: if someone doesn't know how to copyedit then they don't know how to do that, and if someone does know how to copyedit they don't need those vague directions. It's like prompting people as AIs. Gnomingstuff (talk) 18:23, 16 August 2026 (UTC)

(Ironically, the LLM prompt that judged suggestions reads in part You are a strict reviewer. Your job is to find flaws, not to be nice. Really weird feeling to be envious of an LLM, its opinion certainly seems to be taken more seriously and its tone is given much more leeway.) Gnomingstuff (talk) 20:42, 16 August 2026 (UTC)

Who on earth outside the group of people responsible for this is going to read Gnomingstuff's report and consider those suggestions to be "reasonably accurate"?  — Hex • talk 16:02, 16 August 2026 (UTC)

Pre-alpha or not, the WMF shouldn't be experimenting with ways to funnel freeform model suggestions to editors to begin with. Machine models should never be allowed to so directly influence the contents of the project, as even if the suggestions are individually found to be valid and reliable, there will still exist overall biases. An LLM will favor certain sources, certain topics, certain sides. This will be reflected in what suggestions are and are not made, and editors evaluating and implementing individual suggestions will be entirely blind to any larger systemic issues they would be enabling.

Humans have issues with bias too of course, but this can be counteracted on an individual level by self-awareness of this fact, and on a group level by the diversity of our views. A monolithic model has neither, it predicts tokens. fifteen thousand two hundred twenty four (talk) 21:45, 16 August 2026 (UTC)

"Humans have issues with bias too of course, but this can be counteracted on an individual level by self-awareness of this fact - Citation needed: "making people aware of their bias doesn't do anything to mitigate it." Levivich (talk) 22:45, 16 August 2026 (UTC)

Citation: I made it up from first principals and subjective experience, the preprint will be out soon.^[Humor]

I feel entirely comfortable claiming that editors who operate with awareness of their own potential biases will take steps to mitigate them in this structured environment where WP:NPOV serves as a strong guiding force. If you find this unpersuasive, so be it, it is human to disagree. fifteen thousand two hundred twenty four (talk) 23:58, 16 August 2026 (UTC)

The operative word here is can. As human beings that are capable of independent thought and self awareness, we can choose to examine our own biases and seek out information and experiences to change how we think about the world around us and the assumptions we make. Large language models are capable of exactly none of those things, because of the very simple fact that they are computer programs. Claude is exactly as capable of herself awareness as MS Paint is of having an independent thought.

Not every person makes the conscious choice of examining their baises, or is even fortunate enough to exist in a socioeconomic situation to even be able to, but that is beside the point ‑‑gurkubondinn 00:26, 17 August 2026 (UTC)

"Research from Harvard found that the effects of personal interventions such as awareness raising at a personal level are positive, but short-lived."

"And the worst method, the one that actually has no effect at all, is to tell people to be good people, to be egalitarian, and so on. ... It is easy in the sense that it last for a short period of time, but it won't last very long. ... Now, when young people encounter this result, when they see that, yes, they were able to make change, but the change doesn't last, they get very sad, because they want a better world. And I'm not at all sad about that. ... So our brains change, our minds change, associations move around, but they always will gravitate to whatever is your cultural default. And so to bring about actual change, society around us has to change, and then we will move, and then the default will be a new default."

LLMs are biased because humans are biased. Because they're trained by humans, and humans train their biases into the machines. We are no more able to cure bias in machines than we are able to cure it in ourselves. There are plenty of ways in which human intelligence is superior to machine intelligence, but lack of bias isn't one of them (in either direction). Levivich (talk) 02:57, 17 August 2026 (UTC)

Absolutely, I agree with you. That's why the models can never be "neutral" or "unbiased". My point was just that I think you misunderstood 15224's reply, people have the capability to examine and be aware of their biases, but computer programs don't. It's not automatic and it doesn't happen for all people (for various reasons), but we have the ability to examine our own biases because (unlike computer programs) we are conscious beings. It's a bad comparison is what I'm saying, and it anthropomorphises computer programs (that were created by biased people). I have a beef with whoever it was that decided to wrap LLMs in chatbot interfaces. ‑‑gurkubondinn 11:06, 17 August 2026 (UTC)

We're looking forward to getting more deeply into this with you all this week. Before that, I wanted to express gratitude to y'all for the perspectives you are sharing. From the concerns about transparency and community control over models of this sort to issues with specific suggestions you're encountering, please keep the feedback/questions/concerns/etc. coming.

And for anyone who is interested in seeing what these suggestions look like in practice, please do the following...

TRYING EXPERIMENTAL SUGGESTIONS

1. Ensure you have the Suggestion Mode beta feature enabled
2. Enable "experimental" suggestions by pasting the following snippet into your common.js: mw.loader.load( 'https://meta.wikimedia.org/w/index.php?title=User:DLynch_(WMF)/alwaysbesuggesting.js&action=raw&ctype=text/javascript' );
3. Open a page in VE that has one of the 30,000 suggestions available. E.g. https://en.wikipedia.org/w/index.php?title=List_of_examples_of_Stigler%27s_law&veaction=edit

Note: this week, I'm going to see if we can share a spreadsheet so you can see all of the model-generated suggestions in one place rather than having to tap around the wiki looking for them. PPelberg (WMF) (talk) 00:59, 17 August 2026 (UTC)

I wrote a small script so you can view the list of suggested edits on an article without needing to enable the beta feature, switch to VisualEditor, or add that line to common.js to enable "experimental" suggestions: User:DVRTed/sandbox/edit-suggestions.js. — DVRTed (Talk) 03:06, 17 August 2026 (UTC)

The spreadsheet would be helpful, if only because I don't know which of the csvs is the "real" one. In general I think this kind of thing is much easier to review in spreadsheet form than article-by-article. Gnomingstuff (talk) 04:18, 17 August 2026 (UTC)

@Gnomingstuff: what you described makes total sense to me.[i][ii] I've checked in with engineering and it turns out that compiling this list will take a bit of time. Assuming nothing unexpected turns up, you can expect me to return here with a link to a CSV you (and everyone else here!) can review before this week is over.

---

i. Knowing, definitively, what suggestions we need y'alls expertise in reviewing and being able to differentiate those from earlier iterations that we've since discarded.

ii. Seeing all of the suggestions in one place so that you don't have to hunt them down yourself. PPelberg (WMF) (talk) 22:49, 17 August 2026 (UTC)

Is the intent that the U/I would just present these suggestions to the user and let them edit the article themselves if they opt to accept the suggestion? Or is the idea to have a "Make this edit" button that the user could just click? The reason I ask is that if it's the later, it would make sense to include some machine-readable marker in the wikitext (I'm thinking an HTML comment) identifying the source of the inserted text. And/or have a log of such changes. The idea is to make it easier for somebody to audit the performance afterwards. RoySmith (talk) 23:04, 17 August 2026 (UTC)

The latter would fail WP:NOLLM, so that would be a complete nonstarter. The former is not great either. ‑‑gurkubondinn 23:15, 17 August 2026 (UTC)

WP:NOLLM says "Editors are permitted to use LLMs to suggest corrections to their own writing, and to incorporate them after human review. This is limited to spelling, punctuation, capitalisation, grammar, and other simple mistakes." So this would indeed be a starter in those cases. RoySmith (talk) 23:22, 17 August 2026 (UTC)

The suggestions that Gnomingstuff went through go far beyond the very narrow exception in NOLLM. And this exception only applies to [an efitor's] own writing. ‑‑gurkubondinn 23:44, 17 August 2026 (UTC)

The latter would be impossible in the current implementation, anyway, the LLM is not prompted to provide an actual change and the suggestions are usually just stuff like "The sentence is long, contains a grammatical error, and could be expressed more clearly." Gnomingstuff (talk) 05:09, 18 August 2026 (UTC)

@RoySmith: Good question. If/when we (staff + volunteers) come to think these suggestions are reliable and useful, the intention would be to offer a suggestion that would contain the following:

1. A description of the issue and type of fix that is needed. Both of which will need to be generic enough to make sense across the contexts it might appear within while at the same time being concrete enough for the people encountering it to know how to start on the path of acting on it. So for the "Simplify language" suggestion, it might be something like, "Readers might find this text difficult to understand. Try rewriting this using shorter sentences and plain language."

2. A link to the local policy/guideline the suggestion originates from. This serves both as an opportunity for people encountering a suggestion to learn more and also a way to ensure that suggestions are grounded in project consensuses and conventions.

3. Two actions: one action to Dismiss the suggestion and along with it, a way to express why someone has elected that choice. And a second action – maybe we'd label it Rewerite? – that when tapped would A) focus someone's cursor into the span of text the suggestion thinks there is an issue within and B) cause the article text in question to enter a "revising state" so people are clear about where exactly their focus is needed (see screenshot below). From there, the responsibility would be on the person acting on the suggestion to decide what to fix and how to fix it. Said another way: there are NO plans for these suggestions to make fixes with the click of a button let alone to describe specific solutions.

If you'd appreciate a more concise answer, what @Gnomingstuff described here is spot-on. PPelberg (WMF) (talk) 05:32, 18 August 2026 (UTC)

@PPelberg (WMF): So how would you cram enough information in such a tiny area to give the person all the information they need? Are you aware that many guidelines are like 7k words? If you give a newcomer a link to WP:NPOV, that is obviously not enough to have them actually be able to judge the neutrality of an article if they do not have relevant experience and knowledge of the field. And what will happen when inevitably people complain that edits are not improvements? Will this just WP:BITE newcomers even more? Polygnotus (talk) 05:40, 18 August 2026 (UTC)

@Polygnotus: great spot. The questions you're asking sit at the very core of this project.[i]

We seem to be aligned in thinking[ii] that some suggestions may be more harmful than helpful to show to newcomers. They're likely, as you alluded to, too complex and experience-dependent to distill down into a relatively small piece of guidance. If we don't account for this, we could lead newer folks into publishing edits that experienced volunteers revert or respond to with hostility. This could in turn drive these potential contributors away.

And while I don't think we can know for certain which suggestions those are in advance, I think we (staff and volunteers) are develpoiong a pretty good sense that conversations like this one are helping us refine.

I also think we'll learn a lot over time by trying things out, and there are a few features we've put in place to help:

Experimental suggestions: suggestions can be enabled as either default-on or experimental. The latter means that people who have explicitly enabled the soon-to-be-available setting have the space to safely experiment with suggestions. Through that testing, they can decide who—if anyone—an experimental suggestion should be shown to by default.

Tags: all edits in which someone sees and/or acts on a suggestion are tagged. You can see these in action by filtering Special:RecentChanges for Edit Suggestion seen or Edit Suggestion used.

On-wiki configuration: volunteers can independently decide the minimum number of edits someone must have published in order to see a suggestion. If you visit Special:EditChecks and look at the link suggestion, you'll see that volunteers have set the minimumEditCount value to 1000. This means the suggestion will only be shown to editors who have made ≥1,000 edits.

The idea is that, together, the above will let us see the kinds of edits these suggestions lead folks to make in practice and, with that, decide whether, how, where, and to whom they're shown.

How does this sound to you? What, if anything, do you think we might be missing or misunderstanding?

---

i. I hear the questions you're asking as something like: "How might we translate a great deal of nuance and complexity into a format that is simultaneously 1) succinct and simple enough that newcomers will engage with it and 2) explanatory enough that in doing so newcomers will be equipped with the information and know-how they need to act on them in ways experienced volunteers see as constructive?"

ii. Please correct me if I've misinterpreted what you've said PPelberg (WMF) (talk) 05:32, 19 August 2026 (UTC)

I would urge you to please stop developing this feature. It seems like any version of this feature will be detrimental to wikipedia.

A suggestion from an LLM, a suggestion from an official wikipedia tool, will easily be read by an editor as trustworthy, when we know it is not. LLM usage offloads critical thinking to software, meaning the editor is doing less of that themselves. This alone will have a negative impact on quality. And even grammar edits can change the meaning of a sentence, so this is just a non-starter in general.

Please do not pursue this track. TietoTeekkari (talk) 20:00, 20 September 2026 (UTC)

I'm pessimistic about this, but I have a very high bar for pessimism/hopelessness to stop me from supporting a low-stakes experiment. Giving this tool to experienced users to try out seems like one of those low-stakes experiments worth trying, in case there's a way to make it work (again, I'm pessimistic, but possibly with certain constraints regarding task and topic?).
But also, just to put a finer point on something, because I think it does good to repeat it now and then: there's the worry about the quality of these suggestions, but there's also the worry about, for lack of a better word, branding. The branding that the WMF has begun using, "knowledge is human", is an echo of a popular sentiment here. At a time when absolutely every company and every project is cramming in as much AI as possible, Wikipedia is mostly headed in the other direction. That's a good thing IMO, but as a result, any pitch someone has for an LLM-based tool is evaluated with a handicap applied. If you'd otherwise be graded on a scale of 1-10 where 1 is harmful and 10 is helpful, start by subtracting 2 or 3 for "we don't want to be associated with that" (or, alternatively, "we see these as detrimental by default"), and it needs to be really useful to get a critical mass of people behind it. But no complex tool starts its life as an 8+ on that scale, I don't think, so super-low-stakes tests are IMO a good way to start working towards it. FWIW. — Rhododendrites ^talk \\ 22:06, 16 August 2026 (UTC)

I really appreciate and agree with the branding note Rhododendrite gives here. Best, Barkeep49 (talk) 22:48, 16 August 2026 (UTC)

A humorous interlude

I know people love to make fun of AI hallucinations, so I couldn't resist posting this fun map (4:38 in the video). Strange spellings aside, Hoboken, NJ has gotten transported to the midwest, Chicago is on the Pacific coast, and Boston has been relocated to Colorado. I want some of whatever it's smoking. RoySmith (talk) 02:12, 17 August 2026 (UTC)

Also available in Africa. I think I'll stick to the Commons maps. Certes (talk) 14:30, 17 August 2026 (UTC)

These LLM map-fails are legion. My take away from these examples is that they are not only vivid examples of LLM hallucination, and thus LLM unreliability, but they are also vivid examples of the unreliability of humans, because each one of these published hallucinations is only possible because humans obviously failed to check the work--to even look at the maps--before publishing. Humans are unreliable: an important lesson for any crowdsourced project. Levivich (talk) 14:57, 17 August 2026 (UTC)

I don't know about this particular channel but loads of them are (almost) fully automated with no human in the loop. Polygnotus (talk) 15:39, 17 August 2026 (UTC)

Then again, aside from quite a few mistakes, have you all seen the "Chloe vs History" channel on youtube? Quite the advancements in AI lately. Some of the advancements have been made on this channel, which should probably have a Wikipedia page. Randy Kryn (talk) 15:42, 17 August 2026 (UTC)

That Africa one is from a US State Dept presentation. That ain't no YouTube clickbait, that's supposedly professional humans at work. You'd think they'd have looked at the slides before making their presentation. And, I'm speculating here, but I bet multiple humans were involved in that, because when the US State Dept gives a presentation at an int'l conference, I don't think it's just one person creating and making the presentation all by themselves. Maybe. But either way: evidence that at least sometimes, even professional humans presenting at an int'l conference obviously don't bother to check their work. I guess my point is: what makes LLMs unreliable isn't just that they hallucinate, it's that humans won't catch it because sometimes they don't even bother to look. Another well-known example is lawyers submitting briefs to courts with hallucinated citations -- literally licensed professionals in the performance of their profession. So this isn't a problem limited to youtube clickbaiters or "kids on the internet," even trained and licensed professionals, even on a global stage, succumb to laziness. At least the lawyers get fined for it -- now there's a new fundraising channel for the WMF: fine editors for publishing hallucinations on-wiki! Levivich (talk) 16:15, 17 August 2026 (UTC)

At least with hallucinations, they're usually immediately obvious to anybody who bothers to look. This particular example is clearly YouTube clickbait. The bottom-feeders who produce these things don't give a whit about accuracy, just that they can churn out some mildly entertaining garbage videos that collect likes and shares and other revenue-producing metrics. But the fact that people are willing to abuse a tool for their own commercial benefit doesn't mean that the tool is inherently worthless. People abuse wikipedia in all sorts of ways (spam, SEO, reputation management, advertising, etc). Does that mean we should shut Wikipedia down? RoySmith (talk) 15:49, 17 August 2026 (UTC)

PS, it doesn't take AI to generate garbage maps. Some people are able to do it all by themselves with nothing more technologically advanced than a sharpie. RoySmith (talk) 16:22, 17 August 2026 (UTC)

Sure, and wikipedia had hoaxes and plenty of innocent misinformation long before LLMs became popular, but the problem, of course, is one of scale: LLMs make this sort of thing like 100x more common than before. It used to take hours to write a good hoax on wikipedia, now it takes minutes. Levivich (talk) 16:30, 17 August 2026 (UTC)

"Chiiicago". AAND it's in multiple places at once. Ladies,Gentlemen and enbies, The Chicago hivemind. Starlet 01:43, 25 August 2026 (UTC)

Back to business after the humorous interlude, a full ban on AI?

• No AI on Wikipedia and this should be extended to article talk pages, Wikipedia discussions, visible editing suggestions, or draft pages. Too many work-arounds and exceptions have been allowed and these issues and concerns could go on for years. There should be some good faith edit reverts for AI edits made in mainspace while the confusion existed but maybe end the confusion from here-on-in with a blanket ban on AI. Thanks (darn it, the word "thanks" was aided by AI). Randy Kryn (talk) 15:06, 17 August 2026 (UTC)

  I'd support that. I've had enough with AI turning everything to shit, and clearly WMF can't take no for an answer. Tercer (talk) 15:55, 17 August 2026 (UTC)

  I'd say that Polygnotus should be allowed to play around with AI because Polygnotus actually understands its limitations and assumes full responsibility for any edit made with their account.

  The WMF is a different story unfortunately. Polygnotus (talk) 15:59, 17 August 2026 (UTC)

  Does Polygnotus always refer to himself in the third person? RoySmith (talk) 16:03, 17 August 2026 (UTC)

  Polygnotus does not. But Polygnotus could! Polygnotus (talk) 16:05, 17 August 2026 (UTC)

  AI, or LLM? Cluebot, for example uses machine learning (AI but not LLM) to great benefit with remarkably few errors. Certes (talk) 16:36, 17 August 2026 (UTC)

  Generative AI is what bothers me, irrespectively of the underlying technology. I'm fine with cluebot because it doesn't generate content, so there's no risk of poisoning Wikipedia. I would still be fine with it if it were LLM-powered (even if that would be completely pointless). Tercer (talk) 17:46, 17 August 2026 (UTC)

  I would absolutely support a ban on AI (LLMs) for (direct or indirect) content generation or suggestion. Ita140188 (talk) 19:12, 17 August 2026 (UTC)

  I would definitely support this. Cheers, 𝔰𝔥𝔞𝔡𝔢𝔰𝔱𝔞𝔯 ^{(𝔱𝔞𝔩𝔨)} (any/all) In solidarity. 19:14, 17 August 2026 (UTC)

  I would support this on principal, at least until the AI industry stops being complete garbage. —Leaf.Sheap ⇖ /.°°.\ ⇗ (They•Them) 21:18, 17 August 2026 (UTC)

  I would support aswell. I don't think (generative) AI has a place on Earth at all, let alone on wikipedia (which has always been guided by the principles of human knowledge and effort). TheDowningStreetCat (talk) 23:41, 17 August 2026 (UTC)

  100% support. I fail to see how AI slop would actually help the project in any conceivable way. All I’m seeing is negatives… lots of them. 296cherry (talk) 18:29, 20 August 2026 (UTC)

  What would you think of suggestions more along the lines of what I outlined here? ScottishFinnishRadish (talk) 18:32, 20 August 2026 (UTC)

  I would support a ban on LLMs to stop similar things from happening in the future. TietoTeekkari (talk) 20:02, 20 September 2026 (UTC)

• Right now one of our biggest strengths is that we are a large, human written, generative AI free chunk of knowledge. My crystal ball isn't working, so I can't tell you if AI will be good enough to do something cool with Wikipedia in the future. That said, Wikipedia is one of the last places which should incorporate AI because it will give up our unique advantage. Proposals like this from the WMF are not sufficient, are not close, and make me want to use blanket statements to convince the WMF not to keep trying to shoehorn AI into things over and over again. Tazerdadog (talk) 16:59, 17 August 2026 (UTC)

  I feel like it should be pointed out, again, that this is not a proposal. It's an experiment. "Learn whether experienced volunteers at en.wiki think the experimental suggestions are sufficiently reliable" is what they're doing. I think Gnoming helpfully answered that question. Levivich (talk) 17:58, 17 August 2026 (UTC)

• This would be an overreaction. User:Polygnotus and many others have been building various LLM-powered tools, including ones that are used to detect LLM edits (User:Fermiboson/AIlog) and to fight vandalism (Wikishield - LLM is optional). There are more than one hundred editors using these tools. We do want WMF to experiment and implement the best ideas. Alaexis_¿question? 07:06, 18 August 2026 (UTC)
• A full ban does not make sense. We already have a range of community tools that do cool things with Wikipedia using AI. In particular I want the best available tools for review, including those that take advantage of AI for trainable pattern matching and classification. That includes anything that helps with slop-detection, edit checks, citation checks, page linting, and vandal-fighting. This experiment falls under review... with a higher bar for quality I can see a range of suggestions being useful, particularly after a few rounds of focusing on quality. – SJ + 01:07, 19 August 2026 (UTC)
• I would support a ban on generative AI touching any Wiki content regardless of whether LLMs improve. JoelleJay (talk) 11:30, 22 August 2026 (UTC)

Continued discussion

Thank you all for keeping the conversation going, and thank you to Gnomingstuff and others for taking the time to look through outputs. @PPelberg (WMF), @SSalgaonkar-WMF, and I have read the whole thread and tried to pull out some of the most important things we heard and questions being asked. Peter and Sucheta, please add if I missed anything (well, for that matter -- anyone please tell me if I missed anything). I'm just going to list these questions/topics for now, and we'll keep working through them and hopefully you'll keep discussing with us (there are many of you and not as many of us!) This is to build on what I posted above and what Peter posted above.

Firstly, I think we're on the same page about something especially important: we're not going to deploy LLM-backed suggestions to anyone unless this community is supportive of it. And if they do become deployed, they'll be configurable via Special:EditChecks the way existing checks and suggestions are (i.e. community could decide who gets to see them, change what they say, what articles they show up on, or turn them off). I hope that in projects like these, we at WMF are bringing capabilities to volunteers so that they can produce the kinds of checks and suggestions that help both newcomers and experienced editors get the most good wiki work done with the least amount of drudgery. There is also a question of whether these suggestions would be a vehicle for content suggestions -- the answer there is no, we are not designing these to propose text to the user; rather they point out places the user should look and what they should look for. The LLM explanations that Gnomingstuff pointed out initially are in those files as a way for us to understand internally why the LLM is making the suggestion; they would not be shown to editors. (But I know there is a more subtle question here -- when just pointing out a sentence that needs attention in some way exerts that sort of influence).

In that vein, I wanted to say that this current project is more about figuring out what it might be like for checks/suggestions to be generated via LLMs, than about what those exact types of suggestions are. If NPOV is not a good one to pursue (volunteers here have given many good reasons why NPOV is particularly tricky), then we should put that one down in favor of simpler ones to try out. (Worth noting that checks and suggestions are being generated via a bunch of other ways, too, like simple rules, text match, simple models, etc.)

Also just a quick nomenclature thing:

• Checks: this refers to edit checks that react to what the editor is doing at that moment in the editor. e.g. they paste in a blob of text from ChatGPT, and the check pops up right then and says, "Please avoid copying text from other sources".
• Suggestions: this refers to suggestions for improvement that have been pre-calculated and are waiting to show up when someone clicks Edit, e.g. they open up the editor, and there is a box that says, "This link appears more than once in this section." Right now, Suggestion Mode is available on English Wikipedia as a beta feature, and suggestions are available in the feed on Special:Homepage.

You can see all the checks and suggestions and their statuses at Special:EditChecks (note that "experimental" checks are only available to people who have a specific user script installed).

Sorry -- this post is getting long. But on to the list of questions we want to be able to talk about here, raised in the thread above. This list of questions is not something that we just want to provide answers to and then expect that everyone will agree with us. This is a complex area, and we don't have all the answers, but we're trying to figure them out with you.

1. What kind of communication with communities has there been on this project so far?
2. Why are we working on projects like this instead of more work on the backlog of bugs and small improvements for editing?
3. How did we / do we QA lists of suggestions like these?
4. What if communities don't want certain suggestions on their wiki?
5. What determines whether suggestions are good enough to go beyond testing?
6. Is NPOV appropriate to point an LLM at, given its nuance and complexity?
7. If an LLM provides suggestions, how do we make sure that the editor isn't swayed/biased just by virtue of it coming from Wikipedia (a trusted source)?
8. Does it make sense for the message to the world to be "Knowledge is Human", but we are also using AI for certain things?
9. How could the models we use be transparent, open, and have community controls/auditing?

Alright -- more to come as we get into those questions. -- MMiller (WMF) (talk) 23:53, 17 August 2026 (UTC)

@MMiller (WMF): Hi Marshall!

Because of a long history the relation between the community and the WMF is, let's say, far from perfect. And arguably worse than ever.

This is obviously not your fault, but its important to set the scene.

Because Wikipedians are smart and informed they are, generally, skeptical and wary of AI.

This is the correct position in 2026, because we do not yet know the impact of AI on the environment, jobs, and the upcoming resource wars.

The community generates the value and people think they donate to support it. The WMF is terrible at doing the community actually wants and needs. MediaWiki's tech debt is a sad joke. Community members who try to explain what we need from the WMF are routinely ignored. For decades.

Instead of working on the things that are important, the WMF nerds build shiny new toys. Debugging old code sucks. Doing something fun with AI/ML is fun. Because WMF leadership is terrible it seems (from the outside) that no one is working on the important stuff while we get an endless stream of halfbaked projects that waste a lot of time and money. The WMF basically never finishes a project it starts, overcommits at an early stage, and only gets feedback when its too late.

The WMF has a tendency to drop in and reveal they spent a lot of time working on a terrible idea, without asking input from the community, and then are surprised when the community rejects it.

The WMF is terrible at communicating its wishes and goals and what its working on.

The previous WMF attempt to "do something with AI" was a terrible idea and everyone hated it. It proved yet again that the WMF does not understand the Wikipedia community, what writing an encyclopedia means, and (the limitations of) AI.

The WMF did not learn from this and is again presenting a half-baked plan they spent a lot and time and resources on.

The community desperately wants and needs the WMF to succeed and produce great software at a reasonable cost, but that has yet to happen.

Quite a few members of the community are nerds who have spent a lot of time playing around with AI so they know what its limitations are in the context of writing an encyclopedia.

As the person behind the AI Proofreader, AI Source Verifier, AI Editsummary etc I am clearly not some anti-AI luddite.

The WMF is actively making the community more and more anti-AI, and to be honest rightly so. The WMF has no respect for the hard work of countless Wikipedians.

The fact that the WMF does things like have an AI check for NPOV problems shows that the WMF does not understand AI or its limitations or how to write an encyclopedia.

The community could greatly benefit from responsible AI use in a few specific tasks, whereby the human makes the final decision and is responsible for the edit. And the WMF makes it impossible for me to communicate that to people.

At this point, we both know its too late to listen to my feedback. This terrible idea will continue no matter what. The WMF has spent millions on AI related stuff and any benefits to the community were not proportional to the amount of money spent. It doesn't matter to the WMF; they had fun and can put something cool AI-related on their CV and move on.

The WMF has a toxic positivity problem wherein honesty and negative feedback is punished and ignored and all criticism must be hidden below 7 layers of a compliment sandwich. Even people far more diplomatic than I am just can't deal with all the corpo-speak and manipulation.

In an ideal world the WMF would stop what its doing and actually listen.

LLMs present an unique set of challenges and even opportunities and the WMF is fucking it up for everyone else and does not seem to understand how much damage they are causing and can cause in the future.

Is NPOV appropriate to point an LLM at, given its nuance and complexity? No, and that is a silly question. I don't teach my fish braille and ask it to rewrite the bible. Tools are useful in some contexts and bad in others.

What if communities don't want certain suggestions on their wiki? None of the proposed suggestions in its current form would be an improvement.

Does it make sense for the message to the world to be "Knowledge is Human", but we are also using AI for certain things? No, of course not.

How could the models we use be transparent, open, and have community controls/auditing? That is impossible unless you accept the models are terrible compared to competitors. There are no ethical LLMs available. And making something like that would be unethical and require unethical actions.

Why are we working on projects like this instead of more work on the backlog of bugs and small improvements for editing? Because the important work sucks and nerds think its fun to play around with AI. Also the WMF does not understand its role; it should serve and protect the community. In any well-run company you do maybe 90% important stuff and 10% fun stuff.

In the future, the community should be involved at the earliest opportunity, when brainstorming. The WMF should learn from its mistakes. Reflect on Simple Summaries. What went wrong, why, how to avoid that in the future? Allowing random newcomers to act on typofixes proposed by an AI will just make a lot of people very angry, sets newcomers up for failure and degrades the quality of Wikipedia. I use the opposite approach; my software helps experienced Wikipedians to fix typos, and an AI helps filter out things that aren't typos, and no one objects to that.

Polygnotus (talk) 00:41, 18 August 2026 (UTC)

Please assume good faith. No one is doing this to have fun or put "something cool" on their CV and move on.

Wikimedia has actually spent a terrifyingly small amount on AI infra and tooling, which is part of the problem here: when an experiment is run, fast iteration on prompts, benchmarks, evals, and models isn't second nature. There are indeed ethical language models, as others note below, and better orchestration would help choose the best for a given task. – SJ + 16:02, 21 August 2026 (UTC)

Marshall (and colleagues), I will say that from reading your post earlier, I think it reflects a good attitude for approaching the issue. (I'm not going to go down the rabbit hole of the difference between words and actions and trust.) I will also note that I have no idea how much say you have in what projects you work on and how far you take them. (Although if "Senior Director of Product" isn't just a bunch of fancy words, I would guess quite a bit.)

I clicked your link to see what's currently on Special:EditChecks, and I will say that almost all of them seem like really good ideas. None of them (the ones I like, at least), require any AI beyond some if statements in a trenchcoat.

In short: I think we could completely ignore all this AI stuff and you'd have some really great and helpful projects to work on. (Others have been mentioned above.)

I know working with such a large community can be hard. I would certainly like to think that we could give you some advice on how to help us. Ask . We only bite sometimes. Thanks for taking the time to listen. LittlePuppers (talk) 01:47, 18 August 2026 (UTC)

That is an excellent and fitting username. Polygnotus (talk) 01:53, 18 August 2026 (UTC)

One time I said that I didn't know what "Director of Product" means (I am not a native speaker) and the ex-CEO of the WMF started attacking and accusing me because she couldn't handle mild criticism. Polygnotus (talk) 03:11, 18 August 2026 (UTC)

Thanks -- just to take this opportunity to shed a little light on what I do and how we're set up:

• We're set up as a bunch of "cross-functional" product teams. Cross-functional means each team has a few engineers, an engineer manager, a designer, a product manager, a data analyst, a movement communications specialist (give or take -- sometimes a couple teams will share people in certain roles).
• The product manager is responsible for setting the priorities of the team, what order to work on them, deciding what is in and out of scope, deciding whether the results of an experiment show that something is worth pursuing further. They do this in collaboration with their teammates, not unilaterally.
• As a director of product, many of these product managers report up to me. When I started at the foundation, I was the PM of the Growth team, and I've been around for eight years and am now in a management role.
• So in my role, I look across the various teams and play a role in setting the overall priorities -- like which various potential editing projects the various teams working on contributors should prioritize, what the readers teams should prioritize. I work with director counterparts in engineering and design to do this.

MMiller (WMF) (talk) 18:28, 18 August 2026 (UTC)

Re 6, and to some extent 9: Does the WMF have a suitable operational definition of NPOV to measure against? As I understand it, transformer-based models are fairly common in sentiment analysis nowadays, but I don't know if pre-existing models (if any exist) would be well adapted to an encyclopedic context, and I don't think they would be sufficiently interpretable. In any case, I'd say that it is something that is strictly more difficult than tone check, as well as likely being considered higher risk by the community. I can't be certain of course, but I would expect for the community to grant social licence to a model for NPOV specifically would, at minimum, require much better interpretability than typical for current models. Whether the team believes they can achieve that is probably something best answered by the technical staff, the community can only inform as to acceptance criteria. Alpha3031 (t • c) 05:00, 18 August 2026 (UTC)

@Alpha3031 They just take off-the-shelf selfhosted open-weight models, which are way worse than Claude Fable 5 (and other mainstream commercial offerings) like gpt-oss:120b aya:35b / aya-expanse:32b, llama4:scout, qwen3:235b and qwen3:1.7b and then give it a presumably AI generated prompt containing:

 Role: You are an expert Wikipedia Copyeditor and Reviewer.
    Goal: Conduct a professional audit of an article's plaintext so it aligns with Wikipedia core content policies and Manual of Style.

    Context & Constraints:
    - Plaintext environment: non-prose elements may be stripped (infoboxes, references, templates, media, etc.).
    - Non-markup focus: do not suggest wiki-syntax, template, link-formatting, or reference-format edits.
    - Awareness of extraction gaps: if text appears truncated from extraction, do not flag it unless clearly an authorial issue.
    - Focus only on prose, terminology, neutrality, and information structure.

And

    - Neutrality: remove peacock terms, bias, and unnecessary loaded language.

You appear to be overestimating the sophistication of their approach by a rather wide margin.

Polygnotus (talk) 05:14, 18 August 2026 (UTC)

I am aware of that, given that I had read (and in fact posted the link to) said prompts above. Though, I wouldn't say hosted open weights models are necessarily "way worse" than commercial offerings currently given recent and especially upcoming releases (GLM at 700B especially is likely a lot easier to host than the 2.xT models, though still harder than 120B of course). The page does indicate that the team recognises some situations may require bespoke models though. Alpha3031 (t • c) 07:05, 18 August 2026 (UTC)

Open-weights models can be perfectly adequate for some use cases. For source verification the very modest gpt-oss-20b model worked as well as Sonnet 5. Detecting NPOV issues is just a much harder problem, and it needs much more context and probably better models as well. Alaexis_¿question? 07:18, 18 August 2026 (UTC)

I don't disagree that open-weights models (even older or smaller ones) can be adequate for many tasks. I just also wanted to point out that as of the time of the current discussion, there are several open-weights models in the 700 billion to 3 trillion parameter range that are comparable to the frontier in a much wider selection of tasks, namely Kimi, Qwen and the upcoming GLM 5.3 (which being much smaller, is probably going to be cheaper to evaluate).

Incidentally, it does appear that previous WMF research findings (m:Research:Test External AI Models for Integration into the Wikimedia Ecosystem#Evaluation Results and Findings) have pointed out the difficulty of NPOV assessment:

Some policy detection tasks are hard for humans and machines alike. Specifically for NPOV, precision across the three model families is very low. Models overall do better when detecting Peacock behavior. This is because while understanding neutrality requires in-depth reasoning, [bold mine] peacock behavior can be detected via language features.

so it's somewhat odd that they've picked it as a easy, low-risk task in this specific project.

I would say that adequate NPOV assessment is probably the hardest possible task to set as a goal, given that it requires the aforementioned tone check, a comparison with the sources, and also some sort of test to ensure the sources the other models see are an accurate reflection of the body of published RS more generally. It's definitely not suitable for a project intended to explore what can be done with relatively generic models. I think there could also be room to explore, e.g., faster community feedback cycles so that the WMF doesn't feel like it needs a whole year to develop a single task specific model. Alpha3031 (t • c) 08:56, 18 August 2026 (UTC)

Agree that NPOV assessment is the hardest possible task, also for humans. More importantly, the assessment relies on consensus. There is no authority that has the truth about whether something is NPOV. The whole point is that the community reaches a consensus looking at all available reliable sources. Delegating this to an LLM (with the mark of approval of the WMF itself, thus giving it an undue resemblance of authority) defies the premise of Wikipedia, which is based on decisions taken by consensus. Aggravated by the fact that LLMs are not at all unbiased by any definition of the term (and are often just plainly wrong). Ita140188 (talk) 10:27, 18 August 2026 (UTC)

Agreed, NPOV is hard! Alaexis_¿question? 12:52, 18 August 2026 (UTC)

Thanks for the response, this is already much more transparency than the last time around and I appreciate it.

I am a bit confused as to whether these questions are for us and for you. Gnomingstuff (talk) 17:35, 18 August 2026 (UTC)

@Gnomingstuff: oh, good clarification. The questions Marshall posted are a first pass at translating what we're hearing from y'all into a set of questions that we (staff) can then respond to one-by-one. Of course, if you think there are questions we've missed and/or questions that we've misinterpreted, please comment as much. PPelberg (WMF) (talk) 20:05, 18 August 2026 (UTC)

I thought I would ring in to point to another AI-backed project WMF is doing and share our experience working with volunteers on it, given the interest here in this sort of thing. For context, I lead the Product Safety and Integrity (PSI) team here at WMF, we build security and safety features.

One thing we're working on is detecting abusive content using LLM-based models. Specifically to enwiki, last week we deployed a new feature that is only visible to oversighters, at Special:AbuseReview, which uses an open-weight model called CoPE to flag edits that probably need to be suppressed due to containing personal information.

This on-wiki feature is now being used enwiki oversighters to review and take action on what it raises. They tell us the practical accuracy rate they experience in reviewing the output is better than what they see in the reports they get from human users. And, when we were testing the model output in June and July, in batches through an off-wiki process, most of the true positives it was catching were not being caught organically on-wiki.

One thing I want to mention is the iteration and volunteer collaboration that it took for us to make something deployable. There was some initial volunteer skepticism, and we needed to demonstrate not only that it was valuable, but that this was going to be accurate enough to not waste precious volunteer time. That took some internal experimentation on our part to get something that showed enough promise on both those fronts that we could ask for support doing a round of manual labeling -- which is what really pushed it over the edge of accuracy to be a deployable feature.

This collaboration has helped us do something meaningful about doxxing on English Wikipedia over the last few months, that feels good to both WMF and volunteers. That is possible in large part because volunteers were willing to give us space to operate, and to keep an open mind that could be convinced by data. We also needed to be flexible in our own thinking and incorporate volunteer ideas, something PSI has gotten used to doing in our work. EMill-WMF (talk) 21:17, 19 August 2026 (UTC)

Endorsing Eric's statement, as one of the oversighters involved in the testing/iteration he describes. This is catching OS-level material we were not catching otherwise, and is a clear benefit to the encyclopedia. In solidarity, asilvering (talk) 21:54, 19 August 2026 (UTC)

I think there is a role for AI to play on Wikipedia and it's going to be in something kind of like this. Something Eric alluded to but doesn't say directly which I think is important: the first draft the WMF showed us was nowhere close to ready. From that the learning was that we needed to go even farther in how confident . The second draft - which is when we started doing manual labeling - was still not anything which would have been appropriate for use. It did lead to a learning that one element - personal information - was more reliable than the other OS criteria which got us to the third draft and the efforts to then backfill June & July. That got us a lot of incredibly sensitive information that wasn't getting reported and which is now getting appropriately oversighted. I was the one who ran some data after we completed June to compare the true positive rate against email reports and found it to be in the range of 5% higher. I plan to revisit these stats soon and my hope would be that we'd have a higher difference between email reports and these flags for two reasons: 1) the classifier has been improved since I ran those numbers originally 2) some of the "obviously needs OS" tickets we'd have gotten in the past, we won't be getting because OS will be oversighting edits before some other qualified person finds them and reports them. I am really glad we're getting these signals now to stop some very sensitive personal information from being exposed against policy, we wouldn't have been able to do it without the AI classifer model, and it did take an iterative process to get there. Best, Barkeep49 (talk) 23:05, 19 August 2026 (UTC)

Downloaded the most recent csv, picked an NPOV one at random:

• "1358549724,The_Dying_Rooms,10791564,NPOV,Remove biased and profane language describing the Chinese government and replace it with a neutral summary of the government's response.,"In the film, Blewett and others travel to mainland China to visit orphanages housing children abandoned due to the ""one-child policy"". The filmmakers stated that unwanted female and disabled children were left to die of neglect, allowing parents to have another child. Showing that China government is a piece of sh!t for letting this happened and being a coward by lying to our faces that it didn't happened and said that all the footage is fabricated to destroy the reputation of the china government.>",enwiki,df59b690-6c8e-4827-b809-76e90d409f9f,"Editors often revise this kind of wording, saying the tone is unbalanced. You can help rewriting it using a [neutral point of view](https://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view).",Revise tone"

The statement the LLM objects to was in the article for less than a minute and long reverted by the time the suggestion was created. So one can add to all the above problems and errors that it also wastes times and resources by not checking the current version but some snapshot.

Profanity checks in general are a bad idea.

• "1358746317,2024_G20_Rio_de_Janeiro_summit,72163669,NPOV,Rephrase the incident with the first lady using neutral language and omit the explicit profanity.,"During a speech about fake news, Rosângela Lula da Silva, first lady of Brazil, swore at Elon Musk, saying: ""I'm not afraid of you. Fuck you, Elon Musk"" (Eu não tenho medo de você. Inclusive, fuck you, Elon Musk, in Portuguese).",enwiki,fba2904a-9276-47e3-8f5c-27fe5aff71f6,"Editors often revise this kind of wording, saying the tone is unbalanced. You can help rewriting it using a [neutral point of view](https://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view).",Revise tone"

No, we are not going to "omit the explicit profanity" from a quote, and suggesting things like this is a very, very bad idea.

• "1357774252,God_Emperor_Trump,74631506,NPOV,Remove profanity from the description of the phrase on the sword to maintain a neutral and encyclopedic tone.,"According to Fabrizio, the phrase could mean 'here's your fucking tariffs'.",enwiki,0f886b1f-2a4c-4303-b06e-63da6700b71e,"Editors often revise this kind of wording, saying the tone is unbalanced. You can help rewriting it using a [neutral point of view](https://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view).",Revise tone"

Again, it's a quote, from the creator of the sculpture. The AI should not make statements like "Editors often revise this kind of wording, saying the tone is unbalanced. ", which only works to influence newbies by making a false claim to authority.

And then there the internally contradictory advices, indicating the inherent stupidity of LLMs.

• "1356142445,Allan_Segura_(model),80130058,simplify_language,Combine the two sentences about sexual orientation and activism into one concise sentence.,Allan Segura is openly gay. He is a transgender rights activist.,enwiki,d28d67bc-2b32-4cbb-9135-ed391c91c13b,Readers might find this text difficult to understand. Try rewriting this using shorter sentences and plain language. [Learn more](https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style#Vocabulary).,Simplify language"

So do we need to combine these two (very short) sentences into one sentence, or do we need to rewrite this using shorter sentences? Or, just perhaps, our readers are perfectly capable of understanding these two sentences and won't "find this text difficult to understand". What a joke. Fram (talk) 09:01, 18 August 2026 (UTC)

@Fram Also, wasn't the fact that the WMF didn't do content the thing protecting them in lawsuits?

Let's say an article contains negative information about a rich person. If the WMF starts to mess with content, do they not open themselves up to be forced to make changes? I am not a lawyer. Polygnotus (talk) 12:23, 18 August 2026 (UTC)

All good reasons not to use AI for this purpose and probably not for any purpose. Once again, the WMF is wasting what remains of its valuable technical staff (and its ample funds) on trying to drag Wikipedia in completely the wrong direction.

If the bot is criticising vandalism which was only live for a minute, I suspect that it may be reading page history (why?) rather than taking a snapshot. Certes (talk) 12:32, 18 August 2026 (UTC)

A snapshot sample of a large number of articles will also catch revisions that lasted only for a minute. There are lots such edits that are reverted quickly.

Assuming that edit suggestions are not displayed when the content has changed, this particular suggestion would never have been shown - no harm down do anyone. Generating checks on a snapshot rather than live is an architectural decision driven by performance, complexity and other considerations. Alaexis_¿question? 12:51, 18 August 2026 (UTC)

Presumably that means any such system will need to be rerun on each relevant page each time they are edited, in case the edit touched the prompted part? Not sure how the current tasks handle this actually. CMD (talk) 12:59, 18 August 2026 (UTC)

Not necessarily, it can be run once a month, with some kind of caching enabled not to re-check the vast majority of content that stays the same. Then the complex and time-consuming part (LLM calls) is done asynchronously, and the easy part (deterministically checking that the content stayed the same) is done when the user opens visual editor. Alaexis_¿question? 14:34, 18 August 2026 (UTC)

That is indeed how it's currently working. A large batch of the suggestions were pre-generated, and was poured into a fairly simple API that VisualEditor's suggestion mode knows how to query and match up to a document.

Presumably if this was a successful experiment that we decide together to scale up, it'd turn into a more complicated system where we do something like fire off a job that precomputes the suggestion for each new revision of a page. Still avoiding needing to do the expensive generation every time someone opens the editor, but keeping them more up-to-date. DLynch (WMF) (talk) 23:03, 19 August 2026 (UTC)

Again, it's a quote, from the creator of the sculpture.

That's another LLM tic -- at least when it comes to Wikipedia edits/suggestions, they really hate direct quotes and will usually tell you to paraphrase them and/or do so themselves. (Example from a 2026 edit summary: "Paraphrased a lengthy direct quote regarding the skater's performance at the 2025 World Team Trophy into a concise summary. This improves readability and helps maintain a neutral, encyclopedic tone by removing overly detailed personal reflections.") Gnomingstuff (talk) 17:41, 18 August 2026 (UTC)

The examples Fram shows above makes me more resolute in opposing this on principle. The WMF is trying to exert editorial control and hiding it by labelling them as "suggestions". ♠JCW555 (talk)♠ 16:48, 18 August 2026 (UTC)

@JCW555, I guarantee you they are not trying to exert editorial control with this feature. They're trying it because they think it will be helpful for beginner editors. We can tell them they're wrong and we don't like the feature without impugning their motives. In solidarity, asilvering (talk) 21:21, 19 August 2026 (UTC)

But suggesting sentences should be reworded is the WMF trying to exert editorial control. To quote MMiller above "It would just point out the spot in the article that needs attention, e.g. "Does this sentence need to be rewritten to be easier to read?". Whether a sentence/passage needs to be reworded is a matter for the talk page amongst editors, not through the WMF via their AI. No reply from the WMF in this section has done anything to assuage that concern for me. If the WMF comes out and says that these "suggestions" wouldn't touch content at all, no matter how small or big, then I'd be a tad less aggressive in my opposition. ♠JCW555 (talk)♠ 21:44, 19 August 2026 (UTC)

Possible RfC

Given the WMF seems inclined to continue developing these tools regardless of what we say, I propose we hold an RfC establishing that LLM tools may not be deployed without an explicit consensus from the community. If editors think this may be a good idea, I suggest the following as the first draft of the question:

Should we require an explicit consensus before the WMF is permitted to deploy tools involving the use of an LLM? The WMF may deploy tools for user testing, so long as all of the following criteria are met:

A: Testing of the tool concludes after no more than 12 months, after which the tool must be removed unless there is a community consensus for either extended testing or full deployment.

B: Use of the tool is restricted to editors who have opted-in

C: Use of the tool is restricted to editors who have been granted the LLM-tool user right. This would be a new user right created and granted by admins at WP:PERM.

Does anyone see issues with this proposed question? Are there any revisions? BilledMammal (talk) 07:42, 21 August 2026 (UTC)

I'm not fundamentally opposed to this, but I think we need a lot more clarity about what this new LLM-tool user right would mean. A naive reading of the name would be "This user is exempt from WP:LLM" which I suspect is a broader interpretation than you intended. RoySmith (talk) 12:53, 21 August 2026 (UTC)

This seems like policy creep, and not a good approach (we shouldn't have blanket limitations based on the "type of tool" involved). I believe it is also misdirected: the example above is testing what would be an entirely community-configurable tool. Opt-in is appropriate for things that are so new, and I'd say anything that messes with your margins should be configurable. But proliferating user rights is an anti-pattern, and should only be used as a last resort. – SJ + 16:21, 21 August 2026 (UTC)

yeah, I think this may be premature for now, but worth revisiting if there are any disasters Kowal2701 (talk, contribs) 18:24, 22 August 2026 (UTC)

Consistent with the current wording of NOLLM and with active work being done - so far successfully - at helping limit abuse, I would suggest some cleaving of administrative actions and content actions. Best, Barkeep49 (talk) 17:05, 21 August 2026 (UTC)

Solely regarding access control (I'm undecided regarding the overall proposal): I agree with the other commenters that that creating a new user right isn't the best fit. I think tool-specific JSON lists of approved users that are fully protected could suffice, and be more adaptable to allow for per-tool authorization. isaacl (talk) 17:15, 21 August 2026 (UTC)

The whitelist system works well for AWB/JWB. Certes (talk) 21:13, 27 August 2026 (UTC)

I also think that the community should be able to control and configure the tools used in en wiki but I'm not sure an rfc is needed atm and its wording unfortunately isn't clear.

What is a "tool"? What else, other than edit suggestions, is in the scope?

12 months deadline is arbitrary: it's too long for a tool that the community actively opposes and may be too short for an iterated testing of a complex idea.

Some abuse/vandalism prevention tools by definition cannot be opt in. Alaexis_¿question? 20:18, 21 August 2026 (UTC)

It's hard to guess what is a tool, because the AI is developed secretly and added to Wikipedia's software or configuration as a fait accompli. Certes (talk) 21:13, 27 August 2026 (UTC)

I agree with most of the comments above that this is not a well-defined RfC. In addition, the WMF have said this will be community-configurable if it makes it into production so it appears to be unnecessary anyway. Mike Christie (talk - contribs - library) 18:34, 22 August 2026 (UTC)

I assume this is intended to apply to any future implementation that would integrate LLM features in any way, not just this one concept, which I fully support. Pushing it on us without community consent is inappropriate and exactly the type of forced AI we're seeing on every other platform. Wikipedia is supposed to be different.

I'd agree on amending the deadline - I'm not sure what an appropriate initial test phase would look like, but I'd recommend a limit aligned with that and only extended or fully implemented via consensus. Just enough to get a feel for it, not a full beta phase to polish it for release. People with more programming experience than me would probably have a better notion of the timeline.

I do think we need some limitation on access (especially if it is fully implemented) and I think Isaacl's suggestion sounds like a better way to handle it. ChompyTheGogoat (talk) 20:58, 27 August 2026 (UTC)

I think that some level of access control is warranted, but the idea of getting individual approval for every tool that the WMF wants to test would impede testing numbers without much safety gain over a single user list for all beta tools.

Having a time limit on testing is strange, but my exact opinion on it depends on how community consensus is defined here. I think something that both mitigates the issue i think you're trying to solve while not dictating the WMF's development calendar would be something like "Any tool in testing shall be disabled if a consensus is formed at the tool's thread on VPWMF that the tool is causing disruption. At which point the tool will only be re-enabled with consensus." Although, if we got anything near consensus that a tool in testing is disruptive, I think the team working on the tool would disable/fix it very quickly. These are people who chose to work on mediawiki here, not WMF management. ^MetalBreaks_AndBends (One for all) 01:32, 28 August 2026 (UTC)

I think it's only suggesting permissions for LLM tools that would potentially be more prone to abuse, not for any and all in testing. ChompyTheGogoat (talk) 07:13, 28 August 2026 (UTC)

There isn't any limiting language in the comment, so I (and most people, probably) would think it would apply to all LLM tools. ^MetalBreaks_AndBends (One for all) 23:42, 28 August 2026 (UTC)

That is what I meant - all LLM, not all tools period. If we end up with so many LLM tools being tested that they're hard to keep track of I'd consider that a problem unto itself. ChompyTheGogoat (talk) 02:50, 29 August 2026 (UTC)

I'm not saying the issue is that they could be hard to keep track of (though over a long enough time period it could), I'm saying that applying for every beta is an unnecessary hassle. ^MetalBreaks_AndBends (One for all) 03:38, 29 August 2026 (UTC)

I think I would share the view that a full, formal RFC would be unnecessary if prior discussion shows clear consensus to implement, for example (assuming it's advertised to a noticeboard like this one or the cleanup wikiproject). I would expect any community members participating in such discussions to be sufficiently in touch with current attitudes towards LLM tools to bring up anything that would seem potentially controversial and standard pre-RFC discussion processes should function fine (w formal closures and move to full RFC if no clear consensus or if there is consensus there should be an RFC in the specific discussion). Alpha3031 (t • c) 04:05, 29 August 2026 (UTC)

Do you mean a full rfc for each tool or an RFC for this proposal? ^MetalBreaks_AndBends (One for all) 06:25, 29 August 2026 (UTC)

Hey WMF: nice job on communicating in this section

I think this conversation has been healthier than some other recent enwiki–WMF conversations. Just wanted to say thanks to the WMFers that are participating here and doing things like compromising (i.e. getting rid of the NPOV model), replying a lot instead of making one polished statement then leaving, and speaking clearly and honestly.

It's tough because on some issues, WMF and enwiki are very out of sync. So even if everyone does everything right, these conversations may still be tough. But doing things like compromising, having conversations with us that aren't just statements, and speaking clearly and honestly are definitely a good approach. Please keep it up. –Novem Linguae (talk) 19:23, 22 August 2026 (UTC)

Absolutely seconding this! Really happy to see WMF folks take community feedback into account. I understand the task can be much harder than it seems at first, especially as these discussions get sprawling and it can be hard to find a thread that unites the whole range of community opinions together, let alone incorporate it in the team's plans for the project.

The way you managed to navigate it was a very positive surprise, and I'm looking forward to more productive exchanges from both sides! Chaotic Enby (in solidarity · talk · contribs) 19:45, 22 August 2026 (UTC)

Yes, absolutely. Ymblanter (talk) 08:10, 23 August 2026 (UTC)

Absolutely agreed on all of this Gnomingstuff (talk) 19:01, 23 August 2026 (UTC)

+1. Can't speak for anyone else, but pretty much all my complaints about WMF's poor communication are limited to the Trustees and a few Officers. Everyone else at the WMF seems to communicate fine. MMiller and PPelberg are two usernames (among others) I've grown very accustomed to seeing regularly on-wiki, they've communicated often and effectively with volunteers for years. Levivich (talk) 20:22, 23 August 2026 (UTC)

Thank you -- we're glad to hear it -- we are trying hard! Although there are going to keep being times that we all disagree on ideas, the most important thing is that we can discuss constructively to figure out how to best improve/adapt the wikis. Thank you all for being here for these conversations, on top of doing all your usual wiki work. MMiller (WMF) (talk) 05:20, 24 August 2026 (UTC)

I am of the opinion that most -- maybe all -- WMF employees who are not in top management are competent, helpful, want to do the right thing, and are eager to communicate. I attribute the stonewalling we often see with a perfectly reasonable fear that actually having a dialog with the volunteers will never help your career and just might get you fired. If only there was some way that WMF workers could organize and join an entity that works to protect them from being unjustly fired... Let me know if anyone has ever heard about something like that. --Guy Macon (talk) 12:38, 24 August 2026 (UTC)

I find this post amusing. But as a Wikipedian, I can't help but be myself and note that if you define top management as something beyond "C Suite" Marshall is would probably be considered top management. Best, Barkeep49 (talk) 14:47, 24 August 2026 (UTC)

+Whatever we’re on _{In solidarity} Wikipedian12512 (Talking is fine | contribs) 01:05, 6 September 2026 (UTC)