User talk:Trappist the monk/Archive 24

Hello!

This is the regex I have for season conversions: r"(\{\{\s*cit[aeio][^\}]*\|\s*(?:access\-?|archive\-?|doi\-broken\-|lay\-|pmc\-embargo\-|publication\-|air\-?)?date\s*=\s*\d{1,2} +)Summer( +\d{4})", r"\1verë\2"

This^[1] is a citation with a season which should be converted.

regex101 fails on it though and doesn't match. Is it because the reference is too long for regex101? Or have I done any mistakes in the regex? - Klein Muçi (talk) 07:49, 6 September 2022 (UTC)

There are no days in a summer 'month'.

—Trappist the monk (talk) 11:46, 6 September 2022 (UTC)

So, this?
r"(\{\{\s*cit[aeio][^\}]*\|\s*(?:access\-?|archive\-?|doi\-broken\-|lay\-|pmc\-embargo\-|publication\-|air\-?)?date\s*=\s*)Summer( +\d{4})", r"\1verë\2"}} — Klein Muçi (talk) 11:56, 6 September 2022 (UTC)

Yep.

—Trappist the monk (talk) 12:03, 6 September 2022 (UTC)

And what about this then:
r"(\{\{\s*cit[aeio][^\}]*\|\s*(?:access\-?|archive\-?|doi\-broken\-|lay\-|pmc\-embargo\-|publication\-|air\-?)?date\s*=\s*\d{1,2} +)First Quarter( +\d{4})", r"\1Tremujori i parë\2"
Or this:
r"(\{\{\s*cit[aeio][^\}]*\|\s*(?:access\-?|archive\-?|doi\-broken\-|lay\-|pmc\-embargo\-|publication\-|air\-?)?date\s*=\s*\d{1,2} +)Easter( +\d{4})", r"\1pashkë\2"
Also suffering from the same mistake? — Klein Muçi (talk) 12:07, 6 September 2022 (UTC)

yes, but you knew that...

—Trappist the monk (talk) 12:41, 6 September 2022 (UTC)

Valuable confirmation. — Klein Muçi (talk) 14:43, 6 September 2022 (UTC)

This regex
r"(\{\{\s*cit[aeio][^\}]*\|\s*(?:access\-?|archive\-?|doi\-broken\-|lay\-|pmc\-embargo\-|publication\-|air\-?)?date\s*=\s*\d{1,2} +)November( +\d{4})", r"\1nëntor\2"
misses this citation
{{Cite book|first=Colin|last=Nash|title=The History of Aquaculture|url={{google books |plainurl=y |id=glWz131N4i4C}}|date=23 November 2010|publisher=John Wiley & Sons|isbn=978-0-470-95886-5}}
If my assumption is correct is because of the curly brackets inside the URL parameter.
I've never stumbled on this use of that parameter before so can you guide me a bit? Is that a normal practice? Should it be changed? If it is a normal practice, how much of an edge case is it? — Klein Muçi (talk) 10:12, 8 September 2022 (UTC)

As normal as anything else on wikipedia. Used about 30 times at sq.wiki.

—Trappist the monk (talk) 13:24, 8 September 2022 (UTC)

Hmm... I'm actually starting to get more cases "like this":
{{cite web |title=Amsterdam {{!}} History, Population, & Points of Interest |url=https://www.britannica.com/place/Amsterdam |access-date=5 January 2021 |website=Encyclopedia Britannica |language=en}}
Because of the {{!}}, all regexes fail here.
I now remember vaguely discussing this together in the past. Sorry for being a broken record (with an even more broken memory) but any general "guidelines" about including templates inside cite templates? Any crystallized common practice about them in general? Do they occur everywhere or only in regard to certain parameters?
I don't know how to act. I'm not sure if I should do something to them manually, leave them as they are and change my regexes somehow or leave everything as it is and fix the remaining errors manually. :/ — Klein Muçi (talk) 14:39, 8 September 2022 (UTC)

Changing your regexes likely isn't possible. The {{!}} (and also |) are commonly added to |title= by citoid because the Zotero translator simply scrapes <title>...</title> tag contents, replaces any | with {{!}} or | and calls it good; it isn't good, but the editors who use ve and similar tools either don't care or don't know enough to care. This is one reason that automatic citation filling, while a nice idea, in practice is a failure.

I fix 'em as I find 'em. I don't know of a reliable way to automate such fixes because they are so variable.

—Trappist the monk (talk) 15:53, 8 September 2022 (UTC)

┌───────────────────────────┘
I see... Although this wasn't my original question, what about a regex that searches through cite templates and if it finds a {{!}} or | in them converses it to a pipe? Would that be a good idea or would it be prone to false positives? — Klein Muçi (talk) 16:43, 8 September 2022 (UTC)

Taking your example template and converting the {{!}} to | you get this:

{{cite web |title=Amsterdam | History, Population, & Points of Interest |url=https://www.britannica.com/place/Amsterdam |access-date=5 January 2021 |website=Encyclopedia Britannica |language=en}}

"Amsterdam". Encyclopedia Britannica. Retrieved 5 January 2021. {{cite web}}: Text "History, Population, & Points of Interest" ignored (help)

Yeah, you get an error message. Some other editor will 'fix' that error by converting | to {{!}} or |. And your regex will convert the {{!}} or | to | and round and round you go...

One thing that might be done to fix date problems with templates that use {{!}} is to have a single regex at the beginning of the list of date-fixing regexes that converts all instances of {{!}} on a page to a special secret code, perhaps something like: __P1P3__. Then at the end a single regex that converts __P1P3__ to {{!}}.

By the way, the proper fix for the example template is:

{{cite encyclopedia |entry=Amsterdam |entry-url=https://www.britannica.com/place/Amsterdam |access-date=5 janar 2021 |encyclopedia=Encyclopedia Britannica |language=en}}

"Amsterdam". Encyclopedia Britannica. Retrieved 5 janar 2021. {{cite encyclopedia}}: Check date values in: |access-date= (help) yeah, januar doesn't work here...

—Trappist the monk (talk) 17:25, 8 September 2022 (UTC)

That sure is an interesting solution. I'll try to come up with something like that. But I'm kinda confused about the change you made: Why use entry instead of the title? What does "entry" mean as a parameter in itself in this case? I believe this is a language barrier for me. — Klein Muçi (talk) 21:12, 8 September 2022 (UTC)

When I can, I prefer to use parameter names that reflect the nature of the thing that I am citing. |title= is required for {{cite journal}}, {{cite news}}, {{cite magazine}}, etc. For those templates, had it been up to me, would not have used |title= but would have used |article= to hold the article's title. Stands to reason. Because encyclopediae generally contain collections of writings that may or may not be separately authored, may be very brief or 'chapter' length, I like |entry= as a descriptor for those writings. If you don't like |entry=, you can use any of its aliases or even |title= if you prefer.

—Trappist the monk (talk) 22:10, 8 September 2022 (UTC)

No, I fully understand that. I just don't understand the term "entry" in English, what does it mean, the word itself. — Klein Muçi (talk) 22:17, 8 September 2022 (UTC)

Noun entry #9 at Wiktionary.

—Trappist the monk (talk) 22:35, 8 September 2022 (UTC)

Thank you! :) I'll go on with the proposed regexes and the cite change. — Klein Muçi (talk) 22:40, 8 September 2022 (UTC)

How do these regexes look for escaping the {{!}}? I'm almost sure I've done mistakes in them considering the large number of brackets I needed to escape.
(r"(\{\{\s*cit[aeio][^\}]*)(\{\{!\}\})(.*\}\})", r"\1__P1P3__\3"),
(r"(\{\{\s*cit[aeio][^\}]*)(__P1P3__)(.*\}\})", r"\1{{!}}\3") — Klein Muçi (talk) 21:51, 8 September 2022 (UTC)

We must have had an undetected edit conflict. I wouldn't bother with finding {{!}} in cs1|2 templates only; I'd replace them all because there aren't likely to be that many:

(r"\{\{!\}\}", r"__P1P3__"),

(r"__P1P3__", r"{{!}}")

—Trappist the monk (talk) 23:35, 8 September 2022 (UTC)

Oh... So do these changes go as the absolute first and last change in the regex list (even before - and after - other changes not related to dates)? — Klein Muçi (talk) 23:46, 8 September 2022 (UTC)

yes

—Trappist the monk (talk) 00:05, 9 September 2022 (UTC)

Okay then. Thank you! — Klein Muçi (talk) 00:50, 9 September 2022 (UTC)

Hello,

I would like to point out that a certain website owner is constantly trying to promote his website on https://en.wikipedia.org/wiki/Battlegrounds_Mobile_India by claiming some time that the one is sharing an update and some time claiming that it is their official website. In reality, both are wrong.

I have been removing it since a while but the one is editing the page again and again. So, will it be possible for you to do something about it.

I am sorry if I am bothering you. I saw you as the administrator and someone who edited the page, so I decided to let you know about it.

Thank you! DreamySkyy (talk) 14:54, 13 September 2022 (UTC)

Probably the best place for you to take this is to Wikipedia:Requests for page protection. Page protection is not something that I have much experience with.

—Trappist the monk (talk) 14:58, 13 September 2022 (UTC)

Hi @Trappist the monk:, I see that you're upgrading lang-tt to lang-tt-Cyrl template in several articles. On Äxmät İsxaq, I reverted this because lang-tt provides links to the Arabic İske imlâ alphabet alphabet, as well as links to Tatar alphabet for Latin and Cyrillic. That seems more helpful that just the link to Tatar language alone. I'm also not sure that using the translit parameter with lang-tt-Cyrl is best practice since it's the name rendered in the Zamanälif Tatar Latin orthography, not simply a transliteration/romanization from the Tatar Cyrillic alphabet. Is there some benefit to lang-tt-Cyrl that I'm missing that lang-tt doesn't provide? —Carter (Tcr25) (talk) 14:23, 17 September 2022 (UTC)

It is not necessary to ping a user on that user's talk page. Also, adding {{ping}} after the fact does not ping; from the template doc:

The notification will work successfully only if you sign your post in the same edit in which you use this template.

I'm making these changes because the two links to Tatar alphabet for Latin and Cyrillic are sort of WP:EASTEREGGy in that the link labels don't say that they are to the alphabets peculiar to Tatar and in most cases, those links aren't needed. If readers want to know more about Tatar alphabets, those alphabets are linked from / discussed in Tatar language § Writing system. As for the Arabic link, {{lang-tt}} would have us believe that İske imlâ alphabet is the only Arabic alphabet used for Tatar. According to that article, İske imlâ was a (relatively) short-lived alphabet succeeded by Yaña imlâ alphabet.

If you want to link {{lang-tt-Cyrl}} to Tatar alphabet (or some other place, you can write something like this:

{{langx|tt-Cyrl|Әхмәт Исхак|translit=Äxmät İsxaq |label=[[Tatar alphabet]]}} → Tatar alphabet: Әхмәт Исхак, romanized: Äxmät İsxaq

The impetus for the changes arose from Template talk:Lang-tt § Protected edit request on 10 April 2022. The majority of {{lang-tt}} templates used a single parameter, mostly, but not always Cyrillic text. These single-parameter templates get it wrong for Cyrillic text:

{{langx|tt|Әхмәт Исхак}} → Tatar: Әхмәт Исхак – should not be italicized

and when given Latin and Arabic text:

{{langx|tt|Äxmät İsxaq}} → Tatar: Äxmät İsxaq – correct

{{langx|tt|أحمد إسحاق}} → Tatar: أحمد إسحاق – should not be italicized

As part of this exercise, I have discovered many templates that used parameters not supported by {{lang-tt}} – mostly |italics= and |link=; these are parameters supported by most {{lang-??}} templates so editors clearly expect that {{lang-tt}} works the same way. It doesn't.

When I have finished with my awb run, and fixed those templates that my script could not fix, I intend to make {{lang-tt}} a non-specific template much like {{lang-ru}}, {{lang-es}}, etc.

—Trappist the monk (talk) 15:22, 17 September 2022 (UTC)

Thanks for the explanation. I just looked at that edit request and that helps explain a lot of it. It might be a bit late in the game, but the string {{langx|tt-Cyrl|<Cyrillic>|translit=<Latin>}} {{lang|tt-Arab|<Arabic>}} should have a comma between the Latin and Arabic versions. —Carter (Tcr25) (talk) 16:05, 17 September 2022 (UTC)

Yeah, a bit late but there are only 25 articles with that string so if you are looking for something to do ...

—Trappist the monk (talk) 16:11, 17 September 2022 (UTC)

I accept the challenge. Thanks! —Carter (Tcr25) (talk) 16:13, 17 September 2022 (UTC)

I'm confused by the replacement of Template:cite news with Template:cite journal in this edit. Was this in error? I've just partially reverted. Schierbecker (talk) 21:18, 18 September 2022 (UTC)

Between 21 May 2022 and the time of this writing, you've made approximately 195 edits to that page and you are just now asking me this question? Had you asked me in April or even May I might have been able to give you a better answer; as it is, I don't remember making that edit though clearly I did.

Ok, after a bit of hunting about, I discover that I must have visited that article because of this inappropriate url:

https://www-jstor-org.wikipedialibrary.idm.oclc.org/stable/43976559?Search=yes&resultItemClick=true&searchText=%22armored+gun+system%22&searchUri=%2Faction%2FdoBasicSearch%3FQuery%3D%2522armored%2Bgun%2Bsystem%2522%26so%3Dold%26sd%3D1994%26ed%3D1994&ab_segments=0%2Fbasic_phrase_search%2Fcontrol&refreqid=fastly-default%3A73f29ced160178d1558fff87138a694b&seq=1#metadata_info_tab_contents

which was added to the article at this edit – don't do that.

For whatever reason, I'm guessing that I chose {{cite journal}} over {{cite news}} because I associate {{cite news}} with newspapers, with broadcast or online news sources. I suspect that I chose {{cite journal}} over {{cite periodical}} or {{cite magazine}} because JSTOR holds journals and, they even refer to Inside the Army articles as journal articles; see the JSTOR landing page for the above url. I suspect that because of that one improper url, I did an automated search and replace for all of the Inside the Army citations.

—Trappist the monk (talk) 22:36, 18 September 2022 (UTC)

Quick facts Nine years! ...

Precious

Nine years!

Close

--Gerda Arendt (talk) 06:48, 20 September 2022 (UTC)

Dear User:Trappist the monk On August 30, 2022, you deleted part of a reference in the article Arkan Simaan indicating "parameter misuse". A few days later, on September 18, 2022, I undid your modification because it gave a bad result by removing all the informations. Indeed, take the test and you will see that there appears this mention in French: « Cette page ne semble pas exister. Il semble que le lien pointant ici soit défectueux. Et si vous essayiez plutôt de rechercher ? » (“This page does not seem to exist. It looks like the link pointing here is broken. How about trying to search elsewhere instead?”) On the other hand, if you open with the parameters that you have removed, this page opens correctly. I therefore await your response to know whether or not I should undo your contribution again. Thank you for your attention Vicente Seville (talk) 09:19, 4 October 2022 (UTC)

This is the template before my script's edit:

{{Cite web|url=https://web.archive.org/web/20201025131542/http://union-rationaliste.org/index.php/rationalisme-scientifique/emissions/radio-libertaire/132-lecuyer-dhenri-le-navigateur|title=L'écuyer d'Henri le Navigateur|website=Union rationaliste|access-date=2022-07-08|archive-date=2020-10-25|archive-url=https://web.archive.org/web/20201025131542/http://union-rationaliste.org/index.php/rationalisme-scientifique/emissions/radio-libertaire/132-lecuyer-dhenri-le-navigateur|url-status=live}}

"L'écuyer d'Henri le Navigateur". Union rationaliste. Retrieved 2022-07-08. {{cite web}}: Check |archive-url= value (help)CS1 maint: url-status (link)

and this is the template after the edit:

{{Cite web|url=http://union-rationaliste.org/index.php/rationalisme-scientifique/emissions/radio-libertaire/132-lecuyer-dhenri-le-navigateur|title=L'écuyer d'Henri le Navigateur|website=Union rationaliste|access-date=2022-07-08|archive-date=2020-10-25|archive-url=https://web.archive.org/web/20201025131542/http://union-rationaliste.org/index.php/rationalisme-scientifique/emissions/radio-libertaire/132-lecuyer-dhenri-le-navigateur}}

"L'écuyer d'Henri le Navigateur". Union rationaliste. Archived from the original on 2020-10-25. Retrieved 2022-07-08.

You will notice that original template has an error message: {{cite web}}: Check archive-url= value (help). If you follow the error message help link, the reason for the error message is explained there:

This error message is emitted when the value assigned to |archive-url= is the same as the matching title or chapter URL.

In your template, both of |url= and |archive-url= hold the same url. That is wrong. |url= gets the 'original' url of the source; |archive-url= gets the url of an archived snapshot. Yes, the original url is dead and displays "Cette page ne semble pas exister..." when you click on 'the original' link in the rendered citation. This is the normal and correct way that {{cite web}} works at en.wiki.

The script's edit was correct so you should so you should not revert it again.

—Trappist the monk (talk) 11:53, 4 October 2022 (UTC)

Can you explain, please, what is the difference between <br> and <br />? The first one works fine, so why do you change it to the second? -- Ssilvers (talk) 20:32, 8 October 2022 (UTC)

I and others use User:Remember the dot/Syntax highlighter (available at Special:Preferences#mw-prefsection-gadgets → Editing). That is a rather old highlighter that isn't quite clever enough to recognize that <br> is the same as <br />. When the highlighter encounters <br>, everything after it is colored pink. To undo the pink, I have a personal script that converts <br> to <br />. Any excessive pink remaining is usually caused by an unclosed html or html-like tag (<ref> without </ref> etc).

If you right-click → View page source on a article that has <br> in its wiki text and do a Ctrl-F search for <br, you will see that MediaWiki, when it creates the article's html image, has replaced the <br> with </br>.

—Trappist the monk (talk) 21:56, 8 October 2022 (UTC)

Can't someone get the syntax highligher to recognize that <br> is OK? -- Ssilvers (talk) 01:29, 9 October 2022 (UTC)

For performance reasons apparently, the hightlighter's author doesn't want to do that. See mw:User talk:Remember the dot/Syntax highlighter; do a Ctrl-F search for <br.

—Trappist the monk (talk) 13:14, 9 October 2022 (UTC)

OK, thanks. I'm unwatching this page, so if you have any follow-up, please ping me. -- Ssilvers (talk) 16:32, 10 October 2022 (UTC)

I have nominated Rongorongo for a featured article review here. Please join the discussion on whether this article meets the featured article criteria. Articles are typically reviewed for two weeks. If substantial concerns are not addressed during the review period, the article will be moved to the Featured Article Removal Candidates list for a further period, where editors may declare "Keep" or "Delist" in regards to the article's featured status. The instructions for the review process are here. A455bcd9 (talk) 14:34, 17 October 2022 (UTC)

Hello!
Some time ago you helped me take care of pipes in citations with regexes. Ever since our category of automatically translated dates is almost empty. However, there are some stubborn entries there. I thought I can ask you for some of them.
One such article is this: w:sq:Antonio Gramshi. It's fourth reference there keeps being skipped by Smallem but I don't understand exactly why. — Klein Muçi (talk) 13:14, 28 September 2022 (UTC)

Because there are two {{!}} templates in |title=?

{{Cite web|url=http://www.iltorinese.it/italiani-origine-albanese-si-distinti-secoli/|title=Italiani di origine albanese che si sono distinti nei secoli {{!}} Il Torinese {{!}} Quotidiano on line di Informazione Società Cultura|website=www.iltorinese.it|language=it-IT|access-date=2018-05-05|date=8 January 2016}}

The proper fix for that template is:

{{Cite news |url=http://www.iltorinese.it/italiani-origine-albanese-si-distinti-secoli/ |title=Italiani di origine albanese che si sono distinti nei secoli |work=Il Torinese |language=it-IT|access-date=2018-05-05|date=8 January 2016}}

<rant>insert my common complaint about automated citation fillers here.</rant>

We concocted a regex to convert {{!}} to some secret code did we not? Does that regex work for this template at regex101.com?

—Trappist the monk (talk) 13:45, 28 September 2022 (UTC)

We concocted a regex to convert {{!}} to some secret code did we not? Does that regex work for this template at regex101.com?
It does. Shouldn't it work because of that? — Klein Muçi (talk) 14:30, 28 September 2022 (UTC)

If it works at regex101.com then that suggests that there is something wrong with smallem's implementation of the regex... As an experiment, you might duplicate that regex so that it runs twice. What happens?

—Trappist the monk (talk) 14:43, 28 September 2022 (UTC)

I'll try doing that as soon as Smallem finishes its current run and report back with the results. — Klein Muçi (talk) 17:09, 28 September 2022 (UTC)

And, no. It doesn't detect it even with the duplicated regex. Actually, to be completely transparent, it didn't detect any kind of problem at all on that run which is strange. I'm doing another run with the single regex to see if it does detect anything else now. If that happens that would imply that the double regex somehow caused a problem unrelated to the matter we're talking. It could also be that there aren't any problems to solve currently as I've been running it a lot lately and after many-many-many runs, Smallem has finally managed to catch up with SqWiki. From runs that would last more than 1 week with hundreds of edits, they now last 24 hours more or less with ~100 edits in total.

#!/bin/bash
source $HOME/.bash_profile
python3 /data/project/shared/pywikibot/stable/scripts/replace.py -log -ns:0 -transcludes:Moduli:Citation/CS1 r"\{\{!\}\}" r"__P1P3__" r"\{\{!\}\}" r"__P1P3__" -fix:Gjuhë -fix:Data -fix:Përditësime -fix:Shtesa -fix:Padukshëm -fix:Bosh -fix:Dorazi -fix:Referime -fix:Prova r"__P1P3__" r"{{!}}" -summary:"Regexes here only serve with the workflow, they leave no permanent mark." -always

This is the script for that. Doubled pipe conversion code, a series of fixes for languages, dates, etc. (invoked by their names) and the pipe reversion code. (The summary is there only because you can't leave it empty but it doesn't get written anywhere, each fix has its own summary.)

I suppose that's what you meant, no? I also thought about duplicating the reversion code but then I thought that logically there wouldn't be any need of that. And even if it was, it would just leave behind traces of __P1P3__, not not work at all. - Klein Muçi (talk) 10:22, 29 September 2022 (UTC)

Were it me, and I were trying to find out why date fixes aren't working, I would strip everything that isn't required for date fixes from the bash command line and run smallem over just the articles in sq:Kategoria:Mirëmbajtja CS1: Datë e përkthyer automatikisht instead of all-pages-transcluding-Module:Citation/CS1. Surely you can limit smallems scope. Is it known that r"\{\{!\}\}" r"__P1P3__" r"\{\{!\}\}" r"__P1P3__" in the command line is applied to every article that smallem attempts to fix? What if you add the r"\{\{!\}\}" r"__P1P3__" and r"__P1P3__" r"{{!}}" to the beginning and end of the -fix:Data regexes?

—Trappist the monk (talk) 12:53, 29 September 2022 (UTC)

That's what I would normally do as well but it was laziness that killed the cat... and that approach.
I thought given that Smallem now takes less than 1 day to do a full run I could very well try it like that instead of rewriting the script. Sending commands over SSH in Toolforge on a terminal is annoying. But I will do it now anyway. The thing is that there are around 80 articles waiting there and I was hoping to find a quick-fix for this and move on on asking questions about some other cases as well. (I believe 2–3 cases would suffice on dealing with all the ~80 cases.) But this proved more difficult than I expected.
Is it known that r"\{\{!\}\}" r"__P1P3__" r"\{\{!\}\}" r"__P1P3__" in the command line is applied to every article that smallem attempts to fix?
I can say with some degree of certainty that it is.
What if you add the r"\{\{!\}\}" r"__P1P3__" and r"__P1P3__" r"{{!}}" to the beginning and end of the -fix:Data regexes?
I can try that as well but the general idea was that I wanted a universal solution for all lists of fixes in regard to pipes, not only for dates. — Klein Muçi (talk) 15:13, 29 September 2022 (UTC)

Well, the run is complete and Smallem did 38 changes in total and that category went from 84 to 77 but w:sq:Antonio Gramshi was skipped again even with the double regex. I wonder if the pipe regex serves any function at all the way I've set it up. Maybe it needs to be part of the fixes itself to work, as you suggested above. Maybe Smallem has always skipped all citations containing pipes all this time, the number in the category went down because of other fixes that were made. I will try to search for articles containing pipes in their citations and see if Smallem has ever done any changes in their history. — Klein Muçi (talk) 15:26, 29 September 2022 (UTC)

It turns out I don't know how to search for articles containing pipes in their citations and see if Smallem has ever done any changes in their history. Can you help me? :P — Klein Muçi (talk) 12:29, 30 September 2022 (UTC)

this search finds roughly 430 articles.

—Trappist the monk (talk) 13:13, 30 September 2022 (UTC)

This edit shows they do work. I have no more ideas why that article gets skipped. Maybe I've done something wrong on the date regex related to it... - Klein Muçi (talk) 14:41, 30 September 2022 (UTC)

This search finds roughly 80 articles with two {{!}} templates in a cs1|2 template.

If it were your date-fixing regex, I would expect sq:Kategoria:Mirëmbajtja CS1: Datë e përkthyer automatikisht to hold a lot of cs1|2 templates that don't have {{!}} but do have dmy dates or dd January YYYY dates. Is that the case?

—Trappist the monk (talk) 15:06, 30 September 2022 (UTC)

┌─────────────────────────────────┘
I mean, it is easier to check the regex itself than do stat-analysis on this, no?
(r"(\{\{\s*cit[aeio][^\}]*\|\s*(?:access\-?|archive\-?|doi\-broken\-|lay\-|pmc\-embargo\-|publication\-|air\-?)?date\s*=\s*\d{1,2} +)January( +\d{4})", r"\1janar\2")
If I use that regex on regex101 and I remove the 2 pseudo-pipes from the citation, it matches in 62 steps and substitutes it accordingly. — Klein Muçi (talk) 15:17, 30 September 2022 (UTC)

Which is why I asked about the contents of sq:Kategoria:Mirëmbajtja CS1: Datë e përkthyer automatikisht. I looked into the category and of the few articles that I looked at, this one doesn't have internal templates. smallem has visited the article twice where it fixed |access-date=30 Jan 2022 → |access-date=30 jan 2022. On its next visit, smallem ignored |date=30 Jan 2022. Is this because the regexes are case-insensitive? Yeah, I know, off topic... But, my brief and wholly unscientific look in the category suggests that smallem isn't ignoring a lot of dmy dates or dd January YYYY dates.

regex.com is different from smallem but we should be able to say that this date regex is correct and, given the evidence of sq:Special:Diff/2439472, it appears to work as it should (except perhaps for case insensitivity). Therefore something about double pseudo-pipes and our 'fix' for them may not be working correctly in smallem's environment.

Can you run smallem over one or a few articles rather than the whole of sq.wiki articles with cs1|2 templates? Can you put the pseudo-pipe encode/decode regexes into just the date regex list and only run that set of regexes?

—Trappist the monk (talk) 16:00, 30 September 2022 (UTC)

Yes, the regexes are all case-insensitive because templates are rather "case-insensitive" themselves. (So we don't have to write [Cc]ite, etc.) That has been deliberate and so far there hasn't been a problem up until the one you identified now. I wonder if a regex can be written just for that case which would also be case-insensitive. I'll run it on the category now and report back. Then I'll try the other option you mention because that requires to basically write a new script. — Klein Muçi (talk) 10:26, 1 October 2022 (UTC)

I run it only on the category in a way that I could see what was happening (sort of).
As expected, it skipped the Antonio Gramshi article.

Found 1 wikipedia:sq processes running, including this one.
Retrieving 50 pages from wikipedia:sq.
No changes were necessary in [[Akuakultura]]
No changes were necessary in [[Andrew Yang]]
No changes were necessary in [[Antonio Gramshi]]
No changes were necessary in [[Arilena Ara]]

Now... In regard to putting the regexes straight in the date regex list, doubled I guess, no? — Klein Muçi (talk) 10:46, 1 October 2022 (UTC)

Do one run of not doubled and one of doubled? if not doubled works you're done. smallem seems to fix multiple cs1|2 templates for each date regex so the not doubled should fix multiple {{!}} when that regex runs, shouldn't it?

—Trappist the monk (talk) 11:47, 1 October 2022 (UTC)

Just tried, not doubled. Check last 2 edits here. It worked as it should. What does this tell me exactly though? — Klein Muçi (talk) 15:02, 1 October 2022 (UTC)

What do you think it tells you? I think that it tells you that because doubling in the command line did not work, you must add the pseudo pipe regexes to each of your date/language/... regex lists unless... I wonder if bash or whatever, seeing the duplicated regexes in its command line, strips the second, apparently extraneous, regex. Is there a way to ensure that a command line regex is 'global' (doesn't return after the first match)? Does something like this in your command line work?

r"\{\{!\}\}"g – g is a commonly used flag that tells the regex engine to not return after the first match

—Trappist the monk (talk) 15:20, 1 October 2022 (UTC)

I wonder if bash or whatever, seeing the duplicated regexes in its command line, strips the second, apparently extraneous, regex.
Just to be sure, it was the NOT doubled version that worked, not the doubled one. I'm thinking to create 2 other fix-lists called something like "pipes-start" and "pipe-end" and put them in the beginning and end of the current list of fixes. That would be easier and I can use those 2 extra lists for various fixes as well, like {{google books}}, etc. Do you think that would be able to yield the same results? — Klein Muçi (talk) 15:31, 1 October 2022 (UTC)

Yes, I know. But what I'm wondering is: what happens if you modify your command line to remove the duplicates and then flag the remaining fix-pseudo-pipe regexes in the command line with g:

#!/bin/bash
source $HOME/.bash_profile
python3 /data/project/shared/pywikibot/stable/scripts/replace.py -log -ns:0 -transcludes:Moduli:Citation/CS1 r"\{\{!\}\}"g r"__P1P3__" -fix:Gjuhë -fix:Data -fix:Përditësime -fix:Shtesa -fix:Padukshëm -fix:Bosh -fix:Dorazi -fix:Referime -fix:Prova r"__P1P3__"g r"{{!}}" -summary:"Regexes here only serve with the workflow, they leave no permanent mark." -always

Does that work?

Creating separate fixup lists should work as well.

—Trappist the monk (talk) 16:01, 1 October 2022 (UTC)

Just tried it. I reversed the changes in the Andrew Yang's article to see what would happen but Smallem just skipped it again. I'll create the extra 2 fix-lists, fix whatever can be fixed and return to see what's left in that category and how those can be managed. — Klein Muçi (talk) 16:34, 1 October 2022 (UTC)

It worked well. From 77 to 60.
This is the next article in that category: w:sq:Akuakultura
The problem is a Google books template. But unlike the {{!}} template case, this has parameters in it. I don't believe a regex solution can be found in these situations, can it? — Klein Muçi (talk) 16:01, 2 October 2022 (UTC)

At the beginning, this:

r"\{\{(\s*(?:[Gg]oogle [Bb]ooks|[Gg]ooglebooks|[Gg]oogle book|[Gg]books?)[^\}]+)\}\}" r"__0P3N__\1__CL0S3__"

at the ending, this:

r"__0P3N__" r"{{" and r"__CL0S3__" r"}}"

Does that work?

—Trappist the monk (talk) 16:25, 2 October 2022 (UTC)

┌─────────────────────────────────┘
Oh, we can just leave the other parameters as they are. Somehow that never occurred to me. I'll try it and report back. Thank you! — Klein Muçi (talk) 19:16, 2 October 2022 (UTC)

Smallem just finished its run and that category went from 60 to 46 so the regex you gave me is working fine. I was checking what's left and as you've said many times in the past there are a lot of templates that can be found inside the citation templates. (w:sq:Kategoria:Mirëmbajtja CS1: Datë e përkthyer automatikisht - Take a look if you want.) The last regex made me wonder though: Can't we use 1 single general regex to take care of all the "templates inside templates" situations? Only the curly brackets are the real problematic parts, no? What actually makes Smallem skip such replaces. Why not just have a regex that converts those brackets found inside cite templates into some secret codes and then back to brackets when the job is over and be done with it? — Klein Muçi (talk) 23:32, 2 October 2022 (UTC)

You can try this. No guarantees.

r"\{\{((?!\s*(?:[Cc]ite[_\-\s]*(?=ar[Xx]iv|[Aa][Vv] media|[Aa][Vv] media notes|bio[Rr]xiv|book|[Cc]ite[Ss]eer[Xx]|conference|encyclopa?edia|interview|[Jj]ournal|[Mm]agazine|mailing ?list|map|newsgroup|(?:[Nn]ews(?!group|paper))|podcast|press release|report|serial|sign|speech|ssrn|techreport|thesis|[Ww]eb)|[Cc]itation(?=\s*\|)))[^\{\}]*)\}\}" r"__0P3N__\1__CL0S3__"

The regex finds and hides templates that are not the canonical cs1|2 templates (or some common redirects). It replaces their {{ and }} with secret words. Nesting is problematic:

{{cite book |title={{template A|stuff=stuff}}}} – works; {{Template A}} is hidden

{{template A |stuff={{template B|stuff}} and {{Template C}} }} – does not work; {{Template B}} and {{Template C}} are hidden; {{Template A}} is not hidden

—Trappist the monk (talk) 13:06, 3 October 2022 (UTC)

My knee-jerk reaction is to say that there won't be multi-nested templates in citations but I've said stuff like that in the past as well and you/time has proved me wrong. I'm guessing I won't need the regexes for {{!}} and {{Google books}} anymore, will I? I'll try a full run tonight with the new one and report back with the results when I have them. — Klein Muçi (talk) 17:05, 3 October 2022 (UTC)

From 46 to 19. I started another run to see if Smallem can further empty the category without doing any change. Tomorrow I'll check what are the final stubborn entries. — Klein Muçi (talk) 22:48, 4 October 2022 (UTC)

15 is lowest I could get. Still a massive improvement.
Taking a look at what's left here, you can see that Giuseppe Rossi and Ithaca, New York get skipped because they have pseudo-templates with a single pair of curly quotes. Do you think it would be a good idea to change our regex to match even these cases?
Going a bit further down, you see the article for Lepa Mladjenovic in there but I have no idea why that actually gets skipped. Currently I haven't checked further than that. — Klein Muçi (talk) 00:26, 7 October 2022 (UTC)

sq:Giuseppe Rossi and sq:Ithaca, New York skipped because their |url= values have http...{...}.... sq:Lepa Mladjenovic skipped because |year=Apr 18, 2017; your regexes don't account for |year=<whole date> – should be |year=<year>.

—Trappist the monk (talk) 00:46, 7 October 2022 (UTC)

Should I change r"\{\{((?!\s*(?:[Cc]ite[_\-\s]*(?=ar[Xx]iv|[Aa][Vv] media|[Aa][Vv] media notes|bio[Rr]xiv|book|[Cc]ite[Ss]eer[Xx]|conference|encyclopa?edia|interview|[Jj]ournal|[Mm]agazine|mailing ?list|map|newsgroup|(?:[Nn]ews(?!group|paper))|podcast|press release|report|serial|sign|speech|ssrn|techreport|thesis|[Ww]eb)|[Cc]itation(?=\s*\|)))[^\{\}]*)\}\}" r"__0P3N__\1__CL0S3__" somehow to account for single pairs of brackets cases? I'm kinda getting overloaded by its beginning and I'm not really sure what exactly to change.
...Your regexes don't account for |year=<whole date>...
Yes but my memory was telling me that I also had a regex that would change these cases to use the date parameter instead of the year one. My memory was lying me apparently because I just checked and couldn't find it. I believe I should create that though, no? — Klein Muçi (talk) 08:35, 7 October 2022 (UTC)

After you replace template opening and closing curly braces with special secret words you might:

r"([^\{])\{([^\{])" r"\1__L_CURLY__\2" and r"([^\}])\}([^\}])" r"\1__R_CURLY__\2"

At the end before restoring template opening and closing curly braces, replace __L_CURLY__ and __R_CURLY__ with the appropriate character.

It's your bot, you decide what it should do...

—Trappist the monk (talk) 14:28, 7 October 2022 (UTC)

I did and now I'm doing a full run to see what's left again. Thank you!
I asked because I wasn't sure of the details in regard to that. I was thinking 1 regex could be enough to convert years to dates but then I remembered there are different combinations/formats of dates so maybe there are a dozen regexes needed for that and I'm not sure if I want to walk that path now. — Klein Muçi (talk) 10:03, 8 October 2022 (UTC)

Aaand it was disastrous. This is one of 1500 edits Smallem did which I now need to revert. I must have done something wrong (beside the fact that we should have been more specific to only target cite templates). — Klein Muçi (talk) 14:55, 8 October 2022 (UTC)

Why did you not do as I said:

At the end before restoring template opening and closing curly braces, replace __L_CURLY__ and __R_CURLY__ with the appropriate character.

Why did you remove the whitespace between }}} and }}?

—Trappist the monk (talk) 15:00, 8 October 2022 (UTC)

┌─────────────────────────────────┘
I did do as you said. I added these:

(r"__L_CURLY__ ", r"{"),
(r"__R_CURLY__ ", r"}"),

Yet they didn't work. :-/ — Klein Muçi (talk) 22:42, 8 October 2022 (UTC)

Why the parentheses? Have you wrapped regexes in parentheses before? And those regexes worked?

—Trappist the monk (talk) 23:33, 8 October 2022 (UTC)

Yes. Every regex of Smallem is like that. For example:

(r"__L_CURLY__ ", r"{"),
(r"__R_CURLY__ ", r"}"),
(r"__0P3N__", r"{{"),
(r"__CL0S3__", r"}}"),

The open/close ones worked fine leaving no trace behind. I don't know why these did. — Klein Muçi (talk) 23:37, 8 October 2022 (UTC)

Maybe I've done something wrong here?

(r"([^\{])\{([^\{])", r"\1__L_CURLY__\2"),
(r"([^\}])\}([^\}])", r"\1__R_CURLY__\2"),

— Klein Muçi (talk) 23:47, 8 October 2022 (UTC)

I wish you would stop using that factoten thing. It has become incredibly difficult to write a new message and read the clutter of crap that tool produces. I have to flip flop back and forth between what I'm writing and the rendered version of what you wrote. I should not have to do that. Reminds me of the old C obfuscation contests where coders attempted to write the most inscrutable, indecipherable, but working code...

If the __L_CURLY__ and __R_CURLY__ regexes are accurate representations of the smallem code, you have an extraneous space character before the closing quote that should not be there.

—Trappist the monk (talk) 00:31, 9 October 2022 (UTC)

Hahaha, I'm sorry. I actually really love that tool as it basically gives me everything I've wanted for years from discussions on Wikipedia and even more but I do agree in regard to the source code it produces. I've talked with Alexis before about it and if I'm not wrong some adjustments were made but... I'll try to use DT with you. :)

The extra space was causing everything. Good catch!

Now there are only 10 entries left here. None of them has any kind of brackets. Can you tell what might be going wrong? (Apart from those entries that have short months in them, the first 2 if I'm not wrong, because that's a problem I'm planning to solve in the end.) - Klein Muçi (talk) 13:23, 9 October 2022 (UTC)

Thanks.

sq:Chanel Terrero – more than one date parameter with same short-form month name; this is the case-insensitive flaw

sq:Festivali Evropian i Këngës 2021 – {{albumchart}} is not a cs1|2 template but is a wrapper template

sq:Leopardi – {{IUCN}} should be replaced with {{cite iucn}}:

{{cite iucn |author=Stein, A.B. |author2=Athreya, V. |author3=Gerngross, P. |author4=Balme, G. |author5=Henschel, P. |author6=Karanth, U. |author7=Miquelle, D. |author8=Rostro, S. |author9=Kamler, J.F. |author10=Laguardia, A. |year=2016 |title=''Panthera pardus'' |volume=2016 |page=e.T15954A50659089 |doi= |access-date=9 October 2022}}

sq:Lepa Mladjenovic – |year=

sq:Masakra e Kleçkës – |date=25 November 201; year is only three digits. Should it be?

sq:Melburni – {{Census 2006 AUS}} is not a cs1|2 template but is a wrapper template

sq:Misto Treska – sq:Stampa:cito lajm and sq:Stampa:cito uebin are redirects. You can add them to the IS_CS1 regex

sq:MPLS – |year=

sq:Mustafa III – |year=

sq:Qeliza – |year=

—Trappist the monk (talk) 14:41, 9 October 2022 (UTC)

I found out that my memory was actually correct. We did create year-to-date conversion regexes before. Somehow they're not working though. Trying to fix this before I can go on with the last edge cases. Pastebin link. - Klein Muçi (talk) 11:25, 10 October 2022 (UTC)

Oh... It's because of the missing comma I suppose. - Klein Muçi (talk) 11:34, 10 October 2022 (UTC)

That would seem to be true for all of the regexes that handle mdy dates. A comma is required between \d{1,2} +\d{4} → \d{1,2}, +\d{4}.

—Trappist the monk (talk) 13:53, 10 October 2022 (UTC)

Yes. Now they work correctly. Before dealing with the case-insensitive problem, what exactly do you mean with IS_CS1? Klein Muçi (talk) 00:00, 11 October 2022 (UTC)

Shorthand for the regex that looks for all hides all of the non-cs1|2 templates. I have used similar regexes in awb c# code where I set the regex as a constant named IS_CS1 so that I can use it in multiple places but only define it once.

—Trappist the monk (talk) 00:25, 11 October 2022 (UTC)

Hmm... I'm a bit confused by your explanation. What am I supposed to exactly do with those 2 templates? If I'm not wrong, the way we've set up regexes for Smallem are by utilizing cit+vowels and "cito" should also be included by that virtue. Am I misunderstanding something? - Klein Muçi (talk) 10:33, 12 October 2022 (UTC)

The IS_CS1 doesn't support that. I suppose that it could but I wonder if you wouldn't be better served by having smallem convert {{cito ...}} redirects to {{cite ...}} canonical names.

—Trappist the monk (talk) 13:06, 12 October 2022 (UTC)

But... Doesn't this regex for example support that:

(r"(\{\{\s*cit[aeio][^\}]*\|\s*)year(\s*=\s*\d{1,2} +[A-Za-z]+ +\d{4} \– \d{1,2} +[A-Za-z]+ +\d{4})", r"\1date\2"),

The [aeio] part? - Klein Muçi (talk) 13:14, 12 October 2022 (UTC)

Of course it does. But that regex does not care about the {{cit[aeio] ...}} template names. The IS_CS1 regex does care about template names; that is why it includes all of the names. This search suggests that there aren't many {{cito ...}} templates and no {{citi ...}} templates. Here are usage comparisons for those {{cito ...}} templates:

• sq:Stampa:Cito lajm redirects to sq:Stampa:cite news
  • cito lajm ~180 articles
  • cite news ~3590 articles
• sq:Stampa:Cito revistë shk redirects to sq:Stampa:cite journal
  • cito revistë shk ~0 articles
  • cite journal ~2470 articles
• sq:Stampa:Cito revistë shk. → Revista Iliria – with a dot; probably should be subst'd and then deleted
• sq:Stampa:Cito uebin redirects to sq:Stampa:cite web
  • cito uebin ~715 articles
  • cite web ~14120 articles
• sq:Stampa:Cito web redirects to Stampa:cite web
  • cito web ~2330 articles

At en.wiki we have an awb general fixes page that lists instructs awb how to rename redirects to their canonical name. We also have a lot of awb users many of whom run awb with general fixes turned on. Seems that smallem can easily rename these redirects before doing any of its other cs1|2 maintenance tasks.

—Trappist the monk (talk) 14:22, 12 October 2022 (UTC)

I see... The mini-problem is that "cito" is Albanian for "cite" and one may ask why not convert all "cite" into "cito" instead of the vice-versa. Or even better into "citim", Albanian for "citation", the noun instead of the verb, which probably would make more sense syntaxically in Albanian. We have 90% of our technical infrastructure in English because we're dependent on EnWiki for almost everything (your searches showcase that) so a point can be made for each side. Considering all that, I'll leave this point for another time.

This sends us to the last problem in that category: The flaw.

So far I've been using the pywikibot -nocase parameter for the main script of Smallem that invokes all other fix-scripts. This has greatly reduced code size, simplified regexes visually and has never proved problematic... up until this point. Given that the said parameter is located in the main script, removing it would remove it for all regexes which would in turn require me to adjust accordingly every fix-script and you to help me change the Smallem module so that it also starts producing language-regexes in the [Xx] form.

Considering all that, I wonder if a better solution can be found without removing the said parameter. Maybe something like "Convert Jan to jan" would work without removing that parameter? I don't know how regexes usually respond in these cases. Do you think it would catch and make the desired change because I explicitly asked for it or do you think that it would just ignore it because in its eyes there is nothing to change? Klein Muçi (talk) 14:45, 13 October 2022 (UTC)

Yeah, I know that 'cito' is Albanian. That is why I gave you all of those numbers. Clearly the preponderance of cs1|2 templates are written wholly with English parameter names. When you take it upon your selves to convert parameter names to Albanian then at that same time you should convert the template names as well. Mixing just doesn't seem to me to be a good idea.

Maybe the simple solution to the Jan/jan Mar/mar problem is to write a separate process that only does cs1|2 dates for those two abbreviated months.

Can you invoke pywikibot without -nocase and make each regex case insensitive. For example, does this work without -nocase:

(r"(\{\{\s*cit[aeio][^\}]*\|\s*(?:access\-?|archive\-?|doi\-broken\-|lay\-|pmc\-embargo\-|publication\-|air\-?)?date\s*=\s*\d{1,2} +)January( +\d{4})"i, r"\1janar\2") – note the i flag at the end of the regex; working means that it will translate 2 january 1999 as well as translate 2 January 1999

If it does, all of your existing regexes get tweaked to add the case insensitive flag and those that are Jan/jan Mar/mar regexes are evaluated and perhaps rewritten to be case sensitive.

You might ask the pywikibot people if there is a way to force a handful of the 100s of smallem regexes to be case sensitive; is there a case sensitive flag that allows the bot to override -nocase for individual regexes?

We know that the Jan/jan Mar/mar problem isn't a problem when there is only one date with the same abbreviated month. See this diff where smallem converted |access-date=30 Jan 2022 to |access-date=30 jan 2022. The problem is that the template also has |date=30 Jan 2022. Because smallem is case insensitive, it finds |access-date=30 jan 2022 (the replacement from the previous run) as a match and replaces it again.

I don't know what you mean by removing the said parameter et seq.

—Trappist the monk (talk) 15:54, 13 October 2022 (UTC)

So far, Pywikibot doesn't generally work with such flags. I mean, they don't work at all currently but the whole "flag system" looks alien to the current Pywikibot state. As it can be understood from this page, which is where everything that Smallem does is located basically, Pywikibot works with script parameters which tend to affect the whole script. It doesn't support any kind of such modifier. The said page also shortly talks about an advanced use of fixes and utilizing own functions to actually become a wizard but much to my love of Harry Potter, I don't know parseltongue that well.

As for the part that you didn't understand I meant precisely what you have discovered in the diff-example that you brought. I was thinking to create a case insensitive regex that just converts Jan to jan in dates and given that it was explicitly written to convert Jan to jan, even though it was case insensitive, it would still catch such cases and convert them accordingly. Apparently that behavior is already happening, as demonstrated by that diff. No? Why is Smallem selective about that then? Why doesn't it convert some of the dates? I didn't quite understand the reason you gave for it. Is it because they were more than one and it was supposed to do that in the second run but it gets stuck forever in the first one so it never sees the second one? - Klein Muçi (talk) 13:30, 15 October 2022 (UTC)

You must have gotten some of smallem from somewhere other than Manual:Pywikibot/replace.py because I don't find anything on that page that resembles smallem's regexes which all have the form:

r"<regex>", r"<replace>"

Perhaps that place that documents smallem's usage will also answer the regex flag question.

Not quite, it gets stuck forever in the second one so it never sees the first one. This is because [^\}]* is greedy; it consumes as much of the citation as it can when looking for a match so it skips over the first match and goes to the second match. And that suggests a limited solution. Try this at regex.com:

The regex:

(\{\{\s*cit[aeio][^\}]*\|\s*(?:access\-?|archive\-?|doi\-broken\-|lay\-|pmc\-embargo\-|publication\-|air\-?)?date\s*=\s*\d{1,2} +)[Jj]an( +\d{4})

the substitution:

\1jan\2

the test string:

{{cite book |date=1 Jan 1995 |title=Title |access-date=1 Jan 1995}} – both upper
{{cite book |date=1 jan 1995 |title=Title |access-date=1 Jan 1995}} – first lower; second upper
{{cite book |date=1 Jan 1995 |title=Title |access-date=1 jan 1995}} – first upper; second lower
{{cite book |date=1 jan 1995 |title=Title |access-date=1 jan 1995}} – both lower

With that regex, the result is |access-date=1 jan 1995 for all four example citations.

Now try this regex:

(\{\{\s*cit[aeio][^\}]*?\|\s*(?:access\-?|archive\-?|doi\-broken\-|lay\-|pmc\-embargo\-|publication\-|air\-?)?date\s*=\s*\d{1,2} +)[Jj]an( +\d{4})

With the new regex, the result is |date=1 jan 1995 for all four example citations.

The limitation is that this scheme will only work for cs1|2 templates that have one or two date-holding parameters; any cs1|2 template that has |date=, |access-date=, |archive-date= (a very common case) and where all of those dates use the same abbreviated month names 'Jan' or 'Mar', only the first and last date-holding parameters will be 'fixed'.

If you implement this scheme, the [Jj]an part of the regexes should be replaced with Jan; I did that to simplify the explanation.

—Trappist the monk (talk) 14:23, 15 October 2022 (UTC)

Yes, here: mw:Manual:Pywikibot/fixes.py.

I'm confused with the example you brought. I followed your steps and the end result was this:

{{cite book |date=1 Jan 1995 |title=Title |access-date=1 jan 1995}} – both upper
{{cite book |date=1 jan 1995 |title=Title |access-date=1 jan 1995}} – first lower; second upper
{{cite book |date=1 Jan 1995 |title=Title |access-date=1 jan 1995}} – first upper; second lower
{{cite book |date=1 jan 1995 |title=Title |access-date=1 jan 1995}} – both lower

As you can see you still get capital Js.

However, considering that these are only 2 cases, maybe I can create another fix-list just for them which is case sensitive. That should be easier to run than going through all this trouble. How many regexes are we talking about in total? - Klein Muçi (talk) 23:39, 15 October 2022 (UTC)

The results that you posted suggest that you did not try the second regex... and so did not put A and B together to get a solution the two-date Jan/jan Mar/mar problem...

—Trappist the monk (talk) 00:35, 16 October 2022 (UTC)

Yes. I now did and it works as you say. We can apply this scheme but wouldn't just creating some general case sensitive date regexes for Jan/Mar be better? It would work without any such exceptions, no? - Klein Muçi (talk) 10:25, 16 October 2022 (UTC)

So long as you start smallem with the -nocase option, case sensitive regexes aren't possible. You can create a separate 'case-sensitive-smallem' that isn't started with the -nocase option but you have pushed back against the idea of multiple small smallem tasks for as long as we've been talking about smallem. This is why I asked about regexes with the i flag. That is the only way that I know that might make smallem both case sensitive and case insensitive. Someone who knows about pywiki may know the answer to the 'is the i flag supported?' question. Or do an experiment to find out...

—Trappist the monk (talk) 11:51, 16 October 2022 (UTC)

Thank you for being attentive! Yes, that's true however there's one small detail that isn't: Fixes have their own parameters now. So I can just create a new mini-list without the nocase parameter for this case. It's true that I have been reluctant to do such a thing in the past but I already created a similar list for the pipe-problem so that opened the gate. Who knows how many other such cases I'll have to fix the same way? Maybe that mini-list will grow to not be a mini-list in the near future.

I'd just need some general guidance on what such a list would hold. How would the regexes I'm looking for look in this case? Klein Muçi (talk) 12:12, 16 October 2022 (UTC)

All regexes that fix any en.wiki date with 'Jan' or 'Mar' get moved to your new list and generally made case-insensitive except for those month-name abbreviations. Use regex101.com to test your new regexes against templates that use different capitalization.

—Trappist the monk (talk) 13:55, 16 October 2022 (UTC)

So basically, these regexes and other similar ones get moved completely in a new fix-list which is case sensitive, right? They get moved, not copied because they do nothing with the -nocase parameter in the fix-list that they currently are, right?

If this is indeed the case, the date-fix-list would be brutally hacked/chopped in a somewhat random manner. Now I understand why you insisted on having -i flags, that would have been a way more elegant solution. The move would make the said list very hard to read. I'm starting to think I should instead copy them just for readability reasons. - Klein Muçi (talk) 12:01, 17 October 2022 (UTC)

Why do you have a Feb-precedes-Jan regex? Feb–Jan is an invalid date because date order left-to-right is earliest-to-latest; in the real world, February follows January. Auto date translation is only applied to valid dates.

brutally hacked/chopped? Keeping regexes that won't answer all cases seems sort of pointless, especially if they are replaced with regexes elsewhere the will answer all cases. You don't have to move them; you can simply comment them out if that will answer your esthetics concerns. It's your bot, do as you wish. And, I have not insisted on anything. (the flag is i not -i)

—Trappist the monk (talk) 13:02, 17 October 2022 (UTC)

Right. Good catch. Most likely it comes from the early days when I was just getting used to the idea of date regexes and it would make "the aesthetics" appear even worse and unmaintainable. Now I feel a bit more confident. Turns out that happens on every date format so it will take me a while to remove them all but that will make the removal of certain dates easier as the lists are already "hacked".

I'll be back with one final report after I do the task to conclude the conversation. :) Klein Muçi (talk) 14:25, 17 October 2022 (UTC)

Back sooner than expected: Are such regexes correct or should they also include jan-jan, feb-feb, et seq. cases as well?

This is Mdy-Mdy → dMy-dMy.

I have the same question about the pattern in dM-dMy and Md-Mdy → dM-dMy. - Klein Muçi (talk) 12:11, 18 October 2022 (UTC)

Invalid dates are not auto-translated.

—Trappist the monk (talk) 13:00, 18 October 2022 (UTC)

Can you please take a general look at my season regexes and tell me if they look good so I can go on and finish creating the case sensitive ones now that I finished dealing with everything else? - Klein Muçi (talk) 00:49, 20 October 2022 (UTC)

Season dates are only supported by |air-date=, |date=, and |publication-date= so the regexes can be simplified:

(\{\{\s*cit[aeio][^\}]*\|\s*(?:publication\-|air\-?)?date\s*=\s*)Spring +(\d{4}) +[\-–] +Spring +(\d{4})

This same simplification also applies to quarterly and proper name dates.

Otherwise these all appear to be correct.

—Trappist the monk (talk) 13:44, 20 October 2022 (UTC)

Hi Trappist the monk, just a quick note, thanks for your edits on bowl players, Alister Kennedy, Mandy O'Donnell, Lorna Trigwell etc. but you are leaving a https:// out which is causing a check url red message in the reference sections. Thanks Pipesmoking Legend (talk) 17:16, 26 October 2022 (UTC)

Yeah, I'm aware of that. All three of the articles you mention had archive-snapshot urls in |url=. Archive-snapshot urls do not belong in |url=; archive-snapshot urls go in |archive-url=.

It's a garbage-in-garbage-out thing. The archive-snapshot url that was in |url= and now is in |archive-url= is malformed; it is missing the scheme from the original-url portion of the archive-snapshot url. This from Alister Kennedy:

|url=https://web.archive.org/web/20111031201626/www.worldbowlsltd.co.uk/mainresults.php

That missing scheme could be http:// or https://. Because that url is dead regardless of scheme, neither I nor my simple script can know what the scheme in a new |url= value for a properly constructed template should be. So, instead of making a choice, I have elected to have the script fix the template as best it can and let the error message notify editors who care about the article make a determination about what should be done. Clearly that works, because you have fixed |url= in these three cases...

—Trappist the monk (talk) 18:18, 26 October 2022 (UTC)

For fraternities using hebrew letters, what would be the best way to determine if it was Talmudic Hebrew? For Greek Letter Organizations, they are not marked with Modern Greek as the letter pronounciations are clearly closer to that of 2000 years ago. Naraht (talk) 19:37, 14 October 2022 (UTC)

I'm not clear on what you are asking. Discrete Hebrew characters are not a 'language' per se, but are individual elements of a script, are they not? We don't have an article about Talmudic Hebrew so I don't know if that language is/was written with a script that is different from the script(s) used for other forms of Hebrew.

For clarity, hbo → {{lang|fn=name_from_tag|hbo|link=yes}} → Biblical Hebrew. Is that what you mean by Talmudic Hebrew?

—Trappist the monk (talk) 19:49, 14 October 2022 (UTC)

yes, it is. I'm trying to figure out for the fraternity Alpha Yodh He whether it should be viewed as an abbreviation in Modern Hebrew or Biblical Hebrew. Is there a date that it considered to divide on? Note for the Greek Equivalent, even though the abbreviations may have been created in the last 150 years, the usage in the North American Greek Letter system is that of Ancient Greek, so that is used.Naraht (talk) 14:14, 20 October 2022 (UTC)

yes, it is. Yes, what is?

Did you mean Aleph Yodh He? Are those not discrete semitic letters, in this case the Hebrew form, so not a language but just letters using the Hebrew script? Greek-letter fraternities/sororities also not language but initialisms using the Greek alphabet. The names don't appear to me to be language just strings of letters that, dissociated from fraternities/sororities, are just letters.

Instead of {{lang}}, shouldn't you be using {{script|Hebr}} and {{script|Grek}}?

{{script|Hebr|איה}} → איה‎

{{script|Grek|ΦΒΚ}} → ΦΒΚ

I don't know anything about fraternities/sororities or their peculiar names so I don't know anything about division dates. Surely there is/are societies of letter societies whom you could ask?

—Trappist the monk (talk) 14:46, 20 October 2022 (UTC)

Yes, they are specific semetic letters, letters in the Hebrew Script (and similarly for Greek). The question is whether the accessibility given by template:lang is needed as opposed to template:script. And Alpha Yodh He is pretty unique in that it uses exclusively Hebrew Letters (and is no longer active to ask about the original heb vs. hbo question). The Greek Letters Organizations concept in North America is pretty consistent in using Greek which is *not* modern Greek (such as Beta being pronounced as the first letter in the name as "B" in English, not "V".)Naraht (talk) 14:44, 25 October 2022 (UTC)

Alpha and not Aleph?

If Alpha/Aleph Yodh He was a 20th century organization and is no longer active, without any indication to the contrary, I would say that heb is adequate. Are there other semitic societies from which you might take a hint?

If, in American English, Beta is pronounced with a 'B', is it different in British English? If so then, when {{Use British English}} is present in an article, el-GB, and when {{Use American English}} then el-US?

—Trappist the monk (talk) 15:25, 25 October 2022 (UTC)

- The key difference appears to be between English (UK or US) and modern Greek - which according to our article uses a 'V' sound.Nigel Ish (talk) 18:41, 30 October 2022 (UTC)

Notice that the documentation for the URL parameter of citation templates says url: URL of an online location where the text of the publication named by title can be found. – it is not “misuse” to link the URL parameter to a page hosted by the internet archive as a replacement for a 404 link.

You might prefer using the archive-url parameter for that, but from what I can tell there is no sitewide requirement to that effect. Running around with a script making sure to add an extra direct link to every now-404 page with a working internet archive link is a waste of your time which adds extra distraction for readers for little if any benefit. –jacobolus (t) 21:03, 29 October 2022 (UTC)

The guidance at WP:DEADLINK and Help:Citation Style 1 § Web archives would seem to disagree with you. Besides myself, I'm pretty sure that I have seen User:Citation bot, User:InternetArchiveBot, User:GreenC bot, and User:BrownHairedGirl make similar edits.

If you believe that the guidance at WP:DEADLINK and Help:Citation Style 1 is wrong, the place to take that up is at either of WT:LINKROT or Help talk:Citation Style 1 (with the other notified of the discussion).

—Trappist the monk (talk) 22:05, 29 October 2022 (UTC)

Yes, I do make similar edits. I run various searches to make worklists, and every month I extract the original URL from a few hundred refs.

If @Jacobolus wants to pursue their approach, they should open an RFC, rather than complaining to editors who work within long-established guidance.

However, I agree with Jacobolus that parameter misuse is an unhelpfully aggressive way of describing such edits. Much better to simply say "extract original URL": more informative, less bitey. BrownHairedGirl (talk) • (contribs) 12:43, 30 October 2022 (UTC)

The pages you have linked are about repairing dead links (a very useful service, please do that!) not roaming around adding extra dead links (a waste of time). None of those pages says that internet archive links can't go in the "url" parameter of citations templates. –jacobolus (t) 18:03, 30 October 2022 (UTC)

Surely a viable reason for having the archived version in the regular url field is if that is where the editor actually found the information in the first place?Nigel Ish (talk) 18:35, 30 October 2022 (UTC)

Often the internet archive link is where the editor actually found the information. For instance, Google Scholar links directly to internet archive links. –jacobolus (t) 19:13, 30 October 2022 (UTC)

We've got a system and way of doing things, please keep it consistent it will help everyone involved. The idea that we ditch |archive-url= and |archive-date= and only use |url= for everything makes no sense for a couple reasons. Worse would be to only do it sometimes creating needless complexity. -- GreenC 01:04, 30 October 2022 (UTC)

The only “everyone” I care about here are wikipedia readers. Adding a disclaimer with a link that points to a content-free 404 page does not help readers; it just adds clutter. Especially so for the example I reverted, which already also included a DOI pointing at a canonical page for the paper in question. –jacobolus (t) 18:06, 30 October 2022 (UTC)

Interesting that you mentioned |doi=... For the others who can't see my notification list, the reverted edit is this one – wholly script driven, nothing manual about it. Had it been a manual edit, likely I would have simply deleted |url= because the archive snapshot is of a copy of the article originally retrieved from JSTOR but served from a location that is not JSTOR nor the web presence of The American Mathematical Monthly. I'm no lawyer, but JSTOR apparently requires those who retrieve articles from the JSTOR repository to limit distribution of those articles. The archive snapshot fetched the article from the University of California Santa Cruz website for Professor Richard Montgomery. Montgomery may have used the JSTOR-sourced article for classwork which is allowed by the JSTOR license. The JSTOR license has this:

unless otherwise expressly permitted for Early Journal Content, Open JSTOR Collection Content, or Community Collection Content, Institutional Licensees and users may not: ... make available JSTOR Licensed Content or Artstor Collection Content in a manner that would substitute for direct access to such content on the Service platforms

Here is where it gets a bit gray for me. If Montgomery's use for classwork is legitimate, then students in those classes may have free access to the article (no copyright violation). Since en.wiki is not a student, en.wiki is not entitled to have free access to that article via the UCSC website (or the Internet archive) because the distribution rights do not belong to UCSC or Internet Archive. WP:ELNEVER suggests that we should not link to sources in violation of a copyright. As I said, I'm not a lawyer, but were I manually editing this citation, I would link to the publisher and JSTOR copies of the article through |doi=10.1080/00029890.1993.11990430 and/or |jstor=2324297 for the avoidance of copyright violation.

—Trappist the monk (talk) 22:31, 30 October 2022 (UTC)

When something in JSTOR (or similar) has a DOI, especially if it redirects to the same place, just put the DOI and skip the various other article index identifiers; they are redundant and an unhelpful distraction for readers. As for copyright: my understanding is that making hyperlinks is not a copyright violation. If the publisher cares they can take it up with the internet archive. –jacobolus (t) 19:46, 7 November 2022 (UTC)

idea that we ditch |archive-url= and |archive-date= and only use |url= for everything – I never suggested anyone should do this. I am only saying that using a script to hunt up every example where url=archive.org/... is used anywhere on Wikipedia and changing those to use archive-url, etc. instead is a waste of effort. –jacobolus (t) 19:15, 30 October 2022 (UTC)

a waste of effort is my choice, right? After all, it's my effort that I'm 'wasting'.

—Trappist the monk (talk) 22:31, 30 October 2022 (UTC)

It is also a waste of everyone else’s attention. –jacobolus (t) 19:44, 7 November 2022 (UTC)

I do the same sort of edits. Similarly, it's my effort, and now that I have set up my systems to do this, it's very little effort. BrownHairedGirl (talk) • (contribs) 23:19, 30 October 2022 (UTC)

Hello, I would like to know the utility of removing <nowiki> and adding {{=}} on the tables of the List of bridges in Greece. I work on many other lists so I prefer to use the correct form for the next time, thank you.--Glabb (talk) 10:12, 31 October 2022 (UTC)

{{row numbers}} is a hack until MediaWiki decide to do the right thing and make table-row-numbering a native option for wikitables. As originally written, the <nowiki>...</nowiki> tags were required because each pipe (|) in a wikitable looked like another parameter in the {{row numbers}} template. The <nowiki>...</nowiki> tags essentially collapse the entire wikitable into a single parameter value.

What I did not know at the time that I wrote the initial Module:Row numbers was that tables inside <nowiki>...</nowiki> tags did not play nicely with visual editor and various mobile apps. That failing was noted at phab:T203293. Another editor tweaked the module so that numbered wikitables without the <nowiki>...</nowiki> tags work in visual editor and mobile apps. That fix intentionally treats every pipe as the beginning of a new positional parameter. Positional parameters are handled in order (left-to-right, top-to-bottom) so the module can recreate the wikitable with new numbering in place of the special keywords. This requires that = assignment operator must be escaped with {{=}} which converts text that can be interpreted as a named parameter into text that can be interpreted as a positional parameter. Example. If we flatten this (taken from List of bridges in Greece):

{{row indexer|
{| class="wikitable sortable"
|-
! class="unsortable"|
! scope=col |
! scope=col |Name

we get:

{{row indexer|{|class="wikitable sortable"|-! class="unsortable"|! scope=col |! scope=col |Name

That has two positional 'parameters'; one holds { and the other holds Name

The rest of the 'parameters' are named:

|class="wikitable sortable"

|-! class="unsortable"

|! scope=col

|! scope=col

Those named 'parameters' are not known to {{row numbers}} so will be ignored. By escaping the = assignment operator, all of those named 'parameters' become positional 'parameters'.

caveat lector: Assignment operators inside templates inside a wikitable must not be escaped. The assignment operator escape applies only to content between pipes that might make that content look like a named {{row numbers}} parameter. For most, if not all, wikitables, it is only the styling that needs to be escaped. No doubt there are special cases so those must be handled as the need arises.

—Trappist the monk (talk) 14:29, 31 October 2022 (UTC)

Thank you for this complete answer! I will change for the articles I work on.--Glabb (talk) 09:45, 2 November 2022 (UTC)