User:Beastie Bot

Wikipedia editing bot run by Pengo From Wikipedia, the free encyclopedia

This is a bot created, operated and maintainer by Pengo.

Beastie Bot

Purpose: Create lists of species by IUCN category ("conservation status"), by taxon (e.g. mammals, mollusk), and/or by country.
IdeaLab: Beastie Bot on IdeaLab —Please join or give your support
Target articles

More information EX, CR stats ...

	EX	CR stats	EN stats	VU stats	NT stats	LC stats	DD stats
amphibians stats	List of recently extinct amphibians	List of critically endangered amphibians	List of endangered amphibians	List of vulnerable amphibians	List of near threatened amphibians	List of least concern amphibians	List of data deficient amphibians
arthropods stats	List of recently extinct arthropods	List of critically endangered arthropods	List of endangered arthropods	List of vulnerable arthropods	List of near threatened arthropods	List of least concern birds	List of data deficient arthropods
birds stats	(List of recently extinct birds)	List of critically endangered birds	List of endangered birds	List of vulnerable birds	List of near threatened birds	List of least concern birds	List of data deficient birds
fishes stats	List of recently extinct fishes	List of critically endangered fishes	List of endangered fishes	List of vulnerable fishes	List of near threatened fishes	List of least concern fishes	List of data deficient fishes
insects stats	List of recently extinct insects	List of critically endangered insects	List of endangered insects	List of vulnerable insects	List of near threatened insects	List of least concern insects	List of data deficient insects
invertebrates stats	List of recently extinct invertebrates	List of critically endangered invertebrates	List of endangered invertebrates	List of vulnerable invertebrates	List of near threatened invertebrates	List of least concern invertebrates	List of data deficient invertebrates
mammals stats	(List of recently extinct mammals)	List of critically endangered mammals	List of endangered mammals	List of vulnerable mammals	List of near threatened mammals	List of least concern mammals	List of data deficient mammals
molluscs stats	List of recently extinct molluscs	List of critically endangered molluscs	List of endangered molluscs	List of vulnerable molluscs	List of near threatened molluscs	List of least concern molluscs	List of data deficient molluscs
reptiles stats	List of recently extinct reptiles	List of critically endangered reptiles	List of endangered reptiles	List of vulnerable reptiles	List of near threatened reptiles	List of least concern reptiles	List of data deficient reptiles
plants stats	List of recently extinct plants	List of critically endangered plants	List of endangered plants	List of vulnerable plants	List of near threatened plants	List of least concern plants	List of data deficient plants
chromista	—	List of Chromista by conservation status
fungi	—	List of fungi by conservation status

Additionally:

List of animals that are extinct in the wild (EW)
List of plants that are extinct in the wild (EW)
IUCN Red List conservation dependent species (LR/cd)
Conservation-dependent species#Conservation-dependent_animals (LR/cd animals list)

Bot category: Automatic importer
Language: Beastie Bot is written in C#
Bot flag: Yes, now running with a bot flag.
Background
Runtime: The bot will run as needed. It runs unaided.
Status: -Inactive (will manually run when red list is updated, or as improvements made)
Logs: Beastie Bot contributions (includes Stage One run)
Examples: List of critically endangered mammals, List of critically endangered fishes
Stats

Reports

Redirects to same title — binomial names which are listed as separate species in the IUCN Red List but redirect to the same Wikipedia article. (Many need fixing on Wikipedia)
IUCN common name issues
Dupes — common names which are used by more than one species

Rules and exceptions

/exceptions — rules & exceptions for naming and taxonomic hierarchy (found in Beastie Bot source code)
/caps — special-case capitalization rules

Features

Main page: Beastie Bot on IdeaLab

Finds common name from species or taxa name.
Finds correct capitalization
Creates a pie chart
Writes English sentences.
Splitting and clumping of sections to reasonable sizes

To do

To-do list for User:Beastie Bot: edit · history · watch · refresh · Updated 2016-09-23

There are no active tasks for this page

High priority

Auto-add {{main}} for sublists with their own articles (insects within arthropod lists, etc)
Add more reptile common names from other sources
Update Lists of extinct animals

Wikipedia app issues

When lists displayed in Wikipedia's official mobile app:

Pie chart does not display correctly
Scientific names are hidden and cannot be discovered for red links (todo: display scientific name in brackets for red links) — In Progress (will be fixed after 2016-1 pages uploaded. Not fixing for birds and mammals)
Redirect notices aren't displayed, hiding the scientific name and—for example—redirects from a subspecies to species. (need to display scientific names when there's not an exact match?) (now needed for birds and mammals only)
Breakout lists: e.g. split out insects from arthropods list but leave stats and a link in behind. Also gastropods from invertebrate, etc

Missing text/features in existing lists

A key to IUCN abbreviations (though chart should do), e.g. List of extinct and endangered species of Italy, List of mammals of Great Britain
sort List of fungi by conservation status and Chromista by IUCN status
Threatened species estimates: Lower estimate, Best estimate, Upper estimate. — done (partially): Upper estimate added
Note changes (especially "genuine")
Plants lists: split up lists?
Extinction lists: Fix intro text. Custom chart or table. Year last seen and country. Check "since 1500" text. Mention that local extinctions aren't included.

Other lists / extended scope

Plants (find splits used by IUCN?)
List problematic taxoboxes, e.g. taxobox with two species: , or articles with multiple taxoboxes: , missing binomial field , same taxobox used on multiple articles (Dingo and Dingo (taxon)), Type species but no binomial .
Per country lists — e.g. "Threatened species endemic to Peru" (only existing in Peru) and/or "Threatened bats of Peru" (with Peru in their range)
Per major taxon lists (listing all species assessed in all categories, e.g. "Snakes by IUCN red list category" or "List of threatened snakes")
Per threat type: e.g. logging
Minor taxon lists or tables: E.g. just ants, or just wrist-winged gliders.
Generate ref tag for a taxon (IUCN reference) from taxon or red list url. (via JSON interface)
Generate pie graphs as SVG and upload, replacing the current charts
- Regenerate graphs such as File:IUCN Red List 2007.svg which is used in various places
- Incorporate other data which estimates the total number of species for various taxa, so it can be contrasted in charts.
Complete LR/cd list for animals generated (LR/cd is an obsolete category): Conservation-dependent species, but need same for plants
"Possibly extinct", e.g. semi-automatically generated this list: List of recently extinct mammals § Possibly extinct. Tweak and better automate.
Report unusual naming: e.g. if species name used in article title for a family that is otherwise named by common name.
Report of any taxa which are missed in by-category-list and by-taxon-list
Stats on trends (e.g. increasing, decreasing)
Create a plants-only sublist of "Redirects to same title" report for plants wikiproject
List of threatened fishes/species by sea ala Category:Lists of fishes by sea
Find taxa which mention "possibly extinct" in their descriptions (on IUCN and/or wikipedia), but are not tagged by IUCN as such. e.g. Elongate bitterling
Better common name handling for "nominal subspecies" (e.g. Cyclura rileyi rileyi).
Full list of just undescribed species (sp. nov.)

Systemics and common names

Gather taxon groupings and common names from templates, e.g. Template:Murinae (Stenocephalomys–Xeromys)
Incorporate additional taxons (heading levels) from Wikidata and/or CoL
Bug: If a taxon containing only one other taxon is reached, doesn't attempt to split it based on grandchild node contents, e.g. see List of near threatened arthropods#Malacostracans
Add subheading "Order: Lagomorpha" style subheadings ala List of mammals of Great Britain, List of recently extinct birds
List of family/higher taxa in "other" sections, or in brackets after the name

Running the bot

Automatic upload of updated lists
Automatic public logs: warnings and errors
Make a log of non-beastie bot changes to its pages so they can be checked if they're clobbered
Option to only update pages which haven't been manually updated (e.g. don't clobber changes for a minor Beastie Bot update)
Versioning: auto increment version number and make beastie bot aware of its version.
Logging in broken

Open source the code

Database support: allow use of MariaDB Wikipedia mirror (in addition to current Xowa-format SQLite database support)
Remove other subprojects from source code
Move to WMF labs
Don't login if don't need to
Make sure everything works on Linux
Fix hard coded file locations, settings, task lists, etc

Rules lists and tweaking without code

Make rules/exceptions list wiki-editable
Better way of creating intermediate taxa (e.g. new world monkeys, microbats, snakes), and making sure some taxa don't end up unclassified.
Automatic grouping into intermediate taxa when grouping can be made unambiguously (i.e. matching several sources)
Web interface
Special rule type for pair names (pigeons and doves, dragonflies and damselflies)
Create report of unneeded/obsolete rules

Refactoring

Switch to the IUCN JSON API instead of CSV (might use Unicode, might get more regular updates as the CSV export takes days)
Intermediate caching of results rather than regenerating everything each run
Cache common name dupes list
Make template to insert graph data into, e.g. "Template:IUCN pie graph" and then "Template:IUCN pie graph of lizards"
Make pie charts into svg files, e.g. "File:IUCN mammals chart 2016-1.svg"
Clean up exceptions list for easier editing/maintenance (group/sort)
Don't re-generate sections so often (cache)

Open questions

force split turtles/tortoises
force split microbats/macrobats

Documentation

What it's doing
How grouping and splitting works / how to fine tune
How common names are found
Editing the rules/exceptions list

Source code

github.com/pengowray/beastiebot