Talk:Metascience

Source 2^[1] is a webpage containing a podcast. I meant to cite the podcast itself, but I am not sure how to do so. Help would be much appreciated.

The image that come up alongside the metascience page in google's search results is related to the metascience journal.^[1] This is not very good, because the metascience journal covers "the history and philosophy of science and science and technology studies". I see little indication that it is actually associated with the field of metascience. Does anyone know how to change this image? --Wikiman2718 (talk) 19:52, 30 May 2019 (UTC)

@Wikiman2718: I guess it depends on Google's algorithms, but I've added a lead image to this article since that's probably the first thing to try. Sunrise (talk) 01:48, 31 May 2019 (UTC)

@Sunrise: Thanks. Now we just have to wait and see if Google gets around to changing it. Nice edition to the article either way, though. --Wikiman2718 (talk) 01:55, 31 May 2019 (UTC)

Got the issue fixed. The problem was with google's knowledge graph. --Wikiman2718 (talk) 13:10, 18 July 2019 (UTC)

Many sources about topics in metascience do not self-identify as metascientific in nature. In order to cite these sources in the article, we must have another source that identifies them as metascientific. This paper^[1] provides a discussion about which topics fall into the domain of metascience, and can be used as a reference to determine whether or not a given source or subject falls into the domain of metascience.

The need for a merger seems pretty clear to me. --Wikiman2718 (talk) 10:27, 6 June 2019 (UTC)

• @Velella: per WP:PM, obvious mergers can be performed without discussion. Evidence-based research is the same thing as metascience, making the need for a merger obvious. I used a discussion tag anyway just to be polite in case the original editors of the article were still interested in being actively involved. It appears that they are not. I have reversed the revert because it also deleted edits that were not part of the merger. --Wikiman2718 (talk) 09:09, 11 June 2019 (UTC)

This reference may be of interest; it's a secondary source. --Ancheta Wis   (talk | contribs) 06:47, 23 June 2019 (UTC)

I think this may be useful. I read something about how failures to replicate a study finding the ESP works are unpublishable. We may add something about important results that cannot be published for some reason or other, and how this adds bias to literature. --Wikiman2718 (talk) 09:23, 23 June 2019 (UTC)

I forgot to state in my edit summary that the text in the section "Journalology" was pasted from the article on Journalology. --Wikiman2718 (talk) 07:00, 13 July 2019 (UTC)

This is a vital article, but there is no information regarding any success of metascience. We have had almost 20 years since Ioannidis started the ball rolling. Are things getting better? What are the trends? Are there any 'popular science' books or articles on this?

The reason I ask is because the anti-science attitudes and memes that grew during the Covid pandemic are citing the replication crisis as justification. It would seem that the only answer is public acknowledgement that metascience is really reducing the replication crisis.

Thank you. Tfdavisatsnetnet (talk) 11:59, 31 August 2023 (UTC)

I would argue that this article should have a new section on "successes". @Wikiman2718, Ira Leviton, LilyKitty, Carchasm, Nbreznau, Aidybarnett, and Prototyperspective: Please see Talk:Metascience#Is metascience successful? Tfdavisatsnetnet (talk) 01:12, 1 September 2023 (UTC)

I don't think this should be a vital article, and I'm not even sure it should be an article at all. The hatnote says "Not to be confused with Science studies, or with the obsolete synonym 'Meta-science' for the Philosophy of science." but yet Metascience (journal) is a journal in philosophy of science? so this seems wrong. I think perhaps this distinction is WP:MADEUP and the content here should likely be merged somewhere else.

As for any useful follow up on Ioannidis, despite how often I see that paper cited, I think there just hasn't been much? Or I'm not aware of any, at least. - car chasm (talk) 06:48, 1 September 2023 (UTC)

I don't understand your first point. The term metascience has undergone a shift in meaning. This shift is not made up. For example, Karl Popper's definition of meta-science as a philosophy, quoted in Scientific American in the 1990s (a reprint is here^[1]), does not match the definition in the lede of this article. The journal Metascience also predates this shift. If what Ioannidis proposed is today's metascience, then Popper's definition is obsolete. QED.

Your second point is exactly what I am after. How much has been accomplished? Is anyone aware of any progress?

Thanks. Tfdavisatsnetnet (talk) 14:28, 1 September 2023 (UTC) Tfdavisatsnetnet (talk) 14:28, 1 September 2023 (UTC)

Well, I don't think there has been any progress is the thing. Popper's definition is still the WP:COMMONNAME, and the philosophy journal that bears the name Metascience is not defunct and still publishes papers. Meanwhile, Ioannidis' proposal was, as far as I can tell in the intervening time, not adopted. And so we shouldn't have an article on a field of study that never existed, that one researcher 18 years ago thought might be a good idea, and that no one really followed up on in a rigorous enough manner that we can write a wikipedia article on it.

And while I have no doubt that there are other papers that would qualify as "metascience" by Ioannidis' definition, we would need a systematic synthesis of those from a secondary source that refers to what they do as "metascience" in order write an article about it.

Karl Popper's definition of meta-science as a philosophy, quoted in Scientific American in the 1990s (a reprint is here), does not match the definition in the lede of this article.

Yes, this is precisely the problem with the article as it stands, you can't cite the article itself as a proof of your claim that the meaning has changed! There are, as far as I can see, no secondary sources that support this distinction, the citations in the lead all go back to Ioannidis himself. - car chasm (talk) 17:34, 1 September 2023 (UTC)

I don't understand. Are you saying that most of the citations in this article are not secondary? Regarding "the citations in the lead all go back to Ioannidis himself", so what? What about the others throughout the article?

What is the problem with Metascience.com, The Center for Open Science, The Metascience Working Group, scienceplusplus.org, metascience.xyz, The Metascience Research Lab, the Institute for Progress? They all use the definition from this article. So does the journal Nature, which is surely a secondary source if the others are primary. Given this list, Popper's definition is in 2023 not WP:COMMONNAME; also this list of metascience organizations demonstrates that Ioannidis' proposal has been adopted, and the apparent absence of progress does not prove otherwise - to assert that is not NPOV. But believe me, I want to see evidence of progress too. It's my entire motivation for this.

At this point I'm going to delve into what these organizations have published. Tfdavisatsnetnet (talk) 21:58, 1 September 2023 (UTC)

Works for me. If you already believe that such sources do exist, and that they can be used to expand the article, I'm not sure why you pinged me? But while I'm skeptical that you'll be able to find much of value, if you do find useful information to expand this article I certainly don't object. - car chasm (talk) 07:03, 2 September 2023 (UTC)

I think it is entirely possible I will not find much of value: the language is very technical - perhaps too much so for a non-specialist - and the field is still young. Still, the practitioners need to be prodded to show results. Tfdavisatsnetnet (talk) 05:42, 4 September 2023 (UTC)

To return to the original question, I think what you mean by "success" is: has work on metascience improved the quality of scientific research? Well, there are some results in specific sections. For example, the psychology section reports that result-blind peer review has reduced publication bias. Is that success? If yes, how useful would it be to mention that in a separate "success" section? --Macrakis (talk) 21:21, 2 September 2023 (UTC)

I think one of the problems is that metrics of success have to come over time: x number / percent of good papers two years ago, y number / percent last year, etc. In effect metascience must show trends. Tfdavisatsnetnet (talk) 05:45, 4 September 2023 (UTC)

There is no Criticisms section in this article. In my investigation of sources I think I found a very good critical quote.^[1] Is this an RS? May I add it? Tfdavisatsnetnet (talk) 05:47, 4 September 2023 (UTC)

I'm all for including criticism or at least/especially constructive criticism. However, not every article is fit for a section called "Criticism", I don't think there's a Criticism of other fields like 'Physics' or "Biology' (these are much larger and broader and maybe there should be?). But that's more or less just about how such a section would be named or where to put such info.

I think the source you named should definitely be included somehow – at least get briefly mentioned. In fact, I already have doing so on my schedule: I included that paper here: 2023 in science#April at 24 April and I integrate all of the studies I add there with nearly no exception to their respective article/s. I'd be more than happy if people helped out so I don't have to do all that or only improve upon what somebody else edited in. I just was busy doing some other things so I postponed these integrations for a few months.

I think that study is describable as 'Meta-metascience' (or Meta-metaresearch' or similar) – research of metascience. It doesn't necessarily have to all be criticism so a section like this would be more appropriate I think and more easily extendable via other sources. That is, if it doesn't fit well into an existing section. Actually, some info on successes may fit in there quite nicely as well.

However, I don't think it's so simple and maybe not yet possible to measure 'successes' or achievements of metascience so far. It's also only one aspect to view it – metaresearch isn't all about successes in improving science...for example you could also call it a success to develop new tools and methods based on research what is missing/needed and info on such is already briefly included. I don't think there are many other kinds of successes so far and even if there are, it would be very difficult to meaningfully and adequately measure it. For example, see File:Results of trials funded by NHLBI before and after registration was required.jpg isn't a success of metascience, rather such changes are subject of metascience (whether they are effective, how to go about such, etc etc). It would be very interesting if you have any concrete examples of successes to possibly incorporate that aren't already included; until then I guess it's not so meaningful to discuss it.

So yes, it would be nice if you could add some info on/from that source. Most useful would be having an infographic for 'Table 1' (example) of that paper but sadly no such graphic was included in the paper. Good to see somebody thinking about how to improve the article. Prototyperspective (talk) 22:30, 4 September 2023 (UTC)

Currently it is written:

"In 1966, an early meta-research paper examined the statistical methods of 295 papers published in ten high-profile medical journals. It found that "in almost 73% of the reports read ... conclusions were drawn when the justification for these conclusions was invalid."

Following Table 2 in the study by Schor and Karten, the 73 % is related to the number 149, not 295. Hence this number should be corrected in the Wikipedia article. Irazall (talk) 18:31, 28 February 2024 (UTC)