Critical Metascience

Feminist Metascience Madeleine Pownall writes about Carolyn Wood Sherif, a pioneer of feminist psychology. Carolyn Wood Sherif.

Average replicability rates Theories don’t die! Is the replication crisis overblown? Postdoctoral Fellowships Partial replications Berna Devezer and Erkan Buzbas published a preprint disputing the idea that a high average replication rate indicates a healthy literature and a low rate indicates a crisis. They argue that: Aggregating replicability rates across heterogeneous literatures produces averages that conflate incommensurable experimental regimes and lack a stable scientific interpretation.

20:20 hindsight on scientific progress Open research in the humanities The role of scientific sleuths Individual bias and collective criticism Whitney Barlow Robles looks back over the history of scientific progress and notes that a lot of it would look “unscientific” by today’s standards . . . which will look “unscientific” in the future! People sometimes look back at past scientists and ask: “Could they really have been this dumb?

Tyner et al. (2026) recently reported the results of a major study investigating the replicability of claims in the social and behavioural sciences. They found a replication rate of 55.1% of 274 claims. But is this replication rate too low, too high, or just right? They concluded that their findings are consistent with prior work that has found “low observed replication success rates” (Tyner et al., 2026, p. 143). But how do we know that an observed replication rate is “low”?

Workshop on the politics and finances of open science reform Symposium: “Who critiques the critique? Toward a reflexive metascience” Preprint calls for psychologists to establish phenomena before testing theories Systematic review of questionable research practices Sheena Bartscherer, Sven Ulpts, and Joseph Bak-Coleman will be hosting a 2-day workshop in Berlin on 19th and 20th May about the socio-economic architecture of the open science movement.

Welcome to Update #2, which summarizes work on: Barriers and enablers of reproducibility Cataloguing open research practices in the humanities and social sciences Reconsidering reproducibility and questionable research practices in animal-based biomedicine Naming individual studies in metascience work Serge Horbach, Tony Ross-Hellauer and colleagues published the results of a “future studies” analysis of reproducibility.

Papers invited for an open panel on critical metascience at the 2026 meeting of the European Association for the Study of Science and Technology. Many current users of metascience approaches are more interested in questions of speed, with innovation seen as an end in itself, than in questions of direction (see Fortunato et al (2018) for example. We think new approaches to, for example, scientometrics and topic modelling can be used to open up emerging science and innovation to scrutiny.

p-hacking occurs when researchers conduct multiple significance tests and then selectively report tests that yield desired, usually significant, results without correcting for multiple testing. Hence, p-hacking represents a type of undisclosed cherry-picking or fishing for specific (significant) results.

The replication crisis in psychology and other disciplines has made headlines in recent years. Yet crisis discussions are nothing new to psychology. Indeed, if we follow Thomas Kuhn, experiencing crises is not at all unusual in science. There have been repeated declarations of crisis in psychology’s history. But this history is often forgotten, resulting in repeating the same mistakes and raising similar questions over again.

This post is based on a presentation I gave in June 2025 as part of a Metascience 2025 Preconference Virtual Symposium convened by Sven Ulpts and Sheena Bartscherer and including Thomas Hostler, Lai Ma, Lisa Malich, and Carlos Santana. The symposium was called Critical metascience: Does metascience need to change?, and you can find a recording here.