How do Journal Editors Make Decisions About Adopting Open-Science Related Policies or Reforms in their Journal?

Manuscript title: How do Journal Editors Make Decisions About Adopting Open-Science Related Policies or Reforms in their Journal?
Invitation to Review date: 20.02.2026
Review submission date: 23.02.2026
Review type: original submission / revision / other

I have reviewed the manuscript “How do Journal Editors Make Decisions About Adopting Open-Science Related Policies or Reforms in their Journal?” In this stage 1 registered report, the authors describe a study rationale and plan to explore reasons why editors have or have not changed journal policies in line with research ideals / open science practices.

Evaluation: I think that the authors are tackling a very important topic. Academic journals as gatekeepers of publications lie at the heart of our research culture and can be powerful drivers of change. I have a few suggestions for clarification and justification and think that the study would make a valuable contribution to understanding adoption of journal policies.

Recommendation: Revisions required

Remarks in no specific order

1. Clarification of bottom-up versus top-down: I suggest adding references (provided that there is actually support) for some of the core theses such as the first sentence or “much of the change has been bottom-up”. I would also like a clarification of whether bottom versus top refers to career stage, power, or role in the academic system’s hierarchy. I think it becomes clear on page 4 but should be clear from the beginning.
2. To anticipate some of the reasons for not adopting open science-related policies, you may also include literature on how researchers choose target journals and how research is evaluated (e.g., bibliometrics, journal rankings, time from submission to publication, acceptance rates), since these may be related to journal policies.
3. I suggest that you include different models of journals regarding reproducibility checks. For example, Meta-Psychology (a diamond OA journal) conducts in-house checks whereas Nature Human Behavior (and all other Springer-Nature journals for that matter) clearly has the resources to do reproducibility checks on all accepted submissions (https://arxiv.org/abs/2511.04820). Yet, they choose not to conduct them but instead outsource them to volunteers, namely the Institute for Replications, who organize Replication Games and do reproducibility checks for the journal.
4. I think you should also mention Diamond Open Access Standards as standards that editors may pursue. They cover much more ground that TOP factors and include a higher diversity of open access practices (e.g., lay summaries) and not only those that focus on methods: https://diamasproject.eu/diamond-open-access-standard-guide/ I would imagine that if you focus on methodological features only, the barriers that you will learn about are less numerous than if you include non-methods open science facets. For better or worse, I could not fully understand why you chose the specific subject of open science-related journal policies that you did. Can you please elaborate on that?
5. I suggest you spell out the COM-B acronym when you mention it the first time.
6. Recruitment strategy: I suggest also recruiting via TOP factor and possibly citation metrics or APCs. For example, you are likely to encounter editors of journal with a high TOP-factor in JEDI. You could also record the respective journals’ ISSNs and thereby easily determine TOP factor and APCs. Note thought hat TOP factor values are no longer updated since 02/2025.
7. Survey
   1. Please share the survey in a format where there are not multiple empty pages between single lines of text.
   2. I recommend asking for journal ISSNs.
8. I very much like the idea of a registered report for a qualitative study. However, many features that I would have evaluated (e.g., analysis code) are not present in this case. Disclaimer: I have very little experience with qualitative research and suggest that the handling editor (the PCI recommender) invites another reviewer with expertise in qualitative research.
9. This could also be helpful to gain insights into barriers: In the “Road to R2” summaries we documented discussions that were conducted about 6 months before the journal launch. Anybody could join and we invited experts from other journals for some of the meetings: https://osf.io/kupqt/files/kpd2x
10. RR Evaluation Criteria
    1. The scientific validity of the research question(s): I think that you are approaching a very important topic that lies at the heart of research culture change.
    2. The logic, rationale, and plausibility of the proposed hypotheses, as applicable: The research is mostly exploratory with fitting semi-structured interviews.
    3. The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis or alternative sampling plans where applicable): I cannot assess this criterion due to my limited expertise in qualitative research.
    4. Whether the clarity and degree of methodological detail is sufficient to closely replicate the proposed study procedures and analysis pipeline and to prevent undisclosed flexibility in the procedures and analyses: The survey methods are available and clear. The planned analyses are reported.
    5. Whether the authors have considered sufficient outcome-neutral conditions (e.g.,absence of floor or ceiling effects; positive controls; other quality checks) for ensuring that the obtained results are able to test the stated hypotheses or answer the stated research question(s): Does not apply due to semi-structured interviews and no specific hypotheses.

Disclaimers

1. I know JB, PS, AO, SV, and SK through my work at Replication Research and FORRT. I tried not to let this bias my judgment towards their article.
2. I publish all of my reviews in my OSF project (https://osf.io/pbnw7/files/osfstorage) under a CC-BY license.

Recommendations for PCI-RR

1. I recommend also getting a review from a researcher who is experienced in qualitative research.