EASE (European Association of Science Editors) propose (juin 2024) aux rédacteurs de revues scientifiques des outils pour améliorer le peer review (évaluation par les pairs).
Comment améliorer la qualité du peer-review ?
Le peer-review est toujours critiqué, mais il ne semble pas que d’autres méthodes soient meilleures. Ces conseils de EASE aux rédacteurs en chef sont articulés autour de quatre principes :
- Envisagez de demander aux auteurs d’évaluer les rapports de révision qu’ils reçoivent.
- Envisagez de demander aux évaluateurs d’évaluer eux-mêmes les rapports d’évaluation qu’ils soumettent.
- Pensez à évaluer les rapports d’évaluation que vous recevez.
- Envisagez de collaborer avec des chercheurs pour évaluer les rapports d’examen antérieurs de votre revue.
Ce document contient de bonnes références sur ce thème du peer-review.
De bons exemples pour comprendre la difficulté d’évaluer les rapports des évaluateurs
Voici un des exemples :
Imagine a scenario in which four reviewers assessed the following manuscript title: Comparing two approaches for asessing the quality of peer review: a randomised controlled trial/
The assessments were as follows: Reviewer 1: Title is excellent — Reviewer 2: Title is not good — Reviewer 3: Title is not good, and needs modify so that the main finding of the study is shown, i.e. which tool is more better. — Reviewer 4: Title itself shows that the authors are idiots and know nothing of methods of research.
How should one rate the quality of these review comments?
If you were asked to score the above comments using a 5-point grading system, e.g., from 1 (poor) to 5 (excellent), which scores would you give?
Reflect on what elements of the review reports contributed to your score.
Now consider that in this particular case all reviewers failed to detect a spelling error in the title, and that only reviewer 4 recognized that the study was not a randomised trial as the method of random participant allocation was not proper randomisation. Would knowing this information impact the rating you would give? Consider how you would re-rate the above four comments with this new information in mind. Also consider if the language quality or style influenced your score.
