Limits of blind assessments

Limits of blind assessments

The proliferation of fraudulent cases in science raises questions about the limits of the application of some generally accepted criteria for evaluating scientific articles. In particular, the basic assumption guiding peer review procedures is that the arguments and conclusions of an article should be analyzed solely from the perspective of its epistemic value, internal consistency, and theoretical and empirical validity. As a result of this focus on the content of knowledge alone, one should not take into account the personal characteristics of the authors of the publication under review—a criterion that the American sociologist Robert Merton called “universality.”

Despite this principle, it is well known that an article written by someone who is already famous will be evaluated less harshly than an article written by someone who is not well known in the field. Merton also highlighted this phenomenon of accumulating advantages, calling it the “Matthew Effect.” Article published in the magazine sciences In 1968However, we see this as a departure from the principle of universality and it is often condemned as bias, because people's status should not influence the assessment of the quality or validity of their publications.

To eliminate it, it is generally suggested that the authors' names should not be revealed so as not to influence the evaluators. Since the latter are already anonymous, this procedure is called “double blinding”. But being a sociologist and not a moralist, Merton, as a good functionalist, told himself that if the Matthew effect was so generalized, it was because it must have a function within society even if it seemed dysfunctional and unjust at the individual level.

See also  Nova Caledônia: O IRD lançou um projeto "Ciência na Grama" que convida o público em geral a fotografar as ervas ao redor de Noumea

For example, since the time allowed for critical analysis of an article is limited, reviewers will inevitably look—consciously or unconsciously—for contextual clues to help them decide whether an article is credible. From this point of view, it is by no means certain that complete anonymity is reasonable in all disciplines. It is indeed useful to know who proposes a surprising finding in order to judge its plausibility: does this person really have access to the appropriate tools at their institution? Has she or he previously published results in the same discipline or on the same problem? As we can see, far from always creating a harmful bias, this contextual knowledge, on the contrary, contributes to confidence in the results presented. For it is clear that reviewers cannot visit the laboratory or repeat the experiments themselves.

It is therefore legitimate to ask whether we should really treat a new article submitted by someone who has already been convicted of fraud without taking this knowledge into account. As noted, A physicist interviewed in 2023 by a journalist from nature“People who falsify data tend to do so more than once,” so it makes sense that evaluators would be more skeptical of such people’s work.

Individuals who falsify data tend to do so more than once.

Perhaps for these reasons, the most prestigious journals in the natural sciences do not use the double-blind procedure. On the other hand, it is more common in the social sciences and humanities, which are less technologically and instrumentally intensive, have less stable models, and are more diverse. Clearly, objective evaluation, free from any social influence, is commendable and a legitimate organizational model. However, since the sciences are social institutions, we cannot ignore the fact that social facts about credibility and trust also influence judgments—and not only in a negative way.

See also  Em Trieste Coffee Science

You May Also Like

About the Author: Irene Alves

"Bacon ninja. Guru do álcool. Explorador orgulhoso. Ávido entusiasta da cultura pop."

Leave a Reply

Your email address will not be published. Required fields are marked *