Among the things that make science unique is the fact that scientists agree on what they say. There can be disagreement, but it is always understood as a temporary state, because either someone will be proven wrong, or new information will eventually reconcile everyone. Agreement is enforced in many ways, but pre-publication peer review is currently the dominant process, and it has been for over a century.

It is surprising that so little information is available about the efficiency of the peer review process. For instance, there is barely any justification as to why it is by default anonymous. Even more surprising is that people who express their opinion in this regard do not back it up with empirical evidence, because there is essentially no data. Let me clarify something: I do not have any data to show. But I have been signing my reviews for over seven years and I am happy to share this experience with those who wonder what happens when you do this.

How did it start?

I was first contacted by editors to review manuscripts at the time Stack Overflow eclipsed nearly all the forums on the Internet. The forums were supposed to be about sharing – and I truly believe that their creators were genuine humanists – but they invariably turned into a condensate of verbal violence and hate. The participants typically spent more time insulting each other than answering the questions. The insight from Stack Overflow is that distance and anonymity turns us into monsters, even when we have the best intentions. The upside is that very simple incentives, such as a public reputation, are sufficient to make us behave in a civilized manner.

As an author, I had already received my share of pointless violent reviews and I felt that I did not want to inflict this on anyone. Or if I did inadvertently, I wanted to give them the possibility to complain. I reasoned that signing my reviews would force me to write only respectful statements, and in a sense would protect me against the “old forums” effect. Since then, the reasons for signing my reviews have never changed.

It is important to insist that I set out to sign all my reviews, not just the positive ones. I did not see how flexibility on this would do any good to anyone, and I still don’t.

How do the authors respond when you sign your reviews?

I have to admit that I was a little nervous about giving my contact details to the authors in the reviews, because I did not know how they would react. It turns out that they do not react at all. I have never been contacted by an author during the review process, even when the reviews were extremely good or extremely bad.

My interpretation is that authors would never risk to make a faux pas that could make the process look biased from the editor’s point of view. On the other hand, I have sometimes contacted the authors by email during or after the peer review process, most of the time to clarify something I did not understand in the manuscript. I have always found the authors very concise and respectful in their response. I have sometimes contacted the authors after they were rejected, most of the time to give some help on how to fix the issues. So far, the authors have been rather positive about this, they never sent me home with an angry response. Quite the reverse, we sometimes initiated a discussion and got to know each other better.

How do the editors respond when you sign the review?

One thing I did not anticipate was the reaction of some editors. It happened several times that the editor erased my name from the review (without notice). Some journals have a strict anonymity policy, but they usually do not mention this upfront. Nowadays I usually tell the editor that I will sign the review at the time I accept the assignment and I invite him or her to tell me beforehand whether this is a problem. If it is, I politely refuse the review.

An editor has also removed my name without proper explanation when I was a postdoc. It seems that this person wanted to hide from the authors the fact that the review did not come from an established researcher of the field. Later invited as a guest-editor (when I had a faculty position), I realized how hard it can be to find qualified reviewers and I better understood the position of this editor. Peer review is based on volunteering and you cannot force the most appropriate expert to care about a manuscript. And nobody likes to be judged by people that are more junior than them, so it is tempting to hide this fact from the authors.

I came to think that this is a major problem and that anonymity may hide the fact that junior scientists do most of the reviewing (I don't know whether this is case). Using authorship attribution methods, I could become very confident that some of my papers were reviewed by PhD students. Actually, many senior scientists do not hide that they let their students do the review job for them. This is worrisome, to say the least. Anonymity of the peer review process is not the cause, but it may favor it to a large extent.

Can you reject manuscripts if you sign the review?

I heard this question very often. Here is the catch: manuscripts are not rejected by the reviewers; they are rejected by the editors.

In general, I do not express any opinion regarding whether the manuscript should be published. I assume that if the editor needs advice from an expert, he or she is interested in the manuscript in terms of topics and scope. As a reviewer, I sketch a list of strong and weak points that I feel confident to raise. That’s it. Sometimes, I know that some of the weak points are devastating for the manuscript, but I still refrain from using “fatal” or other adjectives to describe them. This is already a judgement, which I leave to the editor. In any event, I try to make sure that rejection, if it happens, will be useful for the authors.

That said, I have to be completely honest, and I find it hard to write a very negative review. It is completely possible that some bias makes me “nicer” than I should, but being aware of this, I typically pay great attention to monitor myself the best I can. At least, this bias is explicit, whereas under anonymity, most biases are implicit.

Finally, I have to say that I sometimes refused some review assignment because I thought that I would not be in capacity of writing a negative review if I had to. This typically happens for manuscript written by close collaborators, but I try to not exclude a review assignment just for that reason. Writing a review is difficult, and even more so when other elements come into play. I think it is a healthy attitude to refuse an assignment when you get the feeling that you will not be able to do a good job.

What are the benefits?

I sign my reviews because this is in line with my ethical values. Even if this was a purely moral decision, it came with benefits that I did not foresee and that are worth mentioning.

The most obvious is that reviewing has become part of my scientific networking. In some cases, I started to collaborate with the authors of a manuscript I reviewed, even when the manuscript was rejected. In hindsight, I find it rather normal that scientists interact over their work. In principle, the reviewer has acquired enough insight to start an in-depth discussion that makes it more valuable than the usual high-level introduction at a conference. This in turn makes reviewing part of the normal scientific interactions, instead of a tax on researchers’ time for the sake of the system (with the possible negative consequence of delegating peer review assignments to the least qualified researchers).

A less obvious benefit is that signing my reviews is the main motivation for improving my reviewer skills, because I can run into serious trouble if I don’t. So I usually seek advice from the editor and we sometimes discuss the review and the scientific merit of the manuscript. This is a great way to get in contact with editors and to know them a little better.

Conclusion

I do not wish to convince you to sign your reviews. I just want to share my experience so that you can take the decision that works best for you. I do not think that signed reviews are more thorough than anonymous reviews, but signing my reviews made me see the review process in a different way. I hope that some of my thoughts will give you some ideas on how to become a better reviewer or how to get more out of the review process.