What constitutes a good peer review?
First impressions
As with many things in life, first impressions are always important. In peer editing, this is true in more ways than one. This does not mean that first impressions determine the value of the article, but they can give a first general idea of what might be found when reading the text. However, it is very important to know that, as in real life, first impressions can be misleading and not give a good picture of what is really happening in the text. I would say that feelings are not allowed at this stage because they can completely distort the final verdict for the article. So in that sense, first impressions are just a simple tool. This stage is the shortest. It’s just a first impression.
- The title. When the title is too broad or too vague, it could be expected to find objectives not very well defined. On the other hand, when the title is too complicated, it could be expected that the text will be complicated, because it could be a way of writing of the author.
- The structure of the article. With a quick look through the article we can understand a bit what the author is trying to do.
- Spelling. This may sound very trivial, but if there are big mistakes in the title, subtitles, or if it is obvious that there are many spelling mistakes that can be seen in a quick look, it means that the final version submitted for peer review has not been read carefully. This is not one of the most important things, but it gives the reviewer an idea that a very final revision was missing.
Abstract
The best way to understand what an article is about si reading the abstract. I think that reading the abstract gives you a very good idea of the article is about. However, it can happen that the author or authors were not able to summarize what they did. So it is important to continue reading and also avoid pre-judgements about the quality of the article.
Objective
If I had to choose where we find the core of the article is here, in the objective. I think we all have struggled with this in writing our final papers and all other pieces of writing. If the objective is not clear, then the whole text is just a mixture of good reflections, quotations and or citations, but not giving the reader that we hit the target.
From my experience in reviewing articles, if the article is not clear, then the whole structure of the text is not strong, the methods are too general, deal with too many things or are the wrong methods. Also, the discussion is a repetition of what other authors have said, and the conclusion is vague or too general. It could also be that the conclusion is again a citation of other authors’ work, but does not explain what this article really concludes. Sometimes conclusions are statements that could be applied to many other articles, that could be copied and pasted at the end of many other articles, and would fit perfectly.
So I think that being able to read, understand, and analyze objectives is the core of peer reviewing. And to do that, a good peer reviewer must also be a good writer, but a good writer who defines clear and concise objectives in his or her own papers. Otherwise, the task of peer reviewing would not be a good one.
Tips
I think that a peer reviewer can follow this tips to do good work:
- Have a first impression with a very quick look
- Read the abstract
- Go directly to the objective
At this stage it could be useful, before reading the whole paper, that the peer reviewer write down what they expect to find in that paper, based on the objective. This is a good way to see how the author could have developed the paper. It also helps the peer reviewer to be better trained on what objectives should look like.
4. Read the full article without making any written comments
Sometimes when I write comments during this first reading of the article, we question many things that are explained later, and usually I have to delete those initial comments. So it is better to have a general idea of the article and then make the written comments on the second and subsequent readings.
5. Be kind and respectful
Often we think we are right and that others are making mistakes. And sometimes peer review is seen as a judging task and not as a learning task. This does not mean that we cannot be as critical as we need to be and that the quality of the article is not being judged, but usually we think that we have a lot of power and therefore feel the right to be mean and make disrespectful comments to the authors.
The best thing for me is to make corrections to others as I would like others to do to me. Then we will find a balance between being a good reviewer and a good colleague who is also in a learning process. We’ll all be learning things until the last minute before we die!
6. Be concise
Our comments as peer reviewers should be as concise as possible. They should be fully understood. We cannot ask for clarity if we are not clear.
7. Be precise
It has to be clear where we are making the comment or the corrections in the text, and what we are correcting.
8. Be honest
I think that reviewers need to be honest with the editor of the journal and refuse to review articles that are beyond their expertise.
Hi Carlos, thanks for sharing your thoughts on peer-editing! I found it interesting that you said that feelings are not allowed at the beginning stages of the peer-review process. For me, they are my guide! Of course I can be logical and rational, but my first impressions of something are also anchored in feeling. For instance, “I feel this point needs to be more precise”, “I feel the rest of the article is going to be about X”, or “I feel this section is well-organized.” Maybe that’s because I often navigate the world through affect (how very typical INFJ of me), or maybe that’s because thinking and feeling are more intertwined than we think, but whatever mental processes are going on in my head, they work.
I also liked your suggestion of writing out what you think the rest of the article will be about after reading the objective. If what I find in the rest of the article drastically doesn’t align with those notes, I can provide that as constructive feedback to the author(s). That will help them say what they mean in the objective.
Finally, I found it interesting that you suggested not making written comments on the first read. While this is fair to the author(s), I find I need to constantly jot down notes, even on a first read. Otherwise, I’ll forget what feedback I want to give. I find my mind is freshest on a first read, so maybe I’ll just need to delete comments later on.
Thanks again for your useful tips!
Carlos –
I’ve reviewed many articles before, and my mantra has always been precise, concise, AND meaningful. I’ve always felt that there are lots of changes we can make in reviewing someone else’s work, but we should ensure that the changes have meaning – both for the author, and to the reader as well.
Is this something that you have observed? How do you ensure that the changes are really being meaningful?
Robb
Hi Robb,
These three words are a good combination: precise, concise and meaningful.
You mentioned a difficult task: “ensure that the changes have meaning – both for the author, and to the reader as well”. I think that this is the most challenging thing. I’ve had that experience. I’d say that if I was chosen for doing peer reviewing, then I would give what I can offer. So my comments will be based on my experience and background and for some articles this means a lot of changes and for others not many changes. I think that to purify this we need to avoid perfectionism. For me that would be the fourth element of your mantra.