How to Review a Scientific Paper
Barak A. Pearlmutter
Purpose of this document
This document is intentionally brief. It is intended as a combination
checklist and reminder. For more complete overviews of the review
process, see the delightful document linked to in the acknowledgments,
and the references therein.
This guide is not meant to be a straight-jacket, but rather something
to turn to when you don't know how to start writing your review, or to
check when you're not sure whether you've finished it.
What is in a review?
A scientific review consists of two parts: a confidential cover
letter and the anonymous referee's report.
The cover letter is addressed to the journal editor, and contains
information that will not be forwarded to the authors. This includes:
The cover letter is also the place for you to
- your name.
- the paper's title, authors, and code number.
- a recommendation (accept, reject, etc.) and brief
- No sneak attacks!
The recommendation belongs in the cover letter, rather than the review
proper, because then your review can be forwarded to the author
directly, without editing, even if your recommendation is not
- describe your expertise in the subject area, especially if
the paper is not in precisely your own line of research.
- say how confident you are of your views of the paper.
- mention how much time you put into this review.
- if you did not actually check equations in the
paper, this is the place to say so.
- remind the editor of any potential conflicts of
interest you may have (naturally you mentioned these earlier,
before agreeing to review the paper!)
- acknowledge anyone who helped you with the review.
- include personal correspondence with the editor.
- The referee's report is usually forwarded to the authors,
and sometimes to other reviewers. It can typically be divided into a
number of sections:
- summarizes the paper succinctly and
dispassionately. This is not the place to criticize, but
rather to show that you understood the paper, and perhaps
discuss how it fits into the big picture.
- general comments
- gives the big critical
picture, before sinking into the details. This is the place
to take a breath, keep your perspective, and explain what the
papers weaknesses are and whether they are serious, or
intrinsic to our current state of knowledge, or whatever.
- constructive criticism
- not only of technical issues, but also organization and clarity.
- table of typos
- and grammatical errors, and minor
textual problems. It's not the reviewer's job to copy edit
so don't go out of your way to look for typos. And if the
paper is a complete mess, just say so---but please be
charitable, especially if English is not the author's native
This document was inspired and influenced by A
guide for new referees in theoretical computer science by Ian Parberry. To quote:
``Desirable traits in a referee include objectivity, fairness,
speed, professionalism, confidentiality, honesty, and
courtesy'' and ``Before submitting a finished report, a wise
referee asks ``Would I be embarrassed if this were to appear in
print with my name on it?'' ''