Jury of Your Peers

If you ever read a popular science article that references papers, you’ll often see the phrase “peer-reviewed” (http://en.wikipedia.org/wiki/Peer_review) Scientists typically only trust peer-reviewed papers, but sometimes the public will accept papers that haven’t been peer-reviewed. With the internet, any yahoo can post an article online. So what does this phrase mean?

Peer Review is the process by which scientific papers get accepted and how the scientific community works in general. Basically, your work isn't accepted until your peers agree with you.

Steps to Publishing your Article

Step 1: Send article to a journal.

Step 2: Article is read by an editor.

Step 3: Either the article is rejected or it gets assigned reviewers.

Step 4: Paper is sent to 2-3 reviewers carefully chosen by the editor.

Step 5: The reviewers anonymously send back comments and recommend the article to be accepted or rejected.

Step 6: Author makes revisions if needed.

Step 7: Article is published!

Cartoon of the process: http://undsci.berkeley.edu/article/howscienceworks_16

Figure 1: An example reviewer. Would you like to be reviewed by this man?

Let’s take a closer look at Step 4: Choosing editors: a complicated process. The reviewers are other professors or scientists in the field.

Factors in the Process:

1)      Rivals of the author: People who will trash the paper

2)      Friends of the author: People who will only praise the paper

3)     Grading scale of the individual reviewer: Some people are negative about everything and some are positive about everything.

Overall, the editor tries to make a fair review committee.Occasionally, there are some issues because reviewers are anonymous and people often have personal biases against certain people or types of research.

One example of this issue is the case of Jo Baeler. Baeler is a math education researcher whose research has been shut down by two other professors who have done shady things to discredit her. Read about it here: http://www.stanford.edu/~joboaler/

So what’s the solution?

Some people have recommended open review where both authors and reviewers identities are revealed. Here is a Nature paper about the pros and cons of open peer review http://www.nature.com/nature/peerreview/debate/nature04991.html

Personally, I’m holding out for robots that can read papers.

Until then, the peer review process will stay in place.