Materials Science and Engineering

Checking for Peer Review

When you find a research article from a web search, it may be difficult to tell whether it is peer-reviewed at first glance. Here are a few steps to figure out if an article is peer-reviewed.

To find out if a journal is peer reviewed (also known as "refereed"), you can use UlrichsWeb. Search for the journal and look for the tiny referee shirt as an indicator that it is a peer-reviewed journal. For open access journals, you can also check to see if it is listed in the Directory of Open Access Journals. Be careful of journals with similar titles! You can search by ISSN Number to disambiguate journal titles.

Example: The Journal of the American Chemical Society is verified to be "refereed" by Ulrichs.
icon of referee shirt
Even within journals which do peer review, not every entry is a peer-reviewed article. Some scholarly journals also publish perspectives, conference notes, news stories, and other items. Look at the full text of the article you're interested in: a peer-reviewed article will show a string of dates to indicate that the article was reviewed and accepted for publication.

Example: A recent research article in the Journal of Chemical Education lists that it was Received 30 April 2021, Revised 25 September 2021, Published 11 November 2021.
Some organizations create fraudulent journals with the intention of deceiving authors and readers, which is referred to as predatory publishing. These journals may claim on their website that they use peer review when they do not. Whenever you are in doubt, investigate the journal itself by searching the journal title elsewhere (even just using Google or Wikipeida can help). Check to see if the journal shows up in reputable databases (like Web of Science). See if the journal has an established history and trusted reputation among researchers in the field.

Example: Biologist and science journalist John Bohannon was able to intentionally publish nonsense research claiming that eating chocolate helps with weight loss in the predatory journal International Archives of Medicine.

🚨 Fraudulent (or "predatory") journal articles can show up in Google Scholar results and look very convincingly like legitimate peer-reviewed articles. Check out the video below to learn about identifying predatory journal articles. 

 

Video Transcript: "Is it Really Peer Reviewed?" by Michelle Nolan

 

You’ve probably found journal articles through Google Scholar before, but are they really peer reviewed? How can you tell?

Peer review is a series of quality tests performed by editors and other researchers in a field prior to accepting a publication in a scholarly journal.However, there’s a problem in the scholarly publishing world right now with what we call predatory publishers.

Predatory journals deceive authors and readers by claiming they have a peer review system when they do not. So you might be reading something that you think is peer reviewed only to find out it has never been reviewed at all.These essentially fraudulent journals trick researchers into submitting their work and paying large sums of money in publication fees. But it’s just a scam.

So why am I mentioning Google Scholar? Unlike some of the subscription databases your Libraries offer which vet journals for legitimacy, these fraudulent journals show up in Google Scholar all the time. And aesthetically, they can look pretty convincing.

Take for example this predatory journal that I get emails from almost every semester. At a glance, it’s very convincing. To an untrained eye, it’s easy to get tricked into thinking this is an actual peer review journal.If I open up one of the articles in it, the saddest thing is that this is probably legitimate research that the authors were tricked into submitting to this predatory journal instead of a real one.So while this really looks really looks like it’s a peer reviewed article, this has not been peer reviewed.

I see students and professors get fooled by this all the time. So, how can you actually check if the article that you’re reading is legitimately peer reviewed? I’m going to give you a couple of tools that you can use.

The first tool that I really like is the UlrichsWeb Global Serials Directory. This directory presents information on different publications, including when the publication started, whether they have a verified peer review system, how often issues are published, and more.

You can search by title in Ulrich’s, but be aware that some predatory journals purposefully choose titles that are very similar to legitimate journals in order to fool you even more.

An unambiguous way to search is to look for a journal’s ISSN number. I’m going to pull that and copy and paste it over. An ISSN number is just like an ISBN number on the back of a book. It’s a number assigned to a publication.

When I search for this ISSN number, you can see by looking for this black and white striped referee shirt (which is not in this column) that this is not a peer review journal.

The reason why they use a referee shirt is because in some fields, peer review is called “refereeing.”

In contrast, if I search for a legitimate journal: this is the ISSN number for the Journal of the American Chemical Society. You’ll see in that column there is the black and white referee shirt which shows that the Journal of the American Chemical Society is indeed peer reviewed.If I open up the entry for the journal, you can find more data about this publication that Ulrich’s has collected.

Now, Ulrich’s isn’t always perfect, especially for new journals that just started. But it’s a good starting indicator to get an idea of what you are looking at. Everyone at the University of Florida has access to Ulrichs. If you’re watching from another institution, check with your librarians about what tools are available to you.

Open access journals are free for the public to read and do not require institutional subscriptions. Since most predatory journals claim to be open access journals, another great tool to check is the Directory of Open Access Journals, or DOAJ. This is a free website that everyone has access to. The DOAJ is a curated list of legitimate, open access, peer review journals. They have strict vetting requirements to get a journal listed. You can search for a journal title in DOAJ to see if they are included in the list.

So once again, if I search for this ISSN number from the [predatory] journal that I had seen before, it does not come up. So this predatory journal is not listed in the Directory of Open Access Journals.

Okay, if you’ve made it this far in this video, here is my all time favorite story related to predatory publishing. In 2005, computer scientists David Mazières and Eddie Kohler responded to an email solicitation from the International Journal of Advanced Computer Technology. Which is a predatory journal.

And this is the manuscript they submitted. The authors were thrilled to learn that this article had been “peer reviewed” and was deemed “excellent research.” Now of course, this manuscript had never been peer reviewed at all. But just look at it. This thing is a work of art.

Something this egregious you would, of course, recognize as not a scholarly research article. But many predatory publications are quite convincing. So always, if you feel an inkling of skepticism about the quality of the source you’re reading, do more searching around until you are satisfied.

I’m Michelle Nolan, Chemical Sciences Librarian at the Marston Science Library, University of Florida. Happy searching.

 

Evaluating Academic Sources

When it comes to evaluating the quality and reliability of a source for your research, there is no "one solution fits all" checklist to follow. Here are some starting questions to consider when evaluating sources.

Is the author an expert on this topic? Check for their institutional affiliations. Google the author to learn more about their work and expertise. Check that the information discussed in the source matches the scope of their expertise. Question whose voices might be left out of the conversation.
Is the publisher reputable? Did you find this source listed in a scientific database? Explore the greater website that is hosting this source. Google the publishing organization to learn more about their reputation.
Consider the type of the source and how the information was created, revised, and shared. Check for conflict of interest statements and other unnamed biases. Is this source mean to inform, teach, sell, or persuade?
Some areas of expertise grow slowly and some change rapidly. How recently was this information published? Is there more recent information available elsewhere? How have perspectives changed over time? Are there any corrections or retraction notices on this source?
Where does the information in this source come from? Has this source been reviewed by other experts? Check the references that the author cited and investigate them further.
Does this source address your research question? Does the context of the information match your needs? Is the information at an appropriate level? Why would you choose this source instead of another?

Examining a Source

The Source

Gordon, C. G.; Mackey, J. L.; Jewett, J. C.; Sletten, E. M.; Houk, K. N.; Bertozzi, C. R. Reactivity of Biarylazacyclooctynones in Copper-Free Click Chemistry. J. Am. Chem. Soc. 2012, 134 (22), 9199-9208. DOI: 10.1021/ja3000936

Who is the author? Are they an expert?

The corresponding author of this article is Dr. Carolyn R. Bertozzi, who is currently a professor in the Department of Chemistry at Stanford University. At the time of publication, she was a professor at the University of California, Berkeley. Dr. Bertozzi has an extensive history of research that has been highly cited by other scientists. She is recognized as a leader in the field of bioorthogonal chemistry. This source reports research that is within her area of expertise.

Where was this source published?

This source was published in the Journal of the American Chemical Society (ISSN 0002-7863), which is the flagship journal of the American Chemical Society (ACS). This journal was founded in 1879 and has an established reputation for publishing scholarly chemistry research. The professional organization that sponsors this journal (ACS) has a large membership and hosts regular conferences. This journal is verified to be refereed in UlrichsWeb. 

What is the purpose of the source?

This article has been peer reviewed, meaning that it was assessed for quality and significance by other researchers in the field. It is meant to report and share scientific research results to an audience of fellow scholars. The authors of this source declared no conflicts of interest or competing financial interest. 

When was this source created?

This article was published in 2012. It has been highly cited by other researchers, meaning it may be important reading for your project, but there are likely newer published articles available that further build upon this research. It may be worth looking at what other articles have cited this source and at more recent research from the authors.

Where can this information be verified? 

The authors included a list of references to support the research in this article, many of which can be accessed online through library databases. 

Is this the most relevant source for your needs?

You would choose to read this article if it provides relevant information to your research questions and makes sense with the context of your research. For example, if you were doing a literature review on copper-free click chemistry, this source may provide information about important compounds. If you were planning to synthesize some of these compounds in a laboratory, this source may provide laboratory protocols to follow. Depending on the context, you may need to look for updated information since this article was published.  

Evaluating a science news story or social media post? Learn more at Fake News: Science Edition.
University of Florida Home Page

This page uses Google Analytics - (Google Privacy Policy)

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.