Source Material Part 1: Source Material at a Glance
No one writes or speaks in a vacuum. Authors reporting on current events, making an argument, or describing new scientific research usually need sources for their information. It's important for authors to clearly, accurately, and completely convey their sources to audience, making it as easy as possible for the reader to find those sources.
How can you be sure that an author's sources are valid and reliable places from which to get information? How can you be sure a source actually supports the claims made by an author? To answer questions like these, we need to dig deeper to find and examine the sources.
This module explores basic ideas about source material, key questions to ask, and shortcuts that can make your analysis a little easier.
Is the content creator providing enough information about the source material?
If you want to check an author's writing against their source material or if you want to dig deeper into the research they briefly touch on, concepts they use, or events the report, it's good to have ready access to their sources.
In written work found online, it's generally easy for publications and authors to link directly to original source material or to otherwise reference and link to sources at the bottom of the piece. If source material seems to have been used, but you have no way of knowing the nature of that work or where that work can be found, ask why that very important information might missing.
There could be a number of reasons for missing or incomplete sources. It could be that the piece was hastily or sloppily written and posted. This can be the case with many social media posts, blog posts, and even many online articles. But it could also be because the piece is written for readability—popular science books, for example, often don't cite sources directly in the text and sometimes don't even reference their sources at the end of the book.
Alternatively, it's possible that the author has neglected to include links or a list of sources because they're trying to hide something (see People & Context module for more to consider about worrying motives and incentives).
Recall, though, that it's hard for you as a reader to detect or prove motives and remember that we as readers will sometimes infer malicious intent (e.g., that the author is being purposefully elusive) when we hold a negative attitude toward the work or the author.
Seek to maintain a charitable stance—if, for instance, haste and malicious intent are equally plausible reasons for missing sources, it's sensible to assume the author has composed their writing hastily. Hanlon's Razor could be useful mental tool here: don't attribute to malice that which could be explained by incompetence.
In any case, red flags may be raised if it's not easy to determine where the author got their information. It might not be a particularly trustworthy source. More importantly, if you can't access the author's sources, you lack a key tool for assessing the validity of the claims the author is making. As such, you'll have to do quite a bit of extra work to grasp the legitimacy of the ideas that are presented.
Kinds of Source Material
Once you've determined whether the author has provided enough information for you to dig into their sources, you can determine the sorts of source material from which they're drawing.
When the author is writing directly about science, making other claims about how the world works, or reporting about historical occurrences, it's good to see that the author is using high quality sources from a scientific literature or another relevant and reputable scholarly literature, such as peer-reviewed journal articles from a reputable field of study
Peer-review is "the process by which scholars critically appraise each other's work to ensure a high level of scholarship in a journal and to improve the quality and readability of a manuscript" (University of Toronto Library; this library site will give you a brief tutorial on the nature of peer-review). If your target article is science-focused, you want to see that the author's sources are peer-reviewed. Red flags should be raised if the author's sources appear science-based but are not peer-reviewed.
It's important to note that peer-review does not guarantee the cited work is high quality. There are certainly innumerable poor quality peer-reviewed journals out there. While determining whether the author's sources are peer-reviewed is relatively easy, determining the quality of the cited work is much harder.
As the ease of self-publishing online increases, things are becoming a little messier with regard to identifying reputable peer-reviewed sources. Academics sometimes share early drafts of unpublished work that is not yet peer-reviewed (these often come in the form of preprints). The current ease of quickly sharing ongoing work in its early stages is great for discerning scholars; however, it can often be confusing for journalists and the general public who may treat this work as established science.
As a shortcut to help you, if it comes in a Word document or looks like a PDF version of a Word document, it's likely not published or peer-reviewed. Alternatively, it may be an alternate form of the work that's been published elsewhere in more formal format. If this is the case, responsible authors will note it prominently—generally on the front page—by the authors. Check whether the authors have indicated somewhere on the first few pages whether it's unpublished work, a pre-print of soon to be published work, or an alternate format for already published work. If the authors are being transparent, that information should be there. If you can't find that information, keep your skepticism high.
It's also noteworthy that if you find one or two sources linked in a piece that aren't high quality, this doesn't necessarily make the entire work problematic. It's all about context—consider what the author is using these sources for. The author might, for example, primarily be basing their work on the science literature, but also use another kind of source for an anecdote or as a basis for comment on current events. Alternatively, they might link a pre-print to unpublished work in the context of discussing ongoing research (without using it as a foundational source for the meat of their piece). Context matters.
Much that you might read online, however, won't be reporting on science. It might, perhaps, be reporting on or offering opinions about current events. These kinds of pieces might depend primarily on other internet sources like blogs, podcasts, news articles, things people have said on Twitter, or interviews they conducted. This makes sense—for current events there isn't an academic literature that backs up the claims.
There are a lot of grey areas in opinion pieces and news stories where authors might feel they have licence to assume things they shouldn't. For example, an article may be a critique of a current trend observed on social media. In such a case, popular media articles, blogs, and other posts would likely be the primary sources for anecdotes. The problem is that if we really want evidence for the social trend, we need more than just anecdotes—we need data and competent analysis of that data by social scientists. And that's where reputable peer-reviewed works are helpful. Without evidence in the data, the author may be assuming a trend where there is none, based on a few extreme or confirmatory cases (i.e., cases that support the author's desired conclusions).
Articles that use individuals' opinions as sources of support for a belief or a position or action that ought to be taken should raise concerns. Quoting people from the general public about their views on policy is neither a good gauge of public opinion nor a good basis for an argument in favour of the policy (visit fallacies in Arguments & Truth Claims section, and in particular consider argument from anecdote). If quotes from non-experts are the basis for an argument or claim, the article might not be a trustworthy source of information.
Interviews with experts are generally better than interviews with the general public, but also need further investigation as they may be based largely on opinion, research that's in it's early stages, or research that is complete but has never been peer-reviewed and published. Always see if you can track down the original research. If you can't, you may not want to let it change your mind.
In summary, if you're reading about science, check if it's drawing on reputable, peer-reviewed sources to support the author's claims. Be careful with and think hard about readings that base their claims on unpublished or non-peer-reviewed work, blogs, social media posts, or other popular press articles.
Learning Check