EVALUATING SOURCES
EVALUATING SOURCES
Once you find books, articles, essays, web pages or other documents on a research topic, it is very important to evaluate those sources of information to try to develop a general understanding of the purpose, value, perspective and quality of any source you use in your research.
Preliminary Evaluation of a Book or Article
Finding Information About Authors
Finding Evaluations of Books from Book Reviews
Questions for Evaluating and Annotating Any Source
Evaluating Journal
Articles: Following
Citations
Evaluating many types of sources can begin even before you have the source in hand by:
If you find a book listed in a library catalog or if you access an article from a periodical database or index, you can initially appraise a source by examining the bibliographic citation. If an abstract is available, that can provide even more information. Sometimes, you can get a sense of a book or an article just from the title. Is it a formal or informal title? Is there a subtitle? Look at all the subject headings listed to get a little better sense of what it's about. The author's name is useful initially if you know something about this person. If you are not familiar with the author, you can look up information about them (as explained below.) Aside from the title, subject headings and author, the bibliographic citation generally includes:
By answering some questions about each of these elements from the citation, you can start to develop a critical evaluation of a source. Let's look at each element more closely.
Date of Publication
When was the source published? If you didn't get the publication date from a bibliographic citation, you can usually find the year of publication for books on the page after the title page. The year of publication may be the same as the copyright date or it could be a later date. For periodicals, the date is usually on the cover and also often at the top of table of contents.
Is the source current or out-of-date for your topic? Topic areas of continuing and rapid development, such as the sciences and technology, demand more current information. On the other hand, historical topics may require material that was written many years ago.
Edition
If the source is a book, is it a first edition or not? Further editions indicate a source has been revised and updated to reflect changes in knowledge, include omissions, and harmonize with its intended reader's needs. Also, many printings or editions may indicate that the work has become a standard source in the area and is reliable.
Publisher
Note the publisher of any book, if you can find the information. If the source is published by a university press, it is likely to be scholarly. Although the fact that the publisher is reputable does not necessarily guarantee quality, it does show that the publisher may have high regard for the source being published.
Title of Journal
Is this a scholarly journal or a popular magazine?
This distinction is important because it indicates different levels
of complexity in conveying ideas. If you need help in distinguishing
scholarly from non-scholarly periodicals, click
here to see the section, "Periodicals and Periodical
Indexes." Check the periodical titles in
the latest edition of Magazines for Libraries (Ready Reference
shelves: Z 6941 .K21 1995) for a brief description. This one-volume
reference book includes important evaluative information, such as the
political perspective and type of articles included, for about 6500
different magazines.
There are numerous questions you should ask yourself in order to evaluate the author (or authors) of any source. Who is the author? What are the author's credentials--educational background, past writings, occupation, position, titles, experience, political perspective? Is the book or article written on a topic in the author's area of expertise? Is the author qualified (or not) to write on the subject?
In articles, brief information about the author is often included on the first or last page. In books, information about the author may be included at the beginning or end of the book, or on the back or on the inside cover. If the book includes an introduction written by someone other than the author, some details about the author will usually be mentioned.
Have you seen the author's name cited in other sources or bibliographies? Respected authors are cited frequently by other scholars. For this reason, always note those names that appear in many different sources.
Author Information from Books, Periodicals or the Internet
Beyond information provided in the work, itself, background information on authors may be found in a variety of other sources.
If the author is an important literary figure or an otherwise very popular individual, entire books may be written about him or her. Books about people can be found in library catalogs by searching by subject. (Remember to always search by the last name first.)
It is more common to find magazine, journal or newspaper articles or Internet documents about authors. To find magazine or journal articles with information about an author, use periodical indexes. Start by searching by subject (last name first) to see if any entire articles are written about the author. Full-text periodical databases and Internet search engines provide more possibilities of at least finding an author's name mentioned in an article or document. In full-text databases, enter the name of the author you are researching first name first. When using Internet search engines, enter the name in quotes first name first.
Author Information from Reference Sources
In addition to looking in books, periodical articles and Internet documents, some special reference sources provide information about all types of authors.
Contemporary Authors (REF Z 1224 C6), often referred to as CA, is one of the most comprehensive sources for basic biographical and bibliographical information on nontechnical writers of all genres-- fiction, nonfiction, poetry, drama, etc.-- who have lived since 1900 and whose work has been published in the U.S. or translated into English. CA includes information on approximately 100,000 modern writers in a set of over 100 print volumes (also available in CD-ROM or online databases.) Authors included in CA have been identified as the most commonly read and studied internationally.
Other biographical reference sources that may
provide useful information about authors include Biography
Index and Who's Who in America. For more help, ask a
reference librarian.
It is particularly useful to find evaluations of a book from other sources. Knowing what other people say about a book may tell you whether the book is worth reading or not. It also can be very helpful to give you an introduction to the author and the ideas presented.
The most common evaluation of books is the book review. A book review is an article that describes and evaluates a specific book. Book reviews are published in magazines, journals and newspapers and are available on the Internet. Book reviews are indexed in most periodical indexes.
To find book review articles, you can use various types of periodical indexes. Book Review Digest and Book Review Index are specialized indexes that exclusively index book reviews. Both are described below. General periodical databases, such as Infotrac Academic ASAP, or the Lexis/Nexis "News Group File, Most Recent Two Years" or "News Group File, Beyond Two Years" files, and newspaper databases such as the Proquest Newspapers database all include book review articles, usually for books of relatively general interest. In order to find reviews of books on more specialized topics, it is often necessary to use subject periodical databases, such as the WilsonWeb Omnifile database, that index periodicals in the book's subject area.
In Infotrac Academic ASAP, WilsonWeb Omnifile database or the Proquest Newspapers database, simply search for the title of a book as if it were your subject.
In the Lexis/Nexis "News Group File, Most Recent Two Years" or "News Group File, Beyond Two Years" files, search for the book title in the "hlead" segment and AND the words "book review".
When using other periodical databases to search
for book reviews, you can either simply do a keyword search just for
the title of the book (if it is unique) or you could do a keyword
search for both the title AND the author. If searching title and
author does not narrow your search enough, in many databases you
could limit your search by ANDing the term "review" with the title.
(In some full-text databases, when ANDing "review", it may be more
effective to limit the term "review" to a field such as "article
type". Check the documentation of the specific database for the
precise terminology to use.)
There are a number of Web sites that specifically provide book reviews written either by professional reviewers or by readers who are interested enough to write and send in their personal reviews. Yahoo has directory of many of these book review sites at http://www.yahoo.com/Arts/Humanities/Literature/Reviews/. A major limitation of most of these sites, however, is that the books reviewed tend to be primarily recent popular works. Newspapers on the Web can be a good source for reviews of recent books. Amazon.com, an online bookstore, lists a huge number of books of all types, many of which include brief reviews. It is very important to check closely the source of reviews or descriptions included in online bookstore sites since they tend to include more favorable descriptions (that may seem like reviews) submitted by publishers. Brief reviews submitted by readers are also often found on these sites. General web search engines (e.g., Google, Northern Light, AltaVista, etc.) can also be used to try to find book reviews on the Web. Try doing a search for just the title of the book (in quotes).
Book Review Digest
One periodical index that specifically indexes book reviews is Book Review Digest, often referred to as "BRD". This database, available on the Web on the WilsonWeb site , includes a brief summary of each book listed and citations for each magazine article that reviews the book. Short excerpts from the reviews are included with most citations so that you can immediately get an idea of reviewers' opinions of the book. The number of words of each complete review article is also given. A significant limitation of BRD is that it indexes reviews from only 80 periodicals and reviews for non-fiction books are only listed in BRD if there have been at least two reviews in the selected periodicals while fiction books are only listed if there have been at least four reviews of the book in those 80 publications.
To look up reviews of a particular book in Book Review Digest:
If the book was published in 1983 or later, use the BRD on the WilsonWeb site. You can search by author or title. There will be one record for the book itself, including a brief description of the book either from publisher information or from one of the review articles, and there will be records for each review article which include key excerpts from the reviews.You can also search for book reviews by the subject of the book. Then, if you find the title(s) of book(s) from a subject search, you can then search for the title(s) of the book(s) to find records for book review articles for the book(s) you found.
For older books published before before 1983, use the BRD cumulative print indexes to look up the year(s) that reviews of the book have been listed in BRD. To find the year(s) that reviews of a book have been listed in BRD, look up the title or author of the book in either the four-volume 1905-1974 index or in the one-volume 1975-1984 index. The index will refer you to the date of the BRD volume in which reviews for that book are cited. Once you know the year(s) that reviews of a book have been listed in BRD, look up the book under the author's name (last name first) in the annual volume for the given year(s).
When you find the listing for the book, you will see the catalog information for the book (including publisher, publication date and subject headings), followed by a brief paragraph that describes the book and then excerpts from various reviews of the book. At the end of each review excerpt is the name and date of the journal from which the review is excerpted, the page number (listed after the colon), the reviewer's name and the number of words of the review.
Beyond finding out about the author and considering the basic citation information, it is very important to do more thorough critical thinking about any source you use in your research. For books, scan the Table of Contents and the Index to get a broad overview of the material covered. Note whether bibliographies are included. Read the chapters that specifically address your topic.
A very useful method of organizing your ideas about a source is to write a critical annotation of that source. An annotation is similar to an abstract, which is a brief summary of an article or book. While an abstract usually just provides a summary of what is included in a source, an annotation generally includes a critical evaluation of a source.
To try to systematize the process of writing critical annotations, librarians at the McIntyre Library, University of Wisconsin-Eau Claire developed a series of questions to ask about any book, article or other publication that is to be critiqued . The questions (slightly revised) are listed below. Linked from the list of questions are some suggestions for how to answer each of the questions in order to evaluate different types of sources and to develop your own annotations.
1. What is the author's purpose for writing the work?
5. What are the author's conclusions?
6. Does the author satisfactorily justify the conclusions? Why or why not?
These questions should be used as general guidelines in the process of analyzing a book, article or other work. It will not always be possible to answer all of these questions about a work. If the subject of the work is relatively new to you, it will be more difficult to answer some of the questions; but the process of trying to answer the questions should help you become more familiar with the subject. As you gain more background on the subject, you will increase your ability to critically evaluate writing on the the subject. Whether or not you can answer all of the questions, however, simply taking the time to think about each question for each book or article that you use in your research can be very helpful in developing your critical thinking skills in the research process.
Gaining some information about an author's background can often shed some light on the author's motivation for writing a work. Although sometimes an author may purposely hide his reasons for writing something, usually authors will openly state their purpose. This will generally be presented within the first several paragraphs of an article. A book's preface or introduction, whether written by the author or not, will usually provide significant information not only on the book's purpose, but also on the author's background, point-of-view and how the book relates to similar works.
The intended audience of a work is rarely stated explicitly, but it can usually be inferred from various clues. The purpose of a work may suggest a particular audience. Other evidence may be found in the writing style, use of jargon and footnotes. Use of a specialized vocabulary, such as medical or legal jargon, for example, would imply that the work is intended for medical or legal practitioners. Information about the periodical in which an article is published will provide a good indication of the audience for a specific article. As mentioned above, Magazines for Libraries (Ready Reference section: Z 6941 .K21 1995) is an excellent reference tool for finding basic descriptions of periodicals.
An author's point-of-view or bias may be his political orientation or a literary, artistic, scientific or other academic school of thought. Familiarity with the subject of a work is especially helpful in detecting an author's viewpoint. Background information about the author will often provide some evidence of his political perspective or school of thought. Understanding the author's purpose in writing the work will commonly suggest his bias on the subject. The perspective of an author may not always be detectable in his writing, but if it is possible to determine the point-of-view of a particular work, that information can be very helpful in understanding the work as a whole. Particular biases or perspectives should not be thought of as correct or incorrect, but as an important piece of data to be considered when analyzing the work.
It is not always easy to separate fact from opinion. Facts can usually be verified; opinions, though they may be based on factual information, evolve from the interpretation of facts. Skilled writers can make you think their interpretations are facts. Assumptions should be reasonable. Note errors or omissions.
The method of obtaining data or conducting research for a work will vary according to the type of work and the discipline. More popularly-oriented articles or books may simply be based on an author's personal opinion, first-hand experiences or on a limited amount of interviewing or library research. More academic works generally involve more extensive research. Depending on the discipline, data-collection techniques may include bibliographic research, laboratory experiments, textual analysis, surveys, empirical observation, etc. Most works use various combinations of techniques and the methods used are commonly specifically stated in the work.
Primary sources are the raw material of the research process. Secondary sources are based on primary sources. For example, if you were researching John F. Kennedy's role in the United States' involvement in Vietnam, Kennedy's own writings would be one of many primary sources available on this topic. Others might include relevant government documents and contemporary newspaper articles. Scholars use this primary material to help generate historical interpretations--a secondary source. Books, encyclopedia articles, and scholarly journal articles about Kennedy's role are considered secondary sources. Choose both primary and secondary sources when you have the opportunity.
The conclusions of a particular work are usually relatively easy to find in a work. In articles, in particular, sentences or paragraphs near the end of the piece will often begin, "In conclusion," "In summary," or with some other explicit wording. Books often include a concluding chapter. Although the conclusions of a work are very important to consider, they should be thought of as one element in the overall analysis of a work.
Determining how well an author justifies his conclusions can be quite difficult for those not very familiar with the subject of a work. For example, those more familiar with the subject area are more likely to recognize significant information that was not included in a work. In considering how well a work's conclusions are justified, it is usually helpful to think about the author's purpose, point-of-view, intended audience and methods of collecting data at the same time as analyzing the conclusions. The general question to ask is: Do the facts in the work fully support the conclusions? Some specific questions to consider about the work are: "Is it internally consistent and does the logic follow? Did the bias determine the method of data collection or the interpretation of the data collected? Was the method of data collection the best one for the original purpose?"
7. Comparison with Similar Works
Although knowledge of the discipline of a work is very helpful in determining how that work relates to the subject matter as a whole, authors often explicitly mention other writers with whom they agree or disagree. In addition, footnotes and bibliographies provide specific references which, in turn, can be researched for comparisons to the original work.
Attachments, such as charts, maps, survey
instruments, appendices or bibliographies, can provide important
clarification or enhancement to the basic text of a work. Often, when
specific data--such as survey responses or statistical data--is
referred to in a work, the actual data may be included or summarized
in an appendix. At the same time, the lack of certain supporting
attachments may suggest a definite limitation in a work. Such
supporting appendices should not be overlooked when evaluating a
work.
Book review articles provide critical analyses of books, but how can you find evaluations of articles from journals? Individual periodical articles are rarely reviewed exclusively in a single reviewing article unless they are extremely significant or groundbreaking new works. While it is rare to find an article solely dedicated to reviewing another article, scholarly journal articles (as opposed to articles in professional journals or popular magazines) commonly refer to and comment on various different authors' articles. Journal articles written in a specific academic research area regularly cite other articles that cover the same area of research. The citations are often used to document support for a particular point being made in an article or, sometimes, to refute a position taken by the article being cited. The way to find evaluations of a particular article, then, is to find other articles that have cited that article. Once you find articles that have cited the article you want to evaluate, read those articles to see what they say about the cited article.
Following the writing of group of researchers who all write about a particular academic research area and who regularly cite each other in their journal articles can be like listening to a scholarly conversation. Each new article usually addresses previous articles and makes new points which may elaborate on or disagree with the articles it cited. This process of finding citations to new articles to be read in a particular research area by following up on references cited in other articles on the topic is probably the most common process of research used by most academic researchers. A researcher may initially find out about a particularly useful article from discussions with a colleague, from listening to a lecture by an expert in the field or by scanning the latest issue of an academic journal in her subject area. As she reads the article, the researcher makes a list of those sources cited in the article which are of interest to her, and she then locates and reads those items. She in turn follows up on any citations listed in those publications which seem promising to her research. Click here to see a diagram showing an overview of this process (shown at the end of the section, "Research Process Models.")
Citation Indexes
A special type of periodical index, called a citation index, can be used as short-cut in this process. (It is important to distinguish these special citation indexes from the more general use of term which is occasionally used simply refer to any index that happens to just include citations.) Citation indexes provide lists of articles that have cited specific previous articles and also list all sources cited in each article included in the index. These indexes are important research tools since they enable researchers to keep track of who is citing whom without necessarily reading the articles. In order to use these indexes most effectively, however, you should first be familiar with the name of at least one author in your subject area.
The three most important citation indexes, each of which covers broad, multidisciplinary subject areas, are Science Citation Index, Social Science Citation Index and Arts and Humanities Citation Index. All three indexes are published by the Institute for Scientific Information. Science Citation Index provides comprehensive indexing of over 3000 journals in all areas of the sciences from 1963 to the present. Social Science Citation Index fully indexes about 1400 journals and selectively indexes approximately 3300 additional journals covering the full range of the social sciences including anthropology, business, psychology, sociology and women's studies. Annual coverage begins in 1969. Arts and Humanities Citation Index indexes 1300 journals in depth and covers another 4500 journals selectively. It covers a wide range of subject areas from architecture through theology going back to 1977. Although all of these indexes index a wide range of articles, no abstracts were included until the early 1990's.
To use one of the citation indexes, you most commonly begin with a citation to a source. The goal is to try to find articles that have cited the source, that is they have referred to or commented on the source in some way. You will be more likely to find citations to sources that are relatively significant to the field.
All three of these citation indexes are available in print and online. Online searching is the easiest method for searching the widest time period in a single search. Using the printed volumes is the most difficult, not only because you have to look in multiple indexes to cover multiple years, but also because the process involves using at least two separate volumes for each year searched.
The printed volumes of these indexes include four basic parts: the "Citation Index," the "Source Index," the "Corporate Index" and the "Permuterm Subject Index." To use the printed volumes, you would start with the most recent cumulative "Citation Index." The first step would be to look up the author of your source, in our example. If an author has more than one publication, a citation (including year of publication, abbreviated journal title, volume number and first page of article) for each publication is listed under the author's name in chronological order. Under the citation for each publication is a listing of articles (in alphabetical order by author) that have cited that publication. To find more information about any of the citing articles, you would look them up in the "Source Index" under the the author's name. In addition to the information included in the "Citation Index," the "Source Index" includes the title of the article, any co-authors, the issue number, pages, the total number of references included in the article and the author's address at the time the article was published. This information is followed by a listing of each of the references included in the article.
Using the online version of these indexes is a simpler process than using the print volumes as long as you are familiar with how to search by field.
The "Permuterm Subject Index" provides a very
limited type of subject access based on keywords appearing in the
titles of articles. This index is computer-generated; it is not based
on any professional indexing or controlled vocabulary. If the title
of an article does not include useful keywords that adequately define
the subject of the article, a successful subject search for the
article will be impossible with this index.
| Home | Syllabus | Assignments | Text | Student Projects | Instructor |
last revised: 3-28-00 by Eric Brenner, Skyline College, San Bruno,
CA
These materials may be used for educational purposes if you inform and credit the author and cite the source as: LSCI 105 Online Research. All commercial rights are reserved. To contact the author, send comments or suggestions to: Eric Brenner at brenner@smcccd.cc.ca.us